Skip to content

Guard package.json test fixtures against drift from the real templates #11

@fernandomg

Description

@fernandomg

Summary

The cleanup tests mock the cloned template's root package.json with hand-transcribed fixtures (mockCantonPackageJson, mockEvmPackageJson in source/__tests__/operations/cleanupFiles.test.ts). These are static snapshots of files that live in other repos (cn-dappbooster, dAppBooster) and, for Canton, move (refType: branch, ref: main). Nothing keeps them in sync — so a real template change that cleanup mishandles can still pass tests. This is exactly how #8 shipped green.

Reproduction

  1. Add to cn-dappbooster@main's root package.json a field cleanup is responsible for pruning (e.g. the workspaces array — the Canton custom mode leaves a dangling workspace entry when a workspace-backed feature is deselected #8 case).
  2. Run the Canton cleanup tests.

Expected

A fixture that no longer matches its upstream package.json (missing a field cleanup must handle, stale scripts, etc.) is caught mechanically.

Actual

The fixture is a stale snapshot, so the tests pass against a model that diverges from reality. In #8 the Canton fixture omitted the workspaces array entirely, so the carpincho-deselected test could not observe the dangling-workspace bug despite exercising the triggering case.

Impact

  1. Silent regressions (like Canton custom mode leaves a dangling workspace entry when a workspace-backed feature is deselected #8) pass CI.
  2. Risk is highest for Canton, which tracks a moving main (vs EVM's pinned tag).
  3. False confidence — green tests on an unfaithful double.

Suggested fix

  • Committed fixture: vendor the real package.json from each template into __tests__/fixtures/ and load it (one canonical, diffable copy).
  • and/or Upstream canary: a scheduled CI job that fetches cn-dappbooster@main and dAppBooster@<latest tag> and asserts the fixture shape still matches (workspaces present + a supported form; every feature paths entry that is also a workspace is listed; scripts referenced by tests still exist).

Out of scope

The v3.3.2 fix already made the Canton fixture faithful and added invariant assertions for the workspaces case — this issue is the general, ongoing drift guard.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions