Skip to content

🎯 Stack tests pass byte-identical in every consumer (config-agnostic + extension-point isolation) #4019

Description

@PierreBrisorgueil

What β€” make every stack test file pass byte-identical in any consumer repo, whatever that consumer configures or extends. Companion of the docs-tests hermeticity work (#4014/#4015), generalized to the whole suite.

Why β€” a consumer that merely configures the stack (custom plans, custom loader component, config-driven flags) or extends it (a module hooked on an extension point) currently has to edit stack test files to keep them green. If configuration alone breaks a test, the test is wrong: it asserts a default value instead of a contract. Today ~13 stack test files carry consumer-side adaptations; each one is a merge landmine at every stack sync (--theirs wipes them β€” it happened, twice).

Doctrine (the 4 rules each sub-issue applies):

  1. Never assert a hardcoded default β€” assert against the loaded config (config.billing.staticContent.packs[0].name), or the test installs its own config fixture (set β†’ assert β†’ restore; the πŸ› fix(public): docs integration tests depend on real on-disk guides β€” make the file consumer-independentΒ #4015 pattern).
  2. Isolate extension points generically β€” stack unit tests mock the hook mechanism (registry / extension point), never a consumer module by name. If a consumer module hooking onOrganizationCreated-class seams changes a stack test's outcome, the stack test lacks isolation.
  3. Env prerequisites are explicit β€” a test that depends on NODE_ENV or an env-gated branch sets and restores it itself.
  4. Project-specific tests live in project-owned files β€” never inside stack test files.

Acceptance (epic-level) β€” a consumer repo with arbitrary config overrides and extra modules runs the full stack test suite green with zero local edits to stack test files. Verified the #4015 way: run the suite once bare, once with a synthetic "configured + extending" fixture profile β€” identical results.

Scope: validated 2026-08-05


Acceptance amendment (2026-08-05, post-audit): the synthetic "configured + extending" whole-suite run is superseded by a real-consumer proof: a consumer repo with its own config overrides and extra modules syncs this epic's changes and runs the full stack test suite green with zero local edits to stack test files. The epic closes on that convergence run.

Doctrine note (rule 3 clarification): rule 3 covers prerequisites a test can set and restore. A live-external-service prerequisite (e.g. a reachable backend for e2e) is check-only: use the shared availability check + graceful-skip pattern, with the CI job that provides the real service as the runtime proof.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions