You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
What β make every stack test file pass byte-identical in any consumer repo, whatever that consumer configures or extends. Companion of the docs-tests hermeticity work (#4014/#4015), generalized to the whole suite.
Why β a consumer that merely configures the stack (custom plans, custom loader component, config-driven flags) or extends it (a module hooked on an extension point) currently has to edit stack test files to keep them green. If configuration alone breaks a test, the test is wrong: it asserts a default value instead of a contract. Today ~13 stack test files carry consumer-side adaptations; each one is a merge landmine at every stack sync (--theirs wipes them β it happened, twice).
Isolate extension points generically β stack unit tests mock the hook mechanism (registry / extension point), never a consumer module by name. If a consumer module hooking onOrganizationCreated-class seams changes a stack test's outcome, the stack test lacks isolation.
Env prerequisites are explicit β a test that depends on NODE_ENV or an env-gated branch sets and restores it itself.
Project-specific tests live in project-owned files β never inside stack test files.
Acceptance (epic-level) β a consumer repo with arbitrary config overrides and extra modules runs the full stack test suite green with zero local edits to stack test files. Verified the #4015 way: run the suite once bare, once with a synthetic "configured + extending" fixture profile β identical results.
Scope: validated 2026-08-05
Acceptance amendment (2026-08-05, post-audit): the synthetic "configured + extending" whole-suite run is superseded by a real-consumer proof: a consumer repo with its own config overrides and extra modules syncs this epic's changes and runs the full stack test suite green with zero local edits to stack test files. The epic closes on that convergence run.
Doctrine note (rule 3 clarification): rule 3 covers prerequisites a test can set and restore. A live-external-service prerequisite (e.g. a reachable backend for e2e) is check-only: use the shared availability check + graceful-skip pattern, with the CI job that provides the real service as the runtime proof.
What β make every stack test file pass byte-identical in any consumer repo, whatever that consumer configures or extends. Companion of the docs-tests hermeticity work (#4014/#4015), generalized to the whole suite.
Why β a consumer that merely configures the stack (custom plans, custom loader component, config-driven flags) or extends it (a module hooked on an extension point) currently has to edit stack test files to keep them green. If configuration alone breaks a test, the test is wrong: it asserts a default value instead of a contract. Today ~13 stack test files carry consumer-side adaptations; each one is a merge landmine at every stack sync (
--theirswipes them β it happened, twice).Doctrine (the 4 rules each sub-issue applies):
config.billing.staticContent.packs[0].name), or the test installs its own config fixture (set β assert β restore; the π fix(public): docs integration tests depend on real on-disk guides β make the file consumer-independentΒ #4015 pattern).onOrganizationCreated-class seams changes a stack test's outcome, the stack test lacks isolation.Acceptance (epic-level) β a consumer repo with arbitrary config overrides and extra modules runs the full stack test suite green with zero local edits to stack test files. Verified the #4015 way: run the suite once bare, once with a synthetic "configured + extending" fixture profile β identical results.
Scope: validated 2026-08-05
Acceptance amendment (2026-08-05, post-audit): the synthetic "configured + extending" whole-suite run is superseded by a real-consumer proof: a consumer repo with its own config overrides and extra modules syncs this epic's changes and runs the full stack test suite green with zero local edits to stack test files. The epic closes on that convergence run.
Doctrine note (rule 3 clarification): rule 3 covers prerequisites a test can set and restore. A live-external-service prerequisite (e.g. a reachable backend for e2e) is check-only: use the shared availability check + graceful-skip pattern, with the CI job that provides the real service as the runtime proof.