ci: stabilize flaky Test job (cap nx parallelism + raise vitest timeouts)#478
Merged
Merged
Conversation
…b in CI The component tests added in #472 render the full Solid + goober tree and take 1-4s locally. Shared CI runners are ~4x slower (the suite runs in ~8s locally but ~31s in CI), which pushes individual tests past vitest's default 5s testTimeout and causes intermittent `@tanstack/devtools:test:lib` failures across unrelated PRs (e.g. #471, #477). Raise testTimeout and hookTimeout to 30s for the devtools package so slow CI runs have headroom. Fast unit tests are unaffected; a real hang still fails (just after 30s instead of 5s).
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughTwo CI stability tweaks: the PR workflow gains a top-level ChangesCI Stability Fixes
Estimated code review effort🎯 1 (Trivial) | ⏱️ ~3 minutes Poem
🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
View your CI Pipeline Execution ↗ for commit aaa71f6
☁️ Nx Cloud last updated this comment at |
More templates
@tanstack/angular-devtools
@tanstack/devtools
@tanstack/devtools-a11y
@tanstack/devtools-client
@tanstack/devtools-ui
@tanstack/devtools-utils
@tanstack/devtools-vite
@tanstack/devtools-event-bus
@tanstack/devtools-event-client
@tanstack/preact-devtools
@tanstack/react-devtools
@tanstack/solid-devtools
@tanstack/svelte-devtools
@tanstack/vue-devtools
commit: |
nx.json sets `parallel: 5`, which is fine for local dev machines but oversubscribes GitHub-hosted runners: each affected `test:lib` (vitest) and `test:types` (tsc) task spawns its own worker pool, so 5 heavy tasks at once exhaust CPU/RAM and intermittently stall or OOM-kill deterministic tasks. That produced flaky failures across unrelated PRs (#471, #477, #478) that pass on a plain re-run. The team already pins `test:e2e` to `--parallel=1` for the same reason; cap the Test job's nx parallelism via NX_PARALLEL=3 (CI-only, leaves local dev at 5). Combined with the raised vitest timeouts, this keeps tasks within the runner's budget.
AlemTuzlak
changed the title
KevinVandy
approved these changes
Jun 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The
Testjob (pnpm run test:pr→nx affected ...) has been failing intermittently across unrelated PRs (#471, #477, and this one), always passing on a plain re-run with no code change. The failing task varied —@tanstack/devtools:test:lib(timeout-shaped) and@tanstack/devtools:test:types(tsc, which is deterministic and cannot fail nondeterministically on real errors). That signature = resource contention on the runner, not a real test/type bug.Root cause
nx.jsonsets"parallel": 5. That's fine on a developer's many-core machine, but a GitHub-hosted runner is far smaller, and each affectedtest:lib(vitest) /test:types(tsc) task spawns its own worker pool. Running 5 such heavy tasks concurrently oversubscribes CPU and RAM, which:testTimeout→ flakytest:lib, andtest:types→ flaky failures that vanish on re-run.The team had already worked around the same contention by pinning
test:e2eto--parallel=1, buttest:prstill ran at the globalparallel: 5.nx-set-shas(correctmain-branch-name) andfetch-depth: 0are already configured correctly, sonx affectedbase/head selection is fine — the only misconfiguration was unbounded CI parallelism.Fix
.github/workflows/pr.yml— setNX_PARALLEL: 3in the workflow env. CI-only; local dev keepsparallel: 5fromnx.json. Keeps concurrent heavy tasks within the runner's CPU/RAM budget.packages/devtools/vite.config.ts— raisetestTimeout/hookTimeoutto 30s so the heavy component renders have headroom even when the runner is loaded. Fast unit tests are unaffected; a genuine hang still fails (after 30s instead of 5s).Belt-and-suspenders: (1) removes the contention that caused both flake shapes; (2) protects
test:libspecifically against residual slowness.Verification
nx test:lib @tanstack/devtools174 tests pass locallynx test:types @tanstack/devtoolspasses locallyNX_PARALLEL=3pathNo published-package code changes, so no changeset.
Summary by CodeRabbit