perf: parallelize the test suite (~7x) and trim slow pre-commit hooks - #65
Conversation
Two complementary changes that make the suite (and commits) fast:
1. pytest-xdist in the dev group + '-n auto' in addopts, so every run is
parallel by default. Coverage is identical (same missing lines).
serial: 622 passed, 27 skipped, 90% cov, 152.9s
-n auto: 622 passed, 27 skipped, 90% cov, 21.96s (~7x faster)
Disable for one serial run with: pytest -n 0
2. Pre-commit now keeps only the fast, auto-fixing hooks (ruff --fix,
ruff-format, cargo-fmt). The slow gates it used to run on every commit
(pyright, a full pytest suite that rebuilt the Rust extension, cargo-clippy,
cargo-test) are owned by CI on every PR. 'pre-commit run --all-files' drops
from ~2.5+min to 0.3s.
- ci.yml: enable uv caching (enable-cache) on all three setup-uv steps so 'uv sync' reuses the package cache across runs; make the test step's '-n auto' explicit (it is also the addopts default). - docs.yml: bump peaceiris/actions-gh-pages v3 -> v4 (v3 runs on the deprecated Node 16 runtime; v4 is the maintained Node 20 release, no breaking changes for the inputs used).
Update CONTRIBUTING.md and docs/installation.md to match the new reality: the suite is parallel by default (~600+ tests, ~20-30s, ~90% cov in the default --extra qc env), and pre-commit runs only the fast lint/format hooks (the full gates run in CI). The stale '324 tests / 40-45s / 94%' figures and the removed pyright/pytest/cargo-clippy/cargo-test hook entries are corrected.
|
Warning Review limit reached
Next review available in: 12 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (6)
Comment |
Makes the offline test suite run ~7x faster on one machine (and ~2x on CI's 2-core runners) with identical coverage, and stops the pre-commit hook from rebuilding the Rust extension + running the full suite on every commit. No tests added, removed, or weakened — same
622 passed / 27 skipped / 90%coverage (default--extra qcenv).Parallel test execution
pytest-xdistin the dev group +-n autoin addopts:pyproject.toml[tool.pytest.ini_options]now defaults to parallel across cores, so every entry point (uv run pytest,make test, CI) benefits from one source of truth. Coverage is collected per-worker and merged bypytest-cov, so the report is the same missing lines.pytest -n 0for a serial run (single-test debugging).Pre-commit
ruff --fix,ruff-format,cargo-fmt. Removedpyright, thepytesthook (ranmaturin develop+ the full suite),cargo-clippy, andcargo-test— all run in CI on every PR, so they duplicated CI and made every commit take minutes.pre-commit run --all-files: ~2.5+ min →0.3s.CI
ci.yml: enabledastral-sh/setup-uv@v4enable-cacheon all three Python jobs souv syncreuses the package cache; made the test step'suv run pytest -n autoexplicit.docs.yml: bumpedpeaceiris/actions-gh-pagesv3→v4(v3 runs on the deprecated Node 16 runtime; no breaking changes for the inputs used).Design
-n autoin addopts (not just CI/Makefile): a single default makes the speedup uniform across every invocation instead of relying on callers remembering-n. The cost is ~1–2s worker spawn on a single-test run, whichpytest -n 0avoids when wanted.pytesthook also aborted commits via the known stale-uv.lockmismatch (forcing--no-verify); the fast hooks have no such failure mode.pull_request. With the slow hooks gone, CI is now the sole runner of pyright/pytest/clippy/cargo-test — fine for this repo's PR-based workflow, but direct pushes tomainwon't run them. (This was already true in practice: the oldpytesthook was routinely bypassed with--no-verify.) Addingpush: branches: [main]toci.ymlwould close the gap if desired — left as a maintainer call since it roughly doubles CI minutes.Docs
CONTRIBUTING.md: corrected the stale324 tests / 40-45s / 94%figures (now ~600+ tests, parallel, ~90% cov) and the "Configured hooks" list (removedpyright/pytest/cargo-clippy/cargo-test).docs/installation.md: softened the pre-commit line — only fast lint/format hooks run locally now.Testing
uv run pytest -q(serial, before) →622 passed, 27 skipped, 90% cov, 152.9s.uv run pytest -q(-n auto, after) →622 passed, 27 skipped, 90% cov (same missing lines), 21.96s(re-run:22.4s).uv run pyright→0 errors, 0 warnings, 0 informations.uv run ruff check .→ all checks passed;uv run ruff format --check .→ 63 files already formatted.cargo fmt --check --manifest-path rust/Cargo.toml→ clean.uv run pre-commit run --all-files→ruff/ruff-format/cargo-fmtPassed in0.33s.uv run --group dev mkdocs build --strict→ built clean (0.35s).uv run pytest --collect-only -q→648 collected(addopts-n autodoesn't break collection);uv run pytest tests/test_enums.py→18 passed(single-test run still works).