Skip to content

perf: parallelize the test suite (~7x) and trim slow pre-commit hooks - #65

Merged
SkyeAv merged 3 commits into
mainfrom
create-a-faster-running-test-suite
Jul 31, 2026
Merged

perf: parallelize the test suite (~7x) and trim slow pre-commit hooks#65
SkyeAv merged 3 commits into
mainfrom
create-a-faster-running-test-suite

Conversation

@SkyeAv

@SkyeAv SkyeAv commented Jul 31, 2026

Copy link
Copy Markdown
Owner

Makes the offline test suite run ~7x faster on one machine (and ~2x on CI's 2-core runners) with identical coverage, and stops the pre-commit hook from rebuilding the Rust extension + running the full suite on every commit. No tests added, removed, or weakened — same 622 passed / 27 skipped / 90% coverage (default --extra qc env).

Parallel test execution

  • pytest-xdist in the dev group + -n auto in addopts: pyproject.toml [tool.pytest.ini_options] now defaults to parallel across cores, so every entry point (uv run pytest, make test, CI) benefits from one source of truth. Coverage is collected per-worker and merged by pytest-cov, so the report is the same missing lines.
  • Escape hatch: pytest -n 0 for a serial run (single-test debugging).

Pre-commit

  • Kept only the fast, auto-fixing hooks: ruff --fix, ruff-format, cargo-fmt. Removed pyright, the pytest hook (ran maturin develop + the full suite), cargo-clippy, and cargo-test — all run in CI on every PR, so they duplicated CI and made every commit take minutes.
  • pre-commit run --all-files: ~2.5+ min → 0.3s.

CI

  • ci.yml: enabled astral-sh/setup-uv@v4 enable-cache on all three Python jobs so uv sync reuses the package cache; made the test step's uv run pytest -n auto explicit.
  • docs.yml: bumped peaceiris/actions-gh-pages v3v4 (v3 runs on the deprecated Node 16 runtime; no breaking changes for the inputs used).

Design

  • Why -n auto in addopts (not just CI/Makefile): a single default makes the speedup uniform across every invocation instead of relying on callers remembering -n. The cost is ~1–2s worker spawn on a single-test run, which pytest -n 0 avoids when wanted.
  • Why remove (not relocate) the slow pre-commit hooks: pre-commit is the wrong layer for whole-repo type/test/lint gates — slow, can't auto-fix, and already duplicated by CI. The removed pytest hook also aborted commits via the known stale-uv.lock mismatch (forcing --no-verify); the fast hooks have no such failure mode.
  • Accepted caveat: CI still triggers only on pull_request. With the slow hooks gone, CI is now the sole runner of pyright/pytest/clippy/cargo-test — fine for this repo's PR-based workflow, but direct pushes to main won't run them. (This was already true in practice: the old pytest hook was routinely bypassed with --no-verify.) Adding push: branches: [main] to ci.yml would close the gap if desired — left as a maintainer call since it roughly doubles CI minutes.

Docs

  • CONTRIBUTING.md: corrected the stale 324 tests / 40-45s / 94% figures (now ~600+ tests, parallel, ~90% cov) and the "Configured hooks" list (removed pyright/pytest/cargo-clippy/cargo-test).
  • docs/installation.md: softened the pre-commit line — only fast lint/format hooks run locally now.

Testing

  • uv run pytest -q (serial, before) → 622 passed, 27 skipped, 90% cov, 152.9s.
  • uv run pytest -q (-n auto, after) → 622 passed, 27 skipped, 90% cov (same missing lines), 21.96s (re-run: 22.4s).
  • uv run pyright0 errors, 0 warnings, 0 informations.
  • uv run ruff check . → all checks passed; uv run ruff format --check . → 63 files already formatted.
  • cargo fmt --check --manifest-path rust/Cargo.toml → clean.
  • uv run pre-commit run --all-filesruff/ruff-format/cargo-fmt Passed in 0.33s.
  • uv run --group dev mkdocs build --strict → built clean (0.35s).
  • uv run pytest --collect-only -q648 collected (addopts -n auto doesn't break collection); uv run pytest tests/test_enums.py18 passed (single-test run still works).

SkyeAv added 3 commits July 31, 2026 14:23
Two complementary changes that make the suite (and commits) fast:

1. pytest-xdist in the dev group + '-n auto' in addopts, so every run is
   parallel by default. Coverage is identical (same missing lines).
     serial:   622 passed, 27 skipped, 90% cov, 152.9s
     -n auto:  622 passed, 27 skipped, 90% cov, 21.96s  (~7x faster)
   Disable for one serial run with: pytest -n 0

2. Pre-commit now keeps only the fast, auto-fixing hooks (ruff --fix,
   ruff-format, cargo-fmt). The slow gates it used to run on every commit
   (pyright, a full pytest suite that rebuilt the Rust extension, cargo-clippy,
   cargo-test) are owned by CI on every PR. 'pre-commit run --all-files' drops
   from ~2.5+min to 0.3s.
- ci.yml: enable uv caching (enable-cache) on all three setup-uv steps so
  'uv sync' reuses the package cache across runs; make the test step's
  '-n auto' explicit (it is also the addopts default).
- docs.yml: bump peaceiris/actions-gh-pages v3 -> v4 (v3 runs on the
  deprecated Node 16 runtime; v4 is the maintained Node 20 release, no
  breaking changes for the inputs used).
Update CONTRIBUTING.md and docs/installation.md to match the new reality:
the suite is parallel by default (~600+ tests, ~20-30s, ~90% cov in the
default --extra qc env), and pre-commit runs only the fast lint/format
hooks (the full gates run in CI). The stale '324 tests / 40-45s / 94%'
figures and the removed pyright/pytest/cargo-clippy/cargo-test hook
entries are corrected.
@coderabbitai

coderabbitai Bot commented Jul 31, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@SkyeAv, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 12 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 17a317b4-1bbb-4d62-a757-47b9f6e40c96

📥 Commits

Reviewing files that changed from the base of the PR and between 5b9ce48 and 1119790.

⛔ Files ignored due to path filters (1)
  • uv.lock is excluded by !**/*.lock
📒 Files selected for processing (6)
  • .github/workflows/ci.yml
  • .github/workflows/docs.yml
  • .pre-commit-config.yaml
  • CONTRIBUTING.md
  • docs/installation.md
  • pyproject.toml

Comment @coderabbitai help to get the list of available commands.

@SkyeAv
SkyeAv merged commit 0f9068c into main Jul 31, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant