perf: cut Windows CI test fixture overhead#174
Merged
Conversation
Windows shards spend most of their 11-13.5 min on a ~15s per-test fixture floor: every test that touches a DB pays unoptimized bundled SQLite, DELETE+FULL journal fsyncs, Defender scanning, and repeated full schema creations. Attack all four: - Cargo.toml: compile libsql-ffi/libsql-sys/libsql-rusqlite at opt-level 2 in dev/test builds (2.8x per-test speedup measured on Linux). - TRACEDECAY_SQLITE_UNSAFE_FAST=1 env hatch (tests/CI only) switching all connections to journal_mode=MEMORY + synchronous=OFF; mode-asserting tests pin the var off via env guards. - ci.yml windows-test-shard: disable Defender real-time scanning, exclude hot paths, redirect TMP/TEMP to the fast RUNNER_TEMP drive, and export the unsafe-fast env for the test step. - Trim per-test fixture excess in the slowest junit offenders: drop dead session-store seeding from scheduler gate tests, isolate the shared per-user global DB out of skill-writer runner tests, pre-create dashboard GlobalDb-schema stores from a cached empty template, and seed branch-drift tracking metadata directly instead of building a branch DB just to delete it.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Windows CI junit data from the latest green master run (28503571229) shows 10 shards at 11-13.5 min each with 3182 tests summing to ~235 CPU-minutes, and a hard ~15s fixture floor: any test that initializes a TraceDecay project or DB costs 15-65s on Windows (p50 for
mcp_handler_testis 22.4s; schema-only tests run in 0.02s). This PR attacks the floor from four directions:Cargo.toml): the bundled SQLite C sources inlibsql-ffiwere compiled at-O0in dev/test builds. Scopedopt-level = 2overrides forlibsql-ffi/libsql-sys/libsql-rusqlitecut measured per-test time 2.8x on Linux (1.69s to 0.60s); the MSVC delta should be at least as large.TRACEDECAY_SQLITE_UNSAFE_FAST=1escape hatch (src/db/connection.rs): tests/CI-only env var that switches every connection tojournal_mode=MEMORY+synchronous=OFF, eliminating the per-transaction rollback-journal create/write/fsync/delete cost of the WindowsDELETE+FULLpairing. In-memory journals never enter WAL mode, so the Windows WAL-teardown crash stays avoided. Tests that assert durable journal/synchronous modes pin the var off via env guards.ci.yml, windows-test-shard only): disable Defender real-time monitoring plus exclusion-path fallback for workspace/temp/cargo dirs, redirectTMP/TEMPoff the slow C: user temp onto theRUNNER_TEMPdrive, and export the unsafe-fast env for the nextest step.TRACEDECAY_GLOBAL_DBat each skill-writer runner test's own store (evidence building was opening the shared per-user global DB, contended by every parallel test process — the 9-of-10-slow signature of that binary); pre-create dashboard GlobalDb-schema stores from a cached empty template so server startup and seeding skip full schema creation (the curation restart test starts the server twice); seed branch-drift tracking metadata directly instead of building and syncing a branch DB only to delete it.Verification
cargo fmt --all -- --check,git diff --check,cargo clippy --workspace --all-targets --locked -- -D warningscargo nextest run --workspace --profile ci --lockedwithTRACEDECAY_SQLITE_UNSAFE_FAST=1: 3272 passed, 0 failedmigration_test,profile_storage_migration_test,corruption_test,src/db/connection.rsunit tests)scripts/check-conventional-commits.sh origin/master..HEADdashboard_session_and_skill_runs_emit_activity_when_evidence_is_unavailable) reproduces identically with these changes stashed under heavy machine load and passes reliably otherwise — pre-existing load-sensitivity flake, not a regression.Test plan
mcp_handler_testp50 22.4s)