ci: add the GitHub Actions pipeline and a toolchain parity gate

[mattwilkinsonn]

The gates already existed and worked. Nothing invoked them remotely: .github/ held only secret_scanning.yml, branches/main/protection was 404, rulesets was []. GitHub reported mergeStateStatus: CLEAN over an empty check array on every PR — indistinguishable from a passing gate, and several lanes nearly reported their PRs as CI-green off that string. This was never an authoring problem; it was a triggering problem.

One job, not a matrix

The central structural choice, and I am not hedging it: gate is a single job running moon run :ci.

A matrix fanning tasks into separate jobs is genuinely better on two axes — per-task check names branch protection could require individually, and failure attribution visible from the check list rather than the log. It was rejected on one decisive ground: a matrix can only be built by enumerating projects (or tasks) in YAML, which makes the workflow a second source of truth for something .moon/workspace.yml already owns. That list goes stale silently.

This is not hypothetical here. .moon/workspace.yml:33-39 documents the identical failure one layer down: a fork tree shipped a functional-CI moon.yml with no entry in the project map, so it read as covered and gated nothing, and no single PR's CI could catch it because the tree and the map arrived on separate branches. A YAML enumeration recreates exactly that at the CI layer — add a project, forget the matrix row, and the gate goes quiet about it while still reporting green.

moon run :ci cannot go quiet. It also directly satisfies the acceptance criterion: the arriving oh-my-pi-fork (#14) needs no edit to this workflow.

The costs are real and accepted: one check name to require, and the failing task named in the log rather than the check list. moon prints the failing target and only its output, so attribution costs one click.

pgtest is a second job — not for attribution, but for blast radius. It is the one lane with an external dependency, and a Postgres service failing to come up must never red the hermetic gate.

Where CI's toolchain comes from

Per the standing decision: native setup-* actions, reading their versions out of .prototools at run time rather than repeating them in YAML, so the file stays the single source.

The nixpkgs-provided tools (buf, protoc, the Go analysis battery, biome, markdownlint) have no setup-* action that could reproduce a nixpkgs pin, and picking a close-enough release from another channel is precisely the drift this PR exists to reject. So one step resolves the same derivations the dev shell does, from the nixpkgs revision devenv.lock pins (tools/toolchain/gate-tools.nix), and puts them on PATH. Nix is the mechanism for that step; it is not the toolchain strategy — the runtimes come from the native actions. The attribute list is parsed out of devenv.nix, never hand-listed, so adding a tool to the dev shell extends CI with no workflow edit.

The parity gate, and the asymmetry it is built around

Reading a version from a file does not prove the runner got it. tools/toolchain/parity.ts runs before the gate (and again inside :ci as toolchain-parity:parity, so the pre-push hook checks it too) and fails the build — never warns.

The two halves of the toolchain are pinned two ways, so they are checked two ways:

half	pin shape	method
.prototools — bun, node, moon, go	a literal version string	self-report: the binary on PATH must report exactly that literal
devenv.nix — everything else	a bare nixpkgs attr, no version literal anywhere in the repo	store-path: realpath of each command must land inside the derivation the devenv.lock-pinned nixpkgs builds

Store-path identity was chosen for the second half for three reasons. It is the only method that works for go-licenses and nilaway, which implement no version flag at all. It is uniform across the whole half. And it is strictly stronger than comparing version strings — it identifies the exact derivation, so an ambient binary that happens to report the same version is still caught (demonstrated below).

Coverage, stated honestly

27 checks, one per command, across every tool the dev shell provides. Per command, not per attribute, because PATH resolves commands and the attribute name often is not the command name (protobuf ships protoc; buf ships the protoc-gen-buf-* plugins) — those extra binaries would be invisible at attribute granularity.

Every tool named in the issue is covered:

• self-report (4): bun, node, moon, go
• store-path (23): proto (+proto-shim), buf (+protoc-gen-buf-breaking, protoc-gen-buf-lint), protobuf (protoc + 7 sibling binaries), protoc-gen-go, protoc-gen-connect-go, golangci-lint, govulncheck, go-licenses, nilaway, biome, markdownlint-cli2, curl (+wcurl)

What it cannot check, and why it is still not a silent gap. Nothing is skipped. A tool the gate cannot verify is reported UNVERIFIABLE and fails the build — a green over an unchecked tool is the exact failure this repo keeps hitting. The two genuine limits, both handled rather than omitted:

1. go-licenses and nilaway cannot self-report a version (go-licenses --version → flag provided but not defined; nilaway likewise). They are fully covered by store-path, which is why that is the method for the whole half — their derivation identity is checkable even though their self-report is not.
2. @bufbuild/protoc-gen-es is not covered. It is a bun devDep resolved through the lockfile, not a PATH tool from either pin source, so neither method reaches it. Its exact-pin discipline is enforced by compass-proto:drift instead — a float restamps every generated header and reds drift on 6 files. Worth flagging: the live ^2.12.1 at packages/compass-client/package.json:16 is currently masked by the lockfile. Deliberately not fixed here — out of scope and separately tracked.

The gate also refuses to run vacuously: if either parse returns zero entries, it exits 1 rather than reporting a pass over nothing.

Demonstrated red-capable — three skews, both halves

A parity gate that has never failed is indistinguishable from one that checks nothing.

A — .prototools half. Skewed against a frozen PATH, which is the CI condition (the runner installs its toolchain independently of the file; editing it locally would otherwise self-heal via proto install on shell entry):

### moon binary CI has on PATH:
moon 2.4.2
### .prototools pin BEFORE skew:
moon = "2.4.2"
### .prototools pin AFTER skew:
moon = "2.4.5"

### GATE OUTPUT (skewed):
  MISMATCH     moon    self-report  expected 2.4.5, got 2.4.2
1 of 27 pinned tools failed parity with the dev shell.
### EXIT CODE: 1

### .prototools REVERTED:
moon = "2.4.2"
### GATE OUTPUT (reverted):
  ok           moon    self-report  2.4.2
All 27 pinned tools match the dev shell.
### EXIT CODE: 0

B — nixpkgs half: an ambient binary shadowing the pinned derivation. The shadow reports the same version the pin resolves to, so a version-string comparison would call this a pass:

### The shadow reports the SAME version the pin resolves to:
Version: 2.5.4
### ...so a version-string comparison would call this a pass.

### GATE OUTPUT — biome row:
  MISMATCH     biome   store-path   expected /nix/store/28nx80wpv7k7bwhwsa357l08bm9p6gj5-biome-2.5.4, got /tmp/shadowbin/biome
1 of 27 pinned tools failed parity with the dev shell.
### EXIT CODE: 1

C — a tool added to the dev shell that CI's PATH lacks. Must be loud, never skipped:

### devenv.nix now also lists:
51:    shellcheck

### GATE OUTPUT — shellcheck row:
  MISMATCH     shellcheck   store-path   expected /nix/store/9phb1flallb4fw685dlblzbqrh9qkphi-shellcheck-0.11.0-bin, got /nix/store/vwvvnr1qxpwn0vffbsn8bkviqcgicizr-shellcheck-0.11.0-bin/bin/shellcheck
1 of 28 pinned tools failed parity with the dev shell.
### EXIT CODE: 1

### REVERTED — both skews undone.
All 27 pinned tools match the dev shell.
### EXIT CODE: 0

Note case C's count moving 27 → 28: adding a tool to devenv.nix extended the gate with no edit to the script, the nix expression, or the workflow.

Per AGENTS.md, the logic is TypeScript run via bun, split pure/shell: parity-core.ts is total functions over strings with no I/O, parity.ts is the execution shell. bun test tools/toolchain/ — 28 pass, 0 fail, including that an unverifiable verdict fails the report and that a sibling store path sharing the expected prefix does not match. It is registered as the toolchain-parity moon project so its own tests are gated: an unverified gate is not a gate.

The one real bug the pipeline caught, on its first run

Run 1's gate failed on compass-go:test — five socket-binding tests, all bind: invalid argument. Triaged rather than papered over:

• Not from this diff. git diff origin/main -- 'go/**' is empty; this PR changes no Go source.
• Root cause measured, not guessed. The runner's default TMPDIR is /home/runner/work/_temp — 22 bytes before the test's own name-derived path — which pushes the socket path to 114 bytes against the 107-byte usable AF_UNIX sun_path budget. Under /tmp the same path is 95.
• Reproduced locally by pointing TMPDIR at an equally deep path: the same test fails with the identical error, and passes without it.
• Fixed at the layer that caused it. TMPDIR: /tmp on both jobs. The cap is real and the code's guards against it live elsewhere (fix(runner): bound the agent socket path under the AF_UNIX sun_path cap #12); one runner's choice of temp root is not a reason to redesign the sockets.

No continue-on-error, no retries. This is the pipeline doing its job on day one.

Verification

• This PR's own CI run: gate green, pgtest RED — and the earlier green was false. The pgtest job originally ran go test … | tee, so the step took tee's exit status and the job was structurally incapable of failing. Run 30329986363 reported pgtest pass while its own log contains --- FAIL: TestServeShutdownWithLiveCommsSubscriberReturnsClean and FAIL github.com/…/go/internal/runnerhub 600.075s (a 10-minute hang). 15b75d8e4 replaced the pipe with capture-then-exit "$rc"; the job has been honestly red ever since. The red is not a regression — it is the gate working for the first time.
• 40 tasks executed on the runner, read off the run log, not assumed: every compass-proto, compass-go, compass-client, compass-agent, compass-ui, and root task, plus toolchain-parity. Baseline 35 + the 5 this PR adds.
• Both fork nix builds confirmed scheduled by :ci on the runner — devenv-fork:build (7m 2s) and nix2container-fork:build, not assumed from the registration.
• pgtest assertions genuinely executed: 722 --- PASS lines against the service Postgres, zero skips. The job asserts both itself — the harness skips rather than fails without a database, so a service that never came up would otherwise pass silently. (The earlier body said 409 … passed; that figure came from the pre-15b75d8e4 run whose exit status was masked, so it counted passes in a job that could not report the concurrent failures.)
• Locally: direnv exec . moon run :ci --force → Tasks: 40 completed, zero failures. bun test tools/toolchain/ → 28 pass.
• Pushed via direnv exec . git push, so the hk pre-push hook ran the gate from the real push context (Tasks: 41 completed).

Required checks on main — for Matt

A workflow that is not required is a gate nobody has to pass. The seal-agent token has admin: false (gh api repos/sealedsecurity/compass --jq .permissions), so this cannot be applied from here. No repo settings were touched.

The check names below are read off the actual run, not inferred from the YAML — a required check whose name matches nothing is a permanent block:

$ gh pr checks 19
gate     pass   8m20s
pgtest   pass  10m49s

Both report under the github-actions app with exactly those names. After merging, apply:

gh api -X POST repos/sealedsecurity/compass/rulesets \
  -f name='main protection' \
  -f target='branch' \
  -f enforcement='active' \
  -F 'conditions[ref_name][include][]=~DEFAULT_BRANCH' \
  -F 'rules[][type]=pull_request' \
  -F 'rules[][type]=required_status_checks' \
  -F 'rules[][parameters][strict_required_status_checks_policy]=true' \
  -f 'rules[][parameters][required_status_checks][][context]=gate' \
  -f 'rules[][parameters][required_status_checks][][context]=pgtest'

Or via Settings → Rules → Rulesets → New branch ruleset: target Default branch, enforcement Active, enable Require a pull request before merging and Require status checks to pass, tick Require branches to be up to date before merging, and add exactly two checks — gate and pgtest — both from the GitHub Actions app.

Order matters: apply this only after this PR merges, or main has a required check no commit on it has ever produced.

Docs

docs/architecture/build-and-ci.md said there was no remote pipeline — a recent correction that this PR makes false again. Swept the tree for CI claims rather than trusting one file: README.md (the "CI scaffold" status line and the gate paragraph), CONTRIBUTING.md (whose coverage list was independently stale — it claimed the -race/lint/vuln battery was still pending when it ships today), and the architecture doc. AGENTS.md's claims were already accurate and are left alone.

The doc is explicit about what the pipeline does not gate: the upstream test suites vendored inside the forks (each fork's registered task is its own nix build), and any live UI↔server path (every compass-ui task runs against fixtures).

Out of scope, deliberately

Per the issue's own list: the Woodpecker generator, the CI image, and the publish script are not ported. The orphaned upstream fork suites, the UI↔server live-path job, and the vendored moon.yml prose sweep stay follow-ups. The packages/compass-client/package.json:16 ^2.12.1 float is left as-is and separately tracked.

Rebase hazard: .moon/workspace.yml conflicts with #20, and a wrong resolution is silent

This PR registers toolchain-parity: 'tools/toolchain' in .moon/workspace.yml. #20 registers stamp-gate: 'tools/stamp-gate' on an adjacent line. git merge-tree reports 2 conflicts (.moon/workspace.yml, bun.lock); package.json auto-merges because both sides add "tools/*" identically.

Both keys are wanted. Resolve by keeping both — never by picking a side.

The reason this needs saying in the body rather than being left to the rebaser: an unregistered project's tasks do not fail, they cease to exist. Measured on this branch by deleting my own key, exactly as a bad resolution would:

key present   moon run :ci --force   ->  40 tasks, 0 failed
key deleted   moon run :ci --force   ->  35 tasks, 0 failed

Five tasks disappear and CI stays green. A rebaser looking at a two-line conflict has no signal that one line is a CI gate, and no signal afterwards either — the build passes, so nothing surfaces the loss. Same class as a rename that carries a defect forward: a correct-looking resolution that removes an enforcement mechanism without reddening anything.

Verified the combined tree works before claiming it: merging both, regenerating bun.lock with bun install --lockfile-only rather than hand-resolving it, gives 45 tasks / 0 failed with toolchain-parity:test 34 pass and stamp-gate:test 9 pass. No ordering is required between #19 and #20 in either direction.

Refs SEA-1507

Co-Authored-By: seal noreply@sealedsecurity.com

[linear-code]

SEA-1507

[mattwilkinsonn]

• ci: add the GitHub Actions pipeline and a toolchain parity gate #19 👈 (View in Graphite)
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[greptile-apps]

Greptile Summary

Adds a GitHub Actions CI pipeline and enforces parity between CI and the repository’s pinned development toolchain.

• Runs the complete Moon CI task graph and real-Postgres test suites in separate jobs.
• Pins external actions and the Postgres service image to immutable revisions.
• Adds a fail-closed toolchain parity project with parser, identity, reporting, and image-parity tests.
• Registers the new project and updates contributor and architecture documentation.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
.github/workflows/ci.yml	Adds the remote gate and Postgres lane; the prior exit-status, mutable-action, and mutable-image issues are fixed at the current head.
tools/toolchain/parity-core.ts	Implements fail-closed parsing and exact runtime/store-path parity checks; the prior silent omission path now throws.
tools/toolchain/parity.ts	Connects repository pin parsing, Nix identity resolution, command discovery, and nonzero parity reporting.
tools/toolchain/parity-core.test.ts	Covers parser rejection, parity mismatch handling, unverifiable tools, store-path boundaries, and Postgres image consistency.
tools/toolchain/gate-tools.nix	Resolves the parsed development-shell attributes from the locked nixpkgs source into the CI tool environment.
go/internal/pgtest/pgtest.go	Pins local container-backed Postgres tests to the same immutable image digest used by CI.
.moon/workspace.yml	Registers the toolchain parity project so its tasks participate in workspace-wide CI.
tools/toolchain/moon.yml	Defines parity tests and checks as CI tasks with inputs covering both Postgres image references.
.github/dependabot.yml	Adds reviewed weekly updates for immutable GitHub Actions pins.

_{Reviews (5): Last reviewed commit: "fix(ci): pin the Postgres service image ..." | Re-trigger Greptile}

[sealedsecurity-bot]

Correcting my own attribution comment on this PR: I wrote that the remaining pgtest reds can't be mine because "my diff touches four files, none of them go/server or runnerhub." That count is wrong — this PR touches sixteen files, and one of them is go/internal/pgtest/pgtest.go, the shared harness every one of those failing suites calls.

The conclusion holds. The evidence I gave for it did not, and the file count is exactly the part that made it sound safe.

The claim that is actually checkable, in three measurements:

1. The harness change is a constant plus comments — no control flow. The whole diff to pgtest.go is pgImage's string and the comment above it.

2. Both CI arms ran the same Postgres bytes, recorded in their own logs. The baseline pulled the mutable tag and the new head pulls the digest, so "same tag" would have been an assumption. It isn't one:

baseline job 90454323446   docker pull postgres:16-alpine
                           Digest: sha256:57c72fd2a128e416c7fcc499958864df5301e940bca0a56f58fddf30ffc07777
new head job 90456831749   docker pull postgres:16-alpine@sha256:57c72fd2a128e416c7fcc499958864df5301e940bca0a56f58fddf30ffc07777

3. The CI pgtest job never evaluates pgImage at all. ci.yml:203 sets COMPASS_TEST_DATABASE_DSN, and RequireDSN returns at the DSN branch before reaching containerCLI(). On the CI path that constant is dead code.

So the harness change cannot be upstream of those failures in CI, and the identical-failure-set comparison across three heads is valid because it was run against identical database bytes — not because the two arms happened to name the same tag.

Worth stating explicitly: the digest pin is a content no-op today. 16-alpine currently resolves to the digest this PR pins, which is what makes the baseline comparison valid. Had I pinned to anything other than the current resolution, the two arms would have run different databases and still reported a clean pre-existing verdict. That is a fact about tonight's timing, not a property of the method.

Credit where due — this was caught by another lane reading the file list instead of my summary of it. A count of files is not a statement about what they do, and the number was the wrong instrument for the claim.