Skip to content

fix(agent): fail-closed seq guard, flap detector, return() cancels Control - #17

Merged
mattwilkinsonn merged 2 commits into
mainfrom
compass-agent-1364-c4b-seq-guard
Jul 30, 2026
Merged

fix(agent): fail-closed seq guard, flap detector, return() cancels Control#17
mattwilkinsonn merged 2 commits into
mainfrom
compass-agent-1364-c4b-seq-guard

Conversation

@mattwilkinsonn

@mattwilkinsonn mattwilkinsonn commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Ported from sealedsecurity/sealed#905. Paths move oss/compass/packages/compass-agent/packages/compass-agent/; no content changes.

11 of that PR's 12 files. The twelfth is docs/designs/product/compass-agent-transport-consolidation/design.md, which stays in sealed — design records do not port, and this repo has no docs/designs/ for one to land in.

What this fixes

The Control stream's dedup cursor assumed the producer stamps a sequence. The Runner on main stamps zero for every opagent.pb.go generates ControlSeq but no Runner path sets it. With the cursor at N, the first post-restart op arrives as seq 1, isApplied(1..N) is 1 <= N → true, and it is counted as a duplicate already applied. The agent goes silently deaf for N ops while continuing to report healthy. The guard now fails closed on an unstamped producer rather than reading its default as an ordering claim.

Reconnect grows a flap detector that resets on a clean open, so a stream that reconnects successfully does not accumulate toward a ceiling it never earned. return() now cancels the Control stream rather than leaving the pump running against a closed buffer.

Base verification (before applying)

Five files pre-existed; the rest are new. All five byte-identical across both repos by blob hash, at sealed@origin/main and compass@05da6b3:

src/transport/control-source.ts       IDENTICAL
src/transport/control-source.test.ts  IDENTICAL
src/transport/index.ts                IDENTICAL
moon.yml                              IDENTICAL
tsconfig.json                         IDENTICAL

A clean git apply would also succeed against a base that drifted outside the patch context, so the hashes are the evidence.

Gate — run locally, in this repo, forced

This repo has no CI. Zero workflows, so CLEAN here means nothing ran. Local testimony, not a gate.

$ moon run compass-agent:ci --force --concurrency 4

compass-agent:test |  157 pass
compass-agent:test |  0 fail
compass-agent:test |  450 expect() calls
compass-agent:test | Ran 157 tests across 10 files. [10.32s]

Tasks: 5 completed
 Time: 10s 504ms

The mutation verifier, and why it ships as code

A reviewer proved the reconnect-pump abort branches survive outright deletion with the suite fully green. A test that passes against both the code and its deletion asserts nothing about it, so each branch is required to die under mutation rather than merely to pass beside one.

$ bun run scripts/verify-abort-mutants.ts

baseline: green

KILLED    top-of-loop abort guard
SURVIVED  catch-side abort guard  (expected — untestable)
KILLED    abortable backoff sleep
KILLED    progress novelty guard (re-ack is not progress)
KILLED    progress counted once per application

all 5 mutants accounted for (1 documented untestable)

The survivor is recorded rather than quietly dropped, because SURVIVED is two findings wearing one label: untested (a real gap — write the test) versus untestable at the public surface (another branch absorbs the difference). Deleting the catch-side guard falls through to the backoff, which returns at once on an already-aborted signal, loops, and the top-of-loop guard returns; buffer.fail() is a no-op after return() closed the buffer. Nothing differs at the public surface. Distinguishing those two is the whole value of running the mutants — a bare count would report a gap that does not exist.

moon.yml adds scripts/**/* to the typecheck/test inputs, which is what pulls verify-abort-mutants.test.ts into the run — 10 test files, not 9. A verifier nothing invokes is exactly the failure it exists to catch.

Everything above was run in this repo, not quoted from sealed: different lockfile resolution and workspace root mean a sealed green is a claim about a tree this does not land on.

Spec-impact: none

Refs SEA-1364

Copy link
Copy Markdown
Contributor Author

This stack of pull requests is managed by Graphite. Learn more about stacking.

@linear-code

linear-code Bot commented Jul 28, 2026

Copy link
Copy Markdown

SEA-1364

@greptile-apps

greptile-apps Bot commented Jul 28, 2026

Copy link
Copy Markdown

Greptile Summary

This PR hardens the Control transport lifecycle and sequencing behavior.

  • Rejects unstamped or invalid Control sequence numbers before deduplication.
  • Adds bounded reconnect handling based on connection uptime and applied progress.
  • Cancels the underlying Control stream when its iterator is returned.
  • Extracts and tests the acknowledgment cursor and asynchronous buffer.
  • Adds mutation-verification tooling and includes scripts in package task inputs.

Confidence Score: 3/5

The PR is not yet safe to merge because repeated reconnects can still terminate Control while an asynchronous operation is actively being applied.

The reconnect pump samples progress only from the acknowledgment cursor after a drop, while a yielded operation does not advance that cursor until the consumer pulls again; multiple drops during one long-running application can therefore exhaust the no-progress budget and fail the Control source.

Files Needing Attention: packages/compass-agent/src/transport/control-source.ts

Important Files Changed

Filename Overview
packages/compass-agent/src/transport/control-source.ts Adds sequence validation, reconnect flap detection, no-progress termination, abortable backoff, and iterator cancellation; the previously reported in-flight progress accounting defect remains.
packages/compass-agent/src/transport/control/ack-cursor.ts Extracts selective acknowledgment state and adds a monotonic count of newly applied operations.
packages/compass-agent/src/transport/control/buffer.ts Extracts the single-consumer asynchronous queue with explicit close and failure semantics.
packages/compass-agent/src/transport/index.ts Extends the Control transport call to accept cancellation and response-header callbacks.
packages/compass-agent/scripts/verify-abort-mutants.ts Adds a manual mutation verifier for abort handling and acknowledgment progress invariants.
packages/compass-agent/src/transport/control-source.test.ts Adds extensive coverage for invalid sequences, reconnect bounds, progress handling, and stream cancellation.
packages/compass-agent/moon.yml Includes package scripts and TypeScript configuration in task inputs.

Reviews (3): Last reviewed commit: "fix(agent): fail-closed seq guard, flap ..." | Re-trigger Greptile

Comment thread packages/compass-agent/src/transport/control-source.ts
@seal-agent
seal-agent force-pushed the compass-agent-1364-c4b-seq-guard branch 2 times, most recently from c998e00 to 0fd9008 Compare July 29, 2026 18:01
…ntrol

The Control stream's dedup cursor assumed the producer stamps a sequence,
and the Runner on main stamps zero for every op. With the cursor at N, the
first post-restart op arrives as seq 1, counts as a duplicate already
applied, and the agent goes silently deaf while continuing to report
healthy. The guard now fails closed on an unstamped producer rather than
treating its default as an ordering claim.

Reconnect grows a flap detector that resets on a clean open, so a stream
that reconnects successfully does not accumulate toward a backoff ceiling
it never earned, and return() now cancels the Control stream rather than
leaving the pump running against a closed buffer.

The abort branches ship with a mutation verifier because a reviewer proved
they survive outright deletion with the suite green. A test that passes
against both the code and its deletion asserts nothing about it, so each
branch is shown to die under mutation rather than merely to pass beside
one. The catch-side guard is recorded as untestable at the public surface
rather than quietly dropped: another branch absorbs the difference, so the
survival is a real finding about reachability, not a missing test.

Ported from sealedsecurity/sealed#905. Paths move
oss/compass/packages/compass-agent/ to packages/compass-agent/; no content
changes. The design record in that PR stays in sealed, per the ruling that
design records do not port. Base verified byte-identical across both repos
for all five pre-existing files before applying.

Co-Authored-By: seal <noreply@sealedsecurity.com>
@seal-agent
seal-agent force-pushed the compass-agent-1364-c4b-seq-guard branch from 0fd9008 to 4ce5d29 Compare July 29, 2026 23:01
…gress

The no-progress reconnect budget measured progress solely as the ack
cursor advancing, but apply-then-ack acks op N only when the consumer
pulls N+1, so a long turn mid-apply advances nothing; >10 socket flaps
during one turn killed a healthy session. Augment the signal to also
treat an op in flight as progress (OR, not replace: the idle residual
still fail-closes at 10).

Refs SEA-1540

Co-Authored-By: seal <noreply@sealedsecurity.com>
@mattwilkinsonn
mattwilkinsonn merged commit 42a56c6 into main Jul 30, 2026
3 checks passed
@mattwilkinsonn
mattwilkinsonn deleted the compass-agent-1364-c4b-seq-guard branch July 30, 2026 02:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants