feat(runner): SEA-1364 C2 - Gateway telemetry-ingest handlers

[mattwilkinsonn]

Ports sealedsecurity/sealed#894 into the standalone repo. Implements Publish and
PostConversationFrame on the Runner Gateway — two of the three handler bodies the
agent↔Runner seam is missing. Control is still unwritten anywhere, so this does not
close the seam on its own.

Stacked on compass-runner-sea-1440-sunpath-cap (#12), and the dependency is measured, not assumed.

Why it must stack on #12

I had this ordered the other way and the measurement reversed it:

this port on main            TestIntegrationProvisionStartRelayToStoreAndBus
                             panic: test timed out — 600s deadline
this port stacked on #12     ok  internal/runnerhub  12.8s

main has neither the sunPathMax guard nor the shortRuntimeDir helper — both live
only on #12 (grep -c sunPathMax → 0 on main, 5 on #12). This PR modifies
integration_pgtest_test.go, and that test cannot pass beneath #12, so the stack is a
correctness requirement rather than a convenience.

The same 600s hang reproduces on clean main with this port stashed, which is what
establishes it is not this diff.

What the rebase collided on

socket.go conflicted: #12 adds the path-length pre-check, this PR adds a cancel
parameter to listenAgentSocket. Independent additions to one function — both kept.

Two callsites carried the pre-cancel signature through the rebase with no file-level
conflict, plus a skeleton comment quoting the same stale shape. The compiler reports only
the first, so I enumerated instead:

socket_test.go:426        3-arg  -> fixed
socket_test.go:439        3-arg  -> fixed   (compiler never mentioned it)
socket_podman_test.go:58  comment, a recipe someone will type -> fixed

Generated files are regenerated here, not patched across

The port carries runner.pb.go, agent.pb.go, agent_gateway.pb.go and two _pb.ts
files. A generated file is only correct relative to the toolchain that produced it, and
sealed's pin is not guaranteed to be this repo's, so I ran moon run compass-proto:gen
in this tree and committed what it emitted.

Base verification

Every one of the 26 files sealed#894 touches was compared between sealed's merge-base
(fe4e8424) and compass origin/main:

total 26 | SAME 26 | DIFF 0

Byte-identical, so the port is a clean apply rather than a hopeful one. git apply --check
returned 0 independently.

Gate: run locally, because this repo has no CI

sealedsecurity/compass has no workflows — a CLEAN PR here means nothing ran, not
everything passed. Everything below is local testimony, at 62b2309, with local ==
remote == PR head confirmed by rev-parse.

go test ./... -count=1              15/15 ok, uncached (real durations)
pgtest internal/runnerhub           ok  10.9s   (live Postgres)
pgtest internal/board               ok   6 named tests, PASS lines verified
compass-proto:drift --force         3 tasks completed

-count=1 is deliberate: moon run compass-go:test --force bypasses moon's cache but
not go test's per-package cache, and it reported 15 packages (cached) — a green
that executed nothing.

internal/board finishes in milliseconds. That is an in-memory suite, not a skip — the six
--- PASS lines are enumerated rather than inferred from ok.

The drift gate is proven capable of failing, not merely observed passing: mutating a
generated header to protoc-gen-go v9.9.9 produces task_runner::run_failed; restored and
re-verified byte-identical.

moon run :ci cannot currently complete in an agent shell

Not this PR, and not any diff — compass-go:fmt fails on clean origin/main:

$ git stash -u && moon run compass-go:fmt --force
gofmt needs to run on:
Error: task_runner::run_failed

The empty file list is the tell. gofmt resolves to ~/.proto/shims/gofmt, which prints
a 165-byte NDJSON banner on stdout ("Detected an AI agent environment"), and the task
is out=$(gofmt -l .); if [ -n "$out" ]; then … exit 1; fi. The banner is captured as
$out. Filtered, gofmt -l . returns 0 filenames on both main and this branch —
nothing is unformatted.

It fires only for agents, so a human sees green and all six of us see red on identical
code. Reported to compass-repo; not fixed here, since go/moon.yml is not my file and
the obvious patch (gofmt -w) would rewrite generated files and redden drift.

Open Questions — two findings this port carries, unfixed

sealed#894 was BLOCKED on two deliberately unresolved review threads. sealedsecurity/compass
has no branch protection (protection → 404, rulesets → []), so nothing here holds
them. Reproducing them rather than letting the move drop them silently:

1. PostConversationFrame acks a frame that is never committed. observedConversationSink
(go/server/sinks.go:48-56) logs and returns nil, so a conversation frame creates no comms
row and no SubscribeComms event while the handler reports success. An ack attests to a
commit that did not happen, and a retry short-circuiting on that key makes the frame
permanently absent. Lands as #900 T3 (compass-server's).

2. That sink's stated blocker is stale. Its comment (sinks.go:37-43) says the real
write-through "needs the session→channel mapping T5's store adds." That mapping already
landed — verified present on compass main. Three lanes read past that comment and
concluded the work was gated; it was gated by prose, not by reality. The comment should die
in the same diff as the fix, restated rather than deleted.

Both are verified live on compass main as of this PR. Neither is introduced here — this
port makes them reachable, which is why they are stated at the top of the merge decision
rather than left in a sealed thread that did not travel.

RunnerSeq could restart mid-session — found by a new test, fixed here

Reviewing this PR's own durable path surfaced a wire-contract defect that is worse than a
gap, because it is silent.

seq lived on sessionPublisher (publisher.go), so any release-then-reacquire inside
one session restarted stamping at 0:

proto/compass/v1/runner.proto:170-172   runner_seq monotonic across this Runner's
                                        whole event stream (not per session)
go/internal/runnerhub/hub.go:230        flags only  seq > lastSeq+1

The hub flags a forward jump and nothing else, so replayed low seqs are accepted and a
genuine in-transit loss inside the replayed range stops being detectable. Not a false alarm
— a silenced one.

What made it live rather than theoretical is in this diff: the durable handler releases the
publisher on every upstream forward failure (post_conversation_frame.go), which it must,
or one failed forward wedges every later durable frame for the session. So a single flaky
forward blinded loss detection.

Fixed by hoisting the counter onto the socket-lifetime Gateway as a seqCounter whose
mutex remains the allocate-and-send critical section — allocation order still equals
emission order, and a replacement publisher's first Send cannot interleave with the
outgoing publisher's close.

relay.go:114-119 defers per-Runner (vs per-session) scope to T9. This does not close
that — it makes the sequence unbroken within one Runner link.

Red-green, by mutation

give newSessionPublisher its own &seqCounter{}   ->  8/10 runs FAIL   (duplicate RunnerSeq)
the hoist as committed                           ->  0/10 runs FAIL

Only TestReleaseDoesNotRestartTheSequence reddens under that mutation.

Note the discriminator: 0/10 without -race, 10/10 with it. The window needs the
scheduler perturbation to be entered at all, which is why no existing test caught it and why
a non-race run is not evidence here.

Two test defects fixed on the way

Both were latent — real, and invisible until something exercised them.

fakeSessions (gateway_test.go)   unsynchronized calls++/gotName; every existing
                                 caller was sequential, so the race was latent
                                 rather than absent -> mutex + snapshot()

the wedge regression test        left a capturePublish handler parked on a bounded
                                 channel at cleanup; httptest.Server.Close then
                                 blocks on its handler WaitGroup
                                 -> release inline + awaitEnded, before cleanup

The second was mine, and it was measured rather than reasoned about:

baseline (this branch's parent)   0/8 runs hang
with the hang present            3/8 runs hang, whole package times out
fixed                            0/8 runs hang, 0/8 fail

No retry, no skip, no raised timeout — the ordering is now deterministic.

Verification claim for this commit

locally-verified, never CI-verified — this repo has no pipeline (.github/workflows
404, rulesets: []), so a green render here is over an empty required set and means nothing.
What I actually ran, in the devenv shell:

gofmt -l .                                    0 files
go build ./...                                clean
go vet ./...                                  clean
go test ./... -count=1                        all packages ok
golangci-lint run ./internal/runner/...       0 issues
go test -race ./internal/runner/gateway/ -count=10, 8 consecutive runs
                                              0/8 hang, 0/8 fail
mutation: per-publisher counter restored      8/10 fail (proves the guard)

Refs SEA-1364

Open Questions — review round (2026-07-29), design forks parked for Matt

A fresh review pass over this diff (correctness + concurrency + security + test-adequacy,
one adversarial reviewer) reconfirmed the two sink findings above and surfaced two more that
are design forks, not mechanical fixes — parked here rather than decided by proxy.

3. The durable ack attests to a buffer, not a commit — and the shared-stream design
makes that inherent, so the T5 sink alone does not close OQ-1. PostConversationFrame
returns success (and caches the idempotency key) the instant pub.forward returns.
pub.forward is only stream.Send(...) on the shared socket-lifetime PublishEvents
client-stream — an HTTP/2 client Send that enqueues subject to flow control. The Server's
terminal per-frame result is produced only by CloseAndReceive, which the durable path
never calls per-frame (the stream stays open across many frames). So any server-side loss
after Send-accept (handler crash, sink error, dropped stream) permanently loses a durable
frame while the agent's retry is suppressed. OQ-1 above frames this as "the sink returns
nil"; this is the sharper statement — even once the T5 committing sink lands, acking on
Send-accept still violates delivered-or-erred, because "buffered by the Runner" ≠ "committed
by the Server." Fork (proto seam, shared with compass-server): (a) give
PostConversationFrame its own unary upstream RPC returning a per-frame commit ack (cleanest
— separates the durable path from the lossy Publish spine), or (b) add an application-level
per-frame ack on PublishEvents the Runner awaits before returning success. The decision
touches proto/compass/v1/runner.proto, so it is settled with compass-server, then ruled by
Matt.

4. releasePublisher holds pubMu across an unbounded CloseAndReceive — a session-wide
liveness hazard — and the ordering rationale that justifies it is narrower than the doc
claims. releasePublisher (publisher.go) holds g.pubMu across pub.close() →
stream.CloseAndReceive(), which blocks on the server's terminal response with no per-close
timeout on a g.baseCtx (socket-lifetime) stream. On an unresponsive-but-connected upstream,
every later Publish/PostConversationFrame blocks on the same mutex → session-wide
telemetry outage from one clean stream-end. The lock is held deliberately — to stop a
second publisher opening while the old stream drains (out-of-order-across-streams). But
acquirePublisher's own seqCounter doc (publisher.go:57-66) already states the opposite
principle ("never hold pubMu across CloseAndReceive because it can block") and closes its
stale publisher outside the lock. The ordering concern is also narrower than stated:
hub.recordSeq (runnerhub/hub.go:227) flags a gap only on a forward jump
(seq > lastSeq+1); a delayed low seq — exactly what a cross-stream reorder produces — is
silently accepted, seenGap stays false. So the reorder the held lock guards against is
not even detected by the hub. Fork (concurrency): bound CloseAndReceive under a
context.WithTimeout derived from baseCtx, and/or capture pub + clear g.pub under
pubMu then close outside the lock (matching the stale-publisher path), having established
the hub tolerates the one-frame reorder. Do not silently keep the unbounded hold. Ruled by
Matt.

5. (medium) No test defends the at-least-once contract. The "delivered-or-erred" tests
exercise only a dead transport (pre-closed server, so Send itself fails). None exercises
the real hazard: a Send that is accepted but whose frame the Server then fails to
process/commit — the exact case OQ-3 describes, under which the unary returns success and
every test stays green. The suite codifies Send-accept as the success criterion, which is
the defect. The fix is contingent on the OQ-3 ruling (the test's shape depends on which ack
model lands), so it is batched with it rather than written now.

These join OQ-1/OQ-2 above as the merge-decision surface. Nothing here is introduced by this
port; the port makes the durable path reachable, which is why the forks are stated at the
top of the merge decision. Tracked: SEA-1364 (this lane); the runner-side residue (OQ-4 lock-across-close +
OQ-5 at-least-once test) as SEA-1545; the sink write-through as compass-server's #900/T3.

Rebased onto post-#32/#42 main + reconciled (2026-07-30)

Reopened per Matt's directive (reopen-in-place, not a fresh PR — keeps number + history) and rebased c040720 → eec89d9 onto current main (f5fb433, post-#32 T3 write-through + #42 sink-ack proto). Stack base is now main directly (the #12 sunpath fix merged).

Sink-seam reconcile — went KEYLESS (ruled by compass-server, owner of the write-through): #32 replaced the T4 observedConversationSink stub with the real commsConversationSink write-through, whose ConversationSink.PostAgentMessage is account-front + keyless: (ctx, account, sessionID, posted, updated). This PR adopts that signature verbatim across the 5 collision files (hub.go, sinks.go, helpers_test.go, integration_pgtest_test.go, board/deliver_integration_test.go). At-most-once dedup ships via the CommitConversationFrame unary keyed path (compass-server's handler + the sink-ack swap), not this Publish-spine sink — which the swap deletes. So agent_caller.go is untouched here (byte-identical to #32).

RunnerEvent.IdempotencyKey (+ PublishEventsRequest.idempotency_key) is retained: the Runner gateway carries the key to the sink boundary (32 gateway tests assert it rides the envelope), but the hub does not thread it into the keyless sink — documented at the field. Whether to drop the now-hub-unconsumed proto field is a follow-up tied to the swap landing, not decided here.

OQ-1/OQ-3 (the observedConversationSink acks-without-committing findings) are RESOLVED by this reconcile: the stub is gone; commsConversationSink commits durable rows via CommitAgentPost/CommitAgentUpdate. The delivered-or-erred hardening (OQ-3 sharpened) ships in the sink-ack swap (SEA-1561) on top of this.

Verification (devenv shell, head eec89d9): go build ./... clean, go vet ./... clean, go test -race -count=1 ./internal/runnerhub/... ./internal/board/... ./internal/runner/... ./server/... all ok, full moon ci gate green through the hk pre-push hook (lint/drift/breaking/vuln/licenses + tests). One contention flake on first push (GOARCH-NDJSON-bleed → compass-go:lint package-load corruption + cascaded nilaway) cleared deterministically on a warm-cache serial retry (compass-go:lint = 0 issues).

#38 (C3 Control) restacked onto eec89d9 (cee7015 → 3852944), pushed green.

[linear-code]

SEA-1364

[greptile-apps]

Durable frames are discarded

When a valid durable conversation frame reaches the production sink, PostAgentMessage only logs that it was observed and returns success without committing or publishing it. The gateway consequently acknowledges the agent and caches the idempotency key even though the message was lost, suppressing retries with that key.

[greptile-apps]

Greptile Summary

This PR implements the Runner Gateway telemetry-ingest seam.

• Adds streaming Publish and unary PostConversationFrame handlers with shared Runner sequencing and idempotency-key forwarding.
• Routes control acknowledgements away from telemetry and adds bounded message and idempotency-cache handling.
• Extends the Runner event protocol, server hub plumbing, generated clients, and integration coverage.

Confidence Score: 2/5

The PR does not appear safe to merge because durable frames can still be acknowledged without being committed, and stalled publisher teardown can block all later telemetry.

The production sink still discards conversation frames while returning success, the gateway treats stream Send completion as durable delivery without a per-frame server acknowledgement, and releasePublisher can retain the shared publisher mutex indefinitely while waiting for CloseAndReceive. The earlier sequence-restart teardown issue is fixed by the Gateway-scoped counter, but the remaining previously reported failures still require correction.

Files Needing Attention: go/server/sinks.go, go/internal/runner/gateway/post_conversation_frame.go, go/internal/runner/gateway/publisher.go

Important Files Changed

Filename	Overview
go/internal/runner/gateway/post_conversation_frame.go	Adds durable-frame validation, forwarding, retry deduplication, and response handling; the previously reported missing commit acknowledgement remains.
go/internal/runner/gateway/publisher.go	Adds shared ordered publishing and preserves sequence numbers across publisher replacement, but the previously reported unbounded teardown lock remains.
go/internal/runner/gateway/publish.go	Adds streaming telemetry forwarding and control-ack routing through the shared publisher.
go/internal/runnerhub/hub.go	Threads durable idempotency keys through Deliver into the conversation sink.
go/server/sinks.go	Extends the production sink signature with the idempotency key while retaining the previously reported no-op write-through.
proto/compass/v1/runner.proto	Adds the durable-frame idempotency key to PublishEventsRequest.

_{Reviews (4): Last reviewed commit: "fix(compass): unshare the publisher stre..." | Re-trigger Greptile}

[mattwilkinsonn]

• feat(runner): SEA-1364 C2 - Gateway telemetry-ingest handlers #24 👈 (View in Graphite)
• fix(runner): bound the agent socket path under the AF_UNIX sun_path cap #12 : 1 other dependent PR (#25 )
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.