fix(broker): keep idle fleet agents active

[khaliqgant]

Summary

• renew the authoritative live-worker inventory once per minute while node control remains connected
• send authoritative snapshots even when empty, clearing stale server-side workers immediately on connect and on periodic refresh
• centralize inventory snapshot construction and disconnect cleanup in one helper shared by connect, mutation, and refresh paths
• preserve the immediate mutation and reconnect inventory behavior added by fix(broker): restore live agent roster after reconnect #1494
• add regression coverage for quiet live workers and stale presence cleanup from empty inventories

What #1494 fixed vs. this residual gap

#1494 made successful launches enter fleet_inventory, published snapshots reliably on changes, and replayed the inventory after reconnect. That fixes agents stranded offline by a provider disconnect. The remaining gap was time based: after the reconnect or launch snapshot, a healthy but idle worker produced no more agent-authenticated activity and no further inventory snapshot.

Root cause

Relaycast expires active agent presence after five minutes without authenticated agent activity. Broker node heartbeats run every 12 seconds, but they renew only the node/provider rows. inventory.sync refreshes each authoritative agent row, yet before this change the broker sent it only on inventory changes and reconnect. A live worker that stayed quiet for five minutes therefore aged offline while its node remained healthy and online.

The broker now replays authoritative inventory every 60 seconds, well inside that lease. Empty snapshots are sent too, both immediately after connecting and periodically, so Relaycast clears workers that no longer exist locally. Worker exit and release paths still remove workers and immediately publish the pruned inventory.

Verification

• Added a red-before regression: without the renewal, the mock Relaycast server times out waiting for the idle workers next inventory.sync
• Added regression coverage proving empty authoritative snapshots are sent on connect and periodic refresh
• Ran an isolated local Relaycast engine on 127.0.0.1:18787 and the patched broker on 127.0.0.1:18788
• Bound a locally minted node token to the exact isolated node ID, then confirmed the node stayed online with accepted 12-second heartbeats
• Spawned an idle /bin/sh worker, sent it no input, and observed its PID remain alive while the agent stayed active beyond the five-minute TTL; last_seen advanced once per minute (11:34:49Z to 11:40:22Z)
• Did not restart, reconfigure, deploy to, or otherwise touch finn-mini, sf-mini, barry, or the Daytona broker

Test plan

• cargo test -p agent-relay-broker (925 library tests passed, 4 ignored; 16 integration tests passed)
• cargo test -p agent-relay-broker node_control::tests:: (51 passed)
• cargo fmt -p agent-relay-broker -- --check
• cargo clippy -p agent-relay-broker --lib -- -D warnings
• git diff --check

Refs #1458. Follow-up to #1494.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 20d3fbf0-eeea-4ad2-9ffb-5ceaf8d6c5c8

📥 Commits

Reviewing files that changed from the base of the PR and between 971065c and eb28e98.

📒 Files selected for processing (2)

• CHANGELOG.md
• crates/broker/src/node_control.rs

🚧 Files skipped from review as they are similar to previous changes (2)

• CHANGELOG.md
• crates/broker/src/node_control.rs

---

📝 Walkthrough

Walkthrough

The broker now sends an initial and periodic worker-inventory synchronization during connected node-control sessions. The refresh interval is 60 seconds. Tests cover idle agents, empty inventories, and reconnect handling. The changelog documents the fix.

Changes

Inventory refresh

Layer / File(s)	Summary
Schedule and send inventory refreshes crates/broker/src/node_control.rs	Connected sessions use a 60-second interval. The initial tick is consumed to prevent a duplicate refresh. Inventory sending is centralized and disconnects after send failure.
Validate inventory replay and reconnect handling crates/broker/src/node_control.rs	Tests validate initial empty inventories, periodic refreshes for idle agents, empty-inventory replay, and reconnect synchronization while filtering heartbeat frames.
Document the patch CHANGELOG.md	The changelog records the unreleased patch and the worker-inventory renewal fix.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Mergeability Score: ⚪ Minimal · up to eb28e

The change periodically renews active idle workers while preserving immediate inventory updates and cleanup behavior; no actionable merge-blocking risk remains after normal checks and review.

Sequence Diagram(s)

sequenceDiagram
  participant FleetBroker
  participant ConnectedSession
  participant NodeControlPeer
  FleetBroker->>ConnectedSession: start session with 60-second refresh interval
  ConnectedSession->>NodeControlPeer: send initial InventorySync
  ConnectedSession->>ConnectedSession: wait for periodic interval
  ConnectedSession->>NodeControlPeer: replay current InventorySync

Loading

Possibly related PRs

• AgentWorkforce/relay#1139: Modifies broker node-control liveness and inventory refresh behavior.
• AgentWorkforce/relay#1462: Modifies node-control heartbeat and reconnect handling.
• AgentWorkforce/relay#1494: Modifies broker inventory synchronization and reconnect handling.

Suggested reviewers: willwashburn

Poem

A rabbit saw the workers stay,
While quiet agents slept away.
The broker sent their roster bright,
Then checked it again by moonlight.
No healthy worker left from sight.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly summarizes the primary change: keeping idle fleet agents active through broker inventory renewal.
Description check	✅ Passed	The description explains the change, root cause, testing, verification, and follow-up context; screenshots are not applicable.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/refresh-live-agent-roster

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[cursor]

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

[cubic-dev-ai]

All reported issues were addressed across 2 files

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>