test(fleet): use load-compatible relaycast engine - #1473
Merged
Conversation
Contributor
📝 WalkthroughWalkthroughThe PR updates the relaycast engine to v7.0.0, documents heartbeat compatibility requirements, and adjusts fleet E2E TTL checks for scheduled mailbox expiry maintenance. ChangesFleet E2E compatibility
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related PRs
Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tests/e2e/fleet/README.md`:
- Around line 59-61: Move the compatibility note above the local test command,
then ensure the RELAYCAST_ENGINE_DIR assignment ends with the line-continuation
backslash as its final character so it correctly combines with
BROKER_BINARY_PATH and npm run test:e2e.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 6802bf5c-4739-4607-90e3-895d85a57708
📒 Files selected for processing (4)
.github/workflows/fleet-e2e.ymltests/e2e/fleet/README.mdtests/e2e/fleet/fleet-e2e.test.tstests/e2e/fleet/harness.ts
Comment on lines
+59
to
61
| RELAYCAST_ENGINE_DIR=/path/to/relaycast \ # must carry relaycast#194 and #307 compat fixes | ||
| BROKER_BINARY_PATH="$PWD/target/release/agent-relay-broker" \ | ||
| npm run test:e2e |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== locate file =="
git ls-files | grep -F 'tests/e2e/fleet/README.md' || true
echo "== relevant README section =="
if [ -f tests/e2e/fleet/README.md ]; then
nl -ba tests/e2e/fleet/README.md | sed -n '1,90p' | sed -n '50,68p'
fi
echo "== bash continuation behavior probe =="
set +e
tmpfile="$(mktemp)"
# emulate the exact line shape in docs: backslash, spaces, inline comment, newline.
cat > "$tmpfile" <<'SH'
#!/usr/bin/env bash
set -u
RELAYCAST_ENGINE_DIR=/path/to/relaycast \ # must carry relaycast#194 and `#307` compat fixes
BROKER_BINARY_PATH="$PWD/target/release/agent-relay-broker" \
printf 'RELAYCAST_ENGINE_DIR=%q has_trailing=%s\n' "$RELAYCAST_ENGINE_DIR" 'trailing' || true
printf 'exit_status=%s\n' "$?"
SH
bash "$tmpfile"
ret=$?
echo "bash_exit=$ret"
rm -f "$tmpfile"
echo "== probe valid continuation equivalent =="
set +e
tmpfile2="$(mktemp)"
cat > "$tmpfile2" <<'SH'
#!/usr/bin/env bash
set -u
# The compatibility note is not part of the command.
RELAYCAST_ENGINE_DIR=/path/to/relaycast \
BROKER_BINARY_PATH="$PWD/target/release/agent-relay-broker" \
printf 'RELAYCAST_ENGINE_DIR=%q has_trailing=%s\n' "$RELAYCAST_ENGINE_DIR" 'trailing' || true
printf 'exit_status=%s\n' "$?"
SH
bash "$tmpfile2"
ret2=$?
echo "bash_exit=$ret2"
rm -f "$tmpfile2"Repository: AgentWorkforce/relay
Length of output: 272
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== locate file =="
git ls-files | grep -F 'tests/e2e/fleet/README.md' || true
echo "== relevant README section =="
if [ -f tests/e2e/fleet/README.md ]; then
nl -ba tests/e2e/fleet/README.md | sed -n '1,90p' | sed -n '50,68p'
fi
echo "== bash continuation behavior probe =="
tmpfile="$(mktemp)"
# emulate the exact line shape in docs: backslash, spaces, inline comment, newline.
cat > "$tmpfile" <<'SH'
#!/usr/bin/env bash
set -u
RELAYCAST_ENGINE_DIR=/path/to/relaycast \ # must carry relaycast#194 and `#307` compat fixes
BROKER_BINARY_PATH="$PWD/target/release/agent-relay-broker" \
printf 'RELAYCAST_ENGINE_DIR=%q has_trailing=%s\n' "$RELAYCAST_ENGINE_DIR" 'trailing' || true
printf 'exit_status=%s\n' "$?"
SH
bash "$tmpfile"
ret=$?
echo "bash_exit=$ret"
rm -f "$tmpfile"
echo "== probe valid continuation equivalent =="
tmpfile2="$(mktemp)"
cat > "$tmpfile2" <<'SH'
#!/usr/bin/env bash
set -u
# The compatibility note is not part of the command.
RELAYCAST_ENGINE_DIR=/path/to/relaycast \
BROKER_BINARY_PATH="$PWD/target/release/agent-relay-broker" \
printf 'RELAYCAST_ENGINE_DIR=%q has_trailing=%s\n' "$RELAYCAST_ENGINE_DIR" 'trailing' || true
printf 'exit_status=%s\n' "$?"
SH
bash "$tmpfile2"
ret2=$?
echo "bash_exit=$ret2"
rm -f "$tmpfile2"Repository: AgentWorkforce/relay
Length of output: 272
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== raw README section =="
awk 'NR>=55 && NR<=65 { printf "%6d %s\n", NR, $0 }' tests/e2e/fleet/README.md
echo "== bash syntax probe exact backslash+spaces+comment =="
tmpfile="$(mktemp)"
cat > "$tmpfile" <<'SH'
set -u
RELAYCAST_ENGINE_DIR=/path/to/relaycast \ # must carry relaycast#194 and `#307` compat fixes
BROKER_BINARY_PATH="$PWD/target/release/agent-relay-broker" \
printf 'RELAYCAST_ENGINE_DIR=%q has_trailing=%s\n' "$RELAYCAST_ENGINE_DIR" 'trailing' || true
printf 'exit_status=%s\n' "$?"
SH
bash -n "$tmpfile"
status=$?
if [ "$status" -eq 0 ]; then
bash "$tmpfile"
else
echo "syntax_check_failed_status=$status"
fi
rm -f "$tmpfile"
echo "== bash syntax probe valid continuation equivalent =="
tmpfile2="$(mktemp)"
cat > "$tmpfile2" <<'SH'
set -u
RELAYCAST_ENGINE_DIR=/path/to/relaycast \
BROKER_BINARY_PATH="$PWD/target/release/agent-relay-broker" \
printf 'RELAYCAST_ENGINE_DIR=%q has_trailing=%s\n' "$RELAYCAST_ENGINE_DIR" 'trailing' || true
printf 'exit_status=%s\n' "$?"
SH
bash -n "$tmpfile2"
status2=$?
if [ "$status2" -eq 0 ]; then
bash "$tmpfile2"
else
echo "syntax_check_failed_status=$status2"
fi
rm -f "$tmpfile2"Repository: AgentWorkforce/relay
Length of output: 950
Fix the shell continuation in the local command.
The backslash on Line 59 is not the last character before the newline, so Bash does not continue to the next command line. Move the compatibility note above the command so RELAYCAST_ENGINE_DIR can be passed to npm run test:e2e.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tests/e2e/fleet/README.md` around lines 59 - 61, Move the compatibility note
above the local test command, then ensure the RELAYCAST_ENGINE_DIR assignment
ends with the line-continuation backslash as its final character so it correctly
combines with BROKER_BINARY_PATH and npm run test:e2e.
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
2 issues found across 4 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="tests/e2e/fleet/fleet-e2e.test.ts">
<violation number="1" location="tests/e2e/fleet/fleet-e2e.test.ts:732">
P3: The two sequential inner waitFor budgets now sum exactly to the test's 35s timeout (25s dead-letter poll + 10s sender-notification poll), with the observer/sendDM setup before the first poll sitting outside that budget. In practice the dead-letter lands within the documented 15s sweep, but on slow CI the scheduled maintenance interval can drift and the dead-letter poll can burn close to its full budget — leaving the 10s notification poll no headroom and causing a spurious vitest timeout. Consider widening the outer test timeout (e.g. 45_000) so the two inner waits plus setup have slack, keeping the reliability improvement from this PR robust.</violation>
</file>
<file name="tests/e2e/fleet/README.md">
<violation number="1" location="tests/e2e/fleet/README.md:59">
P3: The trailing inline comment after the `\` on this line breaks the Bash line continuation (the backslash must be the last character before the newline), so `RELAYCAST_ENGINE_DIR` won't actually be passed to `npm run test:e2e` when following these instructions verbatim. Consider moving the compatibility note above the command instead of appending it after the continuation backslash.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
| expect(failed).toMatchObject({ target_agent_name: 'ttl-recipient' }); | ||
| senderWs.close(); | ||
| }, 25_000); | ||
| }, 35_000); |
Contributor
There was a problem hiding this comment.
P3: The two sequential inner waitFor budgets now sum exactly to the test's 35s timeout (25s dead-letter poll + 10s sender-notification poll), with the observer/sendDM setup before the first poll sitting outside that budget. In practice the dead-letter lands within the documented 15s sweep, but on slow CI the scheduled maintenance interval can drift and the dead-letter poll can burn close to its full budget — leaving the 10s notification poll no headroom and causing a spurious vitest timeout. Consider widening the outer test timeout (e.g. 45_000) so the two inner waits plus setup have slack, keeping the reliability improvement from this PR robust.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At tests/e2e/fleet/fleet-e2e.test.ts, line 732:
<comment>The two sequential inner waitFor budgets now sum exactly to the test's 35s timeout (25s dead-letter poll + 10s sender-notification poll), with the observer/sendDM setup before the first poll sitting outside that budget. In practice the dead-letter lands within the documented 15s sweep, but on slow CI the scheduled maintenance interval can drift and the dead-letter poll can burn close to its full budget — leaving the 10s notification poll no headroom and causing a spurious vitest timeout. Consider widening the outer test timeout (e.g. 45_000) so the two inner waits plus setup have slack, keeping the reliability improvement from this PR robust.</comment>
<file context>
@@ -729,7 +729,7 @@ describe.skipIf(!pre.ok)('bounded durable mailbox', () => {
expect(failed).toMatchObject({ target_agent_name: 'ttl-recipient' });
senderWs.close();
- }, 25_000);
+ }, 35_000);
// Overflow reject-new is enforced by `belowDepthCapSql` (counts queued+delivered
</file context>
Suggested change
| }, 35_000); | |
| }, 45_000); |
| npm run build:core # relay CLI + fleet + harness-driver | ||
| cargo build --release --bin agent-relay-broker # broker | ||
| RELAYCAST_ENGINE_DIR=/path/to/relaycast \ # must carry relaycast#194's compat fixes | ||
| RELAYCAST_ENGINE_DIR=/path/to/relaycast \ # must carry relaycast#194 and #307 compat fixes |
Contributor
There was a problem hiding this comment.
P3: The trailing inline comment after the \ on this line breaks the Bash line continuation (the backslash must be the last character before the newline), so RELAYCAST_ENGINE_DIR won't actually be passed to npm run test:e2e when following these instructions verbatim. Consider moving the compatibility note above the command instead of appending it after the continuation backslash.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At tests/e2e/fleet/README.md, line 59:
<comment>The trailing inline comment after the `\` on this line breaks the Bash line continuation (the backslash must be the last character before the newline), so `RELAYCAST_ENGINE_DIR` won't actually be passed to `npm run test:e2e` when following these instructions verbatim. Consider moving the compatibility note above the command instead of appending it after the continuation backslash.</comment>
<file context>
@@ -55,14 +56,14 @@ engine⇄broker mismatches this E2E surfaced (fixed in relaycast#194).
npm run build:core # relay CLI + fleet + harness-driver
cargo build --release --bin agent-relay-broker # broker
-RELAYCAST_ENGINE_DIR=/path/to/relaycast \ # must carry relaycast#194's compat fixes
+RELAYCAST_ENGINE_DIR=/path/to/relaycast \ # must carry relaycast#194 and #307 compat fixes
BROKER_BINARY_PATH="$PWD/target/release/agent-relay-broker" \
npm run test:e2e
</file context>
Suggested change
| RELAYCAST_ENGINE_DIR=/path/to/relaycast \ # must carry relaycast#194 and #307 compat fixes | |
| # must carry relaycast#194 and #307 compat fixes | |
| RELAYCAST_ENGINE_DIR=/path/to/relaycast \ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
loadRoot cause
0b43dbe3(post-#1445) fails against the old v6.0.1 engine because an unbounded broker-provider heartbeat omitsload; that engine stops applying the heartbeat, soactive_agentsnever advances.1ae49812(pre-#1445) passes. This is the same wire contract covered by relaycast#307 and requires the server release before the relay release.Verification
1ae49812: Fleet E2E exit 0 (27 passed, 4 skipped)0b43dbe3: Fleet E2E exit 1 (heartbeat/load failures)git diff --check: exit 0