fix(adapters,engine,escalation): synthesize marker-less terminal specs + review.on_timeout salvage#273
Merged
Conversation
…s + review.on_timeout salvage Closes #224, closes #271. Part A (#224): a review HALT can finalize the spec frontmatter to status: done while omitting the '## Auto Run Result' marker the harvest scan keys on; every Stop then read no-artifact, stall nudges re-invoked an already-exited workflow (#149), and after max_review_cycles the finished verify-passing work was rolled back. The scan path now synthesizes the result from the authoritative terminal frontmatter (devcontract.find_frontmatter_candidates + the same synthesize_result stories mode already trusts) once the (path, mtime, status) fingerprint holds stable across FM_FALLBACK_MIN_OBS resultless Stops; ambiguity refuses to guess. _post_kill_reconcile applies the same fallback on a single sighting once the window is provably dead. Synthesized results carry synthesized_from_frontmatter and the engine journals session-synthesized-from-frontmatter. Part B (#271): review.on_timeout = retry | salvage-if-done | defer (default retry, unchanged). salvage-if-done converges a timeout-like review verdict (timeout/stalled/over_budget) over an already-finalized, verify-green dev product: commit, reset a mid-review in-review interrupt forward, refile the outstanding follow-up to deferred work (origin review-timeout-salvage), journal review-timeout-salvage; never re-arms followup_review_recommended nor spends a damping grant. env-fault and crashed keep their own routing in every mode; resolved_redrive still re-escalates instead of deferring.
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
tests/test_policy.py (1)
48-50: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winEscape the
.in thematch=pattern.match=is a regex; the unescaped.matches any character (e.g.reviewXon_timeoutwould also match). Static analysis flagged this (RUF043).♻️ Tighten the match pattern
- with pytest.raises(policy.PolicyError, match="review.on_timeout"): + with pytest.raises(policy.PolicyError, match=r"review\.on_timeout"):🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/test_policy.py` around lines 48 - 50, Update the match pattern in test_review_on_timeout_invalid to escape the dot in “review.on_timeout”, ensuring pytest treats it as a literal period rather than a regex wildcard.Source: Linters/SAST tools
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/bmad_loop/data/settings/core.toml`:
- Line 48: Remove the standalone duplicate [[section.field]] declaration in the
settings configuration, leaving only the populated field entry for on_timeout.
Ensure consumers no longer receive an empty section.field object without key or
metadata.
---
Nitpick comments:
In `@tests/test_policy.py`:
- Around line 48-50: Update the match pattern in test_review_on_timeout_invalid
to escape the dot in “review.on_timeout”, ensuring pytest treats it as a literal
period rather than a regex wildcard.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: f4270715-7592-48f4-ae64-806f17beb995
📒 Files selected for processing (12)
CHANGELOG.mdsrc/bmad_loop/adapters/generic.pysrc/bmad_loop/data/settings/core.tomlsrc/bmad_loop/devcontract.pysrc/bmad_loop/engine.pysrc/bmad_loop/escalation.pysrc/bmad_loop/policy.pytests/test_devcontract.pytests/test_engine.pytests/test_escalation.pytests/test_generic_tmux.pytests/test_policy.py
This was referenced Jul 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #224. Closes #271.
Part A — #224: a finished story whose session omitted
## Auto Run Resultno longer livelocks and DEFER-dropsThe review HALT intermittently finalizes the spec's frontmatter to
status: donewithout appending the terminal markerfind_result_artifactkeys on. On the sprint/sweep scan path that spec was invisible: every Stop readno-artifact, stall nudges re-invoked an already-exited workflow (the #149 livelock),_post_kill_reconcilecouldn't rescue it (its read-back hits the same locator), and each RETRY re-stripped the marker (#160) and reproduced the omission — untilmax_review_cyclesdeferred and rolled back finished, verify-passing work. Observed twice consecutively in one run (stories 3-10/3-11).Fix — synthesize from the authoritative terminal frontmatter, exactly what stories mode already trusts (
devcontract.synthesize_result):devcontract.find_frontmatter_candidates: scan for specs with terminal frontmatter (done/blocked), zero real (non-fenced) markers, mtime at/after the session-launch floor. Returns all matches; the caller refuses to guess between several.(path, mtime_ns, status)fingerprint that held stable across 2 consecutive resultless Stops — a single sighting is not enough, because right after a review launch the spec still carries the dev pass'sdonefrontmatter and the review's first write can bump mtime before thein-reviewflip (the Session read-back adopts another story's spec: a review that produced nothing is scored done #261 premature-harvest hazard). Any fingerprint change restarts the count; a status flip clears it._post_kill_reconcilebackstop: a provably dead window harvests on one sighting (dead_window=True); the existing gates (self-consistent, escalation-free, done-only) still apply, and every rescue runs the full deterministic verify downstream (the Hardening: reconcile seam is unreachable for stalled sessions — a finished-but-lost-Stop session cannot be recovered as completed #61 trust model).synthesized_from_frontmatter, the engine journalssession-synthesized-from-frontmatter, and new resultless-stop breadcrumb verdictsterminal-frontmatter-pending/ambiguous-frontmatterrecord the fingerprint's progress.The upstream skill fix (make the review HALT's marker append reliably executed) remains a BMAD-METHOD concern; this is the orchestrator-side safety net the issue argues for given the observed intermittency.
Part B — #271:
review.on_timeoutpolicy knobdecide_review_sessiontreated every non-completed review verdict identically: RETRY untilmax_review_cycles, then DEFER — even when the dev product was already good enough to commit. New knob (validated, settings-UI wired, default preserves today's behavior byte-for-byte):retry(default) — unchanged.salvage-if-done— on a timeout-like verdict (timeout/stalled/over_budget, the same set the post-kill reconcile treats as rescue-eligible), if the spec frontmatter readsdone(orin-review, the mid-review interrupt — reset forward, partial terminal section stripped) and_verify_reviewpasses: commit the work, refile the outstanding follow-up to deferred work (originreview-timeout-salvage), journalreview-timeout-salvage. A timeout never re-armsfollowup_review_recommendednor spends a damping grant. Not applicable → falls back through the extractedreview_retry_or_exhaust; a failing verify journalsreview-timeout-salvage-failedfirst.defer— give up on the first timeout-like verdict.Guard rails: env-fault (#194) still PAUSEs before any mode;
crashedkeeps the default routing in every mode; theresolved_redrivelatch still re-escalates instead of deferring; worktree-isolated runs never salvage (a defer there already preserves the unit's diff). Interaction: with Part A in place, a marker-lessdonespec completes in the adapter and never reaches the timeout path — salvage only ever sees the complementary case (a review that died with non-terminal frontmatter or never wrote the spec).Tests
test_devcontract.py: candidate-finder matrix (terminal statuses, marker/fence exclusion, no-spec-fallback filename, mtime floor, unreadable files).test_generic_tmux.py: two-stable-Stops harvest, premature-harvest guard (mtime bump resets), status-flip clears observations, blocked→CRITICAL, ambiguity never harvests, crash-path compare-only semantics, post-kill backstop (done rescued / blocked & ambiguous & pre-launch kept), wait-loop end-to-end via scripted Stops.test_escalation.py: {timeout, stalled, over_budget} × {retry, salvage-if-done, defer} matrix;crashedand env-fault invariant under every mode; default-mode reasons byte-compatible.test_engine.py: salvage commit + refile + no-rearm,in-reviewreset variant, verify-fail fallback, non-terminal-spec fallback, isolation/no-spec guards, journal event.test_policy.py/test_settings_schema.py: knob parse/validation + core.toml sync.Full suite: 2847 passed; the 2
test_module_skills_sync.pyfailures are pre-existing onmain(verified).trunk checkclean.Summary by CodeRabbit
New Features
Bug Fixes