Skip to content

fix(review-utils): metrics write permission + fail run on missing metrics - #322

Merged
nsheaps merged 1 commit into
mainfrom
claude/code-review-metrics-inconsistency-whxhpf
Aug 11, 2026
Merged

fix(review-utils): metrics write permission + fail run on missing metrics#322
nsheaps merged 1 commit into
mainfrom
claude/code-review-metrics-inconsistency-whxhpf

Conversation

@nsheaps

@nsheaps nsheaps commented Aug 11, 2026

Copy link
Copy Markdown
Owner

Summary

Root cause for the "AI Code Review" check showing failure on greasemonkey-scripts#54 while the underlying Actions run in .ai-agent-henry showed green:

  • run-agent/action.yaml tells the review skill to write metrics to $RUNNER_TEMP/review-metrics.yaml, but $RUNNER_TEMP was never in the claude-code-action settings' permissions.additionalDirectories allowlist. The write was sandbox-denied, so the agent could never emit the metrics file — confirmed in the run's own transcript ("Write to /home/runner/work/_temp/review-metrics.yaml — sandbox restricts writes outside the working directory").
  • read-metrics-and-compute-conclusion.sh treated "metrics missing" as a soft case and exit 0'd after posting conclusion=failure to the PR check. That's why the PR check went red but the Actions job/run stayed green — the script never actually failed the job. The spec (review-dispatch.md §Stage-by-stage bullet 6) already says this should fail the workflow; the implementation just didn't.

Changes

  • plugins/claude-code/review-utils/actions/run-agent/action.yaml: add ${{ runner.temp }} to additionalDirectories so the mandated metrics write is actually permitted.
  • .github/scripts/review-receiver/read-metrics-and-compute-conclusion.sh: exit 1 (was exit 0) when no metrics file is found, and log ::error:: instead of ::warning::. This routes through the receiver's existing if: failure() guard, so the Actions run itself shows red whenever the agent fails to complete a review — not just the PR check.

Test plan

  • bash -n syntax check passes on the modified script (done locally)
  • YAML parses (done locally via yaml.safe_load)
  • Next real PR dispatch to a consumer repo (e.g. re-trigger on greasemonkey-scripts#54) confirms metrics now write successfully and, if it were to fail again, the Actions run shows failed/red

Generated by Claude Code

…ssing metrics

The review agent is told to write review-metrics.yaml to $RUNNER_TEMP, but
that path was never in the claude-code-action settings' additionalDirectories
allowlist, so the write was sandbox-denied — the exact failure seen on
greasemonkey-scripts#54, where the agent reported it could not write the
metrics file at all.

Also: the receiver script exited 0 when metrics were missing, so the PR
check correctly showed failure but the Actions run itself stayed green.
Per the review-dispatch spec ("if absent, the workflow fails"), exit 1
instead so a genuine agent failure is red on the run, not just the PR check.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012gUPPPteashVoJMa4cNDD6
@github-actions

Copy link
Copy Markdown
Contributor

Plugin Version Preview

Preview only — plugin versions and marketplace.json are bumped automatically on merge to main, not in this PR. Manual bumps to higher versions are preserved. See the file annotations for the pending change on each plugin.json.

Plugin Base Current Action
review-utils 0.1.8 0.1.9 Auto-bumped

nsheaps commented Aug 11, 2026

Copy link
Copy Markdown
Owner Author

The test job failure here (@nsheaps/agents-plugins:testError: claude native binary not installed) is pre-existing on main, not caused by this PR — confirmed on run 31435599288 (commit 2d8d152c, predates this branch), same failure. Looks like a bun install postinstall/optional-dependency gap for @anthropic-ai/claude-code's native binary, unrelated to the two files this PR touches. Not fixing it here; will re-check once main's test job recovers.


Generated by Claude Code

@nsheaps
nsheaps marked this pull request as ready for review August 11, 2026 02:24
@nsheaps
nsheaps merged commit 7b41f26 into main Aug 11, 2026
7 of 8 checks passed
@nsheaps
nsheaps deleted the claude/code-review-metrics-inconsistency-whxhpf branch August 11, 2026 02:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants