Skip to content

Incremental PR review: skip/brief/full based on diff fingerprint - #335

Draft
nsheaps wants to merge 7 commits into
mainfrom
claude/ai-mktpl-renovate-reviews-ztkti8
Draft

Incremental PR review: skip/brief/full based on diff fingerprint#335
nsheaps wants to merge 7 commits into
mainfrom
claude/ai-mktpl-renovate-reviews-ztkti8

Conversation

@nsheaps

@nsheaps nsheaps commented Aug 12, 2026

Copy link
Copy Markdown
Owner

Summary

Renovate (and similar bots) force-push rebases on update PRs constantly, and each push re-triggered a full-depth AI review even when the actual diff content hadn't changed. This makes review noise disproportionate to the PR's actual churn and encourages unnecessary re-review.

This PR adds an incremental-review system to the review-utils plugin's review-dispatch pipeline:

  • PR classification (routine-update vs standard) informs how much review depth is warranted.
  • Diff-content fingerprinting (gh pr diff | sha256sum, not head SHA) detects when a force-push changed nothing of substance.
  • Cross-run state cache via actions/cache (restore/save, keyed by repo+PR+run) lets each CI run see the prior run's classification, verdict, and fingerprint.
  • Skip / brief-refresh / full-review decision: if the diff fingerprint is unchanged, skip entirely (nothing posted); if changed but the PR is routine-update, post a brief, proportionate refresh; otherwise run the full review.
  • Approval dismissal moved into the skill itself, gated on the full-review decision — no more unconditional dismissal in review-receiver.yaml before every invocation. Only the skill's own prior APPROVED reviews are dismissed, and only when a full re-review is actually warranted.
  • Brief formatting mode for routine-update refreshes — a one-line verdict + optional follow-up bullets, no badge/scoring scaffolding built for deep reviews.

Plugin review-utils version bumped 0.2.00.3.0 (backwards-compatible new functionality); marketplace.json updated to match.

Changes

  • plugins/claude-code/review-utils/skills/review-code/partials/incremental-review.md — classification, fingerprinting, and skip/brief/full decision tree
  • plugins/claude-code/review-utils/skills/review-code/partials/review-thread-management.md — approval-dismissal procedure, gated to full-review only
  • plugins/claude-code/review-utils/skills/review-code/partials/review-formatting.md — brief-format template for routine-update refreshes
  • plugins/claude-code/review-utils/skills/review-code/SKILL.md — restructured review steps to load/classify/decide before doing any work; review-state emission step
  • plugins/claude-code/review-utils/actions/run-agent/action.yamlREVIEW_STATE_PATH export, cache restore/save around the agent run
  • .github/workflows/review-receiver.yaml — removed the unconditional pre-dismissal step and its dedicated app-token generation
  • .github/scripts/review-receiver/dismiss-prior-approvals.sh — marked superseded, kept for reference
  • .github/scripts/review-receiver/read-metrics-and-compute-conclusion.sh — handles skipped: true metrics (surfaces "previous review still stands" instead of implying a fresh review ran)
  • plugins/claude-code/review-utils/specs/review-dispatch.md — documents the redesign, updated schema/flow/diagram, resolves Open Question 1 (approval-dismissal timing)

Test plan

  • Validated YAML syntax of action.yaml and review-receiver.yaml (yaml.safe_load)
  • Validated JSON syntax of plugin.json/marketplace.json
  • Verified actions/cache SHA 0057852bfaa89a56745cba8c7296529d2fc39830 matches tag v4.3.0 via git ls-remote --tags
  • CI on this PR (validates plugin structure + lint)
  • Manual verification against a live Renovate PR in nsheaps/ai-mktpl once merged: confirm a no-op force-push produces no new review, and a substantive Renovate diff change produces a brief refresh rather than a full re-review

Generated by Claude Code

claude added 2 commits August 12, 2026 01:38
…cision

Renovate/Dependabot force-push rebases were re-triggering the full review
pipeline (dismiss approval -> full-depth review -> dismiss -> repeat) even
when the reviewable diff hadn't changed. This adds a cross-run review-state
cache (actions/cache, keyed per PR) so the review-code skill can classify
each PR (routine-update vs standard), fingerprint the diff content (not
head SHA), and decide to skip, post a brief refresh, or run a full review.

Approval dismissal moves from an unconditional receiver-side pre-step into
the skill itself, firing only when a full re-review is actually warranted.

- New partials/incremental-review.md: classification + skip/brief/full
  decision tree, referencing the review-state cache schema (v1).
- partials/review-thread-management.md: adds "Dismissing a stale prior
  approval", invoked by the skill only in the FULL REVIEW branch.
- partials/review-formatting.md: adds a brief format for routine-update
  BRIEF REFRESH reviews (no mandatory badges/details wrapper).
- SKILL.md: restructured steps to load/classify/decide before doing any
  review work, write review state, and skip posting entirely on SKIP.
- run-agent/action.yaml: exports REVIEW_STATE_PATH, restores/saves the
  cache via actions/cache.
- review-receiver.yaml: removes the unconditional "Dismiss prior approval
  reviews" step and its now-unused App token generation.
- dismiss-prior-approvals.sh: marked superseded (kept for reference).
- read-metrics-and-compute-conclusion.sh: handles the new `skipped` field
  with a distinct check title.
- review-dispatch.md: documents the redesign, resolves Open Question 1
  (approval-dismissal timing).
- review-utils plugin version bump 0.2.0 -> 0.3.0.
…ate-reviews-ztkti8

# Conflicts:
#	.claude-plugin/marketplace.json
#	plugins/claude-code/review-utils/.claude-plugin/plugin.json
#	plugins/claude-code/review-utils/skills/review-code/SKILL.md
@github-actions

github-actions Bot commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Plugin Version Preview

Preview only — plugin versions and marketplace.json are bumped automatically on merge to main, not in this PR. Manual bumps to higher versions are preserved. See the file annotations for the pending change on each plugin.json.

Plugin Base Current Action
review-utils 0.2.3 0.3.0 Already bumped

Comment thread .github/scripts/review-receiver/dismiss-prior-approvals.sh Outdated
Comment thread .github/workflows/review-receiver.yaml Outdated
…orkflow comments

Per review feedback: don't keep dead code around with a "superseded" comment
when git history already tells that story, and don't duplicate design
rationale in workflow YAML comments when the spec already carries it.

- Remove .github/scripts/review-receiver/dismiss-prior-approvals.sh — no
  longer invoked by anything since the 2026-08-12 redesign.
- Trim the "2026-08-12 redesign" comment block in review-receiver.yaml down
  to a one-line pointer at the spec, which already has the full writeup.
- Update the spec's script-inventory section to note the script's removal
  instead of describing a file that no longer exists.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

where's the part that uses actions/cache to save information for PR reviews between runs?

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not in review-receiver.yaml — the cache restore/save lives in the composite action it calls, plugins/claude-code/review-utils/actions/run-agent/action.yaml (restore at lines 222-229, save at 332-337), since that's where ${{ runner.temp }}/review-state.json is populated and where the agent run actually happens. review-receiver.yaml just invokes run-agent — it doesn't touch the cache directly.


Generated by Claude Code

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so it was already present? Can you trigger the review workflow through this PR and test that?

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No — verified via git show main:.../run-agent/action.yaml | grep actions/cache, which returns nothing. The cache restore/save is new in this PR.

Can't easily test it live through this PR though: this repo's own .github/workflows/dispatch-review.yaml (the consumer template copy) calls the reusable decider via uses: nsheaps/agents/.github/workflows/review-dispatch.yaml@9384e01d... — a pinned commit SHA, not @main and not this branch. Reusable workflow_call workflows always execute the code at the ref in uses:, so triggering a review on this PR right now would run that pinned SHA's code (which predates even current main, let alone this branch) — not the cache logic added here. Also this PR is a draft, so the gate wouldn't even fire without the request-review label per its if: condition.

Real options to validate before merge:

  1. Merge this, then validate on the next PR that touches review-utils (dogfooding via this repo's own future PRs).
  2. Temporarily repoint the SHA pin in dispatch-review.yaml to this branch's HEAD, label this PR request-review to force the gate despite draft state, watch it run, then revert the pin before merge. Invasive but does exercise the real code path.

I'd lean toward (1) unless you want the smoke test now — let me know which you'd prefer.


Generated by Claude Code

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can ignore this, this session will solve that: https://claude.ai/code/session_01HfFLavgfxQetUjPgy1VYFR

claude added 3 commits August 12, 2026 01:46
…-ztkti8' into claude/ai-mktpl-renovate-reviews-ztkti8
…ate-reviews-ztkti8

# Conflicts:
#	.claude-plugin/marketplace.json
#	plugins/claude-code/review-utils/.claude-plugin/plugin.json
…ate-reviews-ztkti8

# Conflicts:
#	.claude-plugin/marketplace.json
#	plugins/claude-code/review-utils/.claude-plugin/plugin.json
#	plugins/claude-code/review-utils/skills/review-code/SKILL.md
#	plugins/claude-code/review-utils/specs/review-dispatch.md

nsheaps commented Aug 12, 2026

Copy link
Copy Markdown
Owner Author

Resolved — merged origin/main (d12d9c3, 7 commits ahead incl. #334/#339) into this branch and pushed (da2d4ad). mergeable_state is clean now. main is moving fast right now (10 commits since this PR opened across 3 merge rounds) — I'll keep re-merging as needed until this is ready to land.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants