docs(review): record the META-321 calibration outcome and the required-check decision

[qmarcelle]

Completes META-321. Records what canary PR #23 measured and the branch-protection decision that follows from it.

The short version

The rules work. The check status does not carry the result. Greptile Review is therefore not made a required status, and branch protection is unchanged.

The rules caught every positive control — 3/3

Canary #23 pushed three deliberate producer defects, each mapped to one rule so attribution was unambiguous:

Defect (head 0db1a26)	Rule the reviewer cited	Caught
Refused history refresh made invisible to the caller	history-refresh-refusal-observable	yes, P1
Mining default inverted, ordinary generation reads the commit graph	ordinary-generation-never-mines	yes, P1
compareUtf8 replaced with bare <= in canonical ordering	canonical-utf8-endpoint-order	yes, P1

Each cited Rule Used: … (source: .greptile) — so branch-local repo-owned config is read, not a default profile. pnpm typecheck was clean on all three. None is visible to the compiler.

Why the check is still not required — measured in both directions

• Cannot-ever-fail. On PR tooling(review): repo-owned Greptile review policy for the CLI producer (META-321) #22 the reviewer posted a P1 at 13:30:16 and Greptile Review concluded success at 13:30:18.
• Cannot-ever-pass. On the canary's revert head 2eac40d, whose diff nets to empty against main, no Greptile Review check run was posted at all.

A required context that goes green with an open P1 and can also simply never appear would neither block a bad head nor clear a good one. That is the vacuous-check shape .greptile/rules.md prohibits — so it is not installed here.

What does enforce review, measured not assumed

With the rollup at SUCCESS and the branch MERGEABLE, PR #22 sat at mergeStateStatus: BLOCKED on one unresolved thread and moved to CLEAN the moment it resolved — head SHA and rollup unchanged, so resolution was the only variable.

Conversation resolution is the load-bearing gate. Greptile produces the findings it counts.

No-repeat evidence

The revert head was deliberately not accepted as proof of no-repeat: with a net-empty diff nothing was reviewed, so "no finding" there is absence of evidence. A third head 7fbb87c restored a real diff on the same two files with comment-only changes — Greptile reviewed it, concluded success, added no findings, total stayed at 3.

Sourcery

Non-required status retained on new evidence rather than inheritance: Sourcery review concluded success on the head carrying all three deliberate defects.

Verification on this head

check:architecture passes (100 files, 4 manifests); architecture red suite 21/21.

Release impact — measured

None. No changeset. REVIEW.md is outside packages/; no published bytes, exports, schema, or package metadata change.

Summary by Sourcery

Documentation:

• Update REVIEW.md to capture the canary-based evidence for Greptile Review rules, the non-required status decisions for Greptile and Sourcery, and the current required contexts on main.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[sourcery-ai]

Reviewer's Guide

Documents the META-321 calibration results for Greptile and Sourcery review checks, explains why Greptile Review remains non-required despite effective rules, clarifies what actually enforces review (required conversation resolution), and updates the documented required contexts on main without changing release behavior.

Sequence diagram for review enforcement vs Greptile/Sourcery checks

sequenceDiagram
    actor Developer
    participant GitHub
    participant GreptileReview
    participant SourceryReview

    Developer->>GitHub: open_pull_request
    GitHub->>GreptileReview: run_review_check
    GitHub->>SourceryReview: run_review_check

    GreptileReview-->>GitHub: conclude_success (P1 findings may exist)
    SourceryReview-->>GitHub: conclude_success (even with deliberate defects)

    GitHub-->>Developer: mergeStateStatus BLOCKED [unresolved_review_thread]

    Developer->>GitHub: resolve_conversation_thread
    GitHub-->>Developer: mergeStateStatus CLEAN

    Note over GitHub: Required contexts:
    Note over GitHub: test_20, test_22,
    Note over GitHub: Compatibility_parity_vs_frozen_source,
    Note over GitHub: required_conversation_resolution
    Note over GreptileReview,SourceryReview: Non-required status checks

Loading

File-Level Changes

Change	Details	Files
Clarified that Greptile Review is explicitly not a required branch-protection status based on measured calibration evidence and documented the META-321 canary results.	Updated the Greptile Review row in the required checks table to state it is not required by measured decision and link to the detailed status section. Expanded the Greptile gate section with calibration metadata (date, META-321, canary PR reference) and a narrative that the rules work while the check status does not encode rule outcomes. Added a description of the positive-control canary introducing three deliberate producer defects, each mapped to specific Greptile rules and confirmed caught, along with proof that repo-local configuration was used and defects are not typecheck-visible.	REVIEW.md
Explained why Greptile Review is not made a required check by characterizing its behavior in failure and success edge cases and classifying it as a vacuous check under existing policy.	Described an observed case where Greptile Review concluded success despite an open P1 finding, showing the check cannot be trusted to fail when defects exist. Described an observed case where no Greptile Review check was posted on a revert head with an empty diff, showing the check can simply be absent and thus can never be satisfied if required. Explicitly tied these observations to the vacuous-check shape prohibited by .greptile/rules.md and concluded that Greptile Review is therefore not installed as a required branch-protection status.	REVIEW.md
Documented that required conversation resolution, not Greptile’s status, is the true enforcement mechanism for review and clarified the current required contexts on main.	Added a measurement-based explanation that mergeStateStatus transitioned from BLOCKED to CLEAN solely on conversation resolution with unchanged head SHA and check rollup, proving required thread resolution is the load-bearing gate. Outlined the effective gate as required CI plus reconciliation of all actionable findings on their own threads, positioning Greptile as the producer of findings rather than the enforcing gate. Reiterated the list of existing required contexts on main (test suites and compatibility check with strict and required_conversation_resolution) and noted they are unchanged by this calibration.	REVIEW.md
Recorded calibration evidence and decisions related to Sourcery review and clarified its continued non-required status.	Inserted a short subsection explaining that META-321’s canary head carrying deliberate defects still received a success conclusion from the Sourcery review check. Stated that this outcome reinforces existing calibration and supports keeping Sourcery review as a non-required, defense-in-depth check pending its own calibration. Linked Sourcery’s status to the same evidence-based standard applied to Greptile, rather than inheritance of prior configuration.	REVIEW.md

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sourcery-ai]

Hey - I've left some high level feedback:

• The Greptile calibration narrative is quite long and repeats concepts (e.g., vacuous checks and conversation resolution); consider tightening this section to a shorter, more linear explanation so readers can quickly understand the decision and evidence.
• The exact timestamps and PR numbers used as calibration examples are currently embedded in prose; consider emphasizing the general rule (what conditions make a check eligible to be required) and moving the specific calibration run details into a short, clearly labeled bullet list or table.
• The phrase "Revisit if Greptile's check conclusion becomes a documented function of finding severity" defines a future policy trigger; consider making this a more explicit criterion (e.g., numbered condition in a list) to avoid ambiguity when someone later decides whether the calibration needs to be rerun.

Prompt for AI Agents

Please address the comments from this code review:

## Overall Comments
- The Greptile calibration narrative is quite long and repeats concepts (e.g., vacuous checks and conversation resolution); consider tightening this section to a shorter, more linear explanation so readers can quickly understand the decision and evidence.
- The exact timestamps and PR numbers used as calibration examples are currently embedded in prose; consider emphasizing the general rule (what conditions make a check eligible to be required) and moving the specific calibration run details into a short, clearly labeled bullet list or table.
- The phrase "Revisit if Greptile's check conclusion becomes a documented function of finding severity" defines a future policy trigger; consider making this a more explicit criterion (e.g., numbered condition in a list) to avoid ambiguity when someone later decides whether the calibration needs to be rerun.

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[greptile-apps]

Greptile Summary

Records the META-321 review calibration outcome and explains why Greptile Review remains non-required.

• Documents that all three semantic positive controls were detected.
• Records both failure directions of the Greptile status and identifies conversation resolution as the enforceable review gate.
• Confirms the existing required contexts and Sourcery’s non-required status remain unchanged.

Confidence Score: 5/5

The documentation-only change appears safe to merge with no actionable issues identified.

The updated policy is internally consistent, its named required checks match the checked-in workflow job names, and no committed configuration contradicts the documented gate decision.

Important Files Changed

Filename	Overview
REVIEW.md	Updates the repository’s review policy with calibration evidence, the resulting branch-protection decision, and the current required checks; no actionable defect was identified.

_{Reviews (1): Last reviewed commit: "docs(review): record the calibration out..." | Re-trigger Greptile}