Skip to content

DO NOT MERGE — canary(META-321): calibration of the CLI Greptile policy - #23

Closed
qmarcelle wants to merge 3 commits into
mainfrom
canary/meta-321-greptile-calibration
Closed

DO NOT MERGE — canary(META-321): calibration of the CLI Greptile policy#23
qmarcelle wants to merge 3 commits into
mainfrom
canary/meta-321-greptile-calibration

Conversation

@qmarcelle

@qmarcelle qmarcelle commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Disposable canary. Do not merge. Closed without merging once calibration completes.

This is the positive control for the repo-owned policy that landed in #22 (f11d430). It exists to answer one question: do the CLI-specific rules actually fire on the defects they name? A rule that has not been shown to fail for its own defect is not eligible to become a hard gate.

Injected defects — head 0db1a26

Three deliberate semantic defects, each mapped to exactly one repo-owned rule so caught/not-caught is attributable per rule.

# File Rule under test Defect
D1 packages/cli/src/producer/generate.ts history-refresh-refusal-observable historyRefresh is emitted only when mining produced a block, so an explicit mineHistory: true that refuses returns a successful-looking result carrying the previous revision's counts, with no refusal reported. This is the PR #20 P1 reintroduced verbatim.
D2 packages/cli/src/producer/generate.ts ordinary-generation-never-mines The opt-in default is inverted to mineHistory !== false, so ordinary generation reads the commit graph on every run.
D3 packages/mining-core/src/project.ts canonical-utf8-endpoint-order canonicalPair drops compareUtf8 for bare <=, substituting UTF-16 code unit order for the UTF-8 byte order two independent producers are compared on.

Expected outcome

CI is expected red on this head — that is part of the control, confirming the defects are real rather than cosmetic. Measured locally before pushing:

  • @workspacejson/mining-core — 1 failed / 96 passed (97): project.test.ts > uses UTF-8 BYTE order, not UTF-16 code unit order (D3)
  • @workspacejson/cli — 24 failed, concentrated in generate.integration.test.ts and producer-conformance.test.ts (D1/D2)
  • pnpm typecheckclean on all three packages. None of these defects is a type error.

That last line is the point of the exercise. Type checking cannot see any of them, and D1 in particular is the class of defect that ships a plausible artifact — it is only visible to a reviewer that knows what the field is for.

What is being calibrated

  1. Branch-local .greptile configuration is read, and the review check is tied to the current head.
  2. Each rule catches its own positive control — recorded per rule, caught or not.
  3. A reverted head does not repeat the finding.
  4. A new push retriggers review.
  5. Unresolved actionable findings affect merge eligibility as claimed.

Results are recorded on META-321 as observed counts and states.

Scope

No production change lands from this branch. Nothing here is merged, no producer behavior on main is modified, and no release or publish path is touched.

Summary by Sourcery

Adjust CLI workspace generation to mine history by default and simplify history refresh reporting, and change mining-core project file-pair canonicalization to use standard string ordering instead of UTF-8 byte-order comparison.

Enhancements:

  • Change CLI history mining behavior to default to mining unless explicitly disabled.
  • Simplify CLI history refresh outcome reporting to only emit when a history block is produced.
  • Update mining-core canonical file pair ordering to rely on default string comparison rather than UTF-8 byte-order comparison.

Positive control for the CLI Greptile policy. Each defect maps to exactly one
repo-owned rule so caught/not-caught is attributable per rule.

D1 packages/cli/src/producer/generate.ts -> history-refresh-refusal-observable
   historyRefresh is emitted only when mining produced a block, so an explicit
   mineHistory: true that refuses now returns a successful-looking result
   carrying the previous revision's counts with no refusal reported. This is
   the PR #20 P1 reintroduced verbatim.

D2 packages/cli/src/producer/generate.ts -> ordinary-generation-never-mines
   The opt-in default is inverted to mineHistory !== false, so ordinary
   generation reads the commit graph on every run.

D3 packages/mining-core/src/project.ts -> canonical-utf8-endpoint-order
   canonicalPair drops compareUtf8 for bare <=, substituting UTF-16 code unit
   order for the UTF-8 byte order two producers are compared on.

This branch is disposable and is closed without merging.
Copilot AI lite review requested due to automatic review settings August 12, 2026 13:41

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@sourcery-ai

sourcery-ai Bot commented Aug 12, 2026

Copy link
Copy Markdown
Reviewer's guide (collapsed on small PRs)

Reviewer's Guide

Disposable canary PR that deliberately injects three semantic defects into CLI history mining and mining-core canonicalization to calibrate repo-owned Greptile policy rules and confirm they trigger on real, non-type-check-visible issues.

Sequence diagram for generateWorkspaceJson history mining and refresh outcome (defects D1/D2)

sequenceDiagram
  actor User
  participant Cli as cli_generate
  participant Generator as generateWorkspaceJson
  participant Miner as mineHistoryBlock

  User->>Cli: run workspace JSON generation
  Cli->>Generator: generateWorkspaceJson(options)
  alt options.mineHistory !== false
    Generator->>Miner: mineHistoryBlock(resolvedRoot, refreshDiagnostics)
    Miner-->>Generator: minedHistory
  else options.mineHistory === false
    Generator-->>Generator: minedHistory = undefined
  end

  Generator-->>Generator: preservedHistory = carryForwardHistory(existing)
  Generator-->>Generator: history = minedHistory ?? preservedHistory.history

  alt options.mineHistory === true && minedHistory !== undefined
    Generator-->>Generator: historyRefresh = { requested: true, mined: true, preserved: false }
  else [no mined history or not explicitly requested]
    Generator-->>Generator: historyRefresh = undefined
  end

  Generator-->>Cli: WorkspaceJsonV4 (history, historyRefresh)
Loading

File-Level Changes

Change Details Files
Altered CLI history mining defaults and refresh reporting to reintroduce two semantic defects related to mining behavior and refresh observability.
  • Changed history mining opt-in default so history is mined whenever mineHistory is not explicitly set to false, causing ordinary generation to mine history on every run.
  • Simplified historyRefresh to only report when mining ran and produced a block, dropping refusal reporting and preserved-history signaling when mining is requested but produces no block.
packages/cli/src/producer/generate.ts
Modified mining-core canonical pair ordering to drop UTF-8 byte-order comparison in favor of naive string comparison, reintroducing a defect in endpoint canonicalization.
  • Removed compareUtf8 import from mining-core project module, leaving only type imports.
  • Updated canonicalPair to use direct string <= comparison instead of compareUtf8, switching from UTF-8 byte-order to UTF-16 code unit ordering when canonicalizing file pairs.
packages/mining-core/src/project.ts

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've reviewed your changes and they look great!


Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@greptile-apps

greptile-apps Bot commented Aug 12, 2026

Copy link
Copy Markdown

Greptile Summary

The canary’s three deliberate semantic defects have been reverted. This head only adds explanatory comments documenting refusal diagnostics and canonical UTF-8 endpoint ordering.

  • Clarifies why history-refresh diagnostics must be passed into the mining operation.
  • Documents why artifact-bound endpoint ordering must use compareUtf8.

Confidence Score: 5/5

The current head appears safe to merge because no blocking failure remains.

No blocking failure remains.

Important Files Changed

Filename Overview
packages/cli/src/producer/generate.ts Adds a comment explaining the existing refusal-diagnostics propagation; executable behavior remains correct.
packages/mining-core/src/project.ts Adds a comment documenting the existing UTF-8 comparator requirement; canonicalization still uses compareUtf8.

Reviews (2): Last reviewed commit: "canary(META-321): non-violating head tou..." | Re-trigger Greptile

Comment thread packages/cli/src/producer/generate.ts
Comment thread packages/cli/src/producer/generate.ts Outdated
Comment thread packages/mining-core/src/project.ts
… NOT MERGE

The revert head 2eac40d nets to an empty diff against main, and Greptile posted
no check run for it at all. That is a real result about the gate, but it makes
weak evidence for 'a non-violating head does not repeat the finding': nothing
was re-reviewed.

This head restores a non-empty diff on exactly the two files that carried the
three findings, with comment-only changes that violate no rule. If the findings
do not recur here, they did not recur on reviewed code rather than on skipped
code.
@qmarcelle

Copy link
Copy Markdown
Contributor Author

Calibration complete. Closing without merging — this branch existed only as a positive control.

Results across three heads

Head Diff Greptile check Findings
0db1a26 3 defects non-empty failure (run 94133982853) 3, one per injected defect
2eac40d revert nets to empty vs main no check run posted at all none
7fbb87c non-violating, same two files non-empty success (run 94135738644) none — total stayed 3

Caught / not-caught, per rule

  • D1 → history-refresh-refusal-observablecaught, P1, rule cited by name
  • D2 → ordinary-generation-never-minescaught, P1, rule cited by name
  • D3 → canonical-utf8-endpoint-ordercaught, P1, rule cited by name

3/3. Each finding cited Rule Used: … ([source](.greptile)), so branch-local repo-owned configuration was read, not a default profile. pnpm typecheck was clean on all three defects — none is visible to the type system.

No repeat on a non-violating head

The revert head was deliberately not accepted as proof: with a net-empty diff nothing was reviewed, so "no repeat" there would have been absence of evidence. 7fbb87c restored a real diff on the same two files with comment-only changes; Greptile reviewed it, concluded success, and added no findings. The three original threads went isOutdated: true.

Reconciliation

All three findings were replied to individually on their own threads, each confirming the defect, its rule, and its revert. No bulk resolution was used to produce that state.

Nothing from this branch lands. The branch is deleted.

@qmarcelle qmarcelle closed this Aug 12, 2026
@qmarcelle
qmarcelle deleted the canary/meta-321-greptile-calibration branch August 12, 2026 13:50
qmarcelle added a commit that referenced this pull request Aug 12, 2026
…eck status does not (META-321) (#24)

Canary PR #23 answered the question META-321 actually asked, and the answer is
not the one the staged rollout was set up to expect.

The rules caught 3/3 positive controls, each citing the rule by name and each
sourced from `.greptile`, which also proves branch-local configuration is read.
Typecheck was clean on all three defects, so none of them is visible to the
compiler — the semantic layer earned its place.

The check STATUS is a different question, and it fails in both directions:

- cannot-ever-fail: on PR #22 the reviewer posted a P1 at 13:30:16 and the
  Greptile Review check concluded success at 13:30:18;
- cannot-ever-pass: on the canary's revert head, whose diff nets to empty
  against main, no Greptile Review check run was posted at all.

A required context that goes green with an open P1 and can also never appear
would neither block a bad head nor clear a good one. That is the vacuous-check
shape .greptile/rules.md prohibits, so it is not installed. Branch protection is
unchanged: test (20), test (22), Compatibility parity vs frozen source, strict,
required conversation resolution.

What does enforce review was measured rather than assumed. With the rollup at
SUCCESS and the branch MERGEABLE, PR #22 sat at BLOCKED on one unresolved
thread and moved to CLEAN when that thread resolved, head and rollup unchanged.
Conversation resolution is the load-bearing mechanism; Greptile produces the
findings it counts.

Sourcery keeps its non-required status on new evidence rather than inheritance:
it concluded success on the head carrying all three deliberate defects.

No release impact: REVIEW.md is outside packages/, no published bytes change,
no changeset.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants