evidence: frozen driver for the canonical external co-change run - #21
Merged
Conversation
…A-310) The driver that produced canonical co-change artifacts for three external repositories, committed as the exact bytes that ran. Its sha256 is published in the evidence receipt, so these bytes are the reproduction claim rather than an approximation of it. It lives here rather than beside the artifacts in workspacejson/standard because it imports @workspacejson/cli, and standard sits at the top of the dependency graph — its architecture check correctly refused the file. The invariant is worth more than the convenience of a self-contained directory, and reproduction already requires both repositories pinned, so nothing is lost by the split. meta310-mine.mjs sha256 5be5c814caed895b30a26d6fee697e1b65bc01c95789235dc49ad2a3f805e83c runner-package.json sha256 fed868de3cce3c5e75010fa7062221fb53f155a20ca90c850c5ed47be5a59795 The driver passes no options to mine/score/select: the analysis contract is whatever the pinned source froze, not what the driver adds. It exercises two paths and cross-checks them pair-for-pair — the producer path through the copy of mining-core bundled into the cli tarball, and a receipt path through the separately installed mining-core that exposes basisWindow, exclusions and the selection receipt the artifact does not carry. A disagreement aborts rather than being reconciled. runner-package.json records the clean-install cache defense: cli declares spec and rules by version, and candidate and published packages carry identical version numbers, so overrides force file: resolution. Note for readers: the file contains three NUL bytes, used as separators in the cross-check comparison key. That is sound — a NUL cannot occur in a path — and both sides of every comparison use the same function, so results are unaffected. It was not edited after the run; correcting it cosmetically would change the digest and make this something other than the driver that produced the evidence. Gates: architecture, package-docs, tests, typecheck — all green.
Reviewer's guide (collapsed on small PRs)Reviewer's GuideThis PR adds a frozen, reproducible driver and its runner package definition for the canonical external co-change run, capturing exactly the bytes that produced the evidence artifacts and wiring them to work against pinned workspacejson/cli and mining-core instances via a clean-install, override-based setup. File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
Greptile SummaryThe PR adds the byte-frozen META-310 mining driver and portable clean-room reproduction materials for regenerating canonical co-change evidence.
Confidence Score: 3/5The PR is not yet safe to merge because the frozen driver can still emit successful-looking evidence when its cross-check is absent, incomplete, disagrees, throws, or detects instability. The driver compares only pair fields when an artifact co-change block exists, does not bind that comparison to the evidence basis, and records validation failures without failing the process, leaving both previously reported evidence-integrity defects outstanding. Files Needing Attention: evidence/meta-310/meta310-mine.mjs
|
| Filename | Overview |
|---|---|
| evidence/meta-310/README.md | Documents the portable tarball workflow, pinned inputs, install audit, and calibration procedure. |
| evidence/meta-310/runner-package.json | Replaces machine-local package references with relative tarball dependencies and overrides, resolving the two prior portability findings. |
| evidence/meta-310/meta310-mine.mjs | Adds the frozen evidence driver, but its previously reported cross-check completeness and failure-exit defects remain present. |
Sequence Diagram
sequenceDiagram
participant Reviewer
participant Worktrees
participant Runner
participant Producer as Bundled mining-core
participant Receipt as Installed mining-core
Reviewer->>Worktrees: Check out pinned revisions
Worktrees->>Runner: Pack four verified tarballs
Reviewer->>Runner: Install portable manifest
Runner->>Producer: Generate canonical artifact
Runner->>Receipt: Mine, score, and select receipt data
Producer-->>Runner: Artifact pairs
Receipt-->>Runner: Receipt pairs and basis
Runner->>Runner: Compare results and rerun stability
Runner-->>Reviewer: Artifact and receipt outputs
Reviews (3): Last reviewed commit: "docs(evidence): fix reproduction steps t..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
…d being frozen Review on #21 found that runner-package.json pinned every dependency and override to an absolute /private/tmp/... path from the originating machine. Installed anywhere else it fails before the driver can run, so the file documented the environment while preventing anyone from recreating it — the opposite of what an evidence manifest is for. Correct finding; fixed rather than argued with. Paths are now relative to a tarballs/ directory beside the manifest. Verified end to end rather than by inspection: a fresh directory, tarballs copied in, npm install, then the driver run against standard @ 8e08c8c reproduces history-block 7012352617df with 50 entries, valid, cross-check identical. The install audit still shows 0 registry URLs for @workspacejson/*, all four resolving file:, and exactly 1 copy of @workspacejson/spec. This makes the manifest deliberately NOT byte-frozen, which is a real change to what the receipt claims, so it is stated rather than absorbed: as-run fed868de3cce3c5e75010fa7062221fb53f155a20ca90c850c5ed47be5a59795 now no longer matches that digest The two differ only in the tarball directory prefix. The package set, versions and the overrides that force file: resolution are identical, and no result depends on the prefix — the run depended on the CONTENT of the four tarballs, whose digests are recorded in the evidence receipt and are unchanged. meta310-mine.mjs remains byte-frozen at 5be5c814 and is untouched. The driver's bytes are the reproduction claim; the manifest's absolute paths were machine noise. Freezing noise would have been cargo cult, and freezing it at the cost of reproducibility would have been worse. README.md added: what is frozen and what is not, the four tarball digests, the clean-install cache defense, and the calibration a rerun should confirm before any target is touched. Gates: architecture, tests, typecheck — green.
qmarcelle
added a commit
to workspacejson/standard
that referenced
this pull request
Aug 11, 2026
… driver PR The reproduction block cited the runner manifest by sha256 as though it were byte-frozen alongside the driver. Review on workspacejson/cli#21 found that the manifest as run pinned every dependency to an absolute /private/tmp/... path, which resolves on one machine and fails to install anywhere else — so the digest pinned bytes that actively prevented the reproduction they were meant to enable. The manifest is now portable and explicitly NOT byte-frozen, and this receipt says so rather than quietly dropping the digest. The as-run copy hashed fed868de and the committed file no longer matches it; they differ only in the tarball directory prefix, and no result depends on the prefix — the run depended on the content of the four tarballs, whose digests are recorded here and are unchanged. The driver remains byte-frozen at 5be5c814. Its bytes are the reproduction claim; the manifest's absolute paths were machine noise. Recorded as verified rather than asserted: a fresh install from the portable manifest reproduces calibration history-block 7012352617df with 50 entries, valid, cross-check identical.
Contributor
Author
Valid P1 — fixed in
|
|
Want your agent to iterate on Greptile's feedback? Start a greploop in Claude Code and it will work through the open comments and keep going until this PR reviews clean. |
…teral read Second review finding on #21, also correct. The steps said to pack into a tarballs/ directory "beside this manifest" and then to copy the manifest alone into a fresh directory. Followed literally that puts the tarballs next to the copy in the repository and nothing beside the installed manifest, so file:./tarballs/... resolves to files that are not there and npm install fails before the driver can run. The manifest's relative paths resolve against the directory it is INSTALLED from, not the directory it is stored in. The steps now name a $RUN directory, create $RUN/tarballs first, pack directly into it with --pack-destination, and copy BOTH files into $RUN — which is what the original verification actually did, and what the instructions failed to say. Verified by following the rewritten steps literally in a clean directory rather than by re-reading them: four tarball digests shasum -c against the receipt table: 4/4 OK registry URLs 0 spec copies 1 calibration entries 50 calibration history block 7012352617df... match validation valid, 0 errors Two instructions have now been wrong in the same place for the same underlying reason: a path that was correct on the machine that ran it and meaningless anywhere else. The fix is verification by execution, not by inspection.
Contributor
Author
|
Second P1, also valid, also fixed — The reproduction steps said to pack into a The root cause: the manifest's relative paths resolve against the directory it is installed from, not the directory it is stored in. The steps now name a Verified by executing the rewritten steps literally in a clean directory rather than by re-reading them: Worth naming the pattern, since this is the second finding in the same place: both defects were a path that was correct on the machine that produced the evidence and meaningless anywhere else. The absolute-path manifest was the same bug as the ambiguous pack destination. Neither was caught by inspection, and both were caught immediately by execution from an empty directory — which is now the standard this directory is held to. Thanks for the catches; both were real and both would have shipped an evidence package that only its author could reproduce. |
qmarcelle
added a commit
to workspacejson/standard
that referenced
this pull request
Aug 11, 2026
…es (#31) * evidence: canonical co-change artifacts for three external repositories (META-310) Three artifacts produced by the neutral producer over public commit graphs at pinned revisions, under one frozen contract with no per-repository tuning. Producer workspacejson/cli @ 031c350, standard @ f95c42f, both packed from clean detached worktrees. The spec/rules/cli tarball digests reproduce those recorded in .agents/RECEIPT.md bit for bit; that receipt expected them not to, so the match shows this is the same environment rather than a same-version rebuild. Harness calibrated before any target was mined: run against standard @ 8e08c8c it reproduces the committed artifact's history-block digest 7012352617df exactly, 50 entries, 20,417 bytes, availableTransitions 90. The digest definition was recovered and verified rather than invented — four alternatives were rejected because they did not reproduce the recorded value. FormatJS syncpack Polylith availableTransitions 6,545 919 394 extractedTransitions 500 500 394 windowTruncated yes yes no pairsBeforeCap 713 729 1,658 pairsEmitted 50 50 50 capBound yes yes yes validation valid/0 valid/0 valid/0 All three deterministic across two runs; the bundled producer copy of mining-core agrees pair-for-pair with the standalone copy in every run. Every artifact is the top 50 of a larger population, never "all qualifying pairs". META-289 §1.5 has not landed, so the file-role exclusion set is EMPTY and lockfiles, generated files and documentation are all in play. Recorded as empty rather than as "a recorded set" because it shapes the ranking: release plumbing dominates. Such pairs are valid observations with low outreach novelty, which is a ranking consideration, not grounds to erase evidence. Classifying all 150 emitted pairs by whether both endpoints are source files: FormatJS 0 of 50, syncpack 32 of 50, Polylith 5 of 50. FormatJS produced no source-coupling evidence at all — all 50 pairs are release and dependency plumbing. It is demoted on this output, not on taste; nothing in the prior screening story survives and it is not rescued. syncpack carries the no-import-edge case: the six sibling command modules src/commands/{fix,format,json,lint,list,update}.rs contribute 11 emitted pairs and none of the six references any other, so no import graph relates them. lint.rs <-> list.rs is support 10, occurrences 10 — of the 10 qualifying commits touching either, all 10 touched both. Recorded against overclaiming: instance.rs <-> version_group.rs (support 17) fails the test, because version_group.rs imports instance. Relationships are stated descriptively. No usefulness, adoption or predictive claim is inferred, no publication is authorized, and no maintainer has been contacted. Two gates pushed back during this change and both were obeyed rather than softened: - check-architecture refused the run driver, which imports @workspacejson/cli, under dependency-direction. Standard sits at the top of the graph. The driver now lives in workspacejson/cli at evidence/meta-310/ and is cited here by path and sha256; reproduction already requires both repositories pinned, so the split costs a reader nothing. - check-docs refused the artifacts for carrying an internal tracker identifier. The identifier is producer output, not prose: the miner stamps weightingVersion into every scoring basis and that string names the issue specifying the weighting. Editing it would ship an artifact misreporting which algorithm produced it. docs/evidence/ is exempted for the same reason PROVENANCE_FILES is — an audit trail must be able to name its own source. Watched-red on the widened exemption: an identifier injected into README.md still fails the gate, and passes again on restore, so the exemption is scoped to docs/evidence/ and did not blunt the rule. Gates: spec 264/264, rules 173/173, examples 11/11 + 12/12, architecture, schema, corpus, docs, adr, typecheck — all green. check:conformance requires a locally built candidate and fails identically on main; CI supplies it. * docs(evidence): correct the runner-manifest claim after review on the driver PR The reproduction block cited the runner manifest by sha256 as though it were byte-frozen alongside the driver. Review on workspacejson/cli#21 found that the manifest as run pinned every dependency to an absolute /private/tmp/... path, which resolves on one machine and fails to install anywhere else — so the digest pinned bytes that actively prevented the reproduction they were meant to enable. The manifest is now portable and explicitly NOT byte-frozen, and this receipt says so rather than quietly dropping the digest. The as-run copy hashed fed868de and the committed file no longer matches it; they differ only in the tarball directory prefix, and no result depends on the prefix — the run depended on the content of the four tarballs, whose digests are recorded here and are unchanged. The driver remains byte-frozen at 5be5c814. Its bytes are the reproduction claim; the manifest's absolute paths were machine noise. Recorded as verified rather than asserted: a fresh install from the portable manifest reproduces calibration history-block 7012352617df with 50 entries, valid, cross-check identical. * fix(docs-gate): narrow the evidence exemption from a subtree to one stamped field Review found the isEvidence() exemption overbroad, and it was. It skipped the identifier check for every Markdown, JSON and script under docs/evidence/, so the narrow provenance need — one producer-stamped field — was used to justify waving through arbitrary identifiers in human-authored evidence prose, including files not yet written. Counting what actually needed the exemption shows how far it overreached: *.workspace.json 0 identifiers the artifacts never needed it at all *.receipt.json 1 each all of them "weightingVersion": "META-289 v2.2.1" RECEIPT.md 6 human prose — exactly the overreach So the real need is a single JSON member, not a directory. Two replacements, each matching what the repository already does: - PRODUCER_STAMPED admits weightingVersion by value, on its own line. The string is data: editing it would ship an artifact misreporting which algorithm produced it. Fails closed — minified or reflowed JSON does not match and must be justified rather than silently admitted. - docs/evidence/meta-310/RECEIPT.md is enumerated in PROVENANCE_FILES, one file, next to the ADRs. A future evidence run inherits nothing and has to argue for itself. The mechanism, not just the scope, was the defect. PROVENANCE_FILES is a hand-maintained list precisely so each exemption stays a decision someone made; a directory wildcard auto-exempts whatever anyone drops there later. Convenience was not a good enough reason to depart from that. Watched-red, all four cases, because a gate loosened without being retested is not a gate: unrelated identifier in an evidence receipt JSON fails (the reported risk) unrelated identifier in tracked evidence prose fails identifier in a member OTHER than weightingVersion fails identifier in the enumerated RECEIPT.md passes The third case matters most: it shows the rule keys on the stamped member rather than on the string, so renaming the field does not smuggle an identifier past. Gates: architecture, schema, examples, corpus, docs, adr, tests, typecheck — green.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Companion to workspacejson/standard
evidence/meta-310-canonical-cochange-artifacts, which carries the three artifacts and receipts.The driver that produced canonical co-change artifacts for FormatJS, syncpack and Polylith, committed as the exact bytes that ran — its
sha256is published in the evidence receipt, so these bytes are the reproduction claim rather than an approximation of it.sha256evidence/meta-310/meta310-mine.mjs5be5c814caed895b30a26d6fee697e1b65bc01c95789235dc49ad2a3f805e83cevidence/meta-310/runner-package.jsonfed868de3cce3c5e75010fa7062221fb53f155a20ca90c850c5ed47be5a59795Why it is here and not beside the artifacts
It imports
@workspacejson/cli, andstandardsits at the top of the dependency graph. That repository'scheck-architecturerefused the file underdependency-direction. The gate was right and was not weakened; reproduction already requires both repositories pinned, so the split costs a reader nothing.What it does
Passes no options to
mine/score/select— the analysis contract is whatever the pinned source froze, not what the driver adds. It exercises two paths and cross-checks them pair-for-pair:mining-corebundled into the cli tarball, which yields the canonical artifact;mining-core, which exposesbasisWindow,exclusionsand the selection receipt the artifact does not carry.A disagreement aborts rather than being reconciled. All three runs plus the calibration: identical.
runner-package.jsonrecords the clean-install cache defense — cli declaresspecandrulesby version, and candidate and published packages carry identical version numbers, sooverridesforcefile:resolution.Note for reviewers
The file contains three NUL bytes, used as separators in the cross-check comparison key, so Git shows it as binary. That is sound — a NUL cannot occur in a path — and both sides of every comparison use the same function, so results are unaffected. It was not edited after the run: correcting it cosmetically would change the digest and make this something other than the driver that produced the evidence.
Gates: architecture, package-docs, tests, typecheck — all green.
Summary by Sourcery
Add the frozen package manifest for the canonical external co-change driver used to produce the published evidence artifacts, ensuring the exact dependency and override configuration is captured for reproducibility.
New Features:
Enhancements: