evidence: canonical co-change artifacts for three external repositories - #31
Merged
Merged
Conversation
…es (META-310) Three artifacts produced by the neutral producer over public commit graphs at pinned revisions, under one frozen contract with no per-repository tuning. Producer workspacejson/cli @ 031c350, standard @ f95c42f, both packed from clean detached worktrees. The spec/rules/cli tarball digests reproduce those recorded in .agents/RECEIPT.md bit for bit; that receipt expected them not to, so the match shows this is the same environment rather than a same-version rebuild. Harness calibrated before any target was mined: run against standard @ 8e08c8c it reproduces the committed artifact's history-block digest 7012352617df exactly, 50 entries, 20,417 bytes, availableTransitions 90. The digest definition was recovered and verified rather than invented — four alternatives were rejected because they did not reproduce the recorded value. FormatJS syncpack Polylith availableTransitions 6,545 919 394 extractedTransitions 500 500 394 windowTruncated yes yes no pairsBeforeCap 713 729 1,658 pairsEmitted 50 50 50 capBound yes yes yes validation valid/0 valid/0 valid/0 All three deterministic across two runs; the bundled producer copy of mining-core agrees pair-for-pair with the standalone copy in every run. Every artifact is the top 50 of a larger population, never "all qualifying pairs". META-289 §1.5 has not landed, so the file-role exclusion set is EMPTY and lockfiles, generated files and documentation are all in play. Recorded as empty rather than as "a recorded set" because it shapes the ranking: release plumbing dominates. Such pairs are valid observations with low outreach novelty, which is a ranking consideration, not grounds to erase evidence. Classifying all 150 emitted pairs by whether both endpoints are source files: FormatJS 0 of 50, syncpack 32 of 50, Polylith 5 of 50. FormatJS produced no source-coupling evidence at all — all 50 pairs are release and dependency plumbing. It is demoted on this output, not on taste; nothing in the prior screening story survives and it is not rescued. syncpack carries the no-import-edge case: the six sibling command modules src/commands/{fix,format,json,lint,list,update}.rs contribute 11 emitted pairs and none of the six references any other, so no import graph relates them. lint.rs <-> list.rs is support 10, occurrences 10 — of the 10 qualifying commits touching either, all 10 touched both. Recorded against overclaiming: instance.rs <-> version_group.rs (support 17) fails the test, because version_group.rs imports instance. Relationships are stated descriptively. No usefulness, adoption or predictive claim is inferred, no publication is authorized, and no maintainer has been contacted. Two gates pushed back during this change and both were obeyed rather than softened: - check-architecture refused the run driver, which imports @workspacejson/cli, under dependency-direction. Standard sits at the top of the graph. The driver now lives in workspacejson/cli at evidence/meta-310/ and is cited here by path and sha256; reproduction already requires both repositories pinned, so the split costs a reader nothing. - check-docs refused the artifacts for carrying an internal tracker identifier. The identifier is producer output, not prose: the miner stamps weightingVersion into every scoring basis and that string names the issue specifying the weighting. Editing it would ship an artifact misreporting which algorithm produced it. docs/evidence/ is exempted for the same reason PROVENANCE_FILES is — an audit trail must be able to name its own source. Watched-red on the widened exemption: an identifier injected into README.md still fails the gate, and passes again on restore, so the exemption is scoped to docs/evidence/ and did not blunt the rule. Gates: spec 264/264, rules 173/173, examples 11/11 + 12/12, architecture, schema, corpus, docs, adr, typecheck — all green. check:conformance requires a locally built candidate and fails identically on main; CI supplies it.
Reviewer's GuideAdds canonical co-change evidence artifacts for three external repositories under docs/evidence/meta-310, and adjusts the docs-checking script to exempt evidence files from internal tracker ID rules while reporting them in the check output. File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
|
| Filename | Overview |
|---|---|
| scripts/check-docs.mjs | Narrows internal-identifier handling by enumerating the receipt and recognizing producer-stamped weighting metadata. |
| docs/evidence/meta-310/RECEIPT.md | Documents the evidence-production contract, provenance, calibration, results, reproduction process, and interpretation limits. |
| docs/evidence/meta-310/formatjs.workspace.json | Adds the generated FormatJS workspace evidence artifact at its pinned revision. |
| docs/evidence/meta-310/syncpack.workspace.json | Adds the generated syncpack workspace evidence artifact at its pinned revision. |
| docs/evidence/meta-310/polylith.workspace.json | Adds the generated Polylith workspace evidence artifact at its pinned revision. |
Reviews (3): Last reviewed commit: "fix(docs-gate): narrow the evidence exem..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
… driver PR The reproduction block cited the runner manifest by sha256 as though it were byte-frozen alongside the driver. Review on workspacejson/cli#21 found that the manifest as run pinned every dependency to an absolute /private/tmp/... path, which resolves on one machine and fails to install anywhere else — so the digest pinned bytes that actively prevented the reproduction they were meant to enable. The manifest is now portable and explicitly NOT byte-frozen, and this receipt says so rather than quietly dropping the digest. The as-run copy hashed fed868de and the committed file no longer matches it; they differ only in the tarball directory prefix, and no result depends on the prefix — the run depended on the content of the four tarballs, whose digests are recorded here and are unchanged. The driver remains byte-frozen at 5be5c814. Its bytes are the reproduction claim; the manifest's absolute paths were machine noise. Recorded as verified rather than asserted: a fresh install from the portable manifest reproduces calibration history-block 7012352617df with 50 entries, valid, cross-check identical.
…tamped field
Review found the isEvidence() exemption overbroad, and it was. It skipped the
identifier check for every Markdown, JSON and script under docs/evidence/, so
the narrow provenance need — one producer-stamped field — was used to justify
waving through arbitrary identifiers in human-authored evidence prose, including
files not yet written.
Counting what actually needed the exemption shows how far it overreached:
*.workspace.json 0 identifiers the artifacts never needed it at all
*.receipt.json 1 each all of them "weightingVersion": "META-289 v2.2.1"
RECEIPT.md 6 human prose — exactly the overreach
So the real need is a single JSON member, not a directory. Two replacements,
each matching what the repository already does:
- PRODUCER_STAMPED admits weightingVersion by value, on its own line. The
string is data: editing it would ship an artifact misreporting which
algorithm produced it. Fails closed — minified or reflowed JSON does not
match and must be justified rather than silently admitted.
- docs/evidence/meta-310/RECEIPT.md is enumerated in PROVENANCE_FILES, one
file, next to the ADRs. A future evidence run inherits nothing and has to
argue for itself.
The mechanism, not just the scope, was the defect. PROVENANCE_FILES is a
hand-maintained list precisely so each exemption stays a decision someone made;
a directory wildcard auto-exempts whatever anyone drops there later. Convenience
was not a good enough reason to depart from that.
Watched-red, all four cases, because a gate loosened without being retested is
not a gate:
unrelated identifier in an evidence receipt JSON fails (the reported risk)
unrelated identifier in tracked evidence prose fails
identifier in a member OTHER than weightingVersion fails
identifier in the enumerated RECEIPT.md passes
The third case matters most: it shows the rule keys on the stamped member rather
than on the string, so renaming the field does not smuggle an identifier past.
Gates: architecture, schema, examples, corpus, docs, adr, tests, typecheck — green.
Contributor
Author
|
Valid, and the mechanism was the defect as much as the scope. Fixed in Counting what actually needed the exemption shows how far it overreached:
So the genuine need is one JSON member, not a subtree. I had used a narrow provenance requirement to justify a broad exemption, and the broad part covered files that did not need it plus files not yet written. Two replacements, each matching what this repository already does
The mechanism mattered independently of the scope. Watched-red, four cases
The third case is the one worth noting: the rule keys on the stamped member, not on the string, so renaming the field does not smuggle an identifier past it. One correction to my own testing while verifying this: my first attempt at the prose case reported a pass, but only because the file was untracked and the gate scans tracked files. Re-run with it staged, it fails as it should. Flagging it because a test that passes for the wrong reason is worse than no test. Gates: architecture, schema, examples, corpus, docs, adr, tests, typecheck — all green. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
First commit-history evidence from repositories that are not ours. Companion driver: workspacejson/cli#21.
Three artifacts produced by the neutral producer over public commit graphs at pinned revisions, under one frozen contract, no per-repository tuning. Contract was frozen and posted before any target was mined; revisions were pinned before the first result existed.
Provenance
Producer
workspacejson/cli@031c350, standard @f95c42f, both packed from clean detached worktrees. Thespec/rules/clitarball digests reproduce those in.agents/RECEIPT.mdbit for bit — that receipt expected them not to (mtime in the gzip stream), so the match shows this is the same environment, not a same-version rebuild.Calibrated before any target was mined. Run against standard @
8e08c8c, the harness reproduces the committed artifact's history-block digest7012352617dfexactly, 50 entries, 20,417 bytes,availableTransitions90. The digest definition was recovered and verified, not invented — four alternatives were rejected for not reproducing the recorded value.Results
availableTransitionsextractedTransitionswindowTruncatedpairsBeforeCappairsEmittedcapBoundAll deterministic across two runs; bundled and standalone
mining-coreagree pair-for-pair every time. Every artifact is the top 50 of a larger population, never "all qualifying pairs".The finding
META-289 §1.5 has not landed, so the file-role exclusion set is empty — lockfiles, generated files and documentation are all in play, and release plumbing dominates the ranking. Recorded as empty rather than as "a recorded set", in advance, so it could not later be presented as a discovery.
Classifying all 150 emitted pairs by whether both endpoints are source files:
FormatJS produced no source-coupling evidence at all. It is demoted on this output, not on taste. Nothing in the prior screening story survives, and it is not rescued.
syncpack carries the no-import-edge case. The six sibling command modules
src/commands/{fix,format,json,lint,list,update}.rscontribute 11 emitted pairs, and none of the six references any other — no import graph relates them.lint.rs↔list.rsis support 10, occurrences 10: of the 10 qualifying commits touching either, all 10 touched both. Recorded against overclaiming,instance.rs↔version_group.rs(support 17) fails the test —version_group.rsimportsinstance.Two gates pushed back, both obeyed
dependency-direction. Standard sits at the top of the graph. The driver moved to evidence: frozen driver for the canonical external co-change run cli#21 and is cited here by path and digest.weightingVersioninto every scoring basis, and that string names the issue specifying the weighting. Editing it would ship an artifact misreporting which algorithm produced it.docs/evidence/is exempted for the same reasonPROVENANCE_FILESis.Watched-red on the widened exemption: an identifier injected into
README.mdstill fails the gate and passes again on restore, so the exemption is scoped todocs/evidence/and did not blunt the rule.Limits
Relationships are stated descriptively — counts of qualifying commits in a 500-transition window. No usefulness, adoption or predictive claim is inferred. No publication is authorized. No maintainer has been contacted. Published
spec@0.4.4/rules@0.4.4still reject these artifacts; this is candidate interoperability.Gates: spec 264/264, rules 173/173, examples 11/11 + 12/12, architecture, schema, corpus, docs, adr, typecheck — all green.
check:conformanceneeds a locally built candidate and fails identically onmain; CI supplies it.Summary by Sourcery
Record third-party commit-history evidence artifacts and exempt evidence docs from internal tracker identifier checks.
Enhancements:
Documentation:
Tests: