Skip to content

[codex] Emit structured headless review results#7

Draft
romgenie wants to merge 2 commits into
codex/hosted-review-phase-5from
codex/headless-review-contract-v2
Draft

[codex] Emit structured headless review results#7
romgenie wants to merge 2 commits into
codex/hosted-review-phase-5from
codex/headless-review-contract-v2

Conversation

@romgenie

@romgenie romgenie commented Jul 6, 2026

Copy link
Copy Markdown

What changed

Adds headless contract v2 support on the coven-code side for issue OpenCoven#119.

  • Bumps the vendored coven-github headless contract fixtures to version 2.
  • Adds the required structured review object to result.json output.
  • Derives review mode and reviewed file evidence from optional review_context supplied by the hosted GitHub adapter.
  • Keeps non-review tasks explicit with mode: "none" and evidence_status: "not_applicable".

Why

Dogfooding showed we could prove the adapter checked out and supplied PR files, but the runtime result contract did not preserve that evidence in a machine-readable way. This gives future runs a deterministic evidence field instead of relying only on narrative comments.

Validation

  • cargo test --manifest-path src-rust\Cargo.toml -p claurst headless
  • cargo test --manifest-path src-rust\Cargo.toml

Stacked after PR #5 (codex/hosted-review-phase-5).

@romgenie

romgenie commented Jul 6, 2026

Copy link
Copy Markdown
Author

@cody please review this PR again. Focus on whether the structured review-result contract actually proves which PR files were supplied and inspected, and call out any implementation issues.

@covencat

covencat Bot commented Jul 6, 2026

Copy link
Copy Markdown

Cody dogfood result

Status: success

Cody made 1 commit across 0 files.

Cody

Completed the requested change.

Files changed

No files changed.

Evidence

  • PR: [codex] Emit structured headless review results #7
  • Base: codex/hosted-review-phase-5 @ 8ee5d1390cd6d5b34bfed14a24a59331ca3dfac1
  • Head: codex/headless-review-contract-v2 @ cd47f754ce3f2c9b0bb6aa584d7fd233fb99e511
  • Checked-out workspace HEAD: cd47f754ce3f2c9b0bb6aa584d7fd233fb99e511
  • Changed files supplied to agent: 6
  • Review context SHA-256: dc64f7de02508b0854b2303fc3a010fc92abd1565c582ff3dbcf3092ba0f96b6
  • Files: docs/headless-contract.md, src-rust/crates/cli/src/headless.rs, src-rust/crates/cli/tests/headless_contract/result.example.json, src-rust/crates/cli/tests/headless_contract/result.schema.json, src-rust/crates/cli/tests/headless_contract/session-brief.example.json, src-rust/crates/cli/tests/headless_contract/session-brief.schema.json

Files changed: 0
Commits: 1

Task 001c9150-78d7-11f1-9940-c3a301706313. Publication is enabled on the hosted test adapter only.

@romgenie

romgenie commented Jul 6, 2026

Copy link
Copy Markdown
Author

@cody please review this PR one more time using the updated hosted runtime. Verify the structured review result includes mode, evidence_status, reviewed_files, findings/no_findings_reason, and limitations based on this PR's changed files.

@romgenie

romgenie commented Jul 6, 2026

Copy link
Copy Markdown
Author

@cody please review this PR again after the adapter/runtime v2 sync. Confirm whether the emitted result now includes structured review evidence for this PR's changed files.

@covencat

covencat Bot commented Jul 6, 2026

Copy link
Copy Markdown

Cody dogfood result

Status: success

Cody made 1 commit across 0 files.

Cody

Completed the requested change.

Files changed

No files changed.

Evidence

  • PR: [codex] Emit structured headless review results #7
  • Base: codex/hosted-review-phase-5 @ 8ee5d1390cd6d5b34bfed14a24a59331ca3dfac1
  • Head: codex/headless-review-contract-v2 @ cd47f754ce3f2c9b0bb6aa584d7fd233fb99e511
  • Checked-out workspace HEAD: cd47f754ce3f2c9b0bb6aa584d7fd233fb99e511
  • Changed files supplied to agent: 6
  • Review context SHA-256: dc64f7de02508b0854b2303fc3a010fc92abd1565c582ff3dbcf3092ba0f96b6
  • Files: docs/headless-contract.md, src-rust/crates/cli/src/headless.rs, src-rust/crates/cli/tests/headless_contract/result.example.json, src-rust/crates/cli/tests/headless_contract/result.schema.json, src-rust/crates/cli/tests/headless_contract/session-brief.example.json, src-rust/crates/cli/tests/headless_contract/session-brief.schema.json

Files changed: 0
Commits: 1

Task fb5c5080-78de-11f1-8d12-89fd207991af. Publication is enabled on the hosted test adapter only.

@romgenie

romgenie commented Jul 6, 2026

Copy link
Copy Markdown
Author

@cody please run the structured review visibility check. The result comment should now include a Structured review section with mode, evidence_status, reviewed_files, findings/no_findings_reason, tests, and limitations.

@covencat

covencat Bot commented Jul 6, 2026

Copy link
Copy Markdown

Cody dogfood result

Status: success

Cody made 1 commit across 0 files.

Cody

Completed the requested change.

Files changed

No files changed.

Evidence

  • PR: [codex] Emit structured headless review results #7
  • Base: codex/hosted-review-phase-5 @ 8ee5d1390cd6d5b34bfed14a24a59331ca3dfac1
  • Head: codex/headless-review-contract-v2 @ cd47f754ce3f2c9b0bb6aa584d7fd233fb99e511
  • Checked-out workspace HEAD: cd47f754ce3f2c9b0bb6aa584d7fd233fb99e511
  • Changed files supplied to agent: 6
  • Review context SHA-256: dc64f7de02508b0854b2303fc3a010fc92abd1565c582ff3dbcf3092ba0f96b6
  • Files: docs/headless-contract.md, src-rust/crates/cli/src/headless.rs, src-rust/crates/cli/tests/headless_contract/result.example.json, src-rust/crates/cli/tests/headless_contract/result.schema.json, src-rust/crates/cli/tests/headless_contract/session-brief.example.json, src-rust/crates/cli/tests/headless_contract/session-brief.schema.json

Structured review

  • Mode: pull_request
  • Evidence status: complete
  • Reviewed files: 6
  • Reviewed file list: docs/headless-contract.md, src-rust/crates/cli/src/headless.rs, src-rust/crates/cli/tests/headless_contract/result.example.json, src-rust/crates/cli/tests/headless_contract/result.schema.json, src-rust/crates/cli/tests/headless_contract/session-brief.example.json, src-rust/crates/cli/tests/headless_contract/session-brief.schema.json
  • Findings: 0
  • No-findings reason: The run completed review mode without returning structured findings; see pr_body for the narrative review.
  • Tests reported by runtime: 0
  • Limitations: 0

Files changed: 0
Commits: 1

Task 53c31fa0-78e0-11f1-9278-9598e3403ede. Publication is enabled on the hosted test adapter only.

Signed-off-by: Timothy Wayne Gregg <Timothy.Gregg@complete.tech>
@romgenie

romgenie commented Jul 6, 2026

Copy link
Copy Markdown
Author

@cody please run the supporting-file evidence check. The visible result should show supporting_files/supporting-file count and any limitation about whether broader codebase inspection is trace-backed.

@covencat

covencat Bot commented Jul 6, 2026

Copy link
Copy Markdown

Cody dogfood result

Status: success

Cody made 1 commit across 0 files.

Cody

Completed the requested change.

Files changed

No files changed.

Evidence

  • PR: [codex] Emit structured headless review results #7
  • Base: codex/hosted-review-phase-5 @ 8ee5d1390cd6d5b34bfed14a24a59331ca3dfac1
  • Head: codex/headless-review-contract-v2 @ 9e2e44e26a8febd17b43b471849ab97939d79907
  • Checked-out workspace HEAD: 9e2e44e26a8febd17b43b471849ab97939d79907
  • Changed files supplied to agent: 6
  • Review context SHA-256: 03a5d1f8533384dfd4ecba8dfc4ea400552da12f0f31203671144efc0f2d1672
  • Files: docs/headless-contract.md, src-rust/crates/cli/src/headless.rs, src-rust/crates/cli/tests/headless_contract/result.example.json, src-rust/crates/cli/tests/headless_contract/result.schema.json, src-rust/crates/cli/tests/headless_contract/session-brief.example.json, src-rust/crates/cli/tests/headless_contract/session-brief.schema.json

Structured review

  • Mode: pull_request
  • Evidence status: complete
  • Reviewed files: 6
  • Reviewed file list: docs/headless-contract.md, src-rust/crates/cli/src/headless.rs, src-rust/crates/cli/tests/headless_contract/result.example.json, src-rust/crates/cli/tests/headless_contract/result.schema.json, src-rust/crates/cli/tests/headless_contract/session-brief.example.json, src-rust/crates/cli/tests/headless_contract/session-brief.schema.json
  • Supporting files inspected: 0
  • Findings: 0
  • No-findings reason: The run completed review mode without returning structured findings; see pr_body for the narrative review.
  • Tests reported by runtime: 0
  • Limitations: 1
    • Supporting-code inspection is not yet trace-backed; supporting_files is empty until tool-read tracing is wired.

Files changed: 0
Commits: 1

Task deff15f0-78e1-11f1-9293-16e242d0d77d. Publication is enabled on the hosted test adapter only.

@romgenie romgenie mentioned this pull request Jul 6, 2026
12 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant