fix(probe): run the diagnose egress leak self-check; ship payload schema, not values (#199)

[pbean]

Closes #199.

probe-adapter's report is documented as sanitized and shareable (the adapter-authoring guide's whole point is pasting it into an issue/PR), but it had neither of the two egress protections diagnose relies on for the same expectation: no assert_no_leak backstop over the rendered bytes, and no Pseudonymizer — identifier-shaped values passed the collection scrubbers verbatim by design. probe.py's own docstring tracked the gap and cited #199.

What changed

One audited guard. The guard loop (_guard / _repair_candidates / _assert_clean / LeakDetected) moved from diagnostics.py into sanitize.py as public guard / assert_clean / LeakDetected. Diagnostics delegates and re-exports LeakDetected (noqa-pinned; an identity test catches the ruff-F401-autofix regression). The diagnostics canary/zero-repair/encoding suites pass unchanged — the proof the move preserved semantics byte-for-byte.

Payloads ship as shape, never values. EventCapture.payload (a whole scrubbed hook-payload dict — the widest identifier-shaped surface) is replaced by payload_schema: dotted key paths + leaf types via the existing _walk_paths (tool_input.command:str), which is the part profile authoring actually needs. payload_keys stays but is identifier-gated. --json schema_version bumps 1 → 2 (captured_events[].payload removed — a field removal, per the machine.py additive-only contract).

Guard on the exact emitted bytes. Both probe renderers run sanitize.guard before returning; render_json now dumps ensure_ascii=False (same safety requirement as diagnose — the guard must see the string as itself, and machine.emit/write_document emit it verbatim). cmd_probe registers the project directory name as a project-ns alias (skipped if it collides with the CLI/binary name), routes it into location redaction at collection time (zero repairs on the canonical run), and on a hit fails closed per the #195 contract: message → stderr, empty stdout, exit ≠ 0, no partial --out file. A stray alias original is repaired and disclosed (backstop_repairs + stderr warning), exactly like diagnose.

Collection hardening the guard surfaced immediately. Running the new guard against the existing test suite caught a live leak: transcript-location components that embed the current username shipped verbatim (pytest-of-<user> in every CI run of the PII tests; any user-named $TMPDIR in production). Path components embedding the username are now redacted; also hardened: home-rooted --binary hints render ~-relative in the summary and the not-found warning, credential-shaped dict keys (ghp_… as a map key is identifier-shaped) collapse to <key> in key paths, and --keep-temp prints a ~-relative tmpdir.

Verification

• Full suite: 2590 passed, 1 skipped; trunk check --no-fix --all clean.
• Prove-regression: with src/ stashed to the pre-fix tree, the 13 new/extended egress tests all fail.
• New coverage: fail-closed renderers, CLI refusal contract (both modes, --out absence), _collect_captures schema-not-values, end-to-end egress through the real capture-hook subprocess seam, location aliasing zero-repair invariant, backstop repair of a stray project name in key paths, non-ASCII guard fidelity, guard mechanics ported to test_sanitize.py.
• Live smoke on this repo, both modes: rc 0, schema_version: 2, zero occurrences of the username or project name, zero repairs; --json parses whole.

Out of scope

#202 (diagnose _scrub_policy dict-key gap — sibling class, untouched), #212 (documents.py rename), machine.py behavior, the capture hook script (raw retention inside the transient tmpdir is correct by design), _walk_paths depth-guarding (pre-existing).

Summary by CodeRabbit

• Breaking Changes

  • Probe reports now use schema version 2.
  • Captured hook data now includes payload field/key-path schemas (field names + leaf types), not payload values.
  • Existing captured_events[].payload entries are replaced by captured_events[].payload_schema.

• Security Enhancements

  • Rendered reports are revalidated before emission; sensitive detections fail closed.
  • Added backstop repair behavior for stray pseudonymized originals; tightened transcript/path redaction and credential-shaped key handling.
  • On unsafe output, no report is produced and standard output is suppressed (including JSON mode).

• Documentation

  • Updated adapter-authoring guidance to reflect the new schema-centric privacy model and output validation behavior.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 03502ca1-34c6-4d2c-9049-c2648208f648

📥 Commits

Reviewing files that changed from the base of the PR and between ed9f52f and 1dcc211.

📒 Files selected for processing (2)

• src/bmad_loop/probe.py
• tests/test_probe.py

🚧 Files skipped from review as they are similar to previous changes (1)

• src/bmad_loop/probe.py

---

Walkthrough

Probe reports now emit captured payload schemas instead of values, apply alias-aware redaction, and pass final Markdown or JSON through a shared fail-closed egress guard. CLI output handling, diagnostics integration, documentation, changelog entries, and security tests are updated accordingly.

Changes

Probe egress safety and schema hardening

Layer / File(s)	Summary
Shared sanitizer guard and repair primitives src/bmad_loop/sanitize.py, tests/test_sanitize.py	Adds shared leak detection, bounded pseudonymization repair, clean-output validation, username detection, and related tests.
Schema-only capture and redacted discovery src/bmad_loop/probe.py, tests/test_probe.py	Changes captured events to schema version 2 with payload key paths and leaf types, while tightening transcript, binary, dynamic-key, and username-derived redaction.
Guarded report rendering and shared diagnostics src/bmad_loop/probe.py, src/bmad_loop/diagnostics.py, src/bmad_loop/machine.py, tests/test_diagnostics.py	Validates serialized Markdown and JSON through the shared guard, records repairs, revalidates final bytes, and centralizes LeakDetected.
CLI fail-closed wiring and contract coverage src/bmad_loop/cli.py, tests/test_probe.py	Threads pseudonymization through probing and rendering, refuses leaked output, reports repairs, and validates refusal and schema behavior.
Documentation and release contract docs/adapter-authoring-guide.md, CHANGELOG.md	Documents schema-only capture, guarded emission, redaction rules, and breaking output changes.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant CLI
  participant probe
  participant sanitize.guard
  participant Output
  CLI->>probe: run scan or probe with pseudonymizer
  probe-->>CLI: schema-based finding
  CLI->>sanitize.guard: validate rendered report
  sanitize.guard-->>CLI: clean output or LeakDetected
  CLI->>Output: emit validated bytes

Loading

Possibly related PRs

• bmad-code-org/bmad-loop#136: Both modify probe() fallback-to-scan() wiring.
• bmad-code-org/bmad-loop#187: Both implement rendered-output repair and fail-closed guarding.
• bmad-code-org/bmad-loop#201: Both update probe and diagnose JSON output and leak-check paths.

Poem

I’m a rabbit guarding each report,
No secret crumbs may pass the port.
Keys become shapes, paths disguise,
Clean bytes safely hop the skies.
If leaks appear, I thump and refuse.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 62.50% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title matches the main changes: adding the egress leak self-check and switching probe output to payload schema instead of values.
Linked Issues check	✅ Passed	The PR adds a final-output leak guard, uses schema-only payload capture, and covers identifier-shaped leaks and probe egress as requested.
Out of Scope Changes check	✅ Passed	The changes stay focused on probe egress safety, schema output, redaction hardening, docs, and tests; no unrelated feature work stands out.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/probe-egress-guard-199

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}