Skip to content

Deterministic full-history credential verification - #881

Merged
unbraind merged 2 commits into
mainfrom
fix/full-history-sentry-token-scan
Aug 3, 2026
Merged

Deterministic full-history credential verification#881
unbraind merged 2 commits into
mainfrom
fix/full-history-sentry-token-scan

Conversation

@unbraind

@unbraind unbraind commented Aug 3, 2026

Copy link
Copy Markdown
Owner

Outcome

Fixes the scheduled full-history security regression discovered after #880: provider timeouts no longer turn ordinary PM hash-chain values into nondeterministic unknown secret failures. TruffleHog still scans all Git history and hard-fails provider-verified live credentials plus source scan errors. Deterministic current-tree credential detection remains mandatory through Trivy and pnpm security:scan.

Project management

Verification

  • TruffleHog 3.96.0 full history: 88,655 chunks / 143,087,116 bytes / zero verified findings
  • node scripts/run-tests.mjs test -- tests/integration/ci-workflow-contract.spec.ts — 12/12
  • node scripts/run-tests.mjs coverage — 424 files, 6,773 tests, exact 100/100/100/100
  • pnpm lint — ESLint and zero-clone jscpd
  • pnpm quality:static — 100% docstrings plus SDK/package/token/context gates
  • pnpm security:scan — no credential-like tracked values
  • pnpm changelog:pm:check — unchanged and current on pm-changelog 2026.8.3

Historical report

Supersedes the false-positive scheduled failure in run 30798451130. The previously failing Windows nightly provenance test was independently fixed and is green on merged main #880.

Summary by Sourcery

Tighten the security CI workflow’s TruffleHog integration to keep full-history credential scans deterministic and aligned with documented exclusions.

Enhancements:

  • Clarify inline documentation around provider-verified credential detection and its interaction with other security scanners.

CI:

  • Update the TruffleHog step to only treat verified credentials and scan errors as failures, while still scanning full Git history on scheduled runs.

Tests:

  • Extend the CI workflow contract test to assert the updated TruffleHog arguments and to verify the exact set of excluded paths used during secret scanning.

Summary by cubic

Make full-history credential scanning deterministic by updating trufflesecurity/trufflehog to fail on verified secrets and scan errors, not on unverifiable "unknown" results. This removes flaky scheduled failures from provider timeouts while keeping hard-fail on real credentials; current-tree detection stays enforced via Trivy and pnpm security:scan.

  • Bug Fixes
    • Update .github/workflows/security.yml to use --results=verified --fail-on-scan-errors --exclude-paths=.trufflehog-exclude-paths.txt and keep fetch-depth: 0 for scheduled full-history scans.
    • Add contract tests in tests/integration/ci-workflow-contract.spec.ts to pin these flags and the reviewed exclusion allowlist.

Written for commit 4f096c0. Summary will update on new commits.

Review in cubic

Keep TruffleHog hard-failing provider-verified live credentials and source scan errors across full Git history while relying on mandatory Trivy and repository pattern scans for deterministic current-tree credential detection. Pin the policy and reviewed exclusion allowlist in workflow contract tests, and record complete pm-4ris regression evidence.
@gemini-code-assist

Copy link
Copy Markdown

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@coderabbitai

coderabbitai Bot commented Aug 3, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@unbraind, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 28 seconds

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Repository UI (base), Organization UI (inherited)

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 8bb51c90-f1f5-4e50-b218-3374a9b199e2

📥 Commits

Reviewing files that changed from the base of the PR and between e727e84 and 3b47bad.

📒 Files selected for processing (5)
  • .agents/pm/extensions/.managed-extensions.json
  • .agents/pm/history/pm-4ris.jsonl
  • .agents/pm/tasks/pm-4ris.toon
  • .github/workflows/security.yml
  • tests/integration/ci-workflow-contract.spec.ts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@sourcery-ai

sourcery-ai Bot commented Aug 3, 2026

Copy link
Copy Markdown

Reviewer's Guide

Adjusts the security workflow’s TruffleHog configuration to treat only provider-verified live credentials as findings while failing on scan errors, and extends the CI workflow contract test to enforce the new configuration and the TruffleHog exclusion list for deterministic full-history credential verification.

Flow diagram for updated CI security credential verification

flowchart TD
  A[GitHubActions security job] --> B[Trivy deterministic current-tree scan]
  B --> C[Repository pattern scans]
  C --> D[TruffleHog scan --results=verified --fail-on-scan-errors]

  D --> E{candidate secret}
  E -->|provider verifies live| F[Mark verified credential and fail job]
  E -->|provider timeout or network failure| G[No finding from provider check]

  D --> H{scan error}
  H -->|yes| F
  H -->|no| I[Job succeeds]
Loading

File-Level Changes

Change Details Files
Refine TruffleHog invocation to only fail on provider-verified credentials while still failing on source scan errors.
  • Update TruffleHog documentation comments in the security workflow to describe deterministic current-tree scanning plus full-history rescan behavior and non-fatal provider timeouts.
  • Change TruffleHog extra_args to only include verified results and to fail closed on scan errors, keeping the exclude paths file configured.
.github/workflows/security.yml
Extend CI workflow contract tests to lock in the new TruffleHog configuration and exclusion paths.
  • Read and normalize the TruffleHog exclude paths file within the workflow contract test.
  • Assert that the security workflow YAML contains the updated TruffleHog extra_args string.
  • Assert that the TruffleHog exclusion list exactly matches the expected benign paths.
tests/integration/ci-workflow-contract.spec.ts
Update pm-4ris project management extension and history artifacts (content not shown in diff).
  • Adjust managed extensions configuration related to pm-4ris.
  • Modify pm-4ris historical and task definition artifacts, likely to reflect current project management state.
.agents/pm/extensions/.managed-extensions.json
.agents/pm/history/pm-4ris.jsonl
.agents/pm/tasks/pm-4ris.toon

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@unbraind

unbraind commented Aug 3, 2026

Copy link
Copy Markdown
Owner Author

@greptileai review

@coderabbitai full review

Please review exact head 3b47badde after the post-creation PM evidence commit. Focus on whether the verified-only full-history gate plus mandatory deterministic current-tree Trivy/repository scans preserves strong secret detection without network-dependent false positives.

@coderabbitai

coderabbitai Bot commented Aug 3, 2026

Copy link
Copy Markdown

@unbraind I will perform a full review of pull request #881 at head 3b47badde.

I will verify that the workflow:

  • scans full Git history;
  • fails only for verified credentials and scan errors;
  • keeps deterministic current-tree secret detection mandatory; and
  • avoids provider-timeout false positives.
✅ Action performed

Full review finished.


Your included review limit is currently reached under our Fair Usage Limits Policy. This review may still proceed through usage-based billing if eligible. Your next included review will be available in 10 minutes.

@codspeed-hq

codspeed-hq Bot commented Aug 3, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 11 untouched benchmarks


Comparing fix/full-history-sentry-token-scan (3b47bad) with main (e727e84)

Open in CodSpeed

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • The CI contract test hard-codes the entire .trufflehog-exclude-paths.txt contents, which may be brittle as exclusions evolve; consider asserting a minimal required subset or sourcing the expected patterns from a shared constant to reduce maintenance overhead.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The CI contract test hard-codes the entire `.trufflehog-exclude-paths.txt` contents, which may be brittle as exclusions evolve; consider asserting a minimal required subset or sourcing the expected patterns from a shared constant to reduce maintenance overhead.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@codecov

codecov Bot commented Aug 3, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ All tests successful. No failed tests found.

📢 Thoughts on this report? Let us know!

@unbraind

unbraind commented Aug 3, 2026

Copy link
Copy Markdown
Owner Author

Bot feedback acknowledgement on exact head 3b47badde:

  • Gemini sunset notice 5164884281: acknowledged and up-voted as accurate service-state information.
  • CodeRabbit limit artifacts 5164885489 and 5164889949: down-voted because the paid-plan quota prevented an actionable review; no finding was produced. The explicit review trigger and exact compared commit range are preserved.
  • Sourcery guide 5164886742: up-voted as an accurate change summary. Sourcery review 4842878869: read and down-voted because exact allowlist equality is intentional here—adding a secret-scan exclusion must require explicit security review and a contract update, rather than silently passing through a subset assertion or self-referential shared constant.
  • CodSpeed 5164906727: up-voted; all 11 benchmarks are unchanged.
  • Codecov 5164925705: up-voted; all modified coverable lines are covered and tests passed.

There are zero inline review threads and no actionable code finding from these artifacts.

@unbraind

unbraind commented Aug 3, 2026

Copy link
Copy Markdown
Owner Author

Greptile review limitation: the exact-head review was explicitly dispatched, but the free reviewer returned free_reviews_limit_reached and produced no actionable finding. This is an external free-quota limitation; hosted DeepScan/CodeFactor analysis, CodeQL, Semgrep, Trivy, TruffleHog, actionlint, complete CI, Sourcery, and local exact gates are green.

@unbraind

unbraind commented Aug 3, 2026

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Aug 3, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@unbraind

unbraind commented Aug 3, 2026

Copy link
Copy Markdown
Owner Author

CodeRabbit final retry acknowledgement: after the included-review window reopened, the explicit review completed on exact head 3b47baddef6ea555b5bc418e09e055677bb1a4b7 without an actionable finding. Completion comment 5164985447 is up-voted; earlier rate-limit artifacts remain down-voted as service limitations.

@unbraind
unbraind merged commit 2f9f706 into main Aug 3, 2026
32 checks passed
@unbraind
unbraind deleted the fix/full-history-sentry-token-scan branch August 3, 2026 10:06
@unbraind

unbraind commented Aug 4, 2026

Copy link
Copy Markdown
Owner Author

Tracked in PM: pm-4ris.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant