feat(sdk): surface verifierRejections across reports, CLI output, and findings-file export - #474
Open
CalebKAston wants to merge 13 commits into
Open
feat(sdk): surface verifierRejections across reports, CLI output, and findings-file export#474CalebKAston wants to merge 13 commits into
CalebKAston wants to merge 13 commits into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A finding the verification pass rejects simply disappears today — no count, no reason, visible nowhere in Warden's output. This adds a
verifierRejections: {count, reasons[]}field, computed inverifyFindings()and threaded throughpostProcessFindings()ontoSkillReport.It's surfaced everywhere
failedHunks/failedExtractionsalready are: the terminal report, theReporter's aggregate summary (log + TTY modes), the JSONL run log (per-chunk and run-summary), and the GitHub Action's findings-file export. The run-level aggregate (JSONL summary,Reporter, terminal) intentionally surfaces only acount, not the reasons — matching the existingfailedHunks/failedExtractionsprecedent. Reasons stay available at the per-skill chunk-record level.Real-world verification
This has been running in production for a period feeding an internal data pipeline into an observability platform for tracking skill health. Querying that platform:
Test plan
pnpm lint && pnpm build && pnpm test— all green (93 test files, 1839 passing, 4 pre-existing skips)verify.test.ts— rejection counting, default-reason fallback, multi-finding aggregationoutput.test.ts— findings-file export present/absentterminal.test.ts,reporter.test.ts— per-skill and aggregate warning linesjsonl.test.ts,main.test.ts— schema round-trip through the JSONL chunk stream and run summarytasks.test.ts— end-to-end through the CLI task runner