vectorgen tooling for adding/updating/regenerating test vectors - #268
Open
cpu wants to merge 3 commits into
Open
vectorgen tooling for adding/updating/regenerating test vectors#268cpu wants to merge 3 commits into
cpu wants to merge 3 commits into
Conversation
This was the one file in repo with windows line endings.
davidben
reviewed
Aug 11, 2026
This commit: * Adds a top-level go package (vectorgen) that contains common code for formatting, linting, and modifying test vectors * Implements a tools/vectorgen CLI tool using that library, offering sub commands for formatting, linting, adding new vectors, updating existing vectors, etc. * Reworks the existing CI coverage & formatting to use the above, replacing the Python JSON formatter. * Updates documentation to describe how to contribute vectors using vectorgen, including worked examples based on real world Wycheproof community updates reimagined as vectorgen workflows.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds
vectorgen: a top-level Go library (vectorgen/) and a command line tool using that library (tools/vectorgen) for maintaining the test vector corpus without needing to maintain or re-run centralized generator code.Common contribution workfows are driven by a small JSON envelope describing only the change being made (new tests, a group template, patches), targeted based on source metadata, and applied by
vectorgensubcommand:The tool owns the mechanical parts that previously meant hand-editing JSON:
tcIdassignment, updatingnumberOfTests, maintaining key order, canonical formatting, and schema validation, etc. Critically, untouched portions of vector files avoid spurious diffs and churn. The tooling achieves this by building on encoding/json/v2's ordered values (behind GOEXPERIMENT=jsonv2 until Go 1.27).The
vectorgen fmtsubcommand reproduces the retiredtools/reformat_json.pyPython script byte-for-byte on the existing tree (the oneaes_ff1_radix*carve-out remains), so no one-time reformat is needed and we can drop the one-off Python tool. The oldtools/vectorlinttool is folded into the new tooling as avectorgen lintsubcommand, and CI now runs both formatting and linting checks using the Go tool.The
doc/vectorgen.mddocs file documents the workflows, and consolidates the schema/vector best practices and source-naming guidance formerly in CONTRIBUTING.md. Each workflow also has a worked example undervectorgen/testdata/where I tried to validate the tooling using previous community updates. Theaddandupdatefixtures literally replay #254 and #255 from their envelopes, asserted byte-identical to the committed history. Lastly, for the folks like mysef that have been writing their test case generation code in Go, everything can be done programmatically with thevectrogenlibrary instead of or in addition to using the CLI tool.This is a big diff, and I used Claude for much of the code (though heavily guided/reviewed by myself). Having talked with Filippo about this in the past I plan to merge this as-is without blocking on someone having time to review all the helper code, but I will of course iterate based on feedback from contributors/maintainers, and fix bugs as they arise. If folks do want to provide input I think looking at the documentation updates & the worked examples and providing feedback on missed/clunky usecases would be helpful.
Closes #240
Closes #162