Skip to content

vectorgen tooling for adding/updating/regenerating test vectors - #268

Open
cpu wants to merge 3 commits into
C2SP:mainfrom
cpu:cpu-vectorgen
Open

vectorgen tooling for adding/updating/regenerating test vectors#268
cpu wants to merge 3 commits into
C2SP:mainfrom
cpu:cpu-vectorgen

Conversation

@cpu

@cpu cpu commented Aug 11, 2026

Copy link
Copy Markdown
Member

This PR adds vectorgen: a top-level Go library (vectorgen/) and a command line tool using that library (tools/vectorgen) for maintaining the test vector corpus without needing to maintain or re-run centralized generator code.

Common contribution workfows are driven by a small JSON envelope describing only the change being made (new tests, a group template, patches), targeted based on source metadata, and applied by vectorgen subcommand:

  • add (a whole new file, a new group, or new tests in an existing group),
  • update (add or rewrite fields across existing vectors),
  • replace (swap a regenerated group in place), and
  • scaffold (emit a starter envelope from a schema).

The tool owns the mechanical parts that previously meant hand-editing JSON: tcId assignment, updating numberOfTests, maintaining key order, canonical formatting, and schema validation, etc. Critically, untouched portions of vector files avoid spurious diffs and churn. The tooling achieves this by building on encoding/json/v2's ordered values (behind GOEXPERIMENT=jsonv2 until Go 1.27).

The vectorgen fmt subcommand reproduces the retired tools/reformat_json.py Python script byte-for-byte on the existing tree (the one aes_ff1_radix* carve-out remains), so no one-time reformat is needed and we can drop the one-off Python tool. The old tools/vectorlint tool is folded into the new tooling as a vectorgen lint subcommand, and CI now runs both formatting and linting checks using the Go tool.

The doc/vectorgen.md docs file documents the workflows, and consolidates the schema/vector best practices and source-naming guidance formerly in CONTRIBUTING.md. Each workflow also has a worked example under vectorgen/testdata/ where I tried to validate the tooling using previous community updates. The add and update fixtures literally replay #254 and #255 from their envelopes, asserted byte-identical to the committed history. Lastly, for the folks like mysef that have been writing their test case generation code in Go, everything can be done programmatically with the vectrogen library instead of or in addition to using the CLI tool.

This is a big diff, and I used Claude for much of the code (though heavily guided/reviewed by myself). Having talked with Filippo about this in the past I plan to merge this as-is without blocking on someone having time to review all the helper code, but I will of course iterate based on feedback from contributors/maintainers, and fix bugs as they arise. If folks do want to provide input I think looking at the documentation updates & the worked examples and providing feedback on missed/clunky usecases would be helpful.

Closes #240
Closes #162

This was the one file in repo with windows line endings.
Comment thread tools/vectorgen/main.go Outdated
cpu added 2 commits August 11, 2026 16:56
This commit:

* Adds a top-level go package (vectorgen) that contains common code for
  formatting, linting, and modifying test vectors
* Implements a tools/vectorgen CLI tool using that library, offering
  sub commands for formatting, linting, adding new vectors, updating
  existing vectors, etc.
* Reworks the existing CI coverage & formatting to use the above,
  replacing the Python JSON formatter.
* Updates documentation to describe how to contribute vectors using
  vectorgen, including worked examples based on real world Wycheproof
  community updates reimagined as vectorgen workflows.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Tooling to merge and update test vectors Use a more common JSON formatting

2 participants