Seed the checkpoint scope-key hasher for deterministic restore#526
Open
PratikDhanave wants to merge 1 commit into
Open
Seed the checkpoint scope-key hasher for deterministic restore#526PratikDhanave wants to merge 1 commit into
PratikDhanave wants to merge 1 commit into
Conversation
Checkpoint.UnmarshalJSON rebuilds StateData with scopeKeyHasherImpl, whose Hash used a zero-value maphash.Hash. A zero-value maphash.Hash selects a new random seed on first use, so the same ScopeKey hashes differently on every call. On a map restored from JSON this breaks Load/Delete (the value stored during unmarshal can no longer be found) and stops shared-scope keys from collapsing. state.go's sibling ScopeKey hasher already seeds with a fixed process-wide seed; do the same here. Adds a black-box test that round-trips a Checkpoint through JSON and asserts its StateData is still Loadable — false before this change, true after.
There was a problem hiding this comment.
Pull request overview
This PR fixes a determinism bug in checkpoint JSON restore by ensuring the ScopeKey hasher used to rebuild Checkpoint.StateData is seeded consistently, so restored state remains usable via Load/Delete and key-collapsing behavior works as intended.
Changes:
- Seed the checkpoint-local
ScopeKeyhasher with a process-widemaphash.Seedduring hashing. - Add a regression test that round-trips a
CheckpointJSON payload and asserts restoredStateData.Load(key)succeeds.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| workflow/internal/checkpoint/checkpoint_json.go | Seed the ScopeKey hasher used during JSON unmarshal so hashing is consistent across calls within the process. |
| workflow/internal/checkpoint/json_test.go | Add a test that validates restored StateData remains Load-able after JSON unmarshal. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Checkpoint.UnmarshalJSON(workflow/internal/checkpoint/checkpoint_json.go) rebuildsStateDatawithscopeKeyHasherImpl, whoseHashused a zero-valuemaphash.Hash:A zero-value
maphash.Hashselects a fresh random seed on first use, so the sameScopeKeyhashes to a different value on every call. On thehashmap.Mapproduced by unmarshal this means:Load/Delete/overwrite break — the value stored during unmarshal is hashed with one seed, and a laterLoadre-hashes with a different seed and misses.Equal(shared scope ignores the executor ID) get different hashes and are kept as separate entries, soStateManager.ImportStatere-imports both and which value survives a restore is non-deterministic.The sibling hasher in
workflow/internal/execution/state.goalready seeds with a fixed process-widetheSeed; the checkpoint copy just forgot to.Fix
Add a fixed process-wide
scopeKeyHashSeed = maphash.MakeSeed()andh.SetSeed(scopeKeyHashSeed)inHash, mirroringstate.go.Public API
No exported symbols change.
Tests
Adds
TestCheckpoint_JsonRoundtrip_StateDataRemainsLoadable(black-box, injson_test.go): round-trips aCheckpointthrough JSON and asserts itsStateDatais stillLoad-able. It fails before this change (Loadreturns false) and passes after. (Existing tests missed it because they only consume restoredStateDatavia.All()iteration, which is seed-independent.)