feat(wallet-toolbox): plan noSend batches locally and commit atomically

[ty-everett]

Summary

Add an automatically negotiated Wallet Toolbox fast path for high-volume noSend transaction sequences:

1. reserve a bounded funding pool on the first action;
2. plan, sign, validate, and track dependent actions in memory;
3. atomically persist the complete workspace on the final sendWith;
4. broadcast only after the storage transaction commits.

The BRC-100 wallet interface, CreateActionArgs, SignActionArgs, noSend, noSendChange, sendWith, and result shapes are unchanged. Providers that do not advertise actionBatch: 1 retain the legacy path before a workspace begins.

Why

The existing behavior is correct but latency-bound for long transaction sequences. Every signed noSend action performs a storage createAction call followed by processAction; the final sendWith adds another call. A 250-action sequence therefore serializes 501 storage control calls.

That cost is modest against a local database but dominates against remote storage. At 100 ms RTT, the storage-control component alone is approximately 50.1 seconds before transaction planning, signing, validation, payload transfer, persistence, or broadcast.

This change moves the middle of the workflow to the wallet process, where the keys and staged transactions already live, while preserving storage authority and an atomic remote durability boundary.

Stack status

This draft intentionally targets codex/transaction-pipeline-performance, the head of #285, so its diff contains only the batching work. It reuses the transaction/BEEF performance foundations in #285 and complements the scalable storage work in #283.

After #285 merges, this branch should be rebased or retargeted to main.

Design

Capability negotiation and compatibility

• Add optional storage capability actionBatch: 1.
• Add WalletArgs.actionBatchMode?: 'auto' | 'legacy', defaulting to auto.
• Negotiate once per active provider.
• Fall back to the existing flow before creating any workspace when the provider lacks support.
• Never mix legacy and batch persistence within one workspace.

In-memory planning

• Extract shared action-planning behavior so legacy and batch paths use the same fee, output, change, randomization, and validation rules.
• Maintain one wallet-local workspace containing funding reservations, staged transactions, staged outputs, signing references, consumed inputs, normalized metadata, and a shared BEEF dependency graph.
• Overlay staged state onto relevant list, balance, and abort operations.
• Preserve returnTXIDOnly, knownTxids, commissions, custom inputs, two-step signing, deterministic output order, and noSendChange.
• Compact completed actions into structural records and content-addressed scripts rather than retaining repeated transactions, source transactions, locking scripts, or ancestry.

Reservations and lifecycle

• Add durable action-batch, per-output reservation, and staged-blob records for Knex and IndexedDB providers.
• Reserve only the first action's canonical inputs plus limited headroom.
• Extend adaptively from an EWMA consumption estimate with bounded geometric runway.
• Use 15-minute renewable leases and a 60-minute hard lifetime.
• Recheck reservation ownership and output spendability under row locks.
• Release unused state on commit, abort, cleanup, wallet destruction, or expiry.
• Keep unreserved outputs available to unrelated wallet activity.

Atomic commit and validation

Before persistence, validate:

• manifest and blob digests;
• dependency order and BEEF ancestry;
• transaction structure, TXIDs, input/output mappings, and duplicate spends;
• source amounts and scripts from proven transactions;
• signatures, fees, commissions, and requested output metadata;
• reservations, leases, noSendChange, and storage-controlled outputs.

Persist transactions, outputs, labels, tags, mappings, and proof requests in one storage transaction. Consume reserved inputs and release unused reservations in that same transaction.

Broadcast remains outside the database transaction. Persistence is idempotent by user, batch ID, and semantic manifest digest; retries reconcile the committed result without duplicating wallet state.

Large payload transport

• Inline content-addressed blobs up to 4 MiB.
• For larger manifests, prepare the commit, obtain missing digests, upload authenticated binary blobs, and commit by manifest digest.
• Enforce an 8 MiB per-blob limit and four concurrent uploads by default.
• Accept only digests authorized by the prepared manifest.
• Validate size before hashing and reject digest mismatches, duplicate conflicting uploads, and unprepared blobs.
• Clean incomplete staged blobs with batch expiry.

Main changes

• New wallet-local planner and workspace orchestration.
• New optional storage interfaces for begin, extend, renew, prepare, upload, commit, and abort.
• Knex and IndexedDB schema/migration support.
• Row-locking and reservation-aware allocation for SQL providers.
• User-scoped IndexedDB batch uniqueness.
• Authenticated StorageClient/StorageServer binary upload routes.
• Atomic manifest validation and persistence modules.
• One-minute opportunistic cleanup monitor.
• Staged list/balance/abort coherence.
• Deterministic legacy/batch parity tests.
• Retained benchmark and rollout documentation.
• Benchmark sources are now included in the package lint gate.

Benchmarks

Command:

pnpm --filter @bsv/wallet-toolbox bench:action-batch

Environment: local SQLite provider, Node v25.9.0, pnpm 10.33.2. The measured representative workload is a 250-action dependent chain with 1 KiB scripts and deterministic randomness.

Metric	Legacy	Batch	Change
Storage RPCs	501	2	-99.6%
Database transactions	252	3	-98.8%
Action staging	10,305.7 ms	3,741.2 ms	-63.7%
Planning	6,159.4 ms	118.1 ms	-98.1%
Signing and validation	4,146.2 ms	3,623.0 ms	-12.6%
Final commit	24.7 ms	3,540.0 ms	one atomic batch commit
Stage plus commit	10,330.4 ms	7,281.2 ms	-29.5%
Uploaded/request bytes	1,200,559	791,249	-34.1%
CPU user time	9,665.9 ms	7,011.8 ms	-27.5%
CPU system time	936.9 ms	467.2 ms	-50.1%
Incremental peak heap	372,938,672 B	336,458,528 B	-9.8%

Peak heap is GC-sensitive; the structural assertions are the stronger memory regression gate. They verify that staged records do not retain duplicate source transactions or locking scripts.

The modeled storage-control component for the same 250-action sequence is:

Simulated RTT	Legacy	Batch
25 ms	12,525 ms	50 ms
100 ms	50,100 ms	200 ms
250 ms	125,250 ms	500 ms

The benchmark retains 192 modeled cases across dependent, independent, mixed explicit-input, and two-step workloads; 1, 10, 50, and 250 actions; 1 KiB, 64 KiB, 1 MiB, and 4 MiB scripts; and 25/100/250 ms RTT.

It also physically executes 64 KiB, 1 MiB, and 4 MiB scripts. The 4 MiB one-action batch exercises the prepare/upload/commit path successfully. It is intentionally not presented as a local one-action speedup: on the measured machine, batch stage plus commit was about 2.53 seconds versus 1.47 seconds for legacy because validation and blob staging outweigh avoided local latency. The target benefit is long and/or remote sequences.

Validation

Completed locally:

• pnpm install --frozen-lockfile
• Wallet Toolbox build and lint
• Wallet Toolbox full suite: 127 suites passed; 1,185 tests passed; 27 skipped
• Focused wallet/storage/IndexedDB/authenticated-remote batch suites: 49 passed; 1 skipped
• Retained action-batch benchmark, including physical 4 MiB upload path
• Workspace recursive build, including browser, client, mobile, SDK UMD, and VeriFast packages
• SDK suite: 5,618 passed; 3 skipped
• VeriFast suite: 10 passed; 1 skipped
• Documentation validation: 89 files
• Script conformance validation: 6,650 vectors
• Coverage: 68.46% statements, 57.53% branches, 71.14% functions, 70.52% lines overall; new action-batch modules are approximately 85-95% line-covered

Not yet completed:

• Live MySQL provider E2E. The local Docker daemon was unavailable; Knex/MySQL code compiles and shared storage behavior is exercised through SQLite.
• The full hosted CI workflow does not run while this stacked draft targets codex/transaction-pipeline-performance; the workflow is limited to pull requests targeting main. Socket security checks do run. Full hosted CI will start after the PR is retargeted to main; the equivalent local build and test matrix is listed above.

Reviewer notes

Suggested review order:

1. Compatibility boundary — confirm capability negotiation and legacy fallback in Wallet.ts, WalletStorageManager.ts, and the storage interfaces leave public BRC-100 behavior unchanged.
2. Planner parity — compare shared planning, deterministic transaction parity, returnTXIDOnly, knownTxids, two-step signing, and noSendChange behavior.
3. Reservation concurrency — inspect SQL row locking, per-user uniqueness, lease expiry/reacquisition, cleanup rechecks, and exclusion from legacy allocation.
4. Commit validation — focus on proven source values, signature verification, fees/commissions, requested output metadata, graph ordering, and duplicate-spend rejection.
5. Atomicity and idempotency — verify transaction/output/metadata/proof-request persistence, reservation release, response-loss retries, and broadcaster-failure recovery.
6. Blob authorization — inspect prepared-digest allowlisting, payload caps, hashing order, content addressing, duplicate chunks, and expiry cleanup.
7. Retained memory — verify compact plans and staged outputs hydrate from the shared BEEF/script store instead of retaining full duplicate objects.
8. Provider parity — pay particular attention to MySQL locking semantics and IndexedDB's composite [userId, batchId] key.
9. Benchmark interpretation — distinguish measured local results from modeled network RTT and avoid treating the single-action large-payload path as a throughput win.

Rollout considerations

A server-first rollout is safe:

1. deploy schema and provider capability support;
2. deploy clients with the default auto mode;
3. monitor reservation conflicts, extensions, expiries, commit retries, blob deduplication, commit time, and broadcaster outcomes;
4. retain actionBatchMode: 'legacy' for controlled comparison or rollback.

Intermediate workspaces are intentionally session-scoped. The final commit is the remote durability boundary.

[sonarqubecloud]

Quality Gate passed

Issues
19 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[tonesnotes]

So much good stuff here :-)
This greatly enhances the value of remote storage (which will always have resilience, survivability, and security advantages over local).