Implement a Merkle-proof event integrity system that constructs incremental trees over ingested events, publishes roots on-chain, and serves verifiable inclusion proofs to clients

[Chucks1093]

Summary

The server ingests contract events from Stellar and stores them in the database, but clients have no way to cryptographically verify that the server has not tampered with, omitted, or reordered events. This issue implements a full Merkle-proof integrity layer: the server builds an incremental Merkle tree over all events per (network, contractId) partition, publishes the root hash to a Soroban attestation contract after each partition closes, and exposes an endpoint that returns a Merkle inclusion proof for any event so a client can independently verify the event exists in the committed tree without downloading the full event set.

Scope

1. Event leaf canonicalisation

• Define a canonical byte representation for each event leaf: sha256(ledger || txHash || eventIndex || topic || dataJson) where all fields are length-prefixed big-endian bytes
• Leaf encoding must be deterministic regardless of JSON key ordering — sort all JSON keys recursively before serialisation
• Store the computed leaf hash alongside each event row in a new leafHash column

2. Incremental Merkle tree construction

• Implement a binary Merkle tree using the MerkleTreeState model: store the sparse array of node hashes at each level indexed by position so the tree can be reconstructed or extended without replaying all leaves
• Interior node hash: sha256(leftChild || rightChild); unpaired nodes are promoted as-is (standard Bitcoin-style Merkle tree)
• After each new event is ingested, extend the tree in O(log n) by updating only the nodes on the path from the new leaf to the root — no full rebuild
• The root after N leaves must be reproducible from just the MerkleTreeState rows and the leaf hashes

3. Partition closing and on-chain root publication

• A partition covers a configurable ledger range (default 1000 ledgers) per (network, contractId)
• When a partition closes (ledger range complete), publish the Merkle root to a Soroban attestation contract via publish_root(network, contractId, ledgerStart, ledgerEnd, root: bytes32, eventCount: u32)
• Store the published root and the Soroban transaction hash in a new MerklePartitionRoot table
• Retry publication up to 3 times with exponential backoff; if all retries fail, mark the partition as PUBLICATION_FAILED and alert via structured log

4. Inclusion proof endpoint

• GET /events/:eventId/proof returns a JSON inclusion proof: { leafHash, leafIndex, treeSize, root, siblings: [{ hash, side: 'left'|'right' }][], partitionId, sorobanTxHash }
• The siblings array must be sufficient for a client to recompute the root by hashing up the tree
• Include the on-chain transaction hash so the client can independently verify the root on Stellar
• Return 404 if the event has not yet been included in a closed partition (proof not available for open partitions)

5. Client-side proof verification utility

• Add a verifyInclusionProof(proof, expectedRoot) utility function in src/utils/merkle.ts
• Recompute the root from leafHash and siblings and assert it equals expectedRoot
• This utility is used in integration tests and can be shipped to API consumers

6. Integration tests

• Ingest 8 events, close the partition, publish the root — verify the root matches a reference implementation
• Request a proof for event at index 3 — verify the proof is valid using the utility
• Tamper with an event's dataJson after ingestion — verify the proof for that event fails verification
• Request a proof for an event in an open partition — verify 404 is returned
• Simulate a publication failure — verify the partition is marked PUBLICATION_FAILED and a retry is attempted

Acceptance Criteria

• Leaf hashes deterministic regardless of JSON key order
• Incremental tree extension runs in O(log n) without full rebuild
• Partition root published to Soroban after partition closes with retry logic
• GET /events/:eventId/proof returns a valid inclusion proof for closed partitions
• Inclusion proof verifiable client-side using the utility function
• Tampered event fails proof verification
• Open-partition events return 404 from the proof endpoint
• MerklePartitionRoot table stores root, event count, and Soroban tx hash per partition

ETA: 24 hours

---

Coordinate on Telegram

[TechBroAfrica]

Hi, I would love to work on this issue, kindly assign to me Thanks

[sammycee769]

I’d like to work on this issue by implementing a full Merkle-proof integrity layer for ingested contract events, including deterministic leaf hashing, incremental Merkle tree construction, partition-based root publication to Soroban, and a proof-generation API with client-side verification utilities. I will ensure O(log n) updates, robust retry logic for on-chain publication, and comprehensive integration tests covering correctness, tampering detection, and edge cases. I can complete the implementation within 14–18 hours.

[kingbitnation]

Hello i would love to work on this.

[Alimzy]

Hi! I'd like to work on this issue.

I'm a software engineer with hands-on experience in testing, debugging, and building reliable features across the stack. I'm confident I can take this on and deliver a solid, well-tested solution that follows the project's existing conventions.

I'll make sure to cover the core functionality as well as relevant edge cases to keep things reliable and regression-resistant.

Let me know if I can get started, and I'll provide updates as I progress.