Summary
The current event ingestion pipeline assumes Stellar ledgers are final on first observation. In practice, a network partition or validator disagreement can cause a short ledger rollback (reorg), invalidating previously ingested events. The pipeline must detect when the next ledger's previousLedgerHash does not match the stored hash of the last ingested ledger, trigger a rollback of all affected database records, and re-ingest from the fork point.
Scope
- Store
ledgerSequence and ledgerHash for every ingested event in a LedgerCheckpoint table
- On each new ledger, verify
incoming.previousLedgerHash === checkpoint[incoming.sequence - 1].hash
- On mismatch: identify the fork point by walking backwards through
LedgerCheckpoint until hashes agree, mark all events from the fork point forward as REORGED, reverse their database side effects (key balance changes, holder count changes, transaction history entries) inside a single database transaction, then re-ingest from the fork point
- Emit a
warn-level structured log with forkPoint, reorgedLedgerCount, and affectedEventCount on every reorg
- Gate all buy/sell API responses on confirmed ledger depth ≥ 2 (no response until the ledger is 2 deep) to reduce reorg exposure
- Add integration tests: clean ingestion advances the checkpoint, a simulated reorg triggers rollback and re-ingestion, rolled-back balance changes are correctly reversed, re-ingested events produce the same final state as if the reorg never happened
Acceptance Criteria
ETA: 24 hours
Coordinate on Telegram
Summary
The current event ingestion pipeline assumes Stellar ledgers are final on first observation. In practice, a network partition or validator disagreement can cause a short ledger rollback (reorg), invalidating previously ingested events. The pipeline must detect when the next ledger's
previousLedgerHashdoes not match the stored hash of the last ingested ledger, trigger a rollback of all affected database records, and re-ingest from the fork point.Scope
ledgerSequenceandledgerHashfor every ingested event in aLedgerCheckpointtableincoming.previousLedgerHash === checkpoint[incoming.sequence - 1].hashLedgerCheckpointuntil hashes agree, mark all events from the fork point forward asREORGED, reverse their database side effects (key balance changes, holder count changes, transaction history entries) inside a single database transaction, then re-ingest from the fork pointwarn-level structured log withforkPoint,reorgedLedgerCount, andaffectedEventCounton every reorgAcceptance Criteria
LedgerCheckpointtable stores hash and sequence for every ingested ledgerETA: 24 hours
Coordinate on Telegram