feat(solana-indexer) PR 7.1: wire tx-only decoder run loop#4591
feat(solana-indexer) PR 7.1: wire tx-only decoder run loop#4591squadgazzz wants to merge 1 commit into
Conversation
|
Claude finished @squadgazzz's task in 3m 3s —— View job PR Review: wire decoder run loop, join halves, route instructions
The plumbing reads cleanly — the run loop, the symmetric buffer-or-complete join, the signature guard, and the settlement/SolFlow routing are all reasonable, and the stubbed decoders match the stated scope. Two things worth a look, posted inline:
Nothing blocking; (1) is the one I'd resolve (even if just by confirming) before the account snapshot gets consumed. |
There was a problem hiding this comment.
Code Review
This pull request implements the main run loop for the Decoder struct, which joins transaction and account-update halves using a shared partials map and routes tracked instructions to stubbed per-program decoders. It also adds corresponding unit tests to verify the joining, buffering, and dropping behaviors. A critical issue was identified in the join logic: it assumes a strict 1:1 relationship between transactions and account updates, which does not hold true in Solana and can lead to memory bloat, false-positive dead letters, or lost account updates.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| match self.partials.remove(&(slot, signature)) { | ||
| Some((_, PartialHalf::Tx(tx))) => self.decode(&tx), | ||
| _ => { | ||
| self.partials | ||
| .insert((slot, signature), PartialHalf::Account(account)); | ||
| } | ||
| } |
There was a problem hiding this comment.
The current join logic assumes a strict 1:1 relationship between a transaction and its account updates, which does not hold true in Solana. A single transaction can modify multiple accounts owned by the tracked programs, resulting in multiple SubscribeUpdateAccount messages for the same (slot, signature).
This leads to two major issues:
- If
Txarrives first: The firstAccountupdate removes theTxfrompartialsand processes it. Any subsequentAccountupdates for the same transaction will find noTxinpartialsand will be inserted into the map, where they will remain indefinitely until the watchdog cleans them up. This causes unnecessary memory bloat and false-positive dead letters. - If
Accountupdates arrive first: Each newAccountupdate will overwrite the previous one inpartials(since the_wildcard branch inon_accountremoves the existingAccountand inserts the new one). When theTxupdate finally arrives, it will only be paired with the lastAccountupdate, and all previous ones are lost.
Recommendation
Redesign the partials map or the join logic to support one-to-many relationships. For example, you could buffer multiple account updates in a collection (e.g., Vec<Box<SubscribeUpdateAccountInfo>>) under the (slot, signature) key, or avoid removing the Tx half immediately if more account updates are expected.
| fn on_account( | ||
| &self, | ||
| slot: Slot, | ||
| txn_signature: Option<Signature>, | ||
| account: Box<SubscribeUpdateAccountInfo>, | ||
| ) { | ||
| let Some(signature) = txn_signature else { | ||
| tracing::debug!("account update without a signature, cannot join"); | ||
| return; | ||
| }; | ||
| match self.partials.remove(&(slot, signature)) { | ||
| Some((_, PartialHalf::Tx(tx))) => self.decode(&tx), | ||
| _ => { | ||
| self.partials | ||
| .insert((slot, signature), PartialHalf::Account(account)); | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
The join assumes one account update per transaction, but the ingester subscribes to accounts by owner (accounts = |program| SubscribeRequestFilterAccounts { owner: vec![program.to_string()], .. } in ingester.rs:372). A single settlement/SolFlow tx that writes to several program-owned accounts (PDAs, vaults, …) will emit multiple SubscribeUpdateAccountInfo messages, all sharing the same (slot, txn_signature). With this map keyed on (slot, signature) holding a single PartialHalf::Account:
- If several account halves arrive before the tx, each
on_accountcall clobbers the previously buffered one (the_arm removes + re-inserts), so only the last survives. - Once the pair completes and
decodefires, the tx entry is gone. Any further account updates for that same tx re-buffer as freshAccountpartials that never complete — so once the watchdog lands they'll be dead-lettered asTxUpdateMissingeven though the tx was seen.
This is latent today because decode drops the account snapshot, but the whole join model (and account_snapshot: Option<…> / PartialHalf::Account(Box<…>), both singular) bakes in the one-account assumption. Worth confirming whether a tracked tx can touch >1 program-owned account before the snapshot is actually consumed — if so the value likely needs to be a collection keyed per-account, and "pair complete" needs a definition beyond "first account seen".
| async fn tx_and_account_halves_complete_and_clear_the_partial() { | ||
| let (settlement, solflow) = (pubkey(1), pubkey(2)); | ||
| let (mut decoder, sender, partials) = test_decoder(settlement, solflow); | ||
| let (slot, sig) = (Slot(7), signature(3)); | ||
|
|
||
| sender.send(stream_tx(slot, sig, settlement)).await.unwrap(); | ||
| sender.send(stream_account(slot, Some(sig))).await.unwrap(); | ||
| drop(sender); | ||
|
|
||
| decoder.run().await.unwrap(); | ||
|
|
||
| assert!(partials.is_empty()); | ||
| } |
There was a problem hiding this comment.
Both completion tests send the tx first, so only on_account's completion path (the Some(PartialHalf::Tx(tx)) arm) is exercised. The symmetric path in on_tx — Some((_, PartialHalf::Account(_account))) => self.decode(&tx) at decoder.rs:98 — where the account half arrives first and the tx completes the pair, is never hit. Worth adding a reverse-order test (send stream_account then stream_tx, assert partials.is_empty()) to cover it.
d97fadc to
97e3d14
Compare
97e3d14 to
3cb7a71
Compare
There was a problem hiding this comment.
Code Review
This pull request implements the main loop (run) and routing logic (decode) in the Solana indexer's decoder, which drains the stream of transaction updates and routes relevant instructions to stubbed handlers for settlement and SolFlow programs. Unit tests were also added to verify that the decoder drains transactions until the channel is closed. No critical issues were found, so there is no feedback to provide.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
Description
With the account half of the stream gone (#4601), the decoder's run loop is transaction-only. It drains the channel from the ingester and, for each transaction, resolves the settlement and SolFlow instructions and routes them to per-program decoders. Those decoders are stubs for now: turning instruction bytes into typed events needs the settlement program's on-chain layout, which comes in a later PR.
Changes
Decoder::run: drain the channel until the ingester drops the senderHow to test
New unit test.