Skip to content

feat(solana-indexer) PR 7.1: wire tx-only decoder run loop#4591

Open
squadgazzz wants to merge 1 commit into
solana-indexer/PR6.2-drop-account-halffrom
solana-indexer/PR7-dispatch
Open

feat(solana-indexer) PR 7.1: wire tx-only decoder run loop#4591
squadgazzz wants to merge 1 commit into
solana-indexer/PR6.2-drop-account-halffrom
solana-indexer/PR7-dispatch

Conversation

@squadgazzz

@squadgazzz squadgazzz commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Description

With the account half of the stream gone (#4601), the decoder's run loop is transaction-only. It drains the channel from the ingester and, for each transaction, resolves the settlement and SolFlow instructions and routes them to per-program decoders. Those decoders are stubs for now: turning instruction bytes into typed events needs the settlement program's on-chain layout, which comes in a later PR.

Changes

  • Implement Decoder::run: drain the channel until the ingester drops the sender
  • Route each resolved settlement or SolFlow instruction to a stubbed per-program decoder

How to test

New unit test.

@squadgazzz squadgazzz changed the title feat(solana-indexer): wire decoder run loop, join halves, route instructions feat(solana-indexer) PR 7.1: wire decoder run loop, join halves, route instructions Jul 3, 2026
@squadgazzz squadgazzz marked this pull request as ready for review July 3, 2026 13:11
@squadgazzz squadgazzz requested a review from a team as a code owner July 3, 2026 13:11
@claude

claude Bot commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Claude finished @squadgazzz's task in 3m 3s —— View job


PR Review: wire decoder run loop, join halves, route instructions

  • Read the PR diff and changed files
  • Understand surrounding decoder/ingester/watchdog context
  • Review for correctness, edge cases, style
  • Post inline comments for issues
  • Post summary

The plumbing reads cleanly — the run loop, the symmetric buffer-or-complete join, the signature guard, and the settlement/SolFlow routing are all reasonable, and the stubbed decoders match the stated scope. Two things worth a look, posted inline:

  1. Join assumes one account update per tx (decoder.rs) — the account subscription filters by owner, so a tx touching multiple program-owned accounts emits multiple updates sharing (slot, signature). They clobber each other in the map, and late-arriving ones re-buffer after the pair completes → future false TxUpdateMissing dead-letters. Latent today since the snapshot is dropped, but the singular-account model is baked in here. Worth confirming the assumption holds.
  2. Test coverage gap (tests.rs) — both completion tests send the tx first, so on_tx's account-completion arm (decoder.rs:98) is never exercised. A reverse-order test would cover it.

Nothing blocking; (1) is the one I'd resolve (even if just by confirming) before the account snapshot gets consumed.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements the main run loop for the Decoder struct, which joins transaction and account-update halves using a shared partials map and routes tracked instructions to stubbed per-program decoders. It also adds corresponding unit tests to verify the joining, buffering, and dropping behaviors. A critical issue was identified in the join logic: it assumes a strict 1:1 relationship between transactions and account updates, which does not hold true in Solana and can lead to memory bloat, false-positive dead letters, or lost account updates.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment on lines +118 to +124
match self.partials.remove(&(slot, signature)) {
Some((_, PartialHalf::Tx(tx))) => self.decode(&tx),
_ => {
self.partials
.insert((slot, signature), PartialHalf::Account(account));
}
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The current join logic assumes a strict 1:1 relationship between a transaction and its account updates, which does not hold true in Solana. A single transaction can modify multiple accounts owned by the tracked programs, resulting in multiple SubscribeUpdateAccount messages for the same (slot, signature).

This leads to two major issues:

  1. If Tx arrives first: The first Account update removes the Tx from partials and processes it. Any subsequent Account updates for the same transaction will find no Tx in partials and will be inserted into the map, where they will remain indefinitely until the watchdog cleans them up. This causes unnecessary memory bloat and false-positive dead letters.
  2. If Account updates arrive first: Each new Account update will overwrite the previous one in partials (since the _ wildcard branch in on_account removes the existing Account and inserts the new one). When the Tx update finally arrives, it will only be paired with the last Account update, and all previous ones are lost.

Recommendation

Redesign the partials map or the join logic to support one-to-many relationships. For example, you could buffer multiple account updates in a collection (e.g., Vec<Box<SubscribeUpdateAccountInfo>>) under the (slot, signature) key, or avoid removing the Tx half immediately if more account updates are expected.

Comment on lines +108 to +125
fn on_account(
&self,
slot: Slot,
txn_signature: Option<Signature>,
account: Box<SubscribeUpdateAccountInfo>,
) {
let Some(signature) = txn_signature else {
tracing::debug!("account update without a signature, cannot join");
return;
};
match self.partials.remove(&(slot, signature)) {
Some((_, PartialHalf::Tx(tx))) => self.decode(&tx),
_ => {
self.partials
.insert((slot, signature), PartialHalf::Account(account));
}
}
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The join assumes one account update per transaction, but the ingester subscribes to accounts by owner (accounts = |program| SubscribeRequestFilterAccounts { owner: vec![program.to_string()], .. } in ingester.rs:372). A single settlement/SolFlow tx that writes to several program-owned accounts (PDAs, vaults, …) will emit multiple SubscribeUpdateAccountInfo messages, all sharing the same (slot, txn_signature). With this map keyed on (slot, signature) holding a single PartialHalf::Account:

  • If several account halves arrive before the tx, each on_account call clobbers the previously buffered one (the _ arm removes + re-inserts), so only the last survives.
  • Once the pair completes and decode fires, the tx entry is gone. Any further account updates for that same tx re-buffer as fresh Account partials that never complete — so once the watchdog lands they'll be dead-lettered as TxUpdateMissing even though the tx was seen.

This is latent today because decode drops the account snapshot, but the whole join model (and account_snapshot: Option<…> / PartialHalf::Account(Box<…>), both singular) bakes in the one-account assumption. Worth confirming whether a tracked tx can touch >1 program-owned account before the snapshot is actually consumed — if so the value likely needs to be a collection keyed per-account, and "pair complete" needs a definition beyond "first account seen".

Comment on lines +208 to +220
async fn tx_and_account_halves_complete_and_clear_the_partial() {
let (settlement, solflow) = (pubkey(1), pubkey(2));
let (mut decoder, sender, partials) = test_decoder(settlement, solflow);
let (slot, sig) = (Slot(7), signature(3));

sender.send(stream_tx(slot, sig, settlement)).await.unwrap();
sender.send(stream_account(slot, Some(sig))).await.unwrap();
drop(sender);

decoder.run().await.unwrap();

assert!(partials.is_empty());
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both completion tests send the tx first, so only on_account's completion path (the Some(PartialHalf::Tx(tx)) arm) is exercised. The symmetric path in on_txSome((_, PartialHalf::Account(_account))) => self.decode(&tx) at decoder.rs:98 — where the account half arrives first and the tx completes the pair, is never hit. Worth adding a reverse-order test (send stream_account then stream_tx, assert partials.is_empty()) to cover it.

@squadgazzz squadgazzz marked this pull request as draft July 3, 2026 15:01
@squadgazzz squadgazzz force-pushed the solana-indexer/PR7-dispatch branch from d97fadc to 97e3d14 Compare July 3, 2026 16:57
@squadgazzz squadgazzz force-pushed the solana-indexer/PR7-dispatch branch from 97e3d14 to 3cb7a71 Compare July 6, 2026 10:40
@squadgazzz squadgazzz changed the title feat(solana-indexer) PR 7.1: wire decoder run loop, join halves, route instructions feat(solana-indexer) PR 7.1: wire tx-only decoder run loop Jul 6, 2026
@squadgazzz squadgazzz changed the base branch from solana-indexer/PR6-decoder to solana-indexer/PR6.2-drop-account-half July 6, 2026 10:41
@squadgazzz squadgazzz marked this pull request as ready for review July 6, 2026 14:27

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements the main loop (run) and routing logic (decode) in the Solana indexer's decoder, which drains the stream of transaction updates and routes relevant instructions to stubbed handlers for settlement and SolFlow programs. Unit tests were also added to verify that the decoder drains transactions until the channel is closed. No critical issues were found, so there is no feedback to provide.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant