blip-0073: Path Queries

[brh28]

Path queries are three optional peer messages (query_path, reply_path, reject_query_path) that let nodes cooperatively construct payment paths. Responses apply a node's first-hand knowledge of its own channel balances. This reduces node dependence on the gossiped graph, provides an alternative to probing, and lets routing nodes price payment flows per query, free of gossip's currency and expressivity limits.

This is the second version of the proposal, migrated from its original draft against lightning/bolts. Path queries are optional and pairwise. Since adoption can be partial and strategies are locally decided, the proposal does not require universality and is better suited as a bLIP. The revision also reworks the wire format (TLV streams, MPP path parts, query_id correlation, required expiry_ms in both directions) and substantially expands the rationale, privacy, and DoS analyses.

Reserves message types 32769/32771/32773 and feature bits 264/265 via bLIP 2.

[brh28]

Some open design decisions that I'd like to cover in review discussion:

1. Machine readable error codes for the reject_query_path, indicating things like insufficient liquidity, too many attempts, etc
2. amount_range is included in the Additional Fields section, but should arguably be included in the base TLV as a required field
3. part is a fixed struct with no TLV stream, but should likely include a length-prefixed

[TheBlueMatt]

Isn't the old FoF balance (quantile) query somewhat more privacy-preserving than this? Isn't it preferable for the edge node with limited balances to query their LSP for information and then compute paths from that themselves rather than asking for a whole path?

[brh28]

To my understanding, even though both are query-based and are intended to improve payment reliability, it's really comparing apples and oranges. Foaf is a liquidity management tool; nodes share balance information in their locale, so that they can rebalance with each other. Path queries are a path finding mechanism for payment senders and policy distribution for routing nodes.

Regarding privacy, are you referring to the payment details (e.g a downstream node, amount)? If so, I'm curious what queries you have in mind, but generally yes, the more targeted the query, the more it reveals about itself. That said:

1. Privacy cuts both ways - sharing more liquidity state is less private for routing nodes.
2. As mentioned in Privacy of payment details, the querying node is in control of what it reveals. It can blur amounts, query for sub-paths, split payments, or go nuclear and opt out of the feature.

[TheBlueMatt]

No, foaf balance information is exactly the useful information you want to do pathfinding better if your peers have more channels/liquidity than you do (which is common).

[brh28]

My read of the PR is that foaf is designed to improve reliability by changing network state, rather than supplying payment senders with path knowledge. Perhaps you can point me to a more up-to-date document that describes payment senders using it to discover feasible paths?

Either way, foaf replies are explicitly designed not to compose ("MUST NOT be gossiped and forwarded to other peers") and therefore do not have the same reach as path queries. Foaf messages also do not carry routing policy.

If you think this deserves a section in Related Work, let me know.

[TheBlueMatt]

You can use it for either, its just a general query.

As for the question of asking a random node far away for a path, that's not gonna work as well as you think it will. The only reason to ask someone else for a path is (a) if they're also in your "neighborhood" and (b) (1) they send more payments/probes than you do, and thus have more up-to-date information or (2) they're your LSP and you just want to know the state of their channels. (a) means there's no real reason to send these too far, and (b)(2) is accomplished by foaf. (b)(1) means this isn't as useful as you think in general, I fear.

[brh28]

Anonymous queries definitely require more thought, which is why I left them in the Expanding the protocol section with only a brief introduction. There are some scattered references to this section which provide examples of how it can be used, but may make sense to include them directly in the section.

That said, I disagree on (a) and while both counts of (b) are true to an extent, I would add some context that leads me to a different conclusion:

(a) one of the examples (referenced in the privacy section) is to partition a payment into sub-paths, so that no routing node sees both the source and destination of the full route. In this case, the downstream sub-path(s) is likely outside of your neighborhood. For example:

i. Select a remote node with path query support (T₂)
ii. Query channel peer (T₁) for a path to T₂.
iii. Using an anonymous query, query T₂ for a path from T₂ to final destination (D).
iv. Send payment using aggregate route: S -> T₁ -> ... -> T₂ -> ... -> D

(b) (1) Path queries are designed to bridge this information asymmetry between nodes. The queried node always has direct knowledge of its channels, which is always superior to inferred knowledge from payment results/probes. Beyond its own channels, the queried node can also query (i.e recursion). Yes, large nodes with more inferred knowledge still have an advantage, but cooperation gives nodes the ability to close the gap.

(2) Definitely a valid use case, but more likely accomplished with a regular query. I'd also distinguish; your not asking for channel state, your asking for a pathfinding evaluation, which uses the channel state as input.

---

Personally, my concerns with anonymous queries are:

1. The dependence on onion messages. Reading the delving conversation, it seems rate limiting is already difficult for the relays.
2. The UX. What does it look like to pay for a query response? I'd say this is an implementation detail, but still interesting to consider.

[TheBlueMatt]

If your goal is to partition a payment into subpaths you should use trampoline, not path queries. If your goal is to share knowledge, you should share that knowledge, not make a path query. A path query is a really lazy and bad way to share knowledge rather than just sharing the knowledge built up from probing/payments and letting the client do its own pathfinding with that data.

[brh28]

If your goal is to partition a payment into subpaths you should use trampoline, not path queries.

As described in the Trampoline section, there are trade-offs between path queries and trampoline. To summarize, the payment sender is in the driver's seat; they select the route, the fees, how to handle errors, etc. Can you explain why that control is never worth having?

If your goal is to share knowledge, you should share that knowledge, not make a path query.

The goal is to send a payment. A payment needs a path. A path is the most targeted information a node can ask for, and the least information a responder must reveal to answer it.

rather than just sharing the knowledge built up from probing/payments and letting the client do its own pathfinding with that data.

If I'm following this logic correctly, you're saying nodes should share with their peers, not only their own channel balances, but also the balances they've discovered from others using probes. Feel free to correct.

[TheBlueMatt]

To summarize, the payment sender is in the driver's seat; they select the route, the fees, how to handle errors, etc. Can you explain why that control is never worth having?

Because you want the trampoline hop to be able to retry for you.

If I'm following this logic correctly, you're saying nodes should share with their peers, not only their own channel balances, but also the balances they've discovered from others using probes. Feel free to correct.

Yes. Lightning nodes designed to operate for payers which don't send/probe a lot already have APIs for this (or use trampoline) - https://docs.rs/lightning/latest/lightning/routing/scoring/struct.CombinedScorer.html#method.merge

[brh28]

Because you want the trampoline hop to be able to retry for you.

Yes, that is an argument for trampoline that's explicitly stated in the proposal. I'm asking for critiques on the other side of the argument. Do fees not matter? Is HTLC duration not a cost? Or do path queries fail to deliver on either?

Yes. Lightning nodes designed to operate for payers which don't send/probe a lot already have APIs

Interesting, I didn't know nodes were doing this.

So, it seems we agree that liquidity knowledge is a gap for the payment sender. As for approach, while I can see some value here, merging two global views of the network to perform a single pathfinding operation strikes me as inefficient. Similar to asking Google Maps for the entire map of the US with real-time traffic data just to get directions to your grandma's house. Except in this case, the "real-time traffic data" is someone else's private balance.

I'll also note CombinedScorer::merge only covers ChannelLiquidities but not routing policy, which is half of the Motivation.

[TheBlueMatt]

As for approach, while I can see some value here, merging two global views of the network to perform a single pathfinding operation strikes me as inefficient. Similar to asking Google Maps for the entire map of the US with real-time traffic data just to get directions to your grandma's house. Except in this case, the "real-time traffic data" is someone else's private balance.

Its a tiny file and a few milliseconds of CPU...No, it costs almost nothing.

I'll also note CombinedScorer::merge only covers ChannelLiquidities but not routing policy, which is half of the Motivation.

Huh? It covers everything a node uses to route, ignoring fixed code.

[brh28]

Its a tiny file and a few milliseconds of CPU...No, it costs almost nothing.

It may be small as a one time transfer, but staying accurate requires continuous syncing (i.e liquidity_offset_half_life). A reply_path is the minimum amount of data to fulfill the sender's purpose and can be fetched just-in-time.

It covers everything a node uses to route, ignoring fixed code.

I think we're answering different questions. You're right that merge plus gossip covers the inputs a node routes on today. My claim is about what gossip can express, not whether it's complete.