diff --git a/benchmarks/bridge-quote-latency.yml b/benchmarks/bridge-quote-latency.yml index 0e98a1d9..2f494ca3 100644 --- a/benchmarks/bridge-quote-latency.yml +++ b/benchmarks/bridge-quote-latency.yml @@ -54,7 +54,7 @@ methodology: - "Notional sizes: $5, $50, $300 per quote." - "Cadence: full sweep (4 routes × 3 amounts × N bridges) every 5 minutes for 24 hours." - "Region: eu-west. Single point of measurement; multi-region requires running additional monitor instances." - - "Histogram buckets: 50, 100, 200, 500, 1000, 2000, 5000, 10000 ms." + - "Histogram buckets: 10, 25, 50, 100, 200, 500, 1000, 2000, 5000, 10000 ms. Sub-50ms buckets added 2026-07-13 so solver-cached fast paths (Near Intents on HyperCore in particular) surface their true p50 instead of being floored at the interpolation of the (0, 50] bucket." - "Failures (quote_failed, execution_failed, unsupported route) excluded from latency aggregates and counted toward success rate." # Per-corridor leader rewrite deferred: this YAML does not yet declare @@ -89,7 +89,7 @@ faq: - q: "Are intent and relay bridges always faster than aggregators?" a: "On a single-route query, usually. Intent layers like Relay quote against a pool of pre-positioned solvers, so the API is a thin price-discovery call. Aggregators like LI.FI run a route-search graph over N underlying bridges per request, which adds 100 to 400 ms of irreducible work. On multi-hop or rare-corridor queries the aggregator advantage in coverage matters more than the latency gap, and the comparison flips." - q: "How does OpenChainBench measure bridge quote latency?" - a: "The harness issues identical quote requests against every bridge for the same route and notional, every five minutes, from a single eu-west origin. Each request times wall-clock duration from send to last byte received, then publishes a Prometheus histogram (50, 100, 200, 500, 1000, 2000, 5000, 10000 ms buckets). p50, p90 and p99 are derived via `histogram_quantile` over a rolling 24-hour window. Errored quotes (quote_failed, unsupported route, timeout) are excluded from the latency aggregate and counted toward success rate so a fast-but-broken bridge cannot game the headline." + a: "The harness issues identical quote requests against every bridge for the same route and notional, every five minutes, from a single eu-west origin. Each request times wall-clock duration from send to last byte received, then publishes a Prometheus histogram (10, 25, 50, 100, 200, 500, 1000, 2000, 5000, 10000 ms buckets). p50, p90 and p99 are derived via `histogram_quantile` over a rolling 24-hour window. Errored quotes (quote_failed, unsupported route, timeout) are excluded from the latency aggregate and counted toward success rate so a fast-but-broken bridge cannot game the headline." - q: "Does quote latency affect the price the user gets?" a: "Indirectly, yes. A slower quote API leaves a larger window between the moment the price is quoted and the moment the user signs. On volatile corridors the underlying spot can drift in that window, which surfaces as a quoted-vs-realized gap. Intent layers compensate by re-quoting at signing time; aggregators usually pass through a slippage parameter. The fee benchmark on this site (`/benchmarks/bridge-fee`) measures the realized cost; this page measures the latency half of the same flow." diff --git a/benchmarks/evm-block-builders.yml b/benchmarks/evm-block-builders.yml new file mode 100644 index 00000000..26651f6b --- /dev/null +++ b/benchmarks/evm-block-builders.yml @@ -0,0 +1,232 @@ +# OpenChainBench. Bench № 085 + +slug: evm-block-builders +number: "085" +title: Ethereum block builder market share +seo_title: "Ethereum block builders benchmark 2026, live builder market share" +seo_description: "Live benchmark of who builds Ethereum blocks. Builder market share attributed via block extraData, cross-checked against MEV-Boost relay bidtraces." +subtitle: Share of Ethereum mainnet blocks attributed to each block builder via the extraData self-label, cross-checked against relay bidtraces. Observational, no transactions sent. +category: Trading +status: live +metric: Block share (24h) +unit: pct +higher_is_better: true + +disclaimer: | + Attribution relies on the extraData tag each builder voluntarily stamps into its blocks. A builder can change or strip its tag at any time; unrecognized tags land in the Other row and every raw string is logged so the table grows. High share is a centralization datapoint, not a quality ranking. + +seo_intro: | + Who actually builds Ethereum? Since MEV-Boost, validators outsource + block construction to a handful of specialized builders, and the + builders sign their work: each stamps a self-label into the block's + extraData field ("Titan (titanbuilder.xyz)", "beaverbuild.org", and + so on). This benchmark decodes the extraData of every mainnet block, + attributes it to a curated table of known builder tags, and publishes + the live market share. The result is the concentration story in one + chart: 95 percent plus of Ethereum blocks are built by a handful of + private builders, and the top two regularly build most of the chain. + Blocks carrying an execution client's default tag (geth, reth, + nethermind, besu version strings) are counted as vanilla, meaning the + proposer built locally instead of outsourcing; that row is the live + measure of how much of Ethereum still self-builds. As an independent + cross-check the harness also polls the public bidtrace data APIs of + seven MEV-Boost relays (Flashbots, ultrasound, Agnostic, bloXroute + max-profit and regulated, Titan, Aestus) and counts delivered + payloads per relay, a signal that does not depend on any builder's + self-chosen label. + +abstract: | + We attribute every Ethereum mainnet block to its builder by decoding + the block's extraData field, the 32 bytes a builder uses to sign its + work in human-readable ASCII. The harness polls the head of the chain + through a keyless public RPC once per 12 second slot, backfills any + blocks missed between polls by number, decodes extraData to printable + ASCII and matches it against a curated substring table of known + builder tags (Titan, Quasar, Eureka, BuilderNet, beaverbuild, rsync, + Flashbots, Builder+ and more). Client-default tags (geth, reth, + nethermind, besu, erigon version strings) and empty extraData count + as vanilla, locally built blocks. Anything unrecognized counts as + Other, increments an unattributed counter, and logs the raw string so + the table can grow, the same philosophy as the tip-wallet table in + bench 016. The headline metric is each builder's percentage of all + blocks observed in the last 24 hours. Two companion measurements ship + in the same harness. First, an independent cross-check: the public + bidtrace APIs of seven MEV-Boost relays are polled every five minutes + and delivered payloads are counted per relay, deduped per relay by + slot high-water-mark; the same slot appearing on several relays is + normal because builders multi-home their bids. Second, an L2 + soft-confirmation study: the harness holds the Arbitrum sequencer + feed and the Base flashblocks stream open and measures how far those + preconfirmation channels run ahead of the public RPC head, emitted as + histograms for a future bench page. At inception a 50 block live + sample attributed 98 percent cleanly: Titan 19, Quasar 16, Eureka 6, + BuilderNet 2, beaverbuild 1, Builder+ 1, one vanilla block. + +methodology: + - "Source: keyless public Ethereum RPC (ethereum-rpc.publicnode.com by default, env-overridable). eth_getBlockByNumber(latest) every 12s, matching the mainnet slot time. Gaps between polls are backfilled by block number (capped at 10) so the counters see every block, not just the ones that happen to be head at poll time." + - "Attribution: extraData hex is decoded to printable ASCII and matched, lowercase, against a curated substring table (titan, quasar, eureka, buildernet, beaverbuild, rsync, builder0x69, flashbots, bloxroute, builder+/btcs, penguinbuild and more). First match wins; most specific substrings are ordered first." + - "Vanilla detection: empty extraData or execution-client default tags (geth, reth, nethermind, besu, erigon, go1.x, platform strings) count as builder=vanilla, meaning the proposer built the block locally instead of outsourcing to a builder. This row is the live self-built share of Ethereum." + - "Unattributed handling: any non-empty tag not in the table counts as builder=other, increments ebb_unattributed_total, and logs the raw extraData string. The table grows from operator logs instead of silently under-counting, the bench 016 pattern." + - "Self-labeling caveat: extraData is voluntary. A builder can rebrand, blank its tag, or impersonate another. The relay cross-check below is the independent signal; a large divergence between extraData share and relay-side delivered payloads would surface tag games." + - "Relay cross-check: seven MEV-Boost relay data APIs (Flashbots, ultrasound, Agnostic, bloXroute max-profit, bloXroute regulated, Titan, Aestus) are polled keyless every 5 min via GET /relay/v1/data/bidtraces/proposer_payload_delivered?limit=50. Delivered payloads are counted per relay (ebb_relay_payloads_total), deduped per relay by slot high-water-mark. The same slot on multiple relays is normal: builders multi-home bids, so relay counters measure relay share, not a partition of blocks." + - "Single vantage: one harness in one region observes the chain through one public RPC. Attribution is not latency-sensitive (extraData is consensus data, identical from every vantage), so single-vantage only affects liveness, which the health gauges expose." + - "Companion L2 metrics: the same harness measures Arbitrum sequencer-feed soft-confirmation lag (ebb_arb_softconf_lag_milliseconds, feed arrival to public RPC visibility, offset derived at runtime) and Base flashblocks cadence plus flashblock-to-RPC lag (ebb_base_flashblock_interval_milliseconds, ebb_base_softconf_lag_milliseconds). These are per-chain, not per-builder, so this spec surfaces builder share only; the L2 lag series feed a future bench page." + +findings: + - "{{best_name}} currently builds {{best_p50}} of Ethereum blocks observed over the last 24h, across {{count}} attribution rows. The top two builders regularly account for the majority of the chain, the concentration this bench exists to make visible." + - "{{name:beaverbuild}} sits at {{p50:beaverbuild}}. beaverbuild dominated 2023-2024 alongside rsync; its slide against Titan and the newer Quasar and Eureka operations is the clearest sign the builder market still churns even as it concentrates." + - "{{name:buildernet}} carries {{p50:buildernet}}. BuilderNet is the Flashbots-initiated attempt to mutualize block building into a multi-operator network precisely because of the concentration this page measures; its share is the decentralization counter-signal to watch." + - "{{name:vanilla}} blocks, built locally by the proposer without a builder, sit at {{p50:vanilla}}. This is the live measure of how much of Ethereum still self-builds rather than outsourcing to the MEV supply chain." + - "{{worst_name}} trails at {{worst_p50}}. The long tail matters less than the head here: the bench's story is that a validator set of a million keys funnels block construction through a single-digit number of firms." + +faq: + - q: "How is a block attributed to a builder?" + a: "Builders stamp a self-label into the 32-byte extraData field of every block they construct, for example 'Titan (titanbuilder.xyz)' or 'beaverbuild.org'. The harness decodes extraData to ASCII and matches it against a curated substring table. Empty or client-default extraData (geth, reth, nethermind version strings) means the proposer built the block locally and counts as vanilla. Unrecognized tags count as Other and are logged raw so the table grows. At inception, 98 percent of a live 50-block sample attributed cleanly." + - q: "Can a builder game this by changing its extraData?" + a: "Yes, extraData is voluntary self-labeling, and that is disclosed as the bench's main caveat. A builder could blank its tag (blocks would fall into Other, and the unattributed counter plus raw-string logging would surface the shift within hours) or imitate another builder's tag (detectable by divergence against relay bidtraces, which report the builder's signing pubkey rather than its label). In practice builders keep their tags stable because the label is marketing: searchers pick builders partly on public share stats." + - q: "Why do the p50, p90 and p99 columns show the same number?" + a: "This bench has no latency dimension, so the percentile slots are repurposed, same convention as bench 016. Every aggregate column carries the builder's 24h block share; the sample column carries the raw block count and the success column reports the harness's own RPC poll health, not anything about the builder. The ranking signal is the share itself." + - q: "What is the relay cross-check and why does it not sum to 100 percent?" + a: "Seven MEV-Boost relays expose a public data API listing every payload they delivered to a proposer. The harness counts delivered payloads per relay every five minutes. Because builders submit the same bid through several relays and every relay that carried the winning payload reports it, one block can legitimately appear on multiple relays. Relay counts therefore measure relay market share and act as an attribution sanity check independent of extraData; they are not a partition of blocks and do not sum to the block count." + - q: "Why does builder concentration matter?" + a: "Under MEV-Boost, validators outsource block construction, so whoever builds most blocks decides transaction ordering and inclusion for most of Ethereum. Concentration in two or three private firms creates censorship surface (a dominant builder filtering transactions affects most blocks), a single point of failure for liveness of the fee market, and information asymmetry for anyone trading on-chain. Projects like BuilderNet exist specifically to mutualize this layer; this bench provides the neutral number for whether concentration is getting better or worse." + +source: https://github.com/ChainBench/OpenChainBench/tree/main/harnesses/evm-block-builders + +prometheus: + window: 24h + expected_freshness_seconds: 120 + freshness_metric: ebb_eth_last_block + +# Real metrics emitted by the evm-block-builders harness: +# ebb_blocks_total{builder} counter (attributed blocks) +# ebb_unattributed_total counter +# ebb_eth_poll_health gauge (0|1) +# ebb_eth_last_block gauge (freshness) +# ebb_relay_payloads_total{relay} counter (cross-check) +# ebb_arb_softconf_lag_milliseconds histogram (future bench) +# ebb_base_flashblock_interval_milliseconds histogram (future bench) +# ebb_base_softconf_lag_milliseconds histogram (future bench) +# +# Each "provider" is one builder attribution row. The headline is the +# builder's share of all blocks observed in 24h (pct). p50/p90/p99/mean +# carry the same share scalar (016 convention); sample_size carries the +# raw block count; success carries harness poll health. + +providers: + - slug: titan + name: Titan + tag: titanbuilder.xyz, dominant private builder + formula: "Blocks whose extraData contains the Titan tag, as a percentage of all Ethereum blocks observed in the last 24h." + queries: + p50: 100 * sum(increase(ebb_blocks_total{builder="titan"}[24h])) / scalar(sum(increase(ebb_blocks_total[24h]))) + p90: 100 * sum(increase(ebb_blocks_total{builder="titan"}[24h])) / scalar(sum(increase(ebb_blocks_total[24h]))) + p99: 100 * sum(increase(ebb_blocks_total{builder="titan"}[24h])) / scalar(sum(increase(ebb_blocks_total[24h]))) + mean: 100 * sum(increase(ebb_blocks_total{builder="titan"}[24h])) / scalar(sum(increase(ebb_blocks_total[24h]))) + success: avg_over_time(ebb_eth_poll_health[24h]) + sample_size: sum(increase(ebb_blocks_total{builder="titan"}[24h])) + series: 100 * sum(rate(ebb_blocks_total{builder="titan"}[1h])) / scalar(sum(rate(ebb_blocks_total[1h]))) + + - slug: quasar + name: Quasar + tag: Fast-rising private builder + formula: "Blocks whose extraData contains the Quasar tag, as a percentage of all Ethereum blocks observed in the last 24h." + queries: + p50: 100 * sum(increase(ebb_blocks_total{builder="quasar"}[24h])) / scalar(sum(increase(ebb_blocks_total[24h]))) + p90: 100 * sum(increase(ebb_blocks_total{builder="quasar"}[24h])) / scalar(sum(increase(ebb_blocks_total[24h]))) + p99: 100 * sum(increase(ebb_blocks_total{builder="quasar"}[24h])) / scalar(sum(increase(ebb_blocks_total[24h]))) + mean: 100 * sum(increase(ebb_blocks_total{builder="quasar"}[24h])) / scalar(sum(increase(ebb_blocks_total[24h]))) + success: avg_over_time(ebb_eth_poll_health[24h]) + sample_size: sum(increase(ebb_blocks_total{builder="quasar"}[24h])) + series: 100 * sum(rate(ebb_blocks_total{builder="quasar"}[1h])) / scalar(sum(rate(ebb_blocks_total[1h]))) + + - slug: eureka + name: Eureka + tag: Private builder, newer entrant + formula: "Blocks whose extraData contains the Eureka tag, as a percentage of all Ethereum blocks observed in the last 24h." + queries: + p50: 100 * sum(increase(ebb_blocks_total{builder="eureka"}[24h])) / scalar(sum(increase(ebb_blocks_total[24h]))) + p90: 100 * sum(increase(ebb_blocks_total{builder="eureka"}[24h])) / scalar(sum(increase(ebb_blocks_total[24h]))) + p99: 100 * sum(increase(ebb_blocks_total{builder="eureka"}[24h])) / scalar(sum(increase(ebb_blocks_total[24h]))) + mean: 100 * sum(increase(ebb_blocks_total{builder="eureka"}[24h])) / scalar(sum(increase(ebb_blocks_total[24h]))) + success: avg_over_time(ebb_eth_poll_health[24h]) + sample_size: sum(increase(ebb_blocks_total{builder="eureka"}[24h])) + series: 100 * sum(rate(ebb_blocks_total{builder="eureka"}[1h])) / scalar(sum(rate(ebb_blocks_total[1h]))) + + - slug: buildernet + name: BuilderNet + tag: Flashbots-initiated multi-operator building network + formula: "Blocks whose extraData contains the BuilderNet tag, as a percentage of all Ethereum blocks observed in the last 24h." + queries: + p50: 100 * sum(increase(ebb_blocks_total{builder="buildernet"}[24h])) / scalar(sum(increase(ebb_blocks_total[24h]))) + p90: 100 * sum(increase(ebb_blocks_total{builder="buildernet"}[24h])) / scalar(sum(increase(ebb_blocks_total[24h]))) + p99: 100 * sum(increase(ebb_blocks_total{builder="buildernet"}[24h])) / scalar(sum(increase(ebb_blocks_total[24h]))) + mean: 100 * sum(increase(ebb_blocks_total{builder="buildernet"}[24h])) / scalar(sum(increase(ebb_blocks_total[24h]))) + success: avg_over_time(ebb_eth_poll_health[24h]) + sample_size: sum(increase(ebb_blocks_total{builder="buildernet"}[24h])) + series: 100 * sum(rate(ebb_blocks_total{builder="buildernet"}[1h])) / scalar(sum(rate(ebb_blocks_total[1h]))) + + - slug: beaverbuild + name: beaverbuild + tag: beaverbuild.org, former market leader + formula: "Blocks whose extraData contains the beaverbuild tag, as a percentage of all Ethereum blocks observed in the last 24h." + queries: + p50: 100 * sum(increase(ebb_blocks_total{builder="beaverbuild"}[24h])) / scalar(sum(increase(ebb_blocks_total[24h]))) + p90: 100 * sum(increase(ebb_blocks_total{builder="beaverbuild"}[24h])) / scalar(sum(increase(ebb_blocks_total[24h]))) + p99: 100 * sum(increase(ebb_blocks_total{builder="beaverbuild"}[24h])) / scalar(sum(increase(ebb_blocks_total[24h]))) + mean: 100 * sum(increase(ebb_blocks_total{builder="beaverbuild"}[24h])) / scalar(sum(increase(ebb_blocks_total[24h]))) + success: avg_over_time(ebb_eth_poll_health[24h]) + sample_size: sum(increase(ebb_blocks_total{builder="beaverbuild"}[24h])) + series: 100 * sum(rate(ebb_blocks_total{builder="beaverbuild"}[1h])) / scalar(sum(rate(ebb_blocks_total[1h]))) + + - slug: bobthebuilder + name: Bob The Builder + tag: bobthebuilder.xyz, pseudonymous builder + formula: "Blocks whose extraData contains the bobthebuilder tag, as a percentage of all Ethereum blocks observed in the last 24h." + queries: + p50: 100 * sum(increase(ebb_blocks_total{builder="bobthebuilder"}[24h])) / scalar(sum(increase(ebb_blocks_total[24h]))) + p90: 100 * sum(increase(ebb_blocks_total{builder="bobthebuilder"}[24h])) / scalar(sum(increase(ebb_blocks_total[24h]))) + p99: 100 * sum(increase(ebb_blocks_total{builder="bobthebuilder"}[24h])) / scalar(sum(increase(ebb_blocks_total[24h]))) + mean: 100 * sum(increase(ebb_blocks_total{builder="bobthebuilder"}[24h])) / scalar(sum(increase(ebb_blocks_total[24h]))) + success: avg_over_time(ebb_eth_poll_health[24h]) + sample_size: sum(increase(ebb_blocks_total{builder="bobthebuilder"}[24h])) + series: 100 * sum(rate(ebb_blocks_total{builder="bobthebuilder"}[1h])) / scalar(sum(rate(ebb_blocks_total[1h]))) + + - slug: btcs + name: Builder+ + tag: BTCS Builder+ + formula: "Blocks whose extraData contains the Builder+ or BTCS tag, as a percentage of all Ethereum blocks observed in the last 24h." + queries: + p50: 100 * sum(increase(ebb_blocks_total{builder="btcs"}[24h])) / scalar(sum(increase(ebb_blocks_total[24h]))) + p90: 100 * sum(increase(ebb_blocks_total{builder="btcs"}[24h])) / scalar(sum(increase(ebb_blocks_total[24h]))) + p99: 100 * sum(increase(ebb_blocks_total{builder="btcs"}[24h])) / scalar(sum(increase(ebb_blocks_total[24h]))) + mean: 100 * sum(increase(ebb_blocks_total{builder="btcs"}[24h])) / scalar(sum(increase(ebb_blocks_total[24h]))) + success: avg_over_time(ebb_eth_poll_health[24h]) + sample_size: sum(increase(ebb_blocks_total{builder="btcs"}[24h])) + series: 100 * sum(rate(ebb_blocks_total{builder="btcs"}[1h])) / scalar(sum(rate(ebb_blocks_total[1h]))) + + - slug: vanilla + name: Vanilla + tag: Locally built by the proposer, no builder + formula: "Blocks with empty or execution-client default extraData (geth, reth, nethermind, besu, erigon), meaning the proposer built locally, as a percentage of all blocks observed in 24h." + queries: + p50: 100 * sum(increase(ebb_blocks_total{builder="vanilla"}[24h])) / scalar(sum(increase(ebb_blocks_total[24h]))) + p90: 100 * sum(increase(ebb_blocks_total{builder="vanilla"}[24h])) / scalar(sum(increase(ebb_blocks_total[24h]))) + p99: 100 * sum(increase(ebb_blocks_total{builder="vanilla"}[24h])) / scalar(sum(increase(ebb_blocks_total[24h]))) + mean: 100 * sum(increase(ebb_blocks_total{builder="vanilla"}[24h])) / scalar(sum(increase(ebb_blocks_total[24h]))) + success: avg_over_time(ebb_eth_poll_health[24h]) + sample_size: sum(increase(ebb_blocks_total{builder="vanilla"}[24h])) + series: 100 * sum(rate(ebb_blocks_total{builder="vanilla"}[1h])) / scalar(sum(rate(ebb_blocks_total[1h]))) + + - slug: other + name: Other + tag: Unrecognized extraData tags, logged raw for table growth + formula: "Blocks whose extraData carries a non-empty tag not in the curated table, as a percentage of all blocks observed in 24h. Every raw string is logged so the attribution table grows." + queries: + p50: 100 * sum(increase(ebb_blocks_total{builder="other"}[24h])) / scalar(sum(increase(ebb_blocks_total[24h]))) + p90: 100 * sum(increase(ebb_blocks_total{builder="other"}[24h])) / scalar(sum(increase(ebb_blocks_total[24h]))) + p99: 100 * sum(increase(ebb_blocks_total{builder="other"}[24h])) / scalar(sum(increase(ebb_blocks_total[24h]))) + mean: 100 * sum(increase(ebb_blocks_total{builder="other"}[24h])) / scalar(sum(increase(ebb_blocks_total[24h]))) + success: avg_over_time(ebb_eth_poll_health[24h]) + sample_size: sum(increase(ebb_blocks_total{builder="other"}[24h])) + series: 100 * sum(rate(ebb_blocks_total{builder="other"}[1h])) / scalar(sum(rate(ebb_blocks_total[1h]))) diff --git a/benchmarks/indexer-latency.yml b/benchmarks/indexer-latency.yml new file mode 100644 index 00000000..5a44c1d1 --- /dev/null +++ b/benchmarks/indexer-latency.yml @@ -0,0 +1,128 @@ +# OpenChainBench. Bench № 084 + +slug: indexer-latency +number: "084" +title: Fastest blockchain indexer. Envio HyperSync, The Graph, Mobula +seo_title: "Fastest blockchain indexer 2026: HyperSync vs The Graph vs Mobula" +seo_description: "How fast do indexing pipelines make new onchain events queryable? Envio HyperSync, The Graph and Mobula measured live on organic Ethereum USDC transfers, second by second." +subtitle: Seconds between a USDC transfer confirming on Ethereum and the moment each indexing pipeline first makes that block's data queryable, measured continuously on organic transfers. +category: Aggregators +status: live +metric: Time to queryable +unit: sec +higher_is_better: false + +seo_intro: | + Every dapp backend, analytics product and trading system sits on an + indexing pipeline, and in 2026 the choice of that pipeline is a live + question again. Dune is sunsetting Sim, teams are re-evaluating + their data stack, and Envio markets HyperSync as "10x faster" than + alternatives without publishing comparable numbers. This benchmark + measures the claim the only way that cannot be gamed. We pick a + random, organic USDC transfer from the newest Ethereum block, then + poll each indexing pipeline until that block's data is queryable. + Because the ground truth is a random real transaction there is + nothing a provider could special case, and every measurement is + publicly re-verifiable from the tx hash. Cohort. Envio HyperSync + (low level log query engine), The Graph (decentralized subgraph + network, probed through the canonical Uniswap V3 subgraph) and + Mobula (wallet-history pipeline), all probed on the identical + schedule from the same host. This cohort is indexing + infrastructure; wallet data APIs (Zerion, Moralis, Mobula and + friends) are raced separately in bench 070, with zero provider + overlap between the two. + +abstract: | + We measure time to queryable for indexing infrastructure. The delay + between an organic USDC transfer confirming on Ethereum (T0 = the + instant our own keyless reference RPC first shows the containing + block) and the first poll at which each provider can return that + block's data. One probe event per 5 minutes; each event uses a fresh + random transfer from a fresh block. The poll schedule is front + loaded (1s to 240s, identical for every provider) and detection is + parser free where the API returns the tx hash verbatim. Because the + three pipelines expose different query surfaces, the probed entity + differs per provider and is disclosed. HyperSync is asked for the + exact Transfer log, Mobula for the exact transfer via its indexed + transfers API, The Graph for a block pinned query on the canonical + Uniswap V3 subgraph that only succeeds once the subgraph has indexed + the reference block. The measured question is identical in all three + cases. is block N data queryable at time T? Events not queryable + within 240 seconds count as timeouts and feed the reliability + column. Per provider monthly quota guards keep the probe inside + every free tier. + +methodology: + - "Ground truth: one probe event per 5 minutes. The harness watches new Ethereum blocks through its own keyless reference RPC (publicnode, never a cohort member's endpoint) and picks one random USDC Transfer log from the newest block. T0 is the instant the harness first observes the containing block; the same T0 is used for every provider." + - "Anti-gaming by construction: the probe target is a random organic USDC transfer, different on every event, so no provider can pre-warm a known benchmark entity. Every measurement is re-verifiable by anyone from the public tx hash. No canary contract is deployed; the benchmark piggybacks on the natural high frequency USDC stream at zero onchain cost." + - "Poll schedule: each provider is polled at 1, 2, 3, 5, 8, 12, 20, 30, 45, 60, 90, 120, 180 and 240 seconds after T0. Reported lag is the first poll at which block N's data is queryable, an upper bound with resolution equal to the gap between consecutive polls." + - "Probed entity per provider, disclosed because the pipelines expose different query surfaces: HyperSync is queried via POST /query for the exact Transfer log in block N; Mobula via its wallet-transactions endpoint for the transfer recipient captured from the log topics; The Graph via a block pinned GraphQL query on the canonical Uniswap V3 Ethereum subgraph, which graph-node rejects until the subgraph has indexed block N. The measured question is identical: is block N data queryable at time T." + - "The Graph number therefore includes the full subgraph pipeline (firehose, indexer sync, gateway routing) rather than a raw data lookup. That is the honest comparison: it is the path every subgraph consumer actually waits on, but readers should know it is a pipeline measurement, not a single database read." + - "Detection is parser free where possible: HyperSync and Mobula responses are scanned for the tx hash substring, so no provider gains or loses from response schema differences. The Graph probe succeeds when the block pinned query returns data without a block range error." + - "Classification: found (lag recorded), timeout (not queryable within 240s), error (provider errored on every poll of the event), quota_paused (monthly guard tripped), disabled (credential not configured). Percentiles are computed from the histogram of found lags over 24h; the timeout rate is published alongside because latency without reliability is a misleading ranking signal." + - "Quota fairness: per provider monthly call budgets sized to each free tier with headroom, enforced by a guard that pauses probing at 90 percent. The Graph joins every second event (100k gateway queries/month free plan); HyperSync and Mobula join every event. Sample sizes per provider are published." + - "Disclosure: Mobula operates OpenChainBench. Its poller runs the identical probe schedule, quota guard and tx hash detection as every other cohort member, and every measurement is re-verifiable from the public tx hash. Alchemy was measured during development but is excluded from the live cohort because probing it rides a paid plan, and this bench only races pipelines that cost nothing to probe; a dedicated free tier key would re-qualify it." + - "Excluded from v1 with reasons: Goldsky (requires deploying and paying for your own subgraph, no neutral shared surface to probe), Dune (2500 free credits/month cannot sustain continuous probing), Bitquery (free tier is a one month trial), Moralis (already raced in bench 070 and its free key is saturated there), SubQuery and Ponder (no measurable hosted free tier)." + - "Cohort positioning: this bench races indexing infrastructure. Wallet data APIs (Zerion, Moralis, Allium, GoldRush, Mobula) are a different product layer and are raced separately in bench 070 with an identical T0 and poll design; the two cohorts share zero providers by construction." + - "Scope: Ethereum mainnet, single probe region. Time to queryable is measured in seconds while cross region network deltas are milliseconds, so multi region probing would add cost without changing the ranking." + +findings: + - "{{best_name}} currently leads at {{best_p50}} (p50 of found lags, 24h) on organic Ethereum USDC transfers." + - "{{name:hypersync}} sits at {{p50:hypersync}}. Envio's marketing claims 10x faster retrieval than alternatives; this page is the first continuously measured public number behind that claim." + - "{{name:thegraph}} runs at {{p50:thegraph}} through the canonical Uniswap V3 subgraph. This is a full pipeline measurement (firehose, indexer, gateway), the path every subgraph consumer actually waits on." + - "{{name:mobula}} sits at {{p50:mobula}} on its wallet-history pipeline, the same surface portfolio and activity feeds built on Mobula consume." + - "Read the reliability column before the latency one: a pipeline that occasionally never surfaces an event within 240 seconds breaks downstream consumers in a way a slow median does not." + +faq: + - q: "Which blockchain indexer makes new events queryable fastest?" + a: "Per the live leaderboard above: {{best_name}} at {{best_p50}} (p50 over 24h of organic Ethereum USDC transfers). The ranking re-sorts continuously as probe events land every 5 minutes. Check the reliability column too; a pipeline that occasionally never surfaces an event within 240 seconds is worse for most consumers than one that is a second slower on median." + - q: "How is indexer latency measured here?" + a: "The harness picks a random organic USDC transfer from the newest Ethereum block, records T0 when its own keyless reference RPC first shows the block, then polls each provider on an identical front loaded schedule (1s to 240s) until that block's data is queryable. HyperSync and Mobula are asked for the exact transfer; The Graph is asked a block pinned query on the canonical Uniswap V3 subgraph that only succeeds once block N is indexed. The full harness is open source and each measurement can be re-verified from the public tx hash." + - q: "Is comparing HyperSync, The Graph and Mobula apples to apples?" + a: "Not perfectly, and the asymmetry is disclosed rather than hidden. HyperSync serves raw data and Mobula a wallet-scoped view of it; The Graph serves subgraph entities, so its number includes the whole firehose plus indexer plus gateway pipeline. All three answer the same operational question a builder has: after an event confirms onchain, how long until my query layer can see it. That is the number this page publishes, per pipeline, with the probed entity documented." + - q: "Why are Goldsky, Dune, SubQuery and Ponder missing?" + a: "Each lacks a neutral, continuously probeable free surface in v1. Goldsky requires deploying your own subgraph, so any probe would measure our deployment, not the platform baseline. Dune's free tier (2500 credits/month) cannot sustain a 5 minute cadence. Bitquery's free tier is a one month trial. SubQuery and Ponder have no measurable hosted free tier. Moralis is already raced in the wallet data cohort of bench 070. Providers change tiers often; the cohort is revisited when a fair probe becomes possible." + - q: "Why does indexer latency matter in 2026?" + a: "Because the indexing market is consolidating and re-forming at once. Dune is sunsetting Sim, pushing teams to re-evaluate their data stack, while Envio advertises HyperSync as 10x faster with no public comparable numbers. For trading systems, alerting and activity feeds, the seconds between confirmation and queryability are the product. This page is the only continuously measured, provider neutral comparison of that number across indexing infrastructure." + +source: https://github.com/ChainBench/OpenChainBench/tree/main/harnesses/indexer-latency + +prometheus: + window: 24h + freshness_metric: idx_health + expected_freshness_seconds: 1800 + +providers: + - slug: hypersync + name: Envio HyperSync + tag: Low level log query engine, claims 10x faster retrieval + formula: "Median (p50) over 24h of seconds between our reference RPC first showing an Ethereum block and HyperSync /query returning that block's probe USDC Transfer log, from the shared histogram estimator." + queries: + p50: histogram_quantile(0.5, sum by (le) (increase(idx_latency_seconds_bucket{provider="hypersync"}[24h]))) + p90: histogram_quantile(0.9, sum by (le) (increase(idx_latency_seconds_bucket{provider="hypersync"}[24h]))) + p99: histogram_quantile(0.99, sum by (le) (increase(idx_latency_seconds_bucket{provider="hypersync"}[24h]))) + success: sum(increase(idx_probe_total{provider="hypersync", result="found"}[24h])) / sum(increase(idx_probe_total{provider="hypersync", result=~"found|timeout"}[24h])) + sample_size: sum(increase(idx_probe_total{provider="hypersync", result=~"found|timeout"}[24h])) + series: avg_over_time(idx_latency_last_seconds{provider="hypersync"}[1h]) + - slug: thegraph + name: The Graph + tag: Decentralized subgraph network, probed via Uniswap V3 subgraph + formula: "Median (p50) over 24h of seconds until a block pinned query on the canonical Uniswap V3 subgraph succeeds for the reference block. Includes the full firehose, indexer and gateway pipeline; joins every second event." + queries: + p50: histogram_quantile(0.5, sum by (le) (increase(idx_latency_seconds_bucket{provider="thegraph"}[24h]))) + p90: histogram_quantile(0.9, sum by (le) (increase(idx_latency_seconds_bucket{provider="thegraph"}[24h]))) + p99: histogram_quantile(0.99, sum by (le) (increase(idx_latency_seconds_bucket{provider="thegraph"}[24h]))) + success: sum(increase(idx_probe_total{provider="thegraph", result="found"}[24h])) / sum(increase(idx_probe_total{provider="thegraph", result=~"found|timeout"}[24h])) + sample_size: sum(increase(idx_probe_total{provider="thegraph", result=~"found|timeout"}[24h])) + series: avg_over_time(idx_latency_last_seconds{provider="thegraph"}[1h]) + - slug: mobula + name: Mobula + tag: Wallet-history indexing pipeline (Octopus engine) + formula: "Median (p50) over 24h of seconds between our reference RPC first showing a block and Mobula's wallet-transactions endpoint returning that block's probe USDC transfer for the recipient wallet, from the shared histogram estimator." + queries: + p50: histogram_quantile(0.5, sum by (le) (increase(idx_latency_seconds_bucket{provider="mobula"}[24h]))) + p90: histogram_quantile(0.9, sum by (le) (increase(idx_latency_seconds_bucket{provider="mobula"}[24h]))) + p99: histogram_quantile(0.99, sum by (le) (increase(idx_latency_seconds_bucket{provider="mobula"}[24h]))) + success: sum(increase(idx_probe_total{provider="mobula", result="found"}[24h])) / sum(increase(idx_probe_total{provider="mobula", result=~"found|timeout"}[24h])) + sample_size: sum(increase(idx_probe_total{provider="mobula", result=~"found|timeout"}[24h])) + series: avg_over_time(idx_latency_last_seconds{provider="mobula"}[1h]) diff --git a/benchmarks/l2-block-time.yml b/benchmarks/l2-block-time.yml index 17199113..e65d3f81 100644 --- a/benchmarks/l2-block-time.yml +++ b/benchmarks/l2-block-time.yml @@ -2,12 +2,16 @@ slug: l2-block-time number: "009" -title: Fastest L2 block time, live across Arbitrum, Base, Optimism and 6 more +title: Fastest L2 block time, live across Robinhood, Arbitrum, Base and 7 more seo_title: "Fastest L2 block time 2026" -seo_description: "Fastest L2 sequencer block time live: Arbitrum, Blast, Optimism, Base, Linea and more ranked by newHeads interval." +seo_description: "Fastest L2 sequencer block time live: Robinhood Chain, Arbitrum, Optimism, Base, Linea and more ranked by real block interval." subtitle: Wall-clock interval in milliseconds between two consecutive newHeads events on each L2 sequencer, refreshed continuously. per_chain_explainer: + - slug: robinhood + h2: "Robinhood Chain block time" + body: | + Robinhood Chain block time is {{p50:robinhood}} (p50, 24h), the most aggressive cadence in the cohort: the Arbitrum Orbit stack lets Robinhood's sequencer target ~100 ms blocks for tokenized-stock settlement. The chain launched mainnet on July 1, 2026, pays gas in ETH and settles to Ethereum blobs. Measurement note: Robinhood exposes no public WebSocket, so block time here is derived from block-number deltas over 2-second HTTP poll windows, an accurate average that does not capture per-block jitter. - slug: arbitrum h2: "Arbitrum One block time" body: | @@ -66,6 +70,7 @@ abstract: | outlier sample is capped at 5 min so a reconnect can't pollute p50. methodology: + - "Robinhood Chain exception: the sequencer exposes no public WebSocket, so its block time is derived from eth_blockNumber deltas polled every 2 seconds over HTTP. A window that advanced N blocks contributes N samples of (elapsed / N) ms, which preserves per-block weighting and yields an accurate average; per-block jitter is not captured on this chain and this note is the disclosure." - "Cadence: continuous WebSocket subscription per L2, one persistent connection, no polling." - "Subscribe payload: `{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"eth_subscribe\",\"params\":[\"newHeads\"]}`. No API key, public endpoints (publicnode.com mostly, era.zksync.io/ws for zkSync)." - "Sample: block-time(N) = T_recv(N) − T_recv(N-1), measured in milliseconds with `time.Now()` precision on the harness clock. Block-number duplicates from the same connection are filtered." @@ -101,6 +106,18 @@ faq: a: "Polygon zkEVM and Mode don't expose a public WebSocket endpoint without an API key, so we defer them to v2 when an API-keyed contributor endpoint lands. Starknet uses the Cairo stack with a different RPC shape, it would live in a dedicated bench rather than be shoehorned into the EVM cohort here. Berachain, Sonic and other adjacent chains are L1s, not L2s, so they live in the l1-finality bench." providers: + - slug: robinhood + name: Robinhood Chain + tag: Arbitrum Orbit, ~100ms blocks, HTTP-polled (no public WSS) + formula: "Median milliseconds per block on Robinhood Chain, derived from eth_blockNumber deltas over 2s HTTP poll windows (no public WebSocket), p50 over 24h." + queries: + p50: histogram_quantile(0.50, sum(rate(l2_block_time_milliseconds_histogram_bucket{chain="robinhood"}[24h])) by (le)) + p90: histogram_quantile(0.90, sum(rate(l2_block_time_milliseconds_histogram_bucket{chain="robinhood"}[24h])) by (le)) + p99: histogram_quantile(0.99, sum(rate(l2_block_time_milliseconds_histogram_bucket{chain="robinhood"}[24h])) by (le)) + mean: sum(rate(l2_block_time_milliseconds_histogram_sum{chain="robinhood"}[24h])) / sum(rate(l2_block_time_milliseconds_histogram_count{chain="robinhood"}[24h])) + success: avg_over_time(l2_block_time_health{chain="robinhood"}[24h]) + sample_size: sum(increase(l2_block_time_samples_total{chain="robinhood"}[24h])) + series: avg_over_time(l2_block_time_milliseconds{chain="robinhood"}[1h]) - slug: arbitrum name: Arbitrum One tag: Nitro fast-finality, sub-second sequencer diff --git a/benchmarks/network-fees.yml b/benchmarks/network-fees.yml index 36e65e3a..730f3900 100644 --- a/benchmarks/network-fees.yml +++ b/benchmarks/network-fees.yml @@ -2,10 +2,10 @@ slug: network-fees number: "031" -title: Cheapest blockchain transaction fee, live across 14 L1 and L2 chains +title: Cheapest blockchain transaction fee, live across 15 L1 and L2 chains seo_title: "Cheapest crypto transaction fee 2026" -seo_description: "Cheapest blockchain transaction fee in USD live across 14 chains. Solana, Base, Arbitrum, BNB Chain ranked." -subtitle: "Live USD cost of one native token transaction on 14 Layer 1 and Layer 2 chains, refreshed every 30 seconds." +seo_description: "Cheapest blockchain transaction fee in USD live across 15 chains. Solana, Base, Robinhood Chain, Arbitrum, BNB ranked." +subtitle: "Live USD cost of one native token transaction on 15 Layer 1 and Layer 2 chains, refreshed every 30 seconds." seo_intro: | This page answers one question. How much does it cost in dollars to send one transaction on each major blockchain right now. We track 14 chains in parallel and refresh the number every 30 seconds. The eight Layer 1 chains are Ethereum, Solana, BNB Chain, TRON, Cardano, Sui, Litecoin and Monero. The six Layer 2 rollups are Arbitrum, Base, zkSync Era, Linea, Mantle and Taiko. For every chain we query its own fee market directly (eth_feeHistory for the EVM family, getRecentPrioritizationFees on Solana, koios epoch params on Cardano, get_fee_estimate on Monero, the mempool oracle on Litecoin, getChainParameters on TRON, suix_getReferenceGasPrice on Sui), convert the result to the smallest native unit, then multiply by the live USD price of the chain's native token from Mobula. The output is the actual dollar amount a wallet user pays today. No gwei to lamport conversion, no marketing claim. Compare Ethereum gas now versus Solana fee in USD, see whether Arbitrum is still cheaper than Base today, find out which Layer 1 has the lowest transaction cost this minute. @@ -71,6 +71,10 @@ per_chain_explainer: h2: "Arbitrum transaction fee" body: | Arbitrum One native transfer fee is {{p50:arbitrum}} (p50, 24h). Arbitrum Nitro runs an EIP-1559 fee market against the sequencer at sub-second cadence; native ETH transfers are 21000 gas and sit at fractions of a cent because the sequencer's priority fee is near-zero by design. The L1 data-posting component (Ethereum blobs since Dencun) is not yet included in this figure, only L2 execution cost. Computed via `eth_feeHistory` on the Arbitrum sequencer RPC times live ETH price. + - slug: robinhood + h2: "Robinhood Chain transaction fee" + body: | + Robinhood Chain native transfer fee is {{p50:robinhood}} (p50, 24h). Robinhood's Arbitrum Orbit rollup posts a very low base fee (~0.05 gwei) and zero priority rewards, the typical Orbit pattern where the centralized sequencer needs no tip market; ETH is the gas asset and the L1 blob-posting cost is excluded from this figure like every other L2 on this page. Computed via `eth_feeHistory` on the official sequencer RPC times live ETH price. - slug: base h2: "Base transaction fee" body: | @@ -264,6 +268,18 @@ providers: mean: avg_over_time(tx_fee_native_transfer_usd{chain="arbitrum",tier="std"}[24h]) success: avg_over_time(tx_fee_health{chain="arbitrum"}[24h]) series: tx_fee_native_transfer_usd{chain="arbitrum",tier="std"} + - slug: robinhood + name: Robinhood Chain + layer: l2 + tag: "Arbitrum Orbit rollup Robinhood, 21000 gas" + formula: "Median USD cost of a native ETH transfer on Robinhood Chain over 24h. L2 execution cost only, L1 data posting fee excluded for now." + queries: + p50: quantile_over_time(0.50, tx_fee_native_transfer_usd{chain="robinhood",tier="std"}[24h]) + p90: quantile_over_time(0.90, tx_fee_native_transfer_usd{chain="robinhood",tier="fast"}[24h]) + p99: quantile_over_time(0.99, tx_fee_native_transfer_usd{chain="robinhood",tier="fast"}[24h]) + mean: avg_over_time(tx_fee_native_transfer_usd{chain="robinhood",tier="std"}[24h]) + success: avg_over_time(tx_fee_health{chain="robinhood"}[24h]) + series: tx_fee_native_transfer_usd{chain="robinhood",tier="std"} - slug: base name: Base layer: l2 diff --git a/benchmarks/oracle-deviation.yml b/benchmarks/oracle-deviation.yml index 79e38f0c..6f3ed8dd 100644 --- a/benchmarks/oracle-deviation.yml +++ b/benchmarks/oracle-deviation.yml @@ -135,7 +135,7 @@ providers: p99: quantile_over_time(0.99, ocb_oracle_max_deviation_pct{pair="BTC/USD"}[24h]) * 100 mean: avg_over_time(ocb_oracle_max_deviation_pct{pair="BTC/USD"}[24h]) * 100 success: clamp_min(clamp_max(1 - sum(rate(ocb_oracle_scrape_errors_total{pair="BTC/USD"}[24h])) / (4 * (1/30)), 1), 0) - sample_size: count(count by (source) (ocb_oracle_price{pair="BTC/USD"})) + sample_size: sum(count_over_time(ocb_oracle_price{pair="BTC/USD"}[24h])) series: ocb_oracle_max_deviation_pct{pair="BTC/USD"} * 100 - slug: eth-usd @@ -148,7 +148,7 @@ providers: p99: quantile_over_time(0.99, ocb_oracle_max_deviation_pct{pair="ETH/USD"}[24h]) * 100 mean: avg_over_time(ocb_oracle_max_deviation_pct{pair="ETH/USD"}[24h]) * 100 success: clamp_min(clamp_max(1 - sum(rate(ocb_oracle_scrape_errors_total{pair="ETH/USD"}[24h])) / (4 * (1/30)), 1), 0) - sample_size: count(count by (source) (ocb_oracle_price{pair="ETH/USD"})) + sample_size: sum(count_over_time(ocb_oracle_price{pair="ETH/USD"}[24h])) series: ocb_oracle_max_deviation_pct{pair="ETH/USD"} * 100 - slug: sol-usd @@ -161,7 +161,7 @@ providers: p99: quantile_over_time(0.99, ocb_oracle_max_deviation_pct{pair="SOL/USD"}[24h]) * 100 mean: avg_over_time(ocb_oracle_max_deviation_pct{pair="SOL/USD"}[24h]) * 100 success: clamp_min(clamp_max(1 - sum(rate(ocb_oracle_scrape_errors_total{pair="SOL/USD"}[24h])) / (4 * (1/30)), 1), 0) - sample_size: count(count by (source) (ocb_oracle_price{pair="SOL/USD"})) + sample_size: sum(count_over_time(ocb_oracle_price{pair="SOL/USD"}[24h])) series: ocb_oracle_max_deviation_pct{pair="SOL/USD"} * 100 - slug: bnb-usd @@ -174,7 +174,7 @@ providers: p99: quantile_over_time(0.99, ocb_oracle_max_deviation_pct{pair="BNB/USD"}[24h]) * 100 mean: avg_over_time(ocb_oracle_max_deviation_pct{pair="BNB/USD"}[24h]) * 100 success: clamp_min(clamp_max(1 - sum(rate(ocb_oracle_scrape_errors_total{pair="BNB/USD"}[24h])) / (4 * (1/30)), 1), 0) - sample_size: count(count by (source) (ocb_oracle_price{pair="BNB/USD"})) + sample_size: sum(count_over_time(ocb_oracle_price{pair="BNB/USD"}[24h])) series: ocb_oracle_max_deviation_pct{pair="BNB/USD"} * 100 - slug: avax-usd @@ -187,7 +187,7 @@ providers: p99: quantile_over_time(0.99, ocb_oracle_max_deviation_pct{pair="AVAX/USD"}[24h]) * 100 mean: avg_over_time(ocb_oracle_max_deviation_pct{pair="AVAX/USD"}[24h]) * 100 success: clamp_min(clamp_max(1 - sum(rate(ocb_oracle_scrape_errors_total{pair="AVAX/USD"}[24h])) / (4 * (1/30)), 1), 0) - sample_size: count(count by (source) (ocb_oracle_price{pair="AVAX/USD"})) + sample_size: sum(count_over_time(ocb_oracle_price{pair="AVAX/USD"}[24h])) series: ocb_oracle_max_deviation_pct{pair="AVAX/USD"} * 100 - slug: link-usd @@ -200,7 +200,7 @@ providers: p99: quantile_over_time(0.99, ocb_oracle_max_deviation_pct{pair="LINK/USD"}[24h]) * 100 mean: avg_over_time(ocb_oracle_max_deviation_pct{pair="LINK/USD"}[24h]) * 100 success: clamp_min(clamp_max(1 - sum(rate(ocb_oracle_scrape_errors_total{pair="LINK/USD"}[24h])) / (4 * (1/30)), 1), 0) - sample_size: count(count by (source) (ocb_oracle_price{pair="LINK/USD"})) + sample_size: sum(count_over_time(ocb_oracle_price{pair="LINK/USD"}[24h])) series: ocb_oracle_max_deviation_pct{pair="LINK/USD"} * 100 - slug: matic-usd @@ -213,7 +213,7 @@ providers: p99: quantile_over_time(0.99, ocb_oracle_max_deviation_pct{pair="MATIC/USD"}[24h]) * 100 mean: avg_over_time(ocb_oracle_max_deviation_pct{pair="MATIC/USD"}[24h]) * 100 success: clamp_min(clamp_max(1 - sum(rate(ocb_oracle_scrape_errors_total{pair="MATIC/USD"}[24h])) / (4 * (1/30)), 1), 0) - sample_size: count(count by (source) (ocb_oracle_price{pair="MATIC/USD"})) + sample_size: sum(count_over_time(ocb_oracle_price{pair="MATIC/USD"}[24h])) series: ocb_oracle_max_deviation_pct{pair="MATIC/USD"} * 100 - slug: xrp-usd @@ -226,7 +226,7 @@ providers: p99: quantile_over_time(0.99, ocb_oracle_max_deviation_pct{pair="XRP/USD"}[24h]) * 100 mean: avg_over_time(ocb_oracle_max_deviation_pct{pair="XRP/USD"}[24h]) * 100 success: clamp_min(clamp_max(1 - sum(rate(ocb_oracle_scrape_errors_total{pair="XRP/USD"}[24h])) / (3 * (1/30)), 1), 0) - sample_size: count(count by (source) (ocb_oracle_price{pair="XRP/USD"})) + sample_size: sum(count_over_time(ocb_oracle_price{pair="XRP/USD"}[24h])) series: ocb_oracle_max_deviation_pct{pair="XRP/USD"} * 100 - slug: ada-usd @@ -239,7 +239,7 @@ providers: p99: quantile_over_time(0.99, ocb_oracle_max_deviation_pct{pair="ADA/USD"}[24h]) * 100 mean: avg_over_time(ocb_oracle_max_deviation_pct{pair="ADA/USD"}[24h]) * 100 success: clamp_min(clamp_max(1 - sum(rate(ocb_oracle_scrape_errors_total{pair="ADA/USD"}[24h])) / (3 * (1/30)), 1), 0) - sample_size: count(count by (source) (ocb_oracle_price{pair="ADA/USD"})) + sample_size: sum(count_over_time(ocb_oracle_price{pair="ADA/USD"}[24h])) series: ocb_oracle_max_deviation_pct{pair="ADA/USD"} * 100 - slug: doge-usd @@ -252,5 +252,5 @@ providers: p99: quantile_over_time(0.99, ocb_oracle_max_deviation_pct{pair="DOGE/USD"}[24h]) * 100 mean: avg_over_time(ocb_oracle_max_deviation_pct{pair="DOGE/USD"}[24h]) * 100 success: clamp_min(clamp_max(1 - sum(rate(ocb_oracle_scrape_errors_total{pair="DOGE/USD"}[24h])) / (3 * (1/30)), 1), 0) - sample_size: count(count by (source) (ocb_oracle_price{pair="DOGE/USD"})) + sample_size: sum(count_over_time(ocb_oracle_price{pair="DOGE/USD"}[24h])) series: ocb_oracle_max_deviation_pct{pair="DOGE/USD"} * 100 diff --git a/benchmarks/oracle-freshness.yml b/benchmarks/oracle-freshness.yml new file mode 100644 index 00000000..67aa244d --- /dev/null +++ b/benchmarks/oracle-freshness.yml @@ -0,0 +1,167 @@ +# OpenChainBench. Bench № 082 + +slug: oracle-freshness +number: "082" +title: Chainlink vs Pyth vs RedStone, live oracle freshness +seo_title: "Chainlink vs Pyth vs RedStone oracle staleness 2026" +seo_description: "How old is the oracle price your protocol settles on? Chainlink (Ethereum, Arbitrum, Base), Pyth Hermes and RedStone ranked live by ETH/USD staleness in seconds, p50/p99 over 24h, keyless and reproducible." +subtitle: Seconds since each oracle's own last-update timestamp for ETH/USD, polled every 30 seconds across Chainlink on three chains, Pyth via Hermes and RedStone via its public gateway. Paired with a stale-but-moved flag that fires only when an old price has also been left behind by the market. +category: Trading +status: live +metric: Oracle staleness +unit: sec +higher_is_better: false + +disclaimer: | + Read before ranking. The three oracles use different update models, and that difference IS the measurement. Chainlink pushes on-chain only on a deviation trigger or heartbeat, so hundreds of seconds of staleness in a calm market is by design, not failure. Pyth and RedStone publish continuously off-chain and are pulled on demand. The dangerous state is not "old", it is "old while the market moved": that is the companion `ocb_oracle_stale_but_moved` signal, not the headline. + +seo_intro: | + Every lending market, perp DEX and stablecoin issuer settles on an + oracle price that is, by construction, some number of seconds old. + This page publishes that number live. We read each oracle's own + update timestamp (not our fetch time): Chainlink's on-chain + `latestRoundData().updatedAt` on Ethereum, Arbitrum and Base, Pyth's + Hermes `publish_time`, and the timestamp of RedStone's latest signed + data package, and publish `now - last_update` as a Prometheus gauge + in seconds, quantiled over 24 hours. Why it matters: liquidation + cascades are a freshness problem before they are a price problem. + When ETH drops 3% and the on-chain feed has not updated for ten + minutes, underwater positions sit unliquidated, accruing bad debt; + when the round finally lands, every overdue liquidation fires in the + same block, keepers race, gas spikes, and the forced selling deepens + the move that caused it. The honest complication, stated up front: + Chainlink is a push oracle whose feeds are SUPPOSED to be stale in + calm markets (updates cost gas, so they land only on a 0.5% + deviation or a 1 hour heartbeat), while Pyth and RedStone are + pull-model oracles whose off-chain publication runs at 1 to 25 + second cadence and whose on-chain freshness depends on who pulls. + We are comparing different update models on the same clock, which is + exactly the design decision an integrator is making when they pick + one. The signal that separates "calm market, by design" from "your + liquidation engine is trading on a fossil" is the stale-but-moved + flag: staleness above 300 seconds while the CEX print has moved more + than 0.5% since the oracle's last update. + +abstract: | + We measure oracle freshness by reading each oracle's own + last-update timestamp every 30 seconds and publishing + `ocb_oracle_staleness_seconds{oracle, pair, chain}` (refreshed every + 5 seconds in-harness so the gauge climbs monotonically between + polls). Timestamps come from the source, never from our clock: + Chainlink's `latestRoundData().updatedAt` via keyless `eth_call` on + Ethereum, Arbitrum and Base; Pyth's `publish_time` from the Hermes + REST batch endpoint (Pyth is pull-based, so Hermes IS the price an + integrator would put on-chain, making publish_time the honest + freshness measure); RedStone's signed data-package timestamp from + its keyless per-symbol API. Two companion series complete the + picture: `ocb_oracle_update_events_total` counts observed update + timestamp advances, and `ocb_oracle_stale_but_moved` flags the + dangerous state, staleness above 300 s while the Binance/Coinbase + reference price moved more than 0.5% since the update. The headline + ranks the three oracles by p50 ETH/USD staleness over 24 h + (Chainlink pinned to its Ethereum mainnet feed; the Arbitrum and + Base series are queryable under the `chain` label). Lower is + fresher, but the disclaimer applies: push and pull oracles are + different machines, and the leaderboard quantifies the trade-off + rather than declaring one model wrong. + +methodology: + - "Staleness definition. `staleness = now - last_update_ts`, where last_update_ts is the oracle's OWN declared timestamp, never our fetch time: Chainlink's on-chain `latestRoundData().updatedAt`, Pyth's Hermes `publish_time`, RedStone's signed data-package `timestamp`. Published as `ocb_oracle_staleness_seconds{oracle, pair, chain}` in seconds." + - "Chainlink leg. Keyless `eth_call(latestRoundData())` against the canonical ETH/USD AggregatorV3 contracts on three chains: Ethereum mainnet (0x5f4eC3Df...8419), Arbitrum One (0x639Fe6ab...a612) and Base (0x71041ddd...Bb70), via public RPCs with a fallback per chain. The headline pins `chain=\"ethereum\"`; Arbitrum and Base are live under the same metric's `chain` label." + - "Pyth leg. Hermes REST batch endpoint, no key. Pyth is a pull oracle: there is no continuously-updated on-chain price to read, integrators fetch a signed update from Hermes and post it in their own transaction. Hermes `publish_time` is therefore the freshness of the price you would actually put on-chain, and that is what we measure (`chain=\"hermes\"`). At fetch time the publish_time is typically 1-2 s old; see the observation-lag bullet for what the quantiles report." + - "RedStone leg. Keyless per-symbol price API (`provider=redstone`, the primary production service), which returns the latest signed data package's value and timestamp (`chain=\"gateway\"`). We poll per symbol because the full data-packages gateway payload for the service is ~1.7 MB per request; the per-symbol object is a few hundred bytes. Packages are produced on a roughly 10 s cadence; at fetch time the package timestamp is typically 15-25 s old." + - "Cadence and aggregation. Every (oracle, pair, chain) is polled every 30 s; the staleness gauge is additionally re-published every 5 s in-harness so it climbs between polls instead of stair-stepping. Leaderboard numbers are `quantile_over_time` p50/p90/p99 over 24 h. Total added load is ~8 requests/minute on top of the host harness's 80, all keyless." + - "Observation lag, stated honestly. For oracles that publish faster than our 30 s poll (Pyth, RedStone), the gauge keeps aging until the next poll, so reported quantiles sit about half a poll interval above the true publication age: Pyth's p50 reads mid-teens of seconds even though Hermes is 1-2 s old at every fetch. The numbers are upper bounds with a shared floor set by our cadence, negligible against push-feed staleness measured in hundreds of seconds." + - "Update events. `ocb_oracle_update_events_total` increments each time the source-declared timestamp moves forward between observations. With a 30 s poll it undercounts sources that update faster than the poll (Pyth publishes near-continuously), so treat it as a lower bound on update frequency; it is exact for Chainlink's minutes-to-hours round cadence." + - "Stale-but-moved. `ocb_oracle_stale_but_moved` is 1 when staleness exceeds 300 s AND the CEX reference (Binance, Coinbase fallback, from the same harness's bench 025 pollers) has moved more than 0.5% since the update was observed. 0.5% matches the widest deviation trigger configured on the measured Chainlink feeds: if the market moved past the feed's own trigger and no round landed, the feed is late by its own standard." + - "Threshold rationale. 300 s sits below Chainlink's 1 h heartbeat and above healthy deviation-triggered gaps (feasibility runs saw fresh rounds 398-1069 s apart on quiet hours, sub-minute during moves). Alone it flags nothing; with the 0.5% move condition it isolates the window where liquidations execute on a price the market has left. Cold-start caveat: after a restart the CEX reference snapshots at boot, converging on the first real update." + - "Push vs pull is the comparison, not a flaw. Chainlink pays gas per update so it updates sparsely by design; Pyth and RedStone publish off-chain continuously and shift the on-chain freshness question to whoever pulls. Ranking them on the same staleness clock quantifies the trade-off an integrator accepts with each model. The per-oracle tag states the model so the table is read correctly." + - "Isolation from bench 025. This bench shares the oracle-deviation harness but emits NEW metric families with an `oracle`/`chain` label set. Bench 025's metrics, labels and error counters are untouched; freshness-only sources (RedStone, Arbitrum/Base Chainlink) never enter 025's deviation store, and freshness scrape failures land in `ocb_oracle_freshness_scrape_errors_total`, not 025's error counter." + - "Excluded in v1, with reasons. Chronicle: reads are whitelist-gated on-chain (`toll` access control), so there is no keyless public read to reproduce. API3: the measured-pair proxy addresses we probed were dead or unverifiable without an integration contract. Both are candidates for a follow-up once a reproducible keyless read path exists. Uniswap TWAP is out of scope: it is a derivation of the same CEX prints, not an oracle network with its own update timestamp." + - "Reproducibility. Every leg is a keyless public endpoint: `eth_call` with selector 0xfeaf968c on the published feed addresses, the Hermes batch REST endpoint, and the RedStone per-symbol API. Anyone can replay any sample with curl and get the same timestamps we do." + +findings: + - "{{best_name}} is currently the freshest at {{best_p50}} (p50 ETH/USD staleness, 24 h) across {{count}} measured oracles. Remember the disclaimer before reading this as a quality ranking: the leader is fresh because its update model publishes continuously off-chain, not because the others are broken." + - "{{name:pyth}} sits at {{p50:pyth}}. Hermes aggregates publisher prices at sub-second cadence and its publish_time is 1-2 s old at every fetch; the reported p50 is higher because our gauge ages between 30 s polls (upper-bound semantics, see methodology). Two honest asterisks, then: the number carries our observation lag, and the on-chain freshness of a Pyth integration is ultimately set by how often the integrating protocol pulls." + - "{{name:chainlink}} reads {{p50:chainlink}} on the Ethereum mainnet ETH/USD feed. That number is the deviation-trigger economics made visible: rounds land only when the off-chain aggregate drifts ~0.5% or the 1 h heartbeat expires, so calm hours accumulate hundreds of seconds of staleness by design. The Arbitrum and Base deployments of the same feed are published under the chain label for comparison." + - "{{name:redstone}} lands at {{p50:redstone}}. RedStone's primary production service signs a fresh data package roughly every 10 s, and its per-symbol API served packages 15-25 s old at fetch during feasibility: slower than Hermes, orders of magnitude fresher than a push feed between rounds, with the same observation-lag and pull-model asterisks as Pyth." + - "The alarm metric is not on this leaderboard. `ocb_oracle_stale_but_moved` fires only when a feed is over 300 s old AND the CEX print moved over 0.5% since its last update, which is the state where liquidation engines execute on a price the market has left. In calm markets it should read 0 for every oracle, including a 900-second-stale Chainlink feed." + +faq: + - q: "What is oracle staleness and why should a protocol care?" + a: "Staleness is the age of the oracle price at the moment your contract reads it: now minus the oracle's own last-update timestamp. Every on-chain settlement (a liquidation, a mint, a perp funding tick) executes at that possibly-old price. If ETH moves 3% while a feed sits unchanged for ten minutes, positions that should liquidate do not, bad debt accrues, and when the update finally lands every overdue liquidation fires in the same block. That burst, keepers racing to close stale-priced positions all at once, is how freshness problems become liquidation cascades. This bench publishes the age live, per oracle, so the risk is sized before integration." + - q: "Chainlink shows hundreds of seconds of staleness. Is it broken?" + a: "No, and this is the most important caveat on the page. Chainlink is a push oracle: node operators pay gas to post each round on-chain, so feeds are configured to update only when the off-chain aggregate drifts past a deviation trigger (0.5% on the measured ETH/USD feeds) or a heartbeat expires (1 hour). Between triggers the on-chain price legitimately ages. A 900-second-old Chainlink price in a flat market is the system working as designed and costs integrators nothing. The failure mode worth alarming on is captured by the separate stale_but_moved flag: an old price after the market has moved past the feed's own trigger threshold." + - q: "Why is Pyth measured at Hermes instead of on-chain, and is that fair?" + a: "Because pull oracles have no continuously-updated on-chain price to read. A Pyth integrator fetches a signed price update from Hermes and posts it inside their own transaction; the on-chain contract only stores whatever was last pushed by someone. Measuring Pyth's freshness on-chain would measure the pull habits of whichever protocols happen to share the feed, not the oracle. Hermes publish_time is the age of the price actually available to any integrator at the moment of the pull, which is the like-for-like comparison against Chainlink's updatedAt. The trade-off is stated openly: Hermes is 1-2 s old at pull time (our reported quantiles add up to 30 s of observation lag from the poll cadence), and a protocol that pulls lazily inherits its own staleness on top." + - q: "What exactly does stale_but_moved measure and why 300 seconds and 0.5%?" + a: "It is 1 when two conditions hold at once: the oracle's price is older than 300 seconds, and the CEX reference (Binance, Coinbase fallback) has moved more than 0.5% since that update was observed. 0.5% mirrors the widest deviation trigger on the measured Chainlink feeds, so the flag fires precisely when a feed is late by its own configured standard. 300 s sits above every healthy deviation-triggered gap we observed during fast markets and far below the 1 h heartbeat, isolating genuine lag without penalizing calm-market aging. Both thresholds are documented constants in the harness source and can be recomputed from the raw staleness and price series." + - q: "Why are Chronicle and API3 not on the leaderboard?" + a: "Both failed the bench's reproducibility bar for v1. Chronicle gates on-chain reads behind a whitelist (its toll access control), so there is no keyless public read a third party can replay. For API3, the measured-pair proxy addresses we probed were dead or unverifiable without deploying an integration contract. OpenChainBench only publishes numbers anyone can reproduce with curl and a public RPC, so both are excluded and documented rather than measured badly. If either ships a keyless read path for the measured pairs, they are straightforward to add: the harness's freshness tracker is oracle-agnostic." + +source: https://github.com/ChainBench/OpenChainBench/tree/main/harnesses/oracle-deviation + +prometheus: + window: 24h + freshness_metric: ocb_oracle_staleness_seconds + +# Metric families emitted by the oracle-deviation harness for this bench +# (bench 025's families are separate and untouched): +# ocb_oracle_staleness_seconds{oracle, pair, chain} gauge (s) +# ocb_oracle_update_events_total{oracle, pair, chain} counter +# ocb_oracle_stale_but_moved{oracle, pair, chain} gauge (0/1) +# ocb_oracle_freshness_scrape_errors_total{oracle, pair, chain} counter +# +# oracle is "chainlink" | "pyth" | "redstone". chain is where the +# freshness timestamp lives: ethereum | arbitrum | base for Chainlink +# deployments, "hermes" for Pyth (publish_time), "gateway" for +# RedStone (signed data-package timestamp). +# +# The headline ranks the three oracles by p50 ETH/USD staleness over +# 24h; Chainlink is pinned to its Ethereum mainnet feed and its +# Arbitrum + Base series remain queryable under the chain label. +# Success rate divides freshness scrape failures by the 30s poll +# cadence (1 expected attempt per 30s per series). + +providers: + - slug: chainlink + name: Chainlink + tag: "Push onchain · ETH/USD · ethereum, arbitrum, base" + type: protocol + formula: "p50 over 24h of seconds since the Chainlink ETH/USD aggregator's on-chain updatedAt on Ethereum mainnet. Deviation-triggered push feed: high staleness in calm markets is by design. Arbitrum and Base series live under the chain label." + queries: + p50: quantile_over_time(0.50, ocb_oracle_staleness_seconds{oracle="chainlink", pair="ETH/USD", chain="ethereum"}[24h]) + p90: quantile_over_time(0.90, ocb_oracle_staleness_seconds{oracle="chainlink", pair="ETH/USD", chain="ethereum"}[24h]) + p99: quantile_over_time(0.99, ocb_oracle_staleness_seconds{oracle="chainlink", pair="ETH/USD", chain="ethereum"}[24h]) + mean: avg_over_time(ocb_oracle_staleness_seconds{oracle="chainlink", pair="ETH/USD", chain="ethereum"}[24h]) + success: clamp_min(clamp_max(1 - sum(rate(ocb_oracle_freshness_scrape_errors_total{oracle="chainlink", pair="ETH/USD", chain="ethereum"}[24h])) / (1/30), 1), 0) + sample_size: count_over_time(ocb_oracle_staleness_seconds{oracle="chainlink", pair="ETH/USD", chain="ethereum"}[24h]) + series: ocb_oracle_staleness_seconds{oracle="chainlink", pair="ETH/USD", chain="ethereum"} + + - slug: pyth + name: Pyth + tag: "Pull via Hermes · ETH/USD · publish_time" + type: protocol + formula: "p50 over 24h of seconds since the Hermes publish_time of Pyth's ETH/USD feed. Pull oracle: Hermes is the price an integrator would post on-chain, so publish_time is the freshness of what is available to pull." + queries: + p50: quantile_over_time(0.50, ocb_oracle_staleness_seconds{oracle="pyth", pair="ETH/USD", chain="hermes"}[24h]) + p90: quantile_over_time(0.90, ocb_oracle_staleness_seconds{oracle="pyth", pair="ETH/USD", chain="hermes"}[24h]) + p99: quantile_over_time(0.99, ocb_oracle_staleness_seconds{oracle="pyth", pair="ETH/USD", chain="hermes"}[24h]) + mean: avg_over_time(ocb_oracle_staleness_seconds{oracle="pyth", pair="ETH/USD", chain="hermes"}[24h]) + success: clamp_min(clamp_max(1 - sum(rate(ocb_oracle_freshness_scrape_errors_total{oracle="pyth", pair="ETH/USD", chain="hermes"}[24h])) / (1/30), 1), 0) + sample_size: count_over_time(ocb_oracle_staleness_seconds{oracle="pyth", pair="ETH/USD", chain="hermes"}[24h]) + series: ocb_oracle_staleness_seconds{oracle="pyth", pair="ETH/USD", chain="hermes"} + + - slug: redstone + name: RedStone + tag: "Modular push-pull · ETH/USD · signed package timestamp" + type: protocol + formula: "p50 over 24h of seconds since the timestamp of RedStone's latest signed ETH data package on the keyless per-symbol API (primary production service). Packages are produced on a roughly 10s cadence." + queries: + p50: quantile_over_time(0.50, ocb_oracle_staleness_seconds{oracle="redstone", pair="ETH/USD", chain="gateway"}[24h]) + p90: quantile_over_time(0.90, ocb_oracle_staleness_seconds{oracle="redstone", pair="ETH/USD", chain="gateway"}[24h]) + p99: quantile_over_time(0.99, ocb_oracle_staleness_seconds{oracle="redstone", pair="ETH/USD", chain="gateway"}[24h]) + mean: avg_over_time(ocb_oracle_staleness_seconds{oracle="redstone", pair="ETH/USD", chain="gateway"}[24h]) + success: clamp_min(clamp_max(1 - sum(rate(ocb_oracle_freshness_scrape_errors_total{oracle="redstone", pair="ETH/USD", chain="gateway"}[24h])) / (1/30), 1), 0) + sample_size: count_over_time(ocb_oracle_staleness_seconds{oracle="redstone", pair="ETH/USD", chain="gateway"}[24h]) + series: ocb_oracle_staleness_seconds{oracle="redstone", pair="ETH/USD", chain="gateway"} diff --git a/benchmarks/pm-api-latency.yml b/benchmarks/pm-api-latency.yml index 394efef7..ba19bf0e 100644 --- a/benchmarks/pm-api-latency.yml +++ b/benchmarks/pm-api-latency.yml @@ -101,6 +101,9 @@ per_chain_explainer: - slug: myriad h2: Myriad API latency body: "Myriad's API is a single region Heroku deployment in US East with no order book endpoint and no WebSocket. Its cohort worst {{p50:myriad}} p50 (24h, all regions) is mostly geography: from eu-west and Singapore every request crosses an ocean first. When checking whether the Myriad API is down, look at the us-east region figure, since a global latency rise that spares us-east is network weather, not a venue incident. Myriad also runs the tightest keyless request budget in the cohort, covered in the pm-rate-limits bench." + - slug: predexon + h2: Predexon API latency + body: "Predexon is a multi-venue aggregator that unifies Polymarket, Kalshi and Limitless behind one schema. Its API answers at {{p50:predexon}} p50 (24h, all regions), measured on the same market-price endpoint the harness probes on the native venues, so the direct comparison of a unified aggregator against three venue-native APIs is head to head. Predexon aggregates upstream data; a Predexon latency higher than the sum of its three underliers is the cost of unification. Lower is the value of having one contract instead of three." source: https://github.com/ChainBench/OpenChainBench/tree/main/harnesses/pm-rate-limits @@ -128,14 +131,14 @@ metric_panels: - id: uptime_24h label: Uptime 24h description: "Share of probe cycles that succeeded over the last 24 hours, averaged across the three probe regions. 100 percent means every 5 second probe of the venue's API came back healthy." - metric: 100 * avg(avg_over_time(pmapi_health{source="direct",venue=~"polymarket|kalshi|limitless|manifold|myriad"}[24h])) + metric: 100 * avg(avg_over_time(pmapi_health{source="direct",venue=~"polymarket|kalshi|limitless|manifold|myriad"}[24h])) or 100 * avg(avg_over_time(pmapi_health{source="predexon"}[24h])) label_key: venue unit: pct higher_is_better: true - id: uptime_7d label: Uptime 7d description: "Same health gauge averaged over 7 days. Short outages that vanish from the 24h figure stay visible here for a week." - metric: 100 * avg(avg_over_time(pmapi_health{source="direct",venue=~"polymarket|kalshi|limitless|manifold|myriad"}[7d])) + metric: 100 * avg(avg_over_time(pmapi_health{source="direct",venue=~"polymarket|kalshi|limitless|manifold|myriad"}[7d])) or 100 * avg(avg_over_time(pmapi_health{source="predexon"}[7d])) label_key: venue unit: pct higher_is_better: true @@ -228,3 +231,15 @@ providers: success: clamp_max(sum(rate(pmapi_requests_total{venue="myriad",source="direct",class="price",conn="warm",outcome="ok"}[24h])) / sum(rate(pmapi_requests_total{venue="myriad",source="direct",class="price",conn="warm",outcome!="probe_invalid"}[24h])), 1) sample_size: sum(increase(pmapi_requests_total{venue="myriad",source="direct",class="price",conn="warm",outcome!="probe_invalid"}[24h])) series: 1000 * histogram_quantile(0.50, sum(rate(pmapi_request_duration_seconds_bucket{venue="myriad",source="direct",class="price",conn="warm",cache!="hit"}[1h])) by (le)) + - slug: predexon + name: Predexon + tag: Multi-venue aggregator API (Polymarket, Kalshi, Limitless), unified schema + formula: "p50 of warm, non CDN cached round trips against the Predexon market-price endpoints for the three covered venues, keyless-equivalent (free-tier key), successful requests only, 24h window." + queries: + p50: 1000 * histogram_quantile(0.50, sum(rate(pmapi_request_duration_seconds_bucket{source="predexon",class="price",conn="warm",cache!="hit"}[24h])) by (le)) + p90: 1000 * histogram_quantile(0.90, sum(rate(pmapi_request_duration_seconds_bucket{source="predexon",class="price",conn="warm",cache!="hit"}[24h])) by (le)) + p99: 1000 * histogram_quantile(0.99, sum(rate(pmapi_request_duration_seconds_bucket{source="predexon",class="price",conn="warm",cache!="hit"}[24h])) by (le)) + mean: 1000 * sum(rate(pmapi_request_duration_seconds_sum{source="predexon",class="price",conn="warm",cache!="hit"}[24h])) / sum(rate(pmapi_request_duration_seconds_count{source="predexon",class="price",conn="warm",cache!="hit"}[24h])) + success: clamp_max(sum(rate(pmapi_requests_total{source="predexon",class="price",conn="warm",outcome="ok"}[24h])) / sum(rate(pmapi_requests_total{source="predexon",class="price",conn="warm",outcome!="probe_invalid"}[24h])), 1) + sample_size: sum(increase(pmapi_requests_total{source="predexon",class="price",conn="warm",outcome!="probe_invalid"}[24h])) + series: 1000 * histogram_quantile(0.50, sum(rate(pmapi_request_duration_seconds_bucket{source="predexon",class="price",conn="warm",cache!="hit"}[1h])) by (le)) diff --git a/benchmarks/polkadot-rpc.yml b/benchmarks/polkadot-rpc.yml new file mode 100644 index 00000000..9e8b0ffe --- /dev/null +++ b/benchmarks/polkadot-rpc.yml @@ -0,0 +1,161 @@ +# OpenChainBench. Bench № 086 + +slug: polkadot-rpc +number: "086" +title: Fastest free Polkadot RPC, live no-key endpoint latency +seo_title: "Fastest free Polkadot RPC 2026" +seo_description: "{{best_name}} leads free Polkadot RPC at {{best_p50}} (chain_getHeader p50, 24h). 3 no-key providers measured every 60s from 3 regions." +subtitle: HTTP round-trip latency for chain_getHeader against every free, no-key public Polkadot relay-chain endpoint, audited every 60 seconds from 3 regions. + +category: RPCs +status: live +metric: RPC latency +unit: ms +higher_is_better: false + +seo_intro: | + Polkadot is the first non-EVM, non-Solana chain in the RPC latency + cluster. Substrate-based relay chain, JSON-RPC 2.0 wire but a + different method set: the probe is `chain_getHeader` (returns the + latest header with a hex-encoded block number) rather than + `eth_getBlockByNumber`. Everything else on this page is the same + bench pattern used on Ethereum, Base, Solana and the 20+ other + chains: identical POST every 60 seconds from three regions, latency + measured client-side at millisecond precision, per-region breakdowns + first class. Provider cohort at launch: Parity's official + `rpc.polkadot.io`, OnFinality's public gateway, and PublicNode + (Allnodes). Dwellir + 1RPC audited alongside and excluded (Dwellir + 503 during the launch sweep; 1RPC filters `chain_getHeader` on its + privacy relay); both will be revisited when a stable no-key path + reappears. + +abstract: | + Per-chain member of the RPC latency cluster, extended for Substrate. + We measure the round-trip latency of a single, identical JSON-RPC + call (`chain_getHeader`) against every no-key public Polkadot + relay-chain endpoint that sustains continuous probing, 3 providers + at launch, every 60 seconds, from us-east, eu-west and Singapore. The harness + classifies every response (ok / http_err / jsonrpc_err / stale / + timeout) with a relay-chain-scaled staleness gap (40 blocks ≈ 4 + minutes at Polkadot's ~6 s block time), so the leaderboard rewards + sustained, honest availability rather than a fast error message. + The cross-chain view lives on the parent `rpc-capabilities` + benchmark; this page is the Polkadot-scoped answer with per-region + breakdowns as a first-class dimension. + +methodology: + - "Cadence: every 60 seconds per provider, from each of 3 probe regions (us-east Virginia, eu-west Amsterdam, sgp Singapore). Headline p50/p90/p99 aggregate across all 3 regions via Prometheus `avg(quantile_over_time(...))`; per-region breakdowns are first-class on this page via the region tabs." + - "Payload: `{\"jsonrpc\":\"2.0\",\"id\":,\"method\":\"chain_getHeader\",\"params\":[]}`. Plain HTTP POST, identical for every endpoint, no API key in any request. Non-cacheable by design: the latest-header fetch with a rotating id defeats edge caches that would answer a bare head query without touching a node." + - "Latency: client-side round-trip delta in milliseconds, exposed as both a gauge and a histogram (buckets 50 ms → 10 s), so percentiles are computed via Prometheus `quantile_over_time` over the last 24 hours." + - "Call-result classification: `ok` (HTTP 200 + non-empty result with a parsable hex block number), `http_err`, `jsonrpc_err` (HTTP 200 carrying an error body), `stale` (more than 40 relay blocks behind the cross-provider tip, roughly 4 minutes at Polkadot's ~6 s block time), `timeout`. Latency without reliability is a misleading ranking signal." + - "No archive-depth loop on this chain. Polkadot's state model is accessed via `state_getStorage` keyed by a Blake2-hashed storage key rather than the EVM `eth_getBalance(address, height)` pattern, so the archive probe used on the EVM cluster is not applied here." + - "No cross-provider hash quorum on this chain. `chain_getHeader` returns the parent-block hash rather than the current-block hash (the current hash is derived by Blake2 over the SCALE-encoded header). Bench 083's height-hash quorum check is therefore skipped for Polkadot; the same reliability signal is still captured via `ok` / `stale` classification against the rolling cross-provider tip." + - "This page is part of the per-chain RPC cluster derived from the cross-chain [rpc-capabilities](https://openchainbench.com/benchmarks/rpc-capabilities) benchmark; the identical harness, cadence and exclusion rules apply on every chain." + - "Chain scope: every query on this page is pinned to `chain=\"polkadot\"`. Provider coverage at launch: 3 no-key endpoints (Parity official, OnFinality public gateway, PublicNode). Excluded by the launch audit: 1RPC filters `chain_getHeader` as \"Not Allowed\" on its privacy relay; Dwellir returned 503 during the audit sweep; Ankr paid Polkadot tier only; Chainstack key-gated; RadiumBlock requires a referral header; Grove and Thirdweb return invalid-chain or key-gated." + +findings: + - "{{best_name}} currently leads free Polkadot RPC at {{best_p50}} (`chain_getHeader` p50, 24h) across 3 measured providers." + - "{{name:onfinality}} is a Web3 Foundation grantee that serves a large share of Polkadot ecosystem infra; the bench measures its public gateway under the same probe used for the multi-chain gateways, so the leaderboard is directly comparable to what a chain-agnostic integrator sees." + - "{{name:publicnode}} extends its universal multi-chain footprint onto Substrate without a keyed path; the reliability column is what to read alongside p50 here since Substrate JSON-RPC has different failure modes than EVM (SCALE decode timeouts, sudden peer drops during finality gaps)." + +faq: + - q: "What is the fastest free Polkadot RPC right now?" + a: "{{best_name}} currently leads at {{best_p50}} (`chain_getHeader` p50 over the last 24h), measured against 3 no-key providers probed every 60 seconds from us-east, eu-west and Singapore. The leaderboard re-sorts continuously against fresh Prometheus samples, so the answer on this page is the answer right now, not a quarterly snapshot. Use the region tabs to see the leader from the origin closest to your deployment." + - q: "Which Polkadot RPCs work without an API key?" + a: "Three endpoints sustain continuous keyless probing at launch: Parity's official `rpc.polkadot.io`, OnFinality's public gateway and PublicNode (operated by Allnodes). Every listed endpoint was live-verified with four consecutive `chain_getHeader` probes returning a parsable hex block number before inclusion. Excluded by the audit: 1RPC filters `chain_getHeader` as \"Not Allowed\" on its privacy relay, Dwellir returned 503 Service Unavailable during the sweep (will be revisited once stable), Ankr requires a paid Polkadot key, Chainstack is key-gated on Substrate, and RadiumBlock rejects requests without a referral header." + - q: "Does the fastest Polkadot RPC change by region?" + a: "Frequently, and often more sharply than EVM chains. Polkadot's public infra is heavily weighted toward Web3 Foundation grantees whose primary data centers sit in Europe; a gateway that wins from Amsterdam can lose from Singapore by multiples. The region tabs at the top of the page re-scope every number on the page to a single origin; pick the one closest to where your requests actually originate." + - q: "How is Polkadot RPC latency measured here, technically?" + a: "One identical Substrate JSON-RPC POST (`chain_getHeader`) every 60 seconds against each provider from each of 3 regions, using the same plain HTTP client the EVM cluster uses. The response is a header object containing a hex-encoded relay-block number and the parent-block hash; the harness parses the number (`strconv.ParseUint(strings.TrimPrefix(hdr.Number, \"0x\"), 16, 64)`, same rule as EVM). Wall-clock round-trip is recorded at millisecond precision; p50/p90/p99 are computed via Prometheus `quantile_over_time` over 24 hours. Responses are classified (`ok` / `http_err` / `jsonrpc_err` / `stale` / `timeout`) so an endpoint stuck on an old head or returning errors behind HTTP 200 is never ranked as fastest." + - q: "Why is the probe `chain_getHeader` and not `system_health`?" + a: "Two reasons. First, `chain_getHeader` returns the head block number, which lets the harness plug into the cluster's cross-provider tip machinery for staleness classification without adding a chain-specific code path. Second, `system_health` returns a small object about peer count and sync state that any polite CDN can synthesize without touching the underlying node, defeating the point of a latency probe on the RPC path. `chain_getHeader` forces the endpoint to actually read the head from the runtime, which is what integrators care about." + - q: "Why is there no archive-depth or hash-quorum column for Polkadot?" + a: "Polkadot's state model accesses historical state via `state_getStorage` keyed by a Blake2-hashed storage key, which has no chain-agnostic depth analog for the EVM `eth_getBalance(address, height)` probe the cluster uses on other chains. Similarly, `chain_getHeader` returns the parent-block hash (not the current-block hash: the current hash is derived by Blake2 over the SCALE-encoded header), so the cross-provider height-hash quorum check from bench 083 cannot be run against Polkadot without adding a Substrate-native encoder to the harness. Both columns are omitted honestly rather than filled with a proxy that would misrepresent the chain." + +source: https://github.com/ChainBench/OpenChainBench/tree/main/harnesses/rpc-capabilities + +prometheus: + window: 24h + freshness_metric: rpc_latency_milliseconds + +# Per-cell (region) ranking matrix for scoped badge claims. Chain is +# fixed for the whole bench, so cells key on region alone. +rank_matrix_query: avg by (provider, region) (ocb:rpc_latency_milliseconds:p50_24h{chain="polkadot"}) + +# Region is the only dimension: chain is baked into every query. +dimensions: + region: + - { value: all, label: All regions } + - { value: us-east, label: US-East } + - { value: eu-west, label: EU-West } + - { value: sgp, label: Singapore } + +providers: + - slug: polkadot-official + name: Parity + tag: Parity's official Polkadot relay endpoint + formula: "50th percentile over 24h of client-side round-trip latency (ms) for a single `chain_getHeader` POST sent every 60s from 3 regions (us-east + eu-west + sgp) to Parity's official Polkadot relay endpoint." + queries: + p50: avg(ocb:rpc_latency_milliseconds:p50_24h{provider="polkadot-official", chain="polkadot"}) + p90: avg(ocb:rpc_latency_milliseconds:p90_24h{provider="polkadot-official", chain="polkadot"}) + p99: avg(ocb:rpc_latency_milliseconds:p99_24h{provider="polkadot-official", chain="polkadot"}) + mean: avg(ocb:rpc_latency_milliseconds:mean_24h{provider="polkadot-official", chain="polkadot"}) + success: sum(ocb:rpc_call:ok_rate_24h{provider="polkadot-official", chain="polkadot"}) / sum(ocb:rpc_call:rate_24h{provider="polkadot-official", chain="polkadot"}) + sample_size: sum(ocb:rpc_call:increase_24h{provider="polkadot-official", chain="polkadot"}) + series: avg(avg_over_time(rpc_latency_milliseconds{provider="polkadot-official", chain="polkadot"}[1h])) + regions: + - region: us-east + p50: avg(ocb:rpc_latency_milliseconds:p50_24h{provider="polkadot-official", chain="polkadot", region="us-east"}) + series: avg_over_time(rpc_latency_milliseconds{provider="polkadot-official", chain="polkadot", region="us-east"}[1h]) + - region: eu-west + p50: avg(ocb:rpc_latency_milliseconds:p50_24h{provider="polkadot-official", chain="polkadot", region="eu-west"}) + series: avg_over_time(rpc_latency_milliseconds{provider="polkadot-official", chain="polkadot", region="eu-west"}[1h]) + - region: ap-southeast + p50: avg(ocb:rpc_latency_milliseconds:p50_24h{provider="polkadot-official", chain="polkadot", region="sgp"}) + series: avg_over_time(rpc_latency_milliseconds{provider="polkadot-official", chain="polkadot", region="sgp"}[1h]) + + - slug: onfinality + name: OnFinality + tag: Substrate-native public gateway, Web3 Foundation grantee + formula: "50th percentile over 24h of client-side round-trip latency (ms) for a single `chain_getHeader` POST sent every 60s from 3 regions (us-east + eu-west + sgp) to OnFinality's public Polkadot endpoint." + queries: + p50: avg(ocb:rpc_latency_milliseconds:p50_24h{provider="onfinality", chain="polkadot"}) + p90: avg(ocb:rpc_latency_milliseconds:p90_24h{provider="onfinality", chain="polkadot"}) + p99: avg(ocb:rpc_latency_milliseconds:p99_24h{provider="onfinality", chain="polkadot"}) + mean: avg(ocb:rpc_latency_milliseconds:mean_24h{provider="onfinality", chain="polkadot"}) + success: sum(ocb:rpc_call:ok_rate_24h{provider="onfinality", chain="polkadot"}) / sum(ocb:rpc_call:rate_24h{provider="onfinality", chain="polkadot"}) + sample_size: sum(ocb:rpc_call:increase_24h{provider="onfinality", chain="polkadot"}) + series: avg(avg_over_time(rpc_latency_milliseconds{provider="onfinality", chain="polkadot"}[1h])) + regions: + - region: us-east + p50: avg(ocb:rpc_latency_milliseconds:p50_24h{provider="onfinality", chain="polkadot", region="us-east"}) + series: avg_over_time(rpc_latency_milliseconds{provider="onfinality", chain="polkadot", region="us-east"}[1h]) + - region: eu-west + p50: avg(ocb:rpc_latency_milliseconds:p50_24h{provider="onfinality", chain="polkadot", region="eu-west"}) + series: avg_over_time(rpc_latency_milliseconds{provider="onfinality", chain="polkadot", region="eu-west"}[1h]) + - region: ap-southeast + p50: avg(ocb:rpc_latency_milliseconds:p50_24h{provider="onfinality", chain="polkadot", region="sgp"}) + series: avg_over_time(rpc_latency_milliseconds{provider="onfinality", chain="polkadot", region="sgp"}[1h]) + + - slug: publicnode + name: PublicNode + tag: Allnodes-operated, 70+ chains, includes Substrate + formula: "50th percentile over 24h of client-side round-trip latency (ms) for a single `chain_getHeader` POST sent every 60s from 3 regions (us-east + eu-west + sgp) to PublicNode's no-key Polkadot endpoint." + queries: + p50: avg(ocb:rpc_latency_milliseconds:p50_24h{provider="publicnode", chain="polkadot"}) + p90: avg(ocb:rpc_latency_milliseconds:p90_24h{provider="publicnode", chain="polkadot"}) + p99: avg(ocb:rpc_latency_milliseconds:p99_24h{provider="publicnode", chain="polkadot"}) + mean: avg(ocb:rpc_latency_milliseconds:mean_24h{provider="publicnode", chain="polkadot"}) + success: sum(ocb:rpc_call:ok_rate_24h{provider="publicnode", chain="polkadot"}) / sum(ocb:rpc_call:rate_24h{provider="publicnode", chain="polkadot"}) + sample_size: sum(ocb:rpc_call:increase_24h{provider="publicnode", chain="polkadot"}) + series: avg(avg_over_time(rpc_latency_milliseconds{provider="publicnode", chain="polkadot"}[1h])) + regions: + - region: us-east + p50: avg(ocb:rpc_latency_milliseconds:p50_24h{provider="publicnode", chain="polkadot", region="us-east"}) + series: avg_over_time(rpc_latency_milliseconds{provider="publicnode", chain="polkadot", region="us-east"}[1h]) + - region: eu-west + p50: avg(ocb:rpc_latency_milliseconds:p50_24h{provider="publicnode", chain="polkadot", region="eu-west"}) + series: avg_over_time(rpc_latency_milliseconds{provider="publicnode", chain="polkadot", region="eu-west"}[1h]) + - region: ap-southeast + p50: avg(ocb:rpc_latency_milliseconds:p50_24h{provider="publicnode", chain="polkadot", region="sgp"}) + series: avg_over_time(rpc_latency_milliseconds{provider="publicnode", chain="polkadot", region="sgp"}[1h]) diff --git a/benchmarks/rpc-reliability.yml b/benchmarks/rpc-reliability.yml new file mode 100644 index 00000000..f588fcbf --- /dev/null +++ b/benchmarks/rpc-reliability.yml @@ -0,0 +1,297 @@ +# OpenChainBench. Bench № 083 + +slug: rpc-reliability +number: "083" +title: Most reliable free public RPC, correctness incidents across 7 EVM chains +seo_title: "Most reliable free public RPC 2026" +seo_description: "Free public RPCs ranked by correctness incidents: block-hash quorum disagreements, eth_getLogs completeness and eth_getBalance state integrity, live and keyless." +subtitle: "Correctness incidents per 24 hours for free no-key RPC endpoints: block hashes disagreeing with the cross-provider quorum, eth_getLogs counts deviating from the majority, divergent eth_getBalance state and blocked or gated methods, measured continuously alongside consensus head lag." +category: RPCs +status: live +metric: Reliability incidents +unit: count +higher_is_better: false + +disclaimer: > + A zero row means no incident was caught in the window, not a proof of + correctness. Most providers post zero most days; this bench exists for + the days they do not, and for the errors: a blocked eth_getLogs or a + balance read gated behind a paid key at 40 blocks of depth counts as + an incident because it is one from the caller's seat. + +seo_intro: | + Bench 010 answers which free public RPC is fastest; this page answers + the question that should come first. which one is telling the truth. + A fast endpoint that serves a stale head, drops logs from an + eth_getLogs response, or answers eth_getBalance from a diverged state + view is worse than a slow honest one, and none of that shows up in a + latency percentile. So we cross-examine the same keyless provider + cohort the latency bench probes, continuously, on three axes. First, + consensus head lag: every 60 seconds each provider's reported + (height, hash) pair feeds a per-chain quorum map, and the gap between + a provider's head and the highest head any provider reported is + published as a gauge. Second, hash integrity: when at least two + providers agree on the hash at a height, a provider reporting a + different hash at that same height is counted, once per height, as a + mismatch incident. Third, fixed-vector integrity: every 5 minutes, + one chain at a time, every provider gets the identical eth_getLogs + query for the chain's canonical USDC contract over a 10-block window + behind the tip, plus an eth_getBalance read at the same fixed block; + answers are compared across providers and a deviation from the strict + majority is an incident. Errors count too, deliberately: 1RPC blocks + eth_getLogs outright with a -32000, and PublicNode began gating + keyless reads a few dozen blocks deep behind a personal token + (-32602) in July 2026. Those are reliability failures a latency bench + renders invisible, and here they are the headline. + +abstract: | + We measure the correctness of free no-key RPC endpoints, not their + speed. The harness reuses the bench-010 probe matrix (same providers, + same chains, same 60-second header fetch) and adds three metric + families. rpc_consensus_lag_blocks tracks each provider's head + against the highest head seen across providers for that chain. + rpc_hash_mismatch_total counts same-height hash disagreements with + the two-or-more-provider quorum, at most once per height, with a + strict-plurality rule so a live reorg splitting providers two against + two counts nobody. rpc_integrity_check_total classifies fixed-vector + checks (eth_getLogs for the chain's canonical USDC over a 10-block + window at a daily-rotating depth behind tip, and eth_getBalance of a + fixed well-known address at the same block) as ok, error or disagree + against the cross-provider majority. The headline is the raw + incident count over 24 hours: errors plus disagreements plus hash + mismatches. No weighted composite: weights invite gaming disputes, + and an incident is an incident. + +methodology: + - "Consensus lag: the bench-010 latency probe fetches the full latest header every 60 seconds per (provider, chain); its height and hash, previously discarded, feed this bench. rpc_consensus_lag_blocks is the gap between the provider's reported head and the highest head any probed provider reported for that chain (rolling max). Valid-but-stale responses still count as observations; failed probes delete the series so dead endpoints age out." + - "Hash quorum: per chain, a height-to-hash vote map. When one hash at height H is backed by at least 2 providers AND strictly more than any competing hash, it is canonical; a provider that reported a different hash at H is counted in rpc_hash_mismatch_total, at most once per (provider, height). A 2-2 split, the signature of a live reorg, resolves to no quorum and counts nobody." + - "Fixed-vector integrity, cadence: every 5 minutes ONE chain is checked, rotating through the 7 chains with a canonical high-traffic USDC deployment (Ethereum, Arbitrum, Optimism, Base, Polygon, BNB, Avalanche). Each provider on that chain gets 2 checks per round, so a single-chain provider accrues ~82 checks per day and a 7-chain provider ~575." + - "Logs completeness vector: eth_getLogs for the chain's canonical USDC contract (Circle native deployments; Binance-peg on BNB) over a 10-block window ending at tip minus N. Returned log counts are compared across providers; deviation from the strict majority count increments rpc_logs_disagreement_total and books a disagree incident. Live calibration on Ethereum returned 1371 logs consistently on drpc, tenderly and blastapi over the same fixed range." + - "Anti-gaming rotation: N rotates daily over {20, 30, 40, 50, 60} blocks, so a provider cannot special-case a published fixed range, and request ids rotate on every call (the same edge-cache defeat as the latency probe). All depths stay far inside non-archive territory, so a pruned-but-honest node is never penalized." + - "State consistency vector: eth_getBalance of a fixed well-known address (the same Vitalik address the archive-depth probe uses) at the tip-minus-N block. The hex answer must be byte-identical across providers; divergence from the strict majority increments rpc_state_disagreement_total. Live calibration showed byte-identical balances across all responding providers at a fixed block." + - "Errors are signal: rpc_integrity_check_total{result} is ok, error or disagree. error covers blocked methods (1RPC answers eth_getLogs with -32000), depth gating (PublicNode keyless regressed to under 150 blocks of archive depth in July 2026, returning -32602 asking for a personal token), and transport failures. The headline counts error and disagree alike: a method you cannot call is unreliable regardless of why." + - "Headline: incidents per 24h = increase of integrity checks with result error or disagree, plus increase of hash mismatches, from the eu-west vantage. Raw counts, no weighting; the p90/p99 columns are consensus-lag quantiles in blocks, published as raw companion dimensions." + - "Single-region disclosure: incidents are counted from the eu-west probe only. Correctness, unlike latency, is location-independent in principle, but every provider fronts with anycast, so what we audit is the serving cluster eu-west traffic lands on; a diverged cache in another region is invisible to this bench. The us-east and sgp replicas emit the same metrics for cross-checking on the time-series chart." + - "Quorum honesty: strict plurality everywhere. Two agreeing providers outvote one dissenter; two against two resolves to no quorum and books nothing; a lone answer with no second opinion is unverifiable, not wrong, and books ok. Disagreement requires being outvoted by an established majority." + - "Cohort: 12 keyless providers from the bench-010 audited matrix. Excluded: LlamaRPC (fully down, HTTP 521, already delisted from the harness), 1RPC (delisted 2026-07-09 at under 13 percent success; when probed live it also blocked eth_getLogs with -32000, exactly the incident class this bench counts), Ankr / NodeReal / GetBlock / Chainstack (key-gated). Avalanche's foundation endpoint is probed by the harness and visible on bench 010; it is left out here only to keep the cohort at 12." + - "Provider matrix and vector addresses are documented in harnesses/rpc-capabilities/cmd/script/config.go and integrity.go; the harness is shared with bench 010, so both benches see the identical provider set and probe budget." + +findings: + - "Consensus lag is real and measurable at 60-second resolution: calibration rounds of simultaneous eth_getBlockByNumber across the Ethereum cohort caught individual gateways one block behind the cross-provider tip in 2 of 4 rounds, while every provider at equal height returned byte-identical hashes. The lag gauge turns that from an anecdote into a 24h p90/p99 per provider." + - "The completeness gap is not hypothetical. On the same fixed 10-block USDC range, drpc, tenderly and blastapi each returned exactly 1371 logs, 1RPC refused the method with -32000, and PublicNode's keyless tier had regressed its archive depth to under 150 blocks, answering -32602 and asking for a personal token. Three providers agreed, two could not answer; both failure modes are incidents here." + - "State reads are the quiet good news: eth_getBalance at a fixed recent block came back byte-identical across every responding provider during calibration. rpc_state_disagreement_total exists for the day that stops being true, which is precisely the day you want a public counter." + - "{{name:cloudflare}} shows why errors must count: the endpoint answers HTTP 200 fast and fails inside the JSON-RPC body. A correctness bench that only compared successful answers would score it clean; counting error results books its blocked calls as the incidents they are." + - "Zero is the normal reading and that is the point. Free public RPCs mostly do tell the truth; the bench prices the exceptions, publicly timestamps regressions like PublicNode's July 2026 keyless depth gating, and gives the honest majority a way to prove they stayed honest." + +faq: + - q: "What counts as a reliability incident?" + a: "Three things, summed over 24 hours from the eu-west probe. An integrity check that errored (method blocked, depth gated behind a key, transport failure). An integrity check whose answer deviated from the strict cross-provider majority (an eth_getLogs count differing from the count at least two other providers agree on, or an eth_getBalance hex that is not byte-identical to the majority answer). And a block-hash mismatch, where the provider reported a hash at a height where at least two providers agreed on a different one. Raw counts, no weights; lower is better and zero is common." + - q: "Why do method errors count as incidents when this is a correctness bench?" + a: "Because from the caller's seat a method you cannot call is a reliability failure regardless of the reason. 1RPC blocks eth_getLogs with a -32000 error; PublicNode's keyless tier started answering balance reads a few dozen blocks deep with -32602 and a demand for a personal token. Neither returns wrong data, but both break the contract a developer assumes when pasting a no-key URL into a dapp. Counting them alongside genuine disagreements keeps the headline honest: it measures whether the endpoint reliably does what an EVM JSON-RPC endpoint is expected to do." + - q: "Can a provider game this benchmark by special-casing the test queries?" + a: "The vectors are designed to make that expensive. The eth_getLogs window is anchored at tip minus N where N rotates daily over five depths, so the exact range is never fixed; request ids rotate on every call, defeating body-keyed edge caches; and the USDC contracts are the busiest transfer contracts on each chain, so serving them correctly IS serving real traffic correctly. The head-lag and hash-quorum signals cannot be gamed at all without actually running well-synced nodes, because they are computed from the same latest-header probe the latency bench uses." + - q: "Does a reorg unfairly flag providers as mismatching?" + a: "No, by construction. During a live reorg two providers can legitimately hold different hashes at the same height. The quorum rule requires a strict plurality: one hash backed by at least two providers and strictly more than any competing hash. A two-against-two split resolves to no quorum and counts nobody, and a mismatch is booked at most once per provider per height. What remains after those guards, a provider outvoted by an established majority at a settled height, is the signal this bench exists to record." + - q: "Why is this measured from one region only?" + a: "The headline counts incidents from the eu-west vantage because correctness, unlike latency, does not depend on where the client stands: a node serving the canonical chain serves it to everyone. The honest caveat is anycast: each probe sees the serving cluster its region lands on, so a diverged cache serving only another continent is invisible to the eu-west counter. The us-east and Singapore replicas emit the same metric families, and the per-region series are on the chart for cross-checking; the headline stays single-region so the same incident is never double-counted." + +source: https://github.com/ChainBench/OpenChainBench/tree/main/harnesses/rpc-capabilities + +prometheus: + window: 24h + freshness_metric: rpc_consensus_lag_blocks + +# Real metrics emitted by the rpc-capabilities harness (bench-083 set): +# rpc_consensus_lag_blocks{provider, chain, region} gauge +# rpc_hash_mismatch_total{provider, chain, region} counter +# rpc_logs_count{provider, chain, region} gauge +# rpc_logs_disagreement_total{provider, chain, region} counter +# rpc_state_disagreement_total{provider, chain, region} counter +# rpc_integrity_check_total{provider, chain, region, check, result} +# counter, check in {logs, balance}, result in {ok, error, disagree} +# Counters are zero-initialized for the whole (provider x chain) matrix +# at harness start, so increase() reads 0 (not absent) for clean rows. +# Headline scopes region="eu-west" (see methodology); the chain tab +# injects chain="X" into every query. + +dimensions: + chain: + - { value: all, label: All chains } + - { value: ethereum, label: Ethereum } + - { value: polygon, label: Polygon } + - { value: arbitrum, label: Arbitrum } + - { value: optimism, label: Optimism } + - { value: base, label: Base } + - { value: bnb, label: BNB Chain } + - { value: avalanche, label: Avalanche } + +# clamp_min floor: the cell-rank loader drops samples <= 0 (guard against +# absent-series garbage on latency benches), but 0 incidents is this bench's +# BEST and most common value; the epsilon keeps clean providers ranked. +rank_matrix_query: clamp_min(sum by (provider, chain) (increase(rpc_integrity_check_total{region="eu-west",result=~"error|disagree"}[24h])) + sum by (provider, chain) (increase(rpc_hash_mismatch_total{region="eu-west"}[24h])), 0.001) + +# ~82 integrity checks per provider per chain per day (2 checks per +# 5-min rotation over 7 chains); single-chain providers are the floor. +expected_n: 40 + +ledger_columns: + - { label: "Incidents 24h", slot: p50, unit: count } + - { label: "Head lag p90", slot: p90, unit: count } + - { label: "Head lag p99", slot: p99, unit: count } + +providers: + - slug: publicnode + name: PublicNode + tag: Allnodes-operated; keyless archive depth gated since July 2026 + formula: "Incidents in 24h (eu-west): integrity checks (USDC eth_getLogs count, eth_getBalance bytes) that errored or diverged from the cross-provider majority, plus block-hash quorum disagreements, summed over its measured chains." + queries: + p50: sum(increase(rpc_integrity_check_total{provider="publicnode",region="eu-west",result=~"error|disagree"}[24h])) + sum(increase(rpc_hash_mismatch_total{provider="publicnode",region="eu-west"}[24h])) + p90: avg(quantile_over_time(0.90, rpc_consensus_lag_blocks{provider="publicnode",region="eu-west"}[24h])) + p99: avg(quantile_over_time(0.99, rpc_consensus_lag_blocks{provider="publicnode",region="eu-west"}[24h])) + mean: avg(avg_over_time(rpc_consensus_lag_blocks{provider="publicnode",region="eu-west"}[24h])) + success: sum(increase(rpc_integrity_check_total{provider="publicnode",region="eu-west",result="ok"}[24h])) / sum(increase(rpc_integrity_check_total{provider="publicnode",region="eu-west"}[24h])) + sample_size: sum(increase(rpc_integrity_check_total{provider="publicnode",region="eu-west"}[24h])) + series: sum(increase(rpc_integrity_check_total{provider="publicnode",region="eu-west",result=~"error|disagree"}[1h])) + sum(increase(rpc_hash_mismatch_total{provider="publicnode",region="eu-west"}[1h])) + + - slug: drpc + name: dRPC + tag: Decentralized RPC mesh, consensus-checked routing + formula: "Incidents in 24h (eu-west): integrity checks (USDC eth_getLogs count, eth_getBalance bytes) that errored or diverged from the cross-provider majority, plus block-hash quorum disagreements, summed over its measured chains." + queries: + p50: sum(increase(rpc_integrity_check_total{provider="drpc",region="eu-west",result=~"error|disagree"}[24h])) + sum(increase(rpc_hash_mismatch_total{provider="drpc",region="eu-west"}[24h])) + p90: avg(quantile_over_time(0.90, rpc_consensus_lag_blocks{provider="drpc",region="eu-west"}[24h])) + p99: avg(quantile_over_time(0.99, rpc_consensus_lag_blocks{provider="drpc",region="eu-west"}[24h])) + mean: avg(avg_over_time(rpc_consensus_lag_blocks{provider="drpc",region="eu-west"}[24h])) + success: sum(increase(rpc_integrity_check_total{provider="drpc",region="eu-west",result="ok"}[24h])) / sum(increase(rpc_integrity_check_total{provider="drpc",region="eu-west"}[24h])) + sample_size: sum(increase(rpc_integrity_check_total{provider="drpc",region="eu-west"}[24h])) + series: sum(increase(rpc_integrity_check_total{provider="drpc",region="eu-west",result=~"error|disagree"}[1h])) + sum(increase(rpc_hash_mismatch_total{provider="drpc",region="eu-west"}[1h])) + + - slug: tenderly + name: Tenderly + tag: Multi-chain public gateway, no key + formula: "Incidents in 24h (eu-west): integrity checks (USDC eth_getLogs count, eth_getBalance bytes) that errored or diverged from the cross-provider majority, plus block-hash quorum disagreements, summed over its measured chains." + queries: + p50: sum(increase(rpc_integrity_check_total{provider="tenderly",region="eu-west",result=~"error|disagree"}[24h])) + sum(increase(rpc_hash_mismatch_total{provider="tenderly",region="eu-west"}[24h])) + p90: avg(quantile_over_time(0.90, rpc_consensus_lag_blocks{provider="tenderly",region="eu-west"}[24h])) + p99: avg(quantile_over_time(0.99, rpc_consensus_lag_blocks{provider="tenderly",region="eu-west"}[24h])) + mean: avg(avg_over_time(rpc_consensus_lag_blocks{provider="tenderly",region="eu-west"}[24h])) + success: sum(increase(rpc_integrity_check_total{provider="tenderly",region="eu-west",result="ok"}[24h])) / sum(increase(rpc_integrity_check_total{provider="tenderly",region="eu-west"}[24h])) + sample_size: sum(increase(rpc_integrity_check_total{provider="tenderly",region="eu-west"}[24h])) + series: sum(increase(rpc_integrity_check_total{provider="tenderly",region="eu-west",result=~"error|disagree"}[1h])) + sum(increase(rpc_hash_mismatch_total{provider="tenderly",region="eu-west"}[1h])) + + - slug: nodies + name: Nodies + tag: POKT Network's decentralized public RPC successor + formula: "Incidents in 24h (eu-west): integrity checks (USDC eth_getLogs count, eth_getBalance bytes) that errored or diverged from the cross-provider majority, plus block-hash quorum disagreements, summed over its measured chains." + queries: + p50: sum(increase(rpc_integrity_check_total{provider="nodies",region="eu-west",result=~"error|disagree"}[24h])) + sum(increase(rpc_hash_mismatch_total{provider="nodies",region="eu-west"}[24h])) + p90: avg(quantile_over_time(0.90, rpc_consensus_lag_blocks{provider="nodies",region="eu-west"}[24h])) + p99: avg(quantile_over_time(0.99, rpc_consensus_lag_blocks{provider="nodies",region="eu-west"}[24h])) + mean: avg(avg_over_time(rpc_consensus_lag_blocks{provider="nodies",region="eu-west"}[24h])) + success: sum(increase(rpc_integrity_check_total{provider="nodies",region="eu-west",result="ok"}[24h])) / sum(increase(rpc_integrity_check_total{provider="nodies",region="eu-west"}[24h])) + sample_size: sum(increase(rpc_integrity_check_total{provider="nodies",region="eu-west"}[24h])) + series: sum(increase(rpc_integrity_check_total{provider="nodies",region="eu-west",result=~"error|disagree"}[1h])) + sum(increase(rpc_hash_mismatch_total{provider="nodies",region="eu-west"}[1h])) + + - slug: lava + name: Lava + tag: Decentralized RPC mesh (ETH + Arbitrum no-key) + formula: "Incidents in 24h (eu-west) on Ethereum and Arbitrum: integrity checks (USDC eth_getLogs count, eth_getBalance bytes) that errored or diverged from the cross-provider majority, plus block-hash quorum disagreements." + queries: + p50: sum(increase(rpc_integrity_check_total{provider="lava",region="eu-west",result=~"error|disagree"}[24h])) + sum(increase(rpc_hash_mismatch_total{provider="lava",region="eu-west"}[24h])) + p90: avg(quantile_over_time(0.90, rpc_consensus_lag_blocks{provider="lava",region="eu-west"}[24h])) + p99: avg(quantile_over_time(0.99, rpc_consensus_lag_blocks{provider="lava",region="eu-west"}[24h])) + mean: avg(avg_over_time(rpc_consensus_lag_blocks{provider="lava",region="eu-west"}[24h])) + success: sum(increase(rpc_integrity_check_total{provider="lava",region="eu-west",result="ok"}[24h])) / sum(increase(rpc_integrity_check_total{provider="lava",region="eu-west"}[24h])) + sample_size: sum(increase(rpc_integrity_check_total{provider="lava",region="eu-west"}[24h])) + series: sum(increase(rpc_integrity_check_total{provider="lava",region="eu-west",result=~"error|disagree"}[1h])) + sum(increase(rpc_hash_mismatch_total{provider="lava",region="eu-west"}[1h])) + + - slug: meowrpc + name: MeowRPC + tag: Free public RPC (ETH + Arbitrum) + formula: "Incidents in 24h (eu-west) on Ethereum and Arbitrum: integrity checks (USDC eth_getLogs count, eth_getBalance bytes) that errored or diverged from the cross-provider majority, plus block-hash quorum disagreements." + queries: + p50: sum(increase(rpc_integrity_check_total{provider="meowrpc",region="eu-west",result=~"error|disagree"}[24h])) + sum(increase(rpc_hash_mismatch_total{provider="meowrpc",region="eu-west"}[24h])) + p90: avg(quantile_over_time(0.90, rpc_consensus_lag_blocks{provider="meowrpc",region="eu-west"}[24h])) + p99: avg(quantile_over_time(0.99, rpc_consensus_lag_blocks{provider="meowrpc",region="eu-west"}[24h])) + mean: avg(avg_over_time(rpc_consensus_lag_blocks{provider="meowrpc",region="eu-west"}[24h])) + success: sum(increase(rpc_integrity_check_total{provider="meowrpc",region="eu-west",result="ok"}[24h])) / sum(increase(rpc_integrity_check_total{provider="meowrpc",region="eu-west"}[24h])) + sample_size: sum(increase(rpc_integrity_check_total{provider="meowrpc",region="eu-west"}[24h])) + series: sum(increase(rpc_integrity_check_total{provider="meowrpc",region="eu-west",result=~"error|disagree"}[1h])) + sum(increase(rpc_hash_mismatch_total{provider="meowrpc",region="eu-west"}[1h])) + + - slug: flashbots + name: Flashbots + tag: Private-mempool read proxy, Ethereum only + formula: "Incidents in 24h (eu-west) on Ethereum: integrity checks (USDC eth_getLogs count, eth_getBalance bytes) that errored or diverged from the cross-provider majority, plus block-hash quorum disagreements." + queries: + p50: sum(increase(rpc_integrity_check_total{provider="flashbots",region="eu-west",result=~"error|disagree"}[24h])) + sum(increase(rpc_hash_mismatch_total{provider="flashbots",region="eu-west"}[24h])) + p90: avg(quantile_over_time(0.90, rpc_consensus_lag_blocks{provider="flashbots",region="eu-west"}[24h])) + p99: avg(quantile_over_time(0.99, rpc_consensus_lag_blocks{provider="flashbots",region="eu-west"}[24h])) + mean: avg(avg_over_time(rpc_consensus_lag_blocks{provider="flashbots",region="eu-west"}[24h])) + success: sum(increase(rpc_integrity_check_total{provider="flashbots",region="eu-west",result="ok"}[24h])) / sum(increase(rpc_integrity_check_total{provider="flashbots",region="eu-west"}[24h])) + sample_size: sum(increase(rpc_integrity_check_total{provider="flashbots",region="eu-west"}[24h])) + series: sum(increase(rpc_integrity_check_total{provider="flashbots",region="eu-west",result=~"error|disagree"}[1h])) + sum(increase(rpc_hash_mismatch_total{provider="flashbots",region="eu-west"}[1h])) + + - slug: cloudflare + name: Cloudflare + tag: Permissioned mode; the errors-are-signal exhibit + formula: "Incidents in 24h (eu-west) on Ethereum: integrity checks (USDC eth_getLogs count, eth_getBalance bytes) that errored or diverged from the cross-provider majority, plus block-hash quorum disagreements. Blocked methods count as errors." + queries: + p50: sum(increase(rpc_integrity_check_total{provider="cloudflare",region="eu-west",result=~"error|disagree"}[24h])) + sum(increase(rpc_hash_mismatch_total{provider="cloudflare",region="eu-west"}[24h])) + p90: avg(quantile_over_time(0.90, rpc_consensus_lag_blocks{provider="cloudflare",region="eu-west"}[24h])) + p99: avg(quantile_over_time(0.99, rpc_consensus_lag_blocks{provider="cloudflare",region="eu-west"}[24h])) + mean: avg(avg_over_time(rpc_consensus_lag_blocks{provider="cloudflare",region="eu-west"}[24h])) + success: sum(increase(rpc_integrity_check_total{provider="cloudflare",region="eu-west",result="ok"}[24h])) / sum(increase(rpc_integrity_check_total{provider="cloudflare",region="eu-west"}[24h])) + sample_size: sum(increase(rpc_integrity_check_total{provider="cloudflare",region="eu-west"}[24h])) + series: sum(increase(rpc_integrity_check_total{provider="cloudflare",region="eu-west",result=~"error|disagree"}[1h])) + sum(increase(rpc_hash_mismatch_total{provider="cloudflare",region="eu-west"}[1h])) + + - slug: base-official + name: Base + tag: Coinbase-operated, Base mainnet only + formula: "Incidents in 24h (eu-west) on Base: integrity checks (USDC eth_getLogs count, eth_getBalance bytes) that errored or diverged from the cross-provider majority, plus block-hash quorum disagreements." + queries: + p50: sum(increase(rpc_integrity_check_total{provider="base-official",region="eu-west",result=~"error|disagree"}[24h])) + sum(increase(rpc_hash_mismatch_total{provider="base-official",region="eu-west"}[24h])) + p90: avg(quantile_over_time(0.90, rpc_consensus_lag_blocks{provider="base-official",region="eu-west"}[24h])) + p99: avg(quantile_over_time(0.99, rpc_consensus_lag_blocks{provider="base-official",region="eu-west"}[24h])) + mean: avg(avg_over_time(rpc_consensus_lag_blocks{provider="base-official",region="eu-west"}[24h])) + success: sum(increase(rpc_integrity_check_total{provider="base-official",region="eu-west",result="ok"}[24h])) / sum(increase(rpc_integrity_check_total{provider="base-official",region="eu-west"}[24h])) + sample_size: sum(increase(rpc_integrity_check_total{provider="base-official",region="eu-west"}[24h])) + series: sum(increase(rpc_integrity_check_total{provider="base-official",region="eu-west",result=~"error|disagree"}[1h])) + sum(increase(rpc_hash_mismatch_total{provider="base-official",region="eu-west"}[1h])) + + - slug: arbitrum-official + name: Arbitrum + tag: Arbitrum Foundation RPC, Arbitrum One only + formula: "Incidents in 24h (eu-west) on Arbitrum One: integrity checks (USDC eth_getLogs count, eth_getBalance bytes) that errored or diverged from the cross-provider majority, plus block-hash quorum disagreements." + queries: + p50: sum(increase(rpc_integrity_check_total{provider="arbitrum-official",region="eu-west",result=~"error|disagree"}[24h])) + sum(increase(rpc_hash_mismatch_total{provider="arbitrum-official",region="eu-west"}[24h])) + p90: avg(quantile_over_time(0.90, rpc_consensus_lag_blocks{provider="arbitrum-official",region="eu-west"}[24h])) + p99: avg(quantile_over_time(0.99, rpc_consensus_lag_blocks{provider="arbitrum-official",region="eu-west"}[24h])) + mean: avg(avg_over_time(rpc_consensus_lag_blocks{provider="arbitrum-official",region="eu-west"}[24h])) + success: sum(increase(rpc_integrity_check_total{provider="arbitrum-official",region="eu-west",result="ok"}[24h])) / sum(increase(rpc_integrity_check_total{provider="arbitrum-official",region="eu-west"}[24h])) + sample_size: sum(increase(rpc_integrity_check_total{provider="arbitrum-official",region="eu-west"}[24h])) + series: sum(increase(rpc_integrity_check_total{provider="arbitrum-official",region="eu-west",result=~"error|disagree"}[1h])) + sum(increase(rpc_hash_mismatch_total{provider="arbitrum-official",region="eu-west"}[1h])) + + - slug: optimism-official + name: Optimism + tag: Optimism Foundation RPC, OP mainnet only + formula: "Incidents in 24h (eu-west) on Optimism: integrity checks (USDC eth_getLogs count, eth_getBalance bytes) that errored or diverged from the cross-provider majority, plus block-hash quorum disagreements." + queries: + p50: sum(increase(rpc_integrity_check_total{provider="optimism-official",region="eu-west",result=~"error|disagree"}[24h])) + sum(increase(rpc_hash_mismatch_total{provider="optimism-official",region="eu-west"}[24h])) + p90: avg(quantile_over_time(0.90, rpc_consensus_lag_blocks{provider="optimism-official",region="eu-west"}[24h])) + p99: avg(quantile_over_time(0.99, rpc_consensus_lag_blocks{provider="optimism-official",region="eu-west"}[24h])) + mean: avg(avg_over_time(rpc_consensus_lag_blocks{provider="optimism-official",region="eu-west"}[24h])) + success: sum(increase(rpc_integrity_check_total{provider="optimism-official",region="eu-west",result="ok"}[24h])) / sum(increase(rpc_integrity_check_total{provider="optimism-official",region="eu-west"}[24h])) + sample_size: sum(increase(rpc_integrity_check_total{provider="optimism-official",region="eu-west"}[24h])) + series: sum(increase(rpc_integrity_check_total{provider="optimism-official",region="eu-west",result=~"error|disagree"}[1h])) + sum(increase(rpc_hash_mismatch_total{provider="optimism-official",region="eu-west"}[1h])) + + - slug: binance + name: Binance + tag: BNB Chain dataseed RPC, BNB only + formula: "Incidents in 24h (eu-west) on BNB Chain: integrity checks (USDC eth_getLogs count, eth_getBalance bytes) that errored or diverged from the cross-provider majority, plus block-hash quorum disagreements." + queries: + p50: sum(increase(rpc_integrity_check_total{provider="binance",region="eu-west",result=~"error|disagree"}[24h])) + sum(increase(rpc_hash_mismatch_total{provider="binance",region="eu-west"}[24h])) + p90: avg(quantile_over_time(0.90, rpc_consensus_lag_blocks{provider="binance",region="eu-west"}[24h])) + p99: avg(quantile_over_time(0.99, rpc_consensus_lag_blocks{provider="binance",region="eu-west"}[24h])) + mean: avg(avg_over_time(rpc_consensus_lag_blocks{provider="binance",region="eu-west"}[24h])) + success: sum(increase(rpc_integrity_check_total{provider="binance",region="eu-west",result="ok"}[24h])) / sum(increase(rpc_integrity_check_total{provider="binance",region="eu-west"}[24h])) + sample_size: sum(increase(rpc_integrity_check_total{provider="binance",region="eu-west"}[24h])) + series: sum(increase(rpc_integrity_check_total{provider="binance",region="eu-west",result=~"error|disagree"}[1h])) + sum(increase(rpc_hash_mismatch_total{provider="binance",region="eu-west"}[1h])) diff --git a/benchmarks/token-deployment-cost.yml b/benchmarks/token-deployment-cost.yml index 6c00a430..38f3af8c 100644 --- a/benchmarks/token-deployment-cost.yml +++ b/benchmarks/token-deployment-cost.yml @@ -8,7 +8,7 @@ seo_description: "Live USD cost to create a fungible token on 7 non-EVM L1 chain subtitle: "Live USD cost to bring a fungible token into existence on each chain, using that chain's canonical method. 7 non-EVM L1 chains, 5 minute refresh, no transactions broadcast." seo_intro: | - This page answers a question every memecoin founder, RWA issuer and non-EVM L1 marketing team asks. How much does it actually cost in dollars to create a token on each non-EVM chain right now. We track 7 non-EVM L1 chains in parallel and refresh the number every 5 minutes. Solana uses the SPL mint plus the Associated Token Account plus the Metaplex metadata account so the comparison is fair against ERC20 which embeds name and symbol natively. Sui and Aptos use Move publish gas budgets. Cardano reads coins_per_utxo_size from koios and applies the Conway minUTxO formula. Stellar uses the canonical issuer plus distribution plus trustline two account flow. Cosmos TokenFactory chains (Osmosis, Injective) read denom_creation_fee from the LCD. Every number is reproducible from source. No transactions are broadcast. EVM chains are excluded from this leaderboard for now while we finalise the canonical OpenZeppelin v5 ERC20 artifact bytecode used in eth_estimateGas. + This page answers a question every memecoin founder, RWA issuer and non-EVM L1 marketing team asks. How much does it actually cost in dollars to create a token on each non-EVM chain right now. We track 7 non-EVM L1 chains in parallel and refresh the number every 5 minutes. Solana uses the SPL mint plus the Associated Token Account plus the Metaplex metadata account so the comparison is fair against ERC20 which embeds name and symbol natively. Sui and Aptos use Move publish gas budgets. Cardano reads coins_per_utxo_size from koios and applies the Conway minUTxO formula. Stellar uses the canonical issuer plus distribution plus trustline two account flow at protocol-minimum reserves. Cosmos TokenFactory chains (Osmosis, Injective) read denom_creation_fee from the LCD. Every number is reproducible from source. No transactions are broadcast. EVM chains are excluded from this leaderboard for now while we finalise the canonical OpenZeppelin v5 ERC20 artifact bytecode used in eth_estimateGas. faq: - q: "What does this benchmark measure?" @@ -58,7 +58,7 @@ methodology: - "Aptos. /v1/estimate_gas_price.gas_estimate octas per unit times 150000 gas units for a canonical FA standard publish. Converted to USD via APT price." - "Cosmos TokenFactory (Osmosis, Injective). LCD //tokenfactory/v1beta1/params.denom_creation_fee. Empty array (Osmosis) means gas only, rendered as less than 0.001 dollar. Injective charges a flat 0.1 INJ." - "Cardano. Koios /epoch_params.coins_per_utxo_size. Apply Conway minUTxO formula = (160 byte overhead + 70 byte single asset bundle) × coins_per_utxo_size + 180000 lovelace mint tx fee. Converted to USD via ADA price." - - "Stellar. Horizon /ledgers.base_reserve_in_stroops and base_fee_in_stroops on the latest ledger. Total cost = 3 base reserves (issuer account + distribution account + trustline on distribution) + 2 base fees (create_account + change_trust). Converted to USD via XLM price." + - "Stellar. Horizon /ledgers.base_reserve_in_stroops and base_fee_in_stroops on the latest ledger. Total cost = 5 base reserves (the protocol minimum of 2 per new account for issuer and distribution, plus 1 for the trustline) + 2 base fees (create_account + change_trust). Reserves are locked capital, refundable on account closure, counted here because bringing the asset into existence requires them upfront. Converted to USD via XLM price." - "USD prices. api.mobula.io/api/1/market/multi-data?symbols=SOL,SUI,APT,OSMO,INJ,ADA,XLM polled every 5 minutes." - "Failures. Any upstream error increments token_deployment_samples_total{chain,status=error} and leaves the previous gauge value in place." - "EVM chains excluded. Ethereum, BNB Chain, Avalanche, Polygon, Arbitrum, Optimism, Base, Blast, Mantle, opBNB, Celo, Scroll and Linea were previously exposed but relied on a placeholder init bytecode that undercounted real ERC20 deploy cost by 30 to 500x. Removed until the canonical OZ v5 artifact is finalised." diff --git a/benchmarks/tokenized-stock-arb-latency.yml b/benchmarks/tokenized-stock-arb-latency.yml new file mode 100644 index 00000000..e47159a6 --- /dev/null +++ b/benchmarks/tokenized-stock-arb-latency.yml @@ -0,0 +1,209 @@ +# OpenChainBench. Bench № 080 + +slug: tokenized-stock-arb-latency +number: "080" +title: Tokenized stock arb latency, how fast pools catch a Nasdaq move +seo_title: "Tokenized stock arb latency 2026" +seo_description: "When Nasdaq moves 50 bps, how fast does the onchain pool follow? Live measurement across 11 tokenized equities on Robinhood Chain." +subtitle: "Median seconds between a 50 bps one-minute move on the reference market and the onchain pool coming back within 20 bps, per tokenized equity, regular hours only." + +category: RWA +status: live +metric: Arb latency +unit: sec +higher_is_better: false + +seo_intro: | + The whole promise of tokenized stocks is that the onchain version + tracks the real market. During regular trading hours, that only + works if arbitrageurs close the gap when the Nasdaq moves. This + page measures exactly how long they take. Each time the Yahoo + reference for one of our 11 tracked equities moves 50 basis points + or more in a single minute, the harness starts a timer, and stops + it when the onchain Uniswap v4 pool comes back within 20 bps of + the reference. The published number is the median of those + intervals, per symbol, over the last 24 hours. Fast arbs mean the + market is real. Slow arbs mean the tokenized version is a + screenshot of the real one. + +abstract: | + A derived bench on top of the tokenized-stock-peg series already + in Prometheus. During market_state=\"regular\", the harness tracks + the trailing 1-minute reference move and the current pool-to-ref + deviation, and observes the seconds between a 50 bps trigger and + 20 bps convergence into a histogram, per asset. + +methodology: + - "Trigger: a one-minute reference-price move of at least 50 bps observed while market_state=\"regular\". Only regular Nasdaq hours generate events; pre- and post-market moves are excluded (thin liquidity, unreliable reference)." + - "Convergence: the pool-to-reference deviation drops back below 20 bps. The bench observes seconds from trigger to convergence into a histogram bucket, tagged per asset." + - "Publication: p50, p90 and p99 of the event distribution over 24 hours, per asset. Events that never converge before regular hours end are counted as still_open on a separate counter, not as latency samples." + - "Data source: the tsp_price_reference_usd and tsp_price_onchain_usdg series emitted by the tokenized-stock-peg harness at 60s cadence. The event tracker is in-process; no new external calls." + - "Cohort: the 11 tokenized equities the peg bench measures. Arb latency is a first-order signal of pool quality; the peg deviation number is the second-order signal (how close it stays between moves)." + - "Caveat: the 50 bps trigger filters out normal market chop while catching real moves. Symbols with fewer than 3 events per 24h during quiet market days will show sparse data; the sample_size column is the honest read of statistical weight." + +findings: + - "{{best_name}} closes a 50 bps gap fastest at {{best_p50}} (p50, 24h) across {{count}} tokenized equities." + - "Fast arbs on the liquid low-fee pools validate that the market is real during regular hours; which tickers those are on a given day depends on where the 50 bps moves landed." + - "Slow arbs on the 2 percent fee pools confirm the story the peg bench tells: thin pools do not just deviate more, they take longer to snap back." + - "The still_open counter is the number that matters most for downstream products using the pool price as an oracle: it is the count of moves the arbs did not close before regular hours ended." + +faq: + - q: "What is arb latency and why does it matter?" + a: "It is the time it takes for an arbitrageur to close the gap between the real stock price and the onchain pool price after the real one moves. Short latency means the tokenized version is a real reflection of the real market. Long latency means it is a delayed screenshot with a stale price." + - q: "How are events detected?" + a: "The harness reads the same Yahoo reference and pool spot every 60 seconds. When the reference moves 50 bps in a single minute while the pool is more than 20 bps off it, a new event opens. The event closes when the pool comes back within 20 bps. Time from open to close is observed into a histogram; p50, p90 and p99 are read from that histogram over 24 hours." + - q: "Why 50 bps trigger, 20 bps convergence?" + a: "50 bps is above normal intra-minute noise on liquid stocks and low enough to catch a few events per day per liquid symbol. 20 bps is the convergence band that means the pool is meaningfully close to the reference given the fee tiers of the pools measured (0.3 percent to 2 percent). Both thresholds are disclosed and can be varied in a follow-up dimension." + - q: "What is the still-open counter?" + a: "The number of events where the market session ended (Nasdaq closed) before the pool converged. Those are not counted as latency samples because the reference stops moving after 4 pm ET, but they are the honest measure of how many real-market moves the arbs failed to catch that day." + +source: https://github.com/ChainBench/OpenChainBench/tree/main/harnesses/tokenized-stock-peg + +prometheus: + window: 24h + freshness_metric: tsp_deviation_bps + +providers: + - slug: nvda + name: NVDA + tag: "Nvidia, 0.3% fee, deepest pool" + formula: "Median seconds until the NVDA pool on Robinhood Chain came back within 20 bps of the Yahoo reference after a 50 bps one-minute move, regular hours only, over 24h." + queries: + p50: histogram_quantile(0.50, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="nvda"}[24h]))) + p90: histogram_quantile(0.90, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="nvda"}[24h]))) + p99: histogram_quantile(0.99, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="nvda"}[24h]))) + mean: sum(rate(tsp_arb_latency_seconds_sum{issuer="robinhood", asset="nvda"}[24h])) / sum(rate(tsp_arb_latency_seconds_count{issuer="robinhood", asset="nvda"}[24h])) + success: sum(increase(tsp_arb_event_total{issuer="robinhood", asset="nvda", outcome="converged"}[24h])) / clamp_min(sum(increase(tsp_arb_event_total{issuer="robinhood", asset="nvda"}[24h])), 1) + sample_size: sum(increase(tsp_arb_event_total{issuer="robinhood", asset="nvda", outcome="converged"}[24h])) + series: histogram_quantile(0.50, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="nvda"}[1h]))) + + - slug: tsla + name: TSLA + tag: "Tesla, most active pool of the cohort" + formula: "Median seconds until the TSLA pool on Robinhood Chain came back within 20 bps of the Yahoo reference after a 50 bps one-minute move, regular hours only, over 24h." + queries: + p50: histogram_quantile(0.50, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="tsla"}[24h]))) + p90: histogram_quantile(0.90, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="tsla"}[24h]))) + p99: histogram_quantile(0.99, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="tsla"}[24h]))) + mean: sum(rate(tsp_arb_latency_seconds_sum{issuer="robinhood", asset="tsla"}[24h])) / sum(rate(tsp_arb_latency_seconds_count{issuer="robinhood", asset="tsla"}[24h])) + success: sum(increase(tsp_arb_event_total{issuer="robinhood", asset="tsla", outcome="converged"}[24h])) / clamp_min(sum(increase(tsp_arb_event_total{issuer="robinhood", asset="tsla"}[24h])), 1) + sample_size: sum(increase(tsp_arb_event_total{issuer="robinhood", asset="tsla", outcome="converged"}[24h])) + series: histogram_quantile(0.50, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="tsla"}[1h]))) + + - slug: aapl + name: AAPL + tag: "Apple, 1% fee" + formula: "Median seconds until the AAPL pool on Robinhood Chain came back within 20 bps of the Yahoo reference after a 50 bps one-minute move, regular hours only, over 24h." + queries: + p50: histogram_quantile(0.50, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="aapl"}[24h]))) + p90: histogram_quantile(0.90, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="aapl"}[24h]))) + p99: histogram_quantile(0.99, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="aapl"}[24h]))) + mean: sum(rate(tsp_arb_latency_seconds_sum{issuer="robinhood", asset="aapl"}[24h])) / sum(rate(tsp_arb_latency_seconds_count{issuer="robinhood", asset="aapl"}[24h])) + success: sum(increase(tsp_arb_event_total{issuer="robinhood", asset="aapl", outcome="converged"}[24h])) / clamp_min(sum(increase(tsp_arb_event_total{issuer="robinhood", asset="aapl"}[24h])), 1) + sample_size: sum(increase(tsp_arb_event_total{issuer="robinhood", asset="aapl", outcome="converged"}[24h])) + series: histogram_quantile(0.50, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="aapl"}[1h]))) + + - slug: googl + name: GOOGL + tag: "Alphabet, 1% fee" + formula: "Median seconds until the GOOGL pool on Robinhood Chain came back within 20 bps of the Yahoo reference after a 50 bps one-minute move, regular hours only, over 24h." + queries: + p50: histogram_quantile(0.50, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="googl"}[24h]))) + p90: histogram_quantile(0.90, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="googl"}[24h]))) + p99: histogram_quantile(0.99, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="googl"}[24h]))) + mean: sum(rate(tsp_arb_latency_seconds_sum{issuer="robinhood", asset="googl"}[24h])) / sum(rate(tsp_arb_latency_seconds_count{issuer="robinhood", asset="googl"}[24h])) + success: sum(increase(tsp_arb_event_total{issuer="robinhood", asset="googl", outcome="converged"}[24h])) / clamp_min(sum(increase(tsp_arb_event_total{issuer="robinhood", asset="googl"}[24h])), 1) + sample_size: sum(increase(tsp_arb_event_total{issuer="robinhood", asset="googl", outcome="converged"}[24h])) + series: histogram_quantile(0.50, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="googl"}[1h]))) + + - slug: meta + name: META + tag: "Meta, 0.3% fee" + formula: "Median seconds until the META pool on Robinhood Chain came back within 20 bps of the Yahoo reference after a 50 bps one-minute move, regular hours only, over 24h." + queries: + p50: histogram_quantile(0.50, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="meta"}[24h]))) + p90: histogram_quantile(0.90, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="meta"}[24h]))) + p99: histogram_quantile(0.99, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="meta"}[24h]))) + mean: sum(rate(tsp_arb_latency_seconds_sum{issuer="robinhood", asset="meta"}[24h])) / sum(rate(tsp_arb_latency_seconds_count{issuer="robinhood", asset="meta"}[24h])) + success: sum(increase(tsp_arb_event_total{issuer="robinhood", asset="meta", outcome="converged"}[24h])) / clamp_min(sum(increase(tsp_arb_event_total{issuer="robinhood", asset="meta"}[24h])), 1) + sample_size: sum(increase(tsp_arb_event_total{issuer="robinhood", asset="meta", outcome="converged"}[24h])) + series: histogram_quantile(0.50, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="meta"}[1h]))) + + - slug: amd + name: AMD + tag: "AMD, 1% fee" + formula: "Median seconds until the AMD pool on Robinhood Chain came back within 20 bps of the Yahoo reference after a 50 bps one-minute move, regular hours only, over 24h." + queries: + p50: histogram_quantile(0.50, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="amd"}[24h]))) + p90: histogram_quantile(0.90, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="amd"}[24h]))) + p99: histogram_quantile(0.99, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="amd"}[24h]))) + mean: sum(rate(tsp_arb_latency_seconds_sum{issuer="robinhood", asset="amd"}[24h])) / sum(rate(tsp_arb_latency_seconds_count{issuer="robinhood", asset="amd"}[24h])) + success: sum(increase(tsp_arb_event_total{issuer="robinhood", asset="amd", outcome="converged"}[24h])) / clamp_min(sum(increase(tsp_arb_event_total{issuer="robinhood", asset="amd"}[24h])), 1) + sample_size: sum(increase(tsp_arb_event_total{issuer="robinhood", asset="amd", outcome="converged"}[24h])) + series: histogram_quantile(0.50, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="amd"}[1h]))) + + - slug: pltr + name: PLTR + tag: "Palantir, 1% fee" + formula: "Median seconds until the PLTR pool on Robinhood Chain came back within 20 bps of the Yahoo reference after a 50 bps one-minute move, regular hours only, over 24h." + queries: + p50: histogram_quantile(0.50, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="pltr"}[24h]))) + p90: histogram_quantile(0.90, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="pltr"}[24h]))) + p99: histogram_quantile(0.99, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="pltr"}[24h]))) + mean: sum(rate(tsp_arb_latency_seconds_sum{issuer="robinhood", asset="pltr"}[24h])) / sum(rate(tsp_arb_latency_seconds_count{issuer="robinhood", asset="pltr"}[24h])) + success: sum(increase(tsp_arb_event_total{issuer="robinhood", asset="pltr", outcome="converged"}[24h])) / clamp_min(sum(increase(tsp_arb_event_total{issuer="robinhood", asset="pltr"}[24h])), 1) + sample_size: sum(increase(tsp_arb_event_total{issuer="robinhood", asset="pltr", outcome="converged"}[24h])) + series: histogram_quantile(0.50, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="pltr"}[1h]))) + + - slug: spy + name: SPY + tag: "S&P 500 ETF, 1% fee" + formula: "Median seconds until the SPY pool on Robinhood Chain came back within 20 bps of the Yahoo reference after a 50 bps one-minute move, regular hours only, over 24h." + queries: + p50: histogram_quantile(0.50, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="spy"}[24h]))) + p90: histogram_quantile(0.90, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="spy"}[24h]))) + p99: histogram_quantile(0.99, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="spy"}[24h]))) + mean: sum(rate(tsp_arb_latency_seconds_sum{issuer="robinhood", asset="spy"}[24h])) / sum(rate(tsp_arb_latency_seconds_count{issuer="robinhood", asset="spy"}[24h])) + success: sum(increase(tsp_arb_event_total{issuer="robinhood", asset="spy", outcome="converged"}[24h])) / clamp_min(sum(increase(tsp_arb_event_total{issuer="robinhood", asset="spy"}[24h])), 1) + sample_size: sum(increase(tsp_arb_event_total{issuer="robinhood", asset="spy", outcome="converged"}[24h])) + series: histogram_quantile(0.50, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="spy"}[1h]))) + + - slug: mu + name: MU + tag: "Micron, 1% fee" + formula: "Median seconds until the MU pool on Robinhood Chain came back within 20 bps of the Yahoo reference after a 50 bps one-minute move, regular hours only, over 24h." + queries: + p50: histogram_quantile(0.50, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="mu"}[24h]))) + p90: histogram_quantile(0.90, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="mu"}[24h]))) + p99: histogram_quantile(0.99, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="mu"}[24h]))) + mean: sum(rate(tsp_arb_latency_seconds_sum{issuer="robinhood", asset="mu"}[24h])) / sum(rate(tsp_arb_latency_seconds_count{issuer="robinhood", asset="mu"}[24h])) + success: sum(increase(tsp_arb_event_total{issuer="robinhood", asset="mu", outcome="converged"}[24h])) / clamp_min(sum(increase(tsp_arb_event_total{issuer="robinhood", asset="mu"}[24h])), 1) + sample_size: sum(increase(tsp_arb_event_total{issuer="robinhood", asset="mu", outcome="converged"}[24h])) + series: histogram_quantile(0.50, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="mu"}[1h]))) + + - slug: msft + name: MSFT + tag: "Microsoft, 2% fee, thin pool test" + formula: "Median seconds until the MSFT pool on Robinhood Chain came back within 20 bps of the Yahoo reference after a 50 bps one-minute move, regular hours only, over 24h." + queries: + p50: histogram_quantile(0.50, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="msft"}[24h]))) + p90: histogram_quantile(0.90, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="msft"}[24h]))) + p99: histogram_quantile(0.99, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="msft"}[24h]))) + mean: sum(rate(tsp_arb_latency_seconds_sum{issuer="robinhood", asset="msft"}[24h])) / sum(rate(tsp_arb_latency_seconds_count{issuer="robinhood", asset="msft"}[24h])) + success: sum(increase(tsp_arb_event_total{issuer="robinhood", asset="msft", outcome="converged"}[24h])) / clamp_min(sum(increase(tsp_arb_event_total{issuer="robinhood", asset="msft"}[24h])), 1) + sample_size: sum(increase(tsp_arb_event_total{issuer="robinhood", asset="msft", outcome="converged"}[24h])) + series: histogram_quantile(0.50, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="msft"}[1h]))) + + - slug: amzn + name: AMZN + tag: "Amazon, 2% fee, thin pool test" + formula: "Median seconds until the AMZN pool on Robinhood Chain came back within 20 bps of the Yahoo reference after a 50 bps one-minute move, regular hours only, over 24h." + queries: + p50: histogram_quantile(0.50, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="amzn"}[24h]))) + p90: histogram_quantile(0.90, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="amzn"}[24h]))) + p99: histogram_quantile(0.99, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="amzn"}[24h]))) + mean: sum(rate(tsp_arb_latency_seconds_sum{issuer="robinhood", asset="amzn"}[24h])) / sum(rate(tsp_arb_latency_seconds_count{issuer="robinhood", asset="amzn"}[24h])) + success: sum(increase(tsp_arb_event_total{issuer="robinhood", asset="amzn", outcome="converged"}[24h])) / clamp_min(sum(increase(tsp_arb_event_total{issuer="robinhood", asset="amzn"}[24h])), 1) + sample_size: sum(increase(tsp_arb_event_total{issuer="robinhood", asset="amzn", outcome="converged"}[24h])) + series: histogram_quantile(0.50, sum by (le) (rate(tsp_arb_latency_seconds_bucket{issuer="robinhood", asset="amzn"}[1h]))) + diff --git a/benchmarks/tokenized-stock-peg.yml b/benchmarks/tokenized-stock-peg.yml new file mode 100644 index 00000000..9fa21efc --- /dev/null +++ b/benchmarks/tokenized-stock-peg.yml @@ -0,0 +1,211 @@ +# OpenChainBench. Bench № 076 + +slug: tokenized-stock-peg +number: "076" +title: Tokenized stock price accuracy, live onchain vs Nasdaq across 11 equities +seo_title: "Tokenized stock price tracker 2026" +seo_description: "Do tokenized stocks track the real market? AAPL, TSLA, NVDA and 8 more on Robinhood Chain vs their Nasdaq price, deviation in bps, live and keyless." +subtitle: "Absolute deviation between each tokenized equity's Uniswap v4 pool price on Robinhood Chain and its real market reference price, in basis points, labeled by market session. The only live public measurement of whether onchain stocks actually track their underlying." + +category: RWA +status: live +metric: Price deviation +unit: bps +higher_is_better: false + +seo_intro: | + Robinhood Chain tokenizes ~95 equities, and its Uniswap v4 pools trade + them 24/7 while the Nasdaq closes every evening and every weekend. + This page answers the question that setup begs. how far does the + onchain price drift from the real one. Every 60 seconds we read each + pool's spot price straight from the chain (StateView getSlot0, keyless) + and compare it to the live reference price, labeling every sample with + the market session it was taken in (pre, regular, post, closed). The + regular-hours deviation is the headline ranking; the closed-state + series is the weekend drift nobody else publishes: what AAPL is worth + onchain on a Saturday when no market maker has a reference to arb + against. Of the ~95 tokenized equities only 14 have pools with real + liquidity; we measure the 11 with a listed underlying. SpaceX trades + onchain too, but SpaceX is not listed anywhere, so there is no + reference price to measure against, which says something about + tokenized private equity all by itself. + +abstract: | + Every 60 seconds the harness reads the Uniswap v4 spot price of 11 + tokenized equities on Robinhood Chain (one batched eth_call to the + StateView contract, keyless) and fetches the reference equity price + from Yahoo Finance (one batched spark call). Deviation is + 10000 x |pool - reference| / reference, in basis points, published + with a market_state label derived from Yahoo's holiday-aware session + windows. No API keys, no transactions, both legs reproducible from + public endpoints. + +methodology: + - "Onchain leg: one JSON-RPC batch per tick to StateView (0xF3334192D15450CdD385c8B70e03f9A6bD9E673b) getSlot0(poolId) on the Uniswap v4 PoolManager singleton, keyless against the official Robinhood Chain RPC. sqrtPriceX96 converts to USDG per share with the per-pool currency ordering and the USDG 6 vs stock 18 decimal gap (factor 1e12)." + - "Reference leg: Yahoo Finance v8 spark batch, one call for all 11 symbols per tick, browser User-Agent, no key. regularMarketPrice doubles as the last-close reference when the market is closed, which is exactly the weekend baseline." + - "Market session labels: derived from Yahoo currentTradingPeriod epochs (pre / regular / post / closed), which Yahoo publishes holiday-aware, so the harness maintains no NYSE calendar. The headline ranking pins market_state=\"regular\"; the closed series is the drift panel." + - "Deviation: 10000 x |pool_price - reference_price| / reference_price, sampled every 60 seconds, quantiles over 24h via quantile_over_time." + - "Cohort: the 11 official tokenized equities (name pattern * Robinhood Token, shared verified Stock implementation) whose USDG pool has real liquidity and swap history: NVDA, AAPL, GOOGL, TSLA, PLTR, META, AMD, MSFT, AMZN, SPY, MU. Pool fees range 0.3% to 2%." + - "Excluded, with reasons: SPCX (SpaceX pool holds ~$196k but SpaceX is not listed, no reference price exists), SNDK and QQQ (pool depth under $2k, deviation would be noise), CRCL (pool at zero liquidity), ~80 other official tokens (issuer-seeded placeholder pools at 90-95% fee, zero swaps), HOOD (never issued onchain, all HOOD tokens on the chain are third-party spam)." + - "Thin liquidity is part of the story, not a defect: pool depths run $24k to $330k, so a single mid-size swap can move the onchain price meaningfully off the reference until arbitrage closes it. The bench measures how fast that closure actually happens." + - "Reading grid: an AMM price can sit anywhere inside the pool's fee band without creating an arbitrage opportunity, so a 2% fee pool resting 150 bps off its reference is economically at equilibrium, not broken. Cross-symbol ranking therefore partly reflects each pool's fee tier (disclosed in every tag); the within-symbol trend over time is the purest signal." + - "Quote-asset caveat: pool prices are denominated in USDG and compared against USD references. A USDG peg wobble would appear as a correlated deviation across all 11 symbols simultaneously, which is the signature to check before reading a broad move as tracking error." + - "Caveats: the issuer can pause or blocklist a token (the harness drops the sample and lets the series age out rather than freezing), and the Stock contract carries a split multiplier; a corporate action can look like a one-day deviation spike until reconciled." + +findings: + - "{{best_name}} tracks its reference tightest at {{best_p50}} (p50, 24h) across {{count}} measured tokenized equities." + - "{{name:aapl}} deviates {{p50:aapl}} (p50, 24h) with roughly $330k of pool depth, the deepest tokenized-stock pool on the chain alongside {{name:nvda}} ({{p50:nvda}}, ~$290k)." + - "{{name:tsla}} ({{p50:tsla}}) has the most active pool of the cohort by swap count, which is what keeps an AMM price honest between arbitrage passes." + - "The spread between the regular-hours deviation and the closed-market drift is the number this bench exists for: pools keep trading nights and weekends with no reference to arb against, and the drift until Monday open is measurable here, live." + - "Of the ~95 equities Robinhood tokenized, only 14 have a pool anyone actually trades. The other 80 sit in issuer-seeded placeholder pools at 90-95% fees with zero swaps. Tokenization is easy; markets are hard." + +faq: + - q: "Do tokenized stocks track their real market price?" + a: "During regular Nasdaq hours, mostly yes: arbitrageurs keep the Uniswap v4 pools within tens of basis points of the reference on the liquid names, and this page shows the live per-symbol number. Outside market hours the tether loosens, the pools keep trading around the clock with no authoritative reference, and the closed-session drift measured here is the honest answer to how far a 24/7 stock wanders from its 9:30-to-4 anchor." + - q: "What happens to tokenized stock prices on weekends?" + a: "The pools trade continuously, so the onchain price becomes pure supply and demand against the Friday close. This bench samples that drift every minute all weekend and publishes it as the closed-session series. Monday pre-market usually snaps it back; the interesting data is how far it got before that." + - q: "Why only 11 stocks when Robinhood tokenized about 95?" + a: "Because only 14 of the 95 have a USDG pool with real liquidity and swap activity, and 3 of those are unmeasurable: SpaceX has no listed reference price, and SNDK plus QQQ hold under $2k of depth. The other 80 tokens sit in placeholder pools at 90-95% fees that nobody has ever traded. The gap between tokens issued and markets that exist is one of this page's findings." + - q: "How is the onchain price read?" + a: "One batched keyless eth_call per minute to the StateView contract of the Uniswap v4 singleton on Robinhood Chain, decoding sqrtPriceX96 from getSlot0 for each pool and converting with the pool's currency ordering and the 6-vs-18 decimals gap. No indexer, no API key, reproducible with curl." + - q: "Where does the reference price come from?" + a: "Yahoo Finance's public chart API, one batched call per minute with the session windows read from currentTradingPeriod, which is holiday-aware. When the market is closed the reference is the last regular close, which is exactly the baseline the weekend drift should be measured against." + +source: https://github.com/ChainBench/OpenChainBench/tree/main/harnesses/tokenized-stock-peg + +prometheus: + window: 24h + freshness_metric: tsp_deviation_bps + +# Metrics emitted by the tokenized-stock-peg harness: +# tsp_deviation_bps{asset, market_state} gauge, abs deviation in bps +# tsp_price_onchain_usdg{asset} gauge, v4 pool spot +# tsp_price_reference_usd{asset} gauge, yahoo reference +# tsp_reference_age_seconds{asset} gauge +# tsp_market_session{market_state} gauge, active session flag +# tsp_health{asset} gauge +# tsp_source_call_total{source, result} counter +# tsp_source_latency_milliseconds{source} gauge + +providers: + - slug: nvda + name: NVDA + tag: "Nvidia, ~$290k pool depth, 0.3% fee" + formula: "p50 over 24h of the absolute deviation (bps) between the NVDA Uniswap v4 pool price on Robinhood Chain and the Yahoo reference, regular market hours only." + queries: + p50: quantile_over_time(0.50, tsp_deviation_bps{issuer="robinhood", asset="nvda", market_state="regular"}[24h]) + p90: quantile_over_time(0.90, tsp_deviation_bps{issuer="robinhood", asset="nvda", market_state="regular"}[24h]) + p99: quantile_over_time(0.99, tsp_deviation_bps{issuer="robinhood", asset="nvda", market_state="regular"}[24h]) + mean: avg_over_time(tsp_deviation_bps{issuer="robinhood", asset="nvda", market_state="regular"}[24h]) + success: avg_over_time(tsp_health{asset="nvda"}[24h]) + series: tsp_deviation_bps{issuer="robinhood", asset="nvda", market_state="regular"} + - slug: aapl + name: AAPL + tag: "Apple, ~$330k pool depth, 1% fee" + formula: "p50 over 24h of the absolute deviation (bps) between the AAPL Uniswap v4 pool price on Robinhood Chain and the Yahoo reference, regular market hours only." + queries: + p50: quantile_over_time(0.50, tsp_deviation_bps{issuer="robinhood", asset="aapl", market_state="regular"}[24h]) + p90: quantile_over_time(0.90, tsp_deviation_bps{issuer="robinhood", asset="aapl", market_state="regular"}[24h]) + p99: quantile_over_time(0.99, tsp_deviation_bps{issuer="robinhood", asset="aapl", market_state="regular"}[24h]) + mean: avg_over_time(tsp_deviation_bps{issuer="robinhood", asset="aapl", market_state="regular"}[24h]) + success: avg_over_time(tsp_health{asset="aapl"}[24h]) + series: tsp_deviation_bps{issuer="robinhood", asset="aapl", market_state="regular"} + - slug: googl + name: GOOGL + tag: "Alphabet, ~$280k pool depth, 1% fee" + formula: "p50 over 24h of the absolute deviation (bps) between the GOOGL Uniswap v4 pool price on Robinhood Chain and the Yahoo reference, regular market hours only." + queries: + p50: quantile_over_time(0.50, tsp_deviation_bps{issuer="robinhood", asset="googl", market_state="regular"}[24h]) + p90: quantile_over_time(0.90, tsp_deviation_bps{issuer="robinhood", asset="googl", market_state="regular"}[24h]) + p99: quantile_over_time(0.99, tsp_deviation_bps{issuer="robinhood", asset="googl", market_state="regular"}[24h]) + mean: avg_over_time(tsp_deviation_bps{issuer="robinhood", asset="googl", market_state="regular"}[24h]) + success: avg_over_time(tsp_health{asset="googl"}[24h]) + series: tsp_deviation_bps{issuer="robinhood", asset="googl", market_state="regular"} + - slug: tsla + name: TSLA + tag: "Tesla, most active pool of the cohort, 0.3% fee" + formula: "p50 over 24h of the absolute deviation (bps) between the TSLA Uniswap v4 pool price on Robinhood Chain and the Yahoo reference, regular market hours only." + queries: + p50: quantile_over_time(0.50, tsp_deviation_bps{issuer="robinhood", asset="tsla", market_state="regular"}[24h]) + p90: quantile_over_time(0.90, tsp_deviation_bps{issuer="robinhood", asset="tsla", market_state="regular"}[24h]) + p99: quantile_over_time(0.99, tsp_deviation_bps{issuer="robinhood", asset="tsla", market_state="regular"}[24h]) + mean: avg_over_time(tsp_deviation_bps{issuer="robinhood", asset="tsla", market_state="regular"}[24h]) + success: avg_over_time(tsp_health{asset="tsla"}[24h]) + series: tsp_deviation_bps{issuer="robinhood", asset="tsla", market_state="regular"} + - slug: pltr + name: PLTR + tag: "Palantir, ~$31k pool depth, 1% fee" + formula: "p50 over 24h of the absolute deviation (bps) between the PLTR Uniswap v4 pool price on Robinhood Chain and the Yahoo reference, regular market hours only." + queries: + p50: quantile_over_time(0.50, tsp_deviation_bps{issuer="robinhood", asset="pltr", market_state="regular"}[24h]) + p90: quantile_over_time(0.90, tsp_deviation_bps{issuer="robinhood", asset="pltr", market_state="regular"}[24h]) + p99: quantile_over_time(0.99, tsp_deviation_bps{issuer="robinhood", asset="pltr", market_state="regular"}[24h]) + mean: avg_over_time(tsp_deviation_bps{issuer="robinhood", asset="pltr", market_state="regular"}[24h]) + success: avg_over_time(tsp_health{asset="pltr"}[24h]) + series: tsp_deviation_bps{issuer="robinhood", asset="pltr", market_state="regular"} + - slug: meta + name: META + tag: "Meta, ~$89k pool depth, 0.3% fee" + formula: "p50 over 24h of the absolute deviation (bps) between the META Uniswap v4 pool price on Robinhood Chain and the Yahoo reference, regular market hours only." + queries: + p50: quantile_over_time(0.50, tsp_deviation_bps{issuer="robinhood", asset="meta", market_state="regular"}[24h]) + p90: quantile_over_time(0.90, tsp_deviation_bps{issuer="robinhood", asset="meta", market_state="regular"}[24h]) + p99: quantile_over_time(0.99, tsp_deviation_bps{issuer="robinhood", asset="meta", market_state="regular"}[24h]) + mean: avg_over_time(tsp_deviation_bps{issuer="robinhood", asset="meta", market_state="regular"}[24h]) + success: avg_over_time(tsp_health{asset="meta"}[24h]) + series: tsp_deviation_bps{issuer="robinhood", asset="meta", market_state="regular"} + - slug: amd + name: AMD + tag: "AMD, ~$25k pool depth, 1% fee" + formula: "p50 over 24h of the absolute deviation (bps) between the AMD Uniswap v4 pool price on Robinhood Chain and the Yahoo reference, regular market hours only." + queries: + p50: quantile_over_time(0.50, tsp_deviation_bps{issuer="robinhood", asset="amd", market_state="regular"}[24h]) + p90: quantile_over_time(0.90, tsp_deviation_bps{issuer="robinhood", asset="amd", market_state="regular"}[24h]) + p99: quantile_over_time(0.99, tsp_deviation_bps{issuer="robinhood", asset="amd", market_state="regular"}[24h]) + mean: avg_over_time(tsp_deviation_bps{issuer="robinhood", asset="amd", market_state="regular"}[24h]) + success: avg_over_time(tsp_health{asset="amd"}[24h]) + series: tsp_deviation_bps{issuer="robinhood", asset="amd", market_state="regular"} + - slug: msft + name: MSFT + tag: "Microsoft, ~$50k pool depth, 2% fee" + formula: "p50 over 24h of the absolute deviation (bps) between the MSFT Uniswap v4 pool price on Robinhood Chain and the Yahoo reference, regular market hours only." + queries: + p50: quantile_over_time(0.50, tsp_deviation_bps{issuer="robinhood", asset="msft", market_state="regular"}[24h]) + p90: quantile_over_time(0.90, tsp_deviation_bps{issuer="robinhood", asset="msft", market_state="regular"}[24h]) + p99: quantile_over_time(0.99, tsp_deviation_bps{issuer="robinhood", asset="msft", market_state="regular"}[24h]) + mean: avg_over_time(tsp_deviation_bps{issuer="robinhood", asset="msft", market_state="regular"}[24h]) + success: avg_over_time(tsp_health{asset="msft"}[24h]) + series: tsp_deviation_bps{issuer="robinhood", asset="msft", market_state="regular"} + - slug: amzn + name: AMZN + tag: "Amazon, ~$24k pool depth, 2% fee" + formula: "p50 over 24h of the absolute deviation (bps) between the AMZN Uniswap v4 pool price on Robinhood Chain and the Yahoo reference, regular market hours only." + queries: + p50: quantile_over_time(0.50, tsp_deviation_bps{issuer="robinhood", asset="amzn", market_state="regular"}[24h]) + p90: quantile_over_time(0.90, tsp_deviation_bps{issuer="robinhood", asset="amzn", market_state="regular"}[24h]) + p99: quantile_over_time(0.99, tsp_deviation_bps{issuer="robinhood", asset="amzn", market_state="regular"}[24h]) + mean: avg_over_time(tsp_deviation_bps{issuer="robinhood", asset="amzn", market_state="regular"}[24h]) + success: avg_over_time(tsp_health{asset="amzn"}[24h]) + series: tsp_deviation_bps{issuer="robinhood", asset="amzn", market_state="regular"} + - slug: spy + name: SPY + tag: "S&P 500 ETF, ~$65k pool depth, 1% fee" + formula: "p50 over 24h of the absolute deviation (bps) between the SPY Uniswap v4 pool price on Robinhood Chain and the Yahoo reference, regular market hours only." + queries: + p50: quantile_over_time(0.50, tsp_deviation_bps{issuer="robinhood", asset="spy", market_state="regular"}[24h]) + p90: quantile_over_time(0.90, tsp_deviation_bps{issuer="robinhood", asset="spy", market_state="regular"}[24h]) + p99: quantile_over_time(0.99, tsp_deviation_bps{issuer="robinhood", asset="spy", market_state="regular"}[24h]) + mean: avg_over_time(tsp_deviation_bps{issuer="robinhood", asset="spy", market_state="regular"}[24h]) + success: avg_over_time(tsp_health{asset="spy"}[24h]) + series: tsp_deviation_bps{issuer="robinhood", asset="spy", market_state="regular"} + - slug: mu + name: MU + tag: "Micron, ~$35k pool depth, 1% fee" + formula: "p50 over 24h of the absolute deviation (bps) between the MU Uniswap v4 pool price on Robinhood Chain and the Yahoo reference, regular market hours only." + queries: + p50: quantile_over_time(0.50, tsp_deviation_bps{issuer="robinhood", asset="mu", market_state="regular"}[24h]) + p90: quantile_over_time(0.90, tsp_deviation_bps{issuer="robinhood", asset="mu", market_state="regular"}[24h]) + p99: quantile_over_time(0.99, tsp_deviation_bps{issuer="robinhood", asset="mu", market_state="regular"}[24h]) + mean: avg_over_time(tsp_deviation_bps{issuer="robinhood", asset="mu", market_state="regular"}[24h]) + success: avg_over_time(tsp_health{asset="mu"}[24h]) + series: tsp_deviation_bps{issuer="robinhood", asset="mu", market_state="regular"} diff --git a/benchmarks/tokenized-stock-weekend-drift.yml b/benchmarks/tokenized-stock-weekend-drift.yml new file mode 100644 index 00000000..a5835a4b --- /dev/null +++ b/benchmarks/tokenized-stock-weekend-drift.yml @@ -0,0 +1,199 @@ +# OpenChainBench. Bench № 079 + +slug: tokenized-stock-weekend-drift +number: "079" +title: Tokenized stock weekend drift, how far each equity wanders Friday to Monday +seo_title: "Tokenized stock weekend drift 2026" +seo_description: "How far do onchain tokenized stocks drift from Friday close during the weekend? AAPL, TSLA, NVDA and 8 more measured live on Robinhood Chain, in bps." +subtitle: "Maximum deviation between each tokenized stock's pool price and the last Nasdaq close, measured every minute across the 60-hour closed-market window from Friday 21:00 UTC to Monday 13:30 UTC." + +category: RWA +status: live +metric: Weekend max drift +unit: bps +higher_is_better: false + +seo_intro: | + Nasdaq closes every Friday at 4 pm ET and reopens Monday at 9:30 am, + a 60 hour window with no reference price. The AMM pools on + Robinhood Chain keep trading 24/7 anyway. This page measures how + far the onchain price of each tokenized stock actually wanders + during those 60 hours before the Monday open snaps it back. Same + data pipeline as the tokenized stock peg benchmark, filtered to the + closed session and aggregated per weekend. The interesting rows are + the thin pools: MSFT and AMZN drift multiple hundreds of basis + points on a quiet Sunday because no arbitrageur has a reference to + arb against until the bell rings on Monday. The chart nobody else + publishes. + +abstract: | + Every 60 seconds during the closed market session, the harness + records the deviation between the Uniswap v4 pool price on + Robinhood Chain and the last Yahoo regular-hours close. The + weekend metric is the maximum of that deviation over the full + 60-hour Friday-close-to-Monday-open window, per symbol. Purely + derived from the tokenized-stock-peg series already in Prometheus, + no new probe. + +methodology: + - "Data source: the tsp_deviation_bps series emitted by the tokenized-stock-peg harness with the label market_state=\"closed\". No new probe; the metric is a recording rule on existing samples." + - "Session windows come from Yahoo currentTradingPeriod (holiday aware), which the harness already labels; the closed state runs from post-market close (roughly 21:00 UTC) to the next regular open (roughly 13:30 UTC)." + - "Weekend max drift: max_over_time(tsp_deviation_bps{issuer=\"robinhood\", market_state=\"closed\"}[72h]), aggregated per calendar weekend and labeled with the ISO week. The 72h window covers all pre-market and after-hours slots on either side of the weekend, not just the 60h weekend gap." + - "Cohort: the 11 tokenized equities measured by the tokenized stock peg bench (NVDA, AAPL, GOOGL, TSLA, PLTR, META, AMD, MSFT, AMZN, SPY, MU). Same pool addresses, same reference, same fee tiers." + - "The measurement is a bound, not a fair value: an AMM can sit anywhere inside its fee band without creating arbitrage, so weekend drift on a 2 percent fee pool starts at 200 bps by construction. Read the drift number alongside the pool fee, disclosed on the tokenized-stock-peg page." + - "Not a claim about corporate fair value: the reference is Friday close, not the theoretical opening indication. When earnings, dividends or macro events happen on the weekend, Monday open will differ from Friday close by the same amount whether the pool moved or not." + +findings: + - "{{best_name}} held tightest last weekend at {{best_p50}} (max drift, bps) across {{count}} tokenized equities." + - "{{name:tsla}} ({{p50:tsla}}) has the most active pool of the cohort, and its weekend drift is the closest read on what a 24/7 stock is worth on a Sunday when nobody has a reference to arb against." + - "The largest weekend drifts consistently come from the thin-pool symbols (MSFT, AMZN, MU). That is not a peg failure, it is the arithmetic of a 2 percent fee band on a pool depth in the low tens of thousands." + - "Every reading resets at Monday open. The page is the only place that captures the intra-weekend maximum before the reset happens." + +faq: + - q: "What is the weekend drift on Robinhood Chain tokenized stocks?" + a: "It is the largest gap between the pool price and Friday's Nasdaq close observed anywhere in the 60 hour weekend, per symbol, measured every 60 seconds. Nasdaq is closed all weekend but the AMM keeps trading, and this page shows how far apart the two go before the Monday open snaps them together again." + - q: "Why do pools drift when nobody is trading?" + a: "Some pools do get traded on the weekend, and even a moderate imbalance moves the price meaningfully on a thin depth. But even zero-volume pools can drift on paper if a single one-sided quote lands: the price is the last swap, and there is no reference to arb against until Monday open. That is what the number here is capturing." + - q: "Is a high drift bad?" + a: "It reads as inefficient markets on paper, but during the 60 hour Nasdaq close there is no fair value to converge to, and every drift closes at the next open. The number matters most for downstream products (lending, structured payoffs) that use the tokenized price as an oracle when the real market is closed." + - q: "Which stocks drift the most on weekends?" + a: "The pattern is fee tier times pool depth. On the current cohort, MSFT and AMZN (2 percent fee tier, sub 50k depth) hit the widest weekend drifts, sometimes several hundred basis points. NVDA and TSLA (0.3 percent fee tier, several hundred thousand of depth) drift the least." + +source: https://github.com/ChainBench/OpenChainBench/tree/main/harnesses/tokenized-stock-peg + +prometheus: + window: 24h + freshness_metric: tsp_deviation_bps + +providers: + - slug: nvda + name: NVDA + tag: "Nvidia, ~$290k pool depth, 0.3% fee" + formula: "Maximum deviation between the NVDA pool on Robinhood Chain and Friday's Nasdaq close, over the closed-market window, in bps." + queries: + p50: quantile_over_time(0.50, max_over_time(tsp_deviation_bps{issuer="robinhood", asset="nvda", market_state="closed"}[72h])[7d:6h]) + p90: quantile_over_time(0.90, max_over_time(tsp_deviation_bps{issuer="robinhood", asset="nvda", market_state="closed"}[72h])[7d:6h]) + p99: quantile_over_time(0.99, max_over_time(tsp_deviation_bps{issuer="robinhood", asset="nvda", market_state="closed"}[72h])[7d:6h]) + mean: avg_over_time(max_over_time(tsp_deviation_bps{issuer="robinhood", asset="nvda", market_state="closed"}[72h])[7d:6h]) + success: avg_over_time(tsp_health{asset="nvda"}[24h]) + series: max_over_time(tsp_deviation_bps{issuer="robinhood", asset="nvda", market_state="closed"}[72h]) + + - slug: aapl + name: AAPL + tag: "Apple, ~$330k pool depth, 1% fee" + formula: "Maximum deviation between the AAPL pool on Robinhood Chain and Friday's Nasdaq close, over the closed-market window, in bps." + queries: + p50: quantile_over_time(0.50, max_over_time(tsp_deviation_bps{issuer="robinhood", asset="aapl", market_state="closed"}[72h])[7d:6h]) + p90: quantile_over_time(0.90, max_over_time(tsp_deviation_bps{issuer="robinhood", asset="aapl", market_state="closed"}[72h])[7d:6h]) + p99: quantile_over_time(0.99, max_over_time(tsp_deviation_bps{issuer="robinhood", asset="aapl", market_state="closed"}[72h])[7d:6h]) + mean: avg_over_time(max_over_time(tsp_deviation_bps{issuer="robinhood", asset="aapl", market_state="closed"}[72h])[7d:6h]) + success: avg_over_time(tsp_health{asset="aapl"}[24h]) + series: max_over_time(tsp_deviation_bps{issuer="robinhood", asset="aapl", market_state="closed"}[72h]) + + - slug: googl + name: GOOGL + tag: "Alphabet, ~$280k pool depth, 1% fee" + formula: "Maximum deviation between the GOOGL pool and Friday's Nasdaq close, in bps." + queries: + p50: quantile_over_time(0.50, max_over_time(tsp_deviation_bps{issuer="robinhood", asset="googl", market_state="closed"}[72h])[7d:6h]) + p90: quantile_over_time(0.90, max_over_time(tsp_deviation_bps{issuer="robinhood", asset="googl", market_state="closed"}[72h])[7d:6h]) + p99: quantile_over_time(0.99, max_over_time(tsp_deviation_bps{issuer="robinhood", asset="googl", market_state="closed"}[72h])[7d:6h]) + mean: avg_over_time(max_over_time(tsp_deviation_bps{issuer="robinhood", asset="googl", market_state="closed"}[72h])[7d:6h]) + success: avg_over_time(tsp_health{asset="googl"}[24h]) + series: max_over_time(tsp_deviation_bps{issuer="robinhood", asset="googl", market_state="closed"}[72h]) + + - slug: tsla + name: TSLA + tag: "Tesla, most active pool of the cohort, 0.3% fee" + formula: "Maximum deviation between the TSLA pool and Friday's Nasdaq close, in bps." + queries: + p50: quantile_over_time(0.50, max_over_time(tsp_deviation_bps{issuer="robinhood", asset="tsla", market_state="closed"}[72h])[7d:6h]) + p90: quantile_over_time(0.90, max_over_time(tsp_deviation_bps{issuer="robinhood", asset="tsla", market_state="closed"}[72h])[7d:6h]) + p99: quantile_over_time(0.99, max_over_time(tsp_deviation_bps{issuer="robinhood", asset="tsla", market_state="closed"}[72h])[7d:6h]) + mean: avg_over_time(max_over_time(tsp_deviation_bps{issuer="robinhood", asset="tsla", market_state="closed"}[72h])[7d:6h]) + success: avg_over_time(tsp_health{asset="tsla"}[24h]) + series: max_over_time(tsp_deviation_bps{issuer="robinhood", asset="tsla", market_state="closed"}[72h]) + + - slug: msft + name: MSFT + tag: "Microsoft, ~$50k pool depth, 2% fee" + formula: "Maximum deviation between the MSFT pool and Friday's Nasdaq close, in bps." + queries: + p50: quantile_over_time(0.50, max_over_time(tsp_deviation_bps{issuer="robinhood", asset="msft", market_state="closed"}[72h])[7d:6h]) + p90: quantile_over_time(0.90, max_over_time(tsp_deviation_bps{issuer="robinhood", asset="msft", market_state="closed"}[72h])[7d:6h]) + p99: quantile_over_time(0.99, max_over_time(tsp_deviation_bps{issuer="robinhood", asset="msft", market_state="closed"}[72h])[7d:6h]) + mean: avg_over_time(max_over_time(tsp_deviation_bps{issuer="robinhood", asset="msft", market_state="closed"}[72h])[7d:6h]) + success: avg_over_time(tsp_health{asset="msft"}[24h]) + series: max_over_time(tsp_deviation_bps{issuer="robinhood", asset="msft", market_state="closed"}[72h]) + + - slug: amzn + name: AMZN + tag: "Amazon, ~$24k pool depth, 2% fee" + formula: "Maximum deviation between the AMZN pool and Friday's Nasdaq close, in bps." + queries: + p50: quantile_over_time(0.50, max_over_time(tsp_deviation_bps{issuer="robinhood", asset="amzn", market_state="closed"}[72h])[7d:6h]) + p90: quantile_over_time(0.90, max_over_time(tsp_deviation_bps{issuer="robinhood", asset="amzn", market_state="closed"}[72h])[7d:6h]) + p99: quantile_over_time(0.99, max_over_time(tsp_deviation_bps{issuer="robinhood", asset="amzn", market_state="closed"}[72h])[7d:6h]) + mean: avg_over_time(max_over_time(tsp_deviation_bps{issuer="robinhood", asset="amzn", market_state="closed"}[72h])[7d:6h]) + success: avg_over_time(tsp_health{asset="amzn"}[24h]) + series: max_over_time(tsp_deviation_bps{issuer="robinhood", asset="amzn", market_state="closed"}[72h]) + + - slug: meta + name: META + tag: "Meta, ~$89k pool depth, 0.3% fee" + formula: "Maximum deviation between the META pool and Friday's Nasdaq close, in bps." + queries: + p50: quantile_over_time(0.50, max_over_time(tsp_deviation_bps{issuer="robinhood", asset="meta", market_state="closed"}[72h])[7d:6h]) + p90: quantile_over_time(0.90, max_over_time(tsp_deviation_bps{issuer="robinhood", asset="meta", market_state="closed"}[72h])[7d:6h]) + p99: quantile_over_time(0.99, max_over_time(tsp_deviation_bps{issuer="robinhood", asset="meta", market_state="closed"}[72h])[7d:6h]) + mean: avg_over_time(max_over_time(tsp_deviation_bps{issuer="robinhood", asset="meta", market_state="closed"}[72h])[7d:6h]) + success: avg_over_time(tsp_health{asset="meta"}[24h]) + series: max_over_time(tsp_deviation_bps{issuer="robinhood", asset="meta", market_state="closed"}[72h]) + + - slug: amd + name: AMD + tag: "AMD, ~$25k pool depth, 1% fee" + formula: "Maximum deviation between the AMD pool and Friday's Nasdaq close, in bps." + queries: + p50: quantile_over_time(0.50, max_over_time(tsp_deviation_bps{issuer="robinhood", asset="amd", market_state="closed"}[72h])[7d:6h]) + p90: quantile_over_time(0.90, max_over_time(tsp_deviation_bps{issuer="robinhood", asset="amd", market_state="closed"}[72h])[7d:6h]) + p99: quantile_over_time(0.99, max_over_time(tsp_deviation_bps{issuer="robinhood", asset="amd", market_state="closed"}[72h])[7d:6h]) + mean: avg_over_time(max_over_time(tsp_deviation_bps{issuer="robinhood", asset="amd", market_state="closed"}[72h])[7d:6h]) + success: avg_over_time(tsp_health{asset="amd"}[24h]) + series: max_over_time(tsp_deviation_bps{issuer="robinhood", asset="amd", market_state="closed"}[72h]) + + - slug: pltr + name: PLTR + tag: "Palantir, ~$31k pool depth, 1% fee" + formula: "Maximum deviation between the PLTR pool and Friday's Nasdaq close, in bps." + queries: + p50: quantile_over_time(0.50, max_over_time(tsp_deviation_bps{issuer="robinhood", asset="pltr", market_state="closed"}[72h])[7d:6h]) + p90: quantile_over_time(0.90, max_over_time(tsp_deviation_bps{issuer="robinhood", asset="pltr", market_state="closed"}[72h])[7d:6h]) + p99: quantile_over_time(0.99, max_over_time(tsp_deviation_bps{issuer="robinhood", asset="pltr", market_state="closed"}[72h])[7d:6h]) + mean: avg_over_time(max_over_time(tsp_deviation_bps{issuer="robinhood", asset="pltr", market_state="closed"}[72h])[7d:6h]) + success: avg_over_time(tsp_health{asset="pltr"}[24h]) + series: max_over_time(tsp_deviation_bps{issuer="robinhood", asset="pltr", market_state="closed"}[72h]) + + - slug: spy + name: SPY + tag: "S&P 500 ETF, ~$65k pool depth, 1% fee" + formula: "Maximum deviation between the SPY pool and Friday's NYSE close, in bps." + queries: + p50: quantile_over_time(0.50, max_over_time(tsp_deviation_bps{issuer="robinhood", asset="spy", market_state="closed"}[72h])[7d:6h]) + p90: quantile_over_time(0.90, max_over_time(tsp_deviation_bps{issuer="robinhood", asset="spy", market_state="closed"}[72h])[7d:6h]) + p99: quantile_over_time(0.99, max_over_time(tsp_deviation_bps{issuer="robinhood", asset="spy", market_state="closed"}[72h])[7d:6h]) + mean: avg_over_time(max_over_time(tsp_deviation_bps{issuer="robinhood", asset="spy", market_state="closed"}[72h])[7d:6h]) + success: avg_over_time(tsp_health{asset="spy"}[24h]) + series: max_over_time(tsp_deviation_bps{issuer="robinhood", asset="spy", market_state="closed"}[72h]) + + - slug: mu + name: MU + tag: "Micron, ~$35k pool depth, 1% fee" + formula: "Maximum deviation between the MU pool and Friday's Nasdaq close, in bps." + queries: + p50: quantile_over_time(0.50, max_over_time(tsp_deviation_bps{issuer="robinhood", asset="mu", market_state="closed"}[72h])[7d:6h]) + p90: quantile_over_time(0.90, max_over_time(tsp_deviation_bps{issuer="robinhood", asset="mu", market_state="closed"}[72h])[7d:6h]) + p99: quantile_over_time(0.99, max_over_time(tsp_deviation_bps{issuer="robinhood", asset="mu", market_state="closed"}[72h])[7d:6h]) + mean: avg_over_time(max_over_time(tsp_deviation_bps{issuer="robinhood", asset="mu", market_state="closed"}[72h])[7d:6h]) + success: avg_over_time(tsp_health{asset="mu"}[24h]) + series: max_over_time(tsp_deviation_bps{issuer="robinhood", asset="mu", market_state="closed"}[72h]) diff --git a/benchmarks/usdy-nav-basis.yml b/benchmarks/usdy-nav-basis.yml new file mode 100644 index 00000000..9b756be2 --- /dev/null +++ b/benchmarks/usdy-nav-basis.yml @@ -0,0 +1,90 @@ +# OpenChainBench. Bench № 078 + +slug: usdy-nav-basis +number: "078" +title: USDY NAV basis, live market price vs official redemption rate +seo_title: "USDY price vs NAV 2026" +seo_description: "Does USDY trade at its NAV? Live basis between Ondo's tokenized treasury market price (Orca, Pyth) and its official redemption rate, in bps, keyless." +subtitle: "Signed basis between USDY's onchain market price and the official Ondo redemption rate published on Pyth, in basis points. Tokenized treasuries are the largest RWA segment; this is the live answer to whether the market actually prices the NAV." + +category: RWA +status: live +metric: NAV basis +unit: bps +higher_is_better: false + +seo_intro: | + USDY is Ondo's yield-bearing tokenized treasury note, one of the + largest RWA tokens by float. Its official value is the redemption + rate Ondo publishes (the NAV a redeeming holder receives), but its + price on the open market is whatever the pools say. This page + measures the gap live: the Orca USDY/USDC whirlpool on Solana (the + deepest genuine venue, about $2.9M) and the Pyth market composite, + each against the Pyth redemption rate feed, every 60 seconds, + keyless on every leg. A persistent negative basis means the market + discounts the NAV (liquidity preference, exit friction); a positive + one means buyers pay a premium over redemption value. Most tokenized + treasuries (OUSG, BUIDL, BENJI) are transfer-restricted and never + trade on open pools, so their NAV can never be market-tested; USDY + is the rare one where the question is measurable at all. + +abstract: | + Every 60 seconds the harness reads the Ondo redemption rate and the + Pyth USDY/USD market composite from one Hermes call, and the Orca + USDY/USDC whirlpool price from one Solana getAccountInfo (sqrtPrice + decoded from the account bytes). Basis = (market - NAV) / NAV in + signed basis points per venue. No keys, no transactions. + +methodology: + - "NAV leg: the Pyth Hermes feed Crypto.USDY/USD.RR (the redemption rate Ondo publishes onchain), fetched keyless. The same Hermes call carries the Pyth USDY/USD market composite, which doubles as a venue row." + - "Market leg, Orca: the USDY/USDC whirlpool on Solana (~$2.9M, the deepest genuine USDY pool anywhere), price decoded straight from getAccountInfo bytes (sqrtPrice u128 at offset 65, both mints 6 decimals). Keyless against a public RPC." + - "Basis: signed, (market - NAV) / NAV x 10000. Negative = the market discounts the NAV. The ranking sorts by absolute basis; the sign is the story and both are displayed." + - "Excluded, verified 2026-07-13: the Arbitrum Camelot USDY/USDC pool holds 232 USDY against 7M USDC, effectively drained, its stale price sitting ~345bps under NAV with near zero volume. Kept out of the ranking as the textbook example of why pool depth gates peg quality; revisited monthly." + - "Also not measurable, disclosed: OUSG, BUIDL and BENJI are transfer-restricted mint/redeem instruments with no genuine open pools, so no market test of their NAV exists to publish." + - "USDY is yield accruing: the redemption rate rises daily, so a naive USD peg comparison would show permanent drift. Comparing against the live RR feed removes that by construction." + +findings: + - "{{best_name}} trades closest to NAV at {{best_p50}} (p50 absolute basis, 24h) across {{count}} measured venues." + - "{{name:orca-solana}} ({{p50:orca-solana}}) is the deepest genuine USDY venue anywhere at about $2.9M; its basis is the closest thing to a market verdict on Ondo's published NAV." + - "{{name:pyth-market}} ({{p50:pyth-market}}) aggregates offchain and onchain USDY trading into one composite; its spread against the RR feed is the cleanest single number for the NAV discount." + - "The excluded Camelot pool on Arbitrum is the finding that did not make the table: drained to 232 USDY, price frozen ~345bps under NAV. Depth is not a detail in RWA pricing, it is the whole game." + +faq: + - q: "Does USDY trade at its NAV?" + a: "Close to it, with a measurable basis that this page tracks live. At verification the Orca pool sat within a few bps of the redemption rate and the Pyth market composite about 10bps under. A persistent discount reflects exit friction and liquidity preference, not a broken product; the point is that it is now measured instead of assumed." + - q: "Why only two venues?" + a: "Because genuine USDY liquidity is rarer than the token's float suggests. The Arbitrum Camelot pool is drained and excluded (232 USDY left, price frozen well under NAV), and most other tokenized treasuries never trade openly at all. Two honest venues beat five misleading ones." + - q: "What does a negative basis mean?" + a: "The market prices USDY under its redemption value. Redeeming through Ondo takes time and has minimums, so sellers who want out now accept a small discount. The size and persistence of that discount is exactly what this bench publishes." + - q: "Why is USDY measurable when OUSG and BUIDL are not?" + a: "OUSG, BUIDL and BENJI are transfer-restricted: they move between allowlisted addresses and cannot trade on open AMMs, so no market price exists to compare against NAV. USDY circulates freely and has genuine pools, which makes it the one tokenized treasury where the NAV question can be answered by measurement." + +source: https://github.com/ChainBench/OpenChainBench/tree/main/harnesses/usdy-nav-basis + +prometheus: + window: 24h + freshness_metric: usdy_basis_bps + +providers: + - slug: orca-solana + name: Orca (Solana) + tag: "USDY/USDC whirlpool, ~$2.9M, deepest genuine USDY venue" + formula: "p50 over 24h of the absolute signed basis (bps) between the Orca whirlpool USDY/USDC price and the Pyth redemption rate." + queries: + p50: quantile_over_time(0.50, abs(usdy_basis_bps{venue="orca-solana"})[24h:]) + p90: quantile_over_time(0.90, abs(usdy_basis_bps{venue="orca-solana"})[24h:]) + p99: quantile_over_time(0.99, abs(usdy_basis_bps{venue="orca-solana"})[24h:]) + mean: avg_over_time(abs(usdy_basis_bps{venue="orca-solana"})[24h:]) + success: avg_over_time(usdy_health{venue="orca-solana"}[24h]) + series: usdy_basis_bps{venue="orca-solana"} + - slug: pyth-market + name: Pyth market composite + tag: "Crypto.USDY/USD aggregate vs the RR feed, same oracle network" + formula: "p50 over 24h of the absolute signed basis (bps) between the Pyth USDY/USD market composite and the Pyth redemption rate." + queries: + p50: quantile_over_time(0.50, abs(usdy_basis_bps{venue="pyth-market"})[24h:]) + p90: quantile_over_time(0.90, abs(usdy_basis_bps{venue="pyth-market"})[24h:]) + p99: quantile_over_time(0.99, abs(usdy_basis_bps{venue="pyth-market"})[24h:]) + mean: avg_over_time(abs(usdy_basis_bps{venue="pyth-market"})[24h:]) + success: avg_over_time(usdy_health{venue="pyth-market"}[24h]) + series: usdy_basis_bps{venue="pyth-market"} diff --git a/benchmarks/ws-head-latency.yml b/benchmarks/ws-head-latency.yml new file mode 100644 index 00000000..49a42ebd --- /dev/null +++ b/benchmarks/ws-head-latency.yml @@ -0,0 +1,127 @@ +# OpenChainBench. Bench № 081 + +slug: ws-head-latency +number: "081" +title: Fastest WebSocket newHeads push, live block-push lag across RPC providers +seo_title: "Fastest WebSocket RPC newHeads push 2026" +seo_description: "{{best_name}} pushes Ethereum newHeads first. Live WebSocket block-push lag for PublicNode, dRPC and Tenderly, measured per block against the earliest arrival in the cohort." +subtitle: Per-block WebSocket push lag in milliseconds versus the earliest provider to deliver the same Ethereum head, measured continuously from one eu-west vantage point. + +category: RPCs +status: live +metric: Block push lag +unit: ms +higher_is_better: false + +seo_intro: | + Every trading bot, MEV searcher, liquidation keeper and live dashboard + that subscribes to `eth_subscribe("newHeads")` cares about one number + first: how many milliseconds after the network produces a block does my + RPC provider actually push it to me? Marketing pages say "real-time + WebSocket streaming"; this page says who delivers block N first and by + how much the others trail. The harness holds one persistent WebSocket + per provider from the same eu-west host, timestamps every newHeads + frame on receipt, and scores each Ethereum block as a race: the + earliest arrival sets T0, every other provider's lag is its arrival + minus T0. The headline is pinned to Ethereum, where all three keyless + providers compete; the same harness also races Base heads and Solana + slots for cross-chain context. Because the measurement is relative, + the leader reads near zero by construction, so the win-rate and the + shape of the trailing distribution (p50 vs p99) are the numbers to + read, not the absolute magnitude. {{best_name}} currently leads at + {{best_p50}} (p50, 24 h) across {{count}} measured providers. + +abstract: | + We race RPC providers on WebSocket block delivery. One harness holds a + persistent `eth_subscribe("newHeads")` connection to each provider from + the same host and timestamps every head frame on receipt. Per block, + the earliest arrival across providers defines T0; each provider's + sample is its own arrival minus T0, in milliseconds. A block's cohort + stays open 5 seconds after the first arrival, then it is scored: + lag histogram per provider, a win for the earliest, and a gap counter + for any live provider that never delivered the block. The measurement + is relative to the fastest provider observed from our vantage point, + not an absolute network latency; the same-path subtraction is exactly + what makes the ordering fair. Solana runs the identical race on + `slotSubscribe` slot notifications. + +methodology: + - "Providers measured (keyless WebSocket endpoints): PublicNode (wss://ethereum-rpc.publicnode.com), dRPC (wss://eth.drpc.org), Tenderly (wss://mainnet.gateway.tenderly.co). Keyed providers (Alchemy, Infura, Chainstack) join the cohort when contributor keys are wired; the harness reads them from env vars and skips them cleanly when unset." + - "Chains: Ethereum (headline, all three providers compete), Base via newHeads and Solana via slotSubscribe as secondary cohorts. Headline queries on this page are pinned to chain=ethereum so the ranking never mixes block cadences." + - "Race scoring: the first provider to deliver block N sets T0. Each provider's sample for block N is arrival(N) minus T0 in milliseconds. The cohort closes 5 seconds after the first arrival; a provider arriving later than that is scored as a missed block (ws_block_gap_total), not as a huge latency sample." + - "Relative, not absolute: a single vantage point cannot separate its own network path from provider pipeline time, so we subtract the two arrivals over the same path instead. Consequence: the fastest provider reads ~0 by construction, and the honest readings are win rate, the trailers' lag distribution, and gap counts, not the leader's absolute number." + - "Blocks where only one provider delivered within the window emit no lag samples and no win: a one-horse race carries no relative information. Gap counters still increment for live providers that missed the block." + - "Connection discipline: one persistent WebSocket per (provider, chain), client ping every 30s, read deadline 60s, plus a 90s head watchdog that forces a reconnect when a provider keeps the socket heartbeat alive but silently drops the subscription (a failure mode we have observed in production on other benches). Reconnects use exponential backoff 2s to 60s and are counted in ws_reconnects_total." + - "Coverage score: ws_head_blocks_seen_total per provider divided by the cohort maximum over the same 24h window, clamped to 1. A provider that pushes fast but skips blocks loses on this column rather than hiding in the latency percentiles." + - "Aggregation: per-provider lag histogram (5ms to 10s exponential buckets), p50/p90/p99 via histogram_quantile over the 24h rate. Duplicate announcements of the same block on one connection are dropped; per-provider liveness is exported as ws_health (1 = connected and delivered a message in the last 120s)." + - "Cheat vector disclosure: a provider could theoretically pre-announce heads it has not fully validated to win the race. We keyed the cohort on block number as pushed; hash-level cross-checking against a reference node is on the v2 list. Reading win rate together with the gap counter already flags a provider that pushes early but wrong (retracted heads show up as cohort inconsistencies and reorg noise)." + - "Vantage point: a single eu-west host. Geography moves absolute arrival times but affects all providers over the same path; still, a provider whose nearest edge is far from eu-west is disadvantaged, and a us-east probe is the planned second vantage before any cross-region claim is made." + +findings: + - "{{best_name}} leads the Ethereum cohort at {{best_p50}} (p50, 24 h) across {{count}} measured providers. Remember the construction: the per-block winner defines zero, so the leader's near-zero p50 mostly says it wins most races; the informative numbers are the trailers' medians and the p99 tails." + - "{{name:drpc}} trails the winner by {{p50:drpc}} at the median. That is the extra time a liquidation bot or MEV searcher on dRPC's keyless endpoint waits for block N after the fastest feed already delivered it." + - "{{name:tenderly}} trails by {{p50:tenderly}} (p50, 24 h). Tenderly's gateway is built around simulation and developer tooling rather than raw head-push fan-out, and the push pipeline depth shows up in this race." + - "The p99 column is the integration-grade number: {{p99:publicnode}} / {{p99:drpc}} / {{p99:tenderly}}. A provider with a tight p50 but a multi-second p99 will occasionally hand your bot a stale head exactly when the chain is busiest." + - "Win rate and block coverage are the robust readings. A provider can look decent on median lag while silently skipping blocks; the success column (blocks seen vs cohort max) and ws_block_gap_total keep that visible." + +source: https://github.com/ChainBench/OpenChainBench/tree/main/harnesses/ws-head-latency + +expected_n: 7000 + +prometheus: + window: 24h + freshness_metric: ws_head_blocks_seen_total + +faq: + - q: "Which RPC provider pushes new Ethereum blocks first over WebSocket?" + a: "{{best_name}} currently wins most per-block races, with a median lag of {{best_p50}} versus the earliest arrival (p50, 24 h) across {{count}} measured keyless providers. The leaderboard re-scores on every Ethereum block, roughly every 12 seconds, so the answer reflects sustained live measurement, not a one-off test. Check the win rate and the trailers' p99 before wiring a latency-sensitive bot to any single provider." + - q: "Why does the fastest provider show a lag near zero?" + a: "By construction. Each block is scored as a race: the earliest arrival among providers defines T0 and every sample is measured against it, so the provider that wins most races accumulates mostly zero samples. This is deliberate. A single vantage point cannot measure absolute chain-to-client latency without a reference node co-located with block producers; it can measure, fairly, which provider delivers the same block earlier over the same network path. Read the leader's win rate and the trailers' distributions, not the leader's absolute number." + - q: "Is this absolute latency from block production to my client?" + a: "No. The number is relative lag versus the fastest provider observed from our eu-west host. Absolute delivery latency includes the propagation time from the block producer to each provider's infrastructure plus the path to you, which shifts with your geography. What transfers to your integration is the ordering and the spread: if a provider trails the winner by 800 ms at the median from our vantage point, its pipeline is structurally behind, and that gap does not disappear because you probe from another region." + - q: "Could a provider cheat by pushing heads before validating them?" + a: "In principle yes: pre-announcing an unvalidated head wins the race at the cost of occasionally pushing a block that gets reorged or retracted. That is why the bench exports more than the lag histogram. A provider that pushes early but wrong shows up as cohort inconsistencies, elevated reorg noise and gaps, and the coverage column (blocks seen versus the cohort maximum) penalises skipped or retracted blocks. Hash-level cross-checking against an independent reference node is planned for v2 of the harness." + - q: "How is WebSocket head lag measured on OpenChainBench?" + a: "One Go harness holds a persistent eth_subscribe('newHeads') WebSocket per provider from the same eu-west host, plus slotSubscribe for the Solana cohort. Every frame is timestamped on receipt with time.Now() precision. A block's cohort stays open for 5 seconds after the first arrival, then each provider's lag versus the earliest is observed into a Prometheus histogram (5 ms to 10 s exponential buckets); p50/p90/p99 come from histogram_quantile over the 24 h window. Connections carry a 90 s silent-subscription watchdog and exponential-backoff reconnects so a flaky endpoint degrades to health=0 instead of corrupting the race." + - q: "Why are only PublicNode, dRPC and Tenderly listed?" + a: "V1 measures the keyless tier: endpoints anyone can open a WebSocket to without an account, which is also the tier where claims are hardest to verify and marketing is loudest. Alchemy, Infura and Chainstack require API keys; the harness already accepts them via environment variables and they join the leaderboard once contributor keys are wired into the deployment. Keyed tiers usually run on better-provisioned infrastructure, so expect the keyless numbers here to be the floor, not the ceiling, of each vendor's performance." + +providers: + - slug: publicnode + name: PublicNode + tag: Keyless WS, wss://ethereum-rpc.publicnode.com + formula: "Median ms behind the earliest provider to push each Ethereum newHeads frame, from one persistent WebSocket per provider on the same eu-west host, histogram_quantile over 24h. The per-block winner scores 0 by construction." + queries: + p50: histogram_quantile(0.50, sum by (le) (rate(ws_head_lag_milliseconds_bucket{provider="publicnode", chain="ethereum"}[24h]))) + p90: histogram_quantile(0.90, sum by (le) (rate(ws_head_lag_milliseconds_bucket{provider="publicnode", chain="ethereum"}[24h]))) + p99: histogram_quantile(0.99, sum by (le) (rate(ws_head_lag_milliseconds_bucket{provider="publicnode", chain="ethereum"}[24h]))) + mean: sum(rate(ws_head_lag_milliseconds_sum{provider="publicnode", chain="ethereum"}[24h])) / sum(rate(ws_head_lag_milliseconds_count{provider="publicnode", chain="ethereum"}[24h])) + success: clamp_max(sum(increase(ws_head_blocks_seen_total{provider="publicnode", chain="ethereum"}[24h])) / max(sum by (provider) (increase(ws_head_blocks_seen_total{chain="ethereum"}[24h]))), 1) + sample_size: sum(increase(ws_head_lag_milliseconds_count{provider="publicnode", chain="ethereum"}[24h])) + series: histogram_quantile(0.50, sum by (le) (rate(ws_head_lag_milliseconds_bucket{provider="publicnode", chain="ethereum"}[1h]))) + + - slug: drpc + name: dRPC + tag: Keyless WS, wss://eth.drpc.org + formula: "Median ms behind the earliest provider to push each Ethereum newHeads frame, from one persistent WebSocket per provider on the same eu-west host, histogram_quantile over 24h. Lag is relative to the per-block winner." + queries: + p50: histogram_quantile(0.50, sum by (le) (rate(ws_head_lag_milliseconds_bucket{provider="drpc", chain="ethereum"}[24h]))) + p90: histogram_quantile(0.90, sum by (le) (rate(ws_head_lag_milliseconds_bucket{provider="drpc", chain="ethereum"}[24h]))) + p99: histogram_quantile(0.99, sum by (le) (rate(ws_head_lag_milliseconds_bucket{provider="drpc", chain="ethereum"}[24h]))) + mean: sum(rate(ws_head_lag_milliseconds_sum{provider="drpc", chain="ethereum"}[24h])) / sum(rate(ws_head_lag_milliseconds_count{provider="drpc", chain="ethereum"}[24h])) + success: clamp_max(sum(increase(ws_head_blocks_seen_total{provider="drpc", chain="ethereum"}[24h])) / max(sum by (provider) (increase(ws_head_blocks_seen_total{chain="ethereum"}[24h]))), 1) + sample_size: sum(increase(ws_head_lag_milliseconds_count{provider="drpc", chain="ethereum"}[24h])) + series: histogram_quantile(0.50, sum by (le) (rate(ws_head_lag_milliseconds_bucket{provider="drpc", chain="ethereum"}[1h]))) + + - slug: tenderly + name: Tenderly + tag: Keyless WS, wss://mainnet.gateway.tenderly.co + formula: "Median ms behind the earliest provider to push each Ethereum newHeads frame, from one persistent WebSocket per provider on the same eu-west host, histogram_quantile over 24h. Lag is relative to the per-block winner." + queries: + p50: histogram_quantile(0.50, sum by (le) (rate(ws_head_lag_milliseconds_bucket{provider="tenderly", chain="ethereum"}[24h]))) + p90: histogram_quantile(0.90, sum by (le) (rate(ws_head_lag_milliseconds_bucket{provider="tenderly", chain="ethereum"}[24h]))) + p99: histogram_quantile(0.99, sum by (le) (rate(ws_head_lag_milliseconds_bucket{provider="tenderly", chain="ethereum"}[24h]))) + mean: sum(rate(ws_head_lag_milliseconds_sum{provider="tenderly", chain="ethereum"}[24h])) / sum(rate(ws_head_lag_milliseconds_count{provider="tenderly", chain="ethereum"}[24h])) + success: clamp_max(sum(increase(ws_head_blocks_seen_total{provider="tenderly", chain="ethereum"}[24h])) / max(sum by (provider) (increase(ws_head_blocks_seen_total{chain="ethereum"}[24h]))), 1) + sample_size: sum(increase(ws_head_lag_milliseconds_count{provider="tenderly", chain="ethereum"}[24h])) + series: histogram_quantile(0.50, sum by (le) (rate(ws_head_lag_milliseconds_bucket{provider="tenderly", chain="ethereum"}[1h]))) diff --git a/benchmarks/xstocks-peg.yml b/benchmarks/xstocks-peg.yml new file mode 100644 index 00000000..9f7d8d8e --- /dev/null +++ b/benchmarks/xstocks-peg.yml @@ -0,0 +1,207 @@ +# OpenChainBench. Bench № 077 + +slug: xstocks-peg +number: "077" +title: xStocks price accuracy, live onchain vs Nasdaq across 12 equities on Solana +seo_title: "xStocks price tracker 2026" +seo_description: "Do Backed's xStocks track the real market? AAPLx, TSLAx, NVDAx and 9 more on Solana vs their Nasdaq price, deviation in bps, live and keyless." +subtitle: "Absolute deviation between each xStock's executable Jupiter price on Solana and its real market reference, in basis points, labeled by market session. Companion to the Robinhood Chain tokenized stock bench: same methodology, different issuer, so the two pages read as a head to head." + +category: RWA +status: live +metric: Price deviation +unit: bps +higher_is_better: false + +seo_intro: | + Backed's xStocks are the other big tokenized equity program: 20+ US + stocks and ETFs issued on Solana and distributed through Kraken and + Bybit, trading around the clock on Raydium and Orca with pool depths + that reach several million dollars. This page measures how closely + each xStock's executable price tracks its real market reference, + every 60 seconds, keyless on both legs. The price measured is the + Jupiter aggregated quote for actually swapping one share in each + direction, so it is an executable mid, not a theoretical pool spot. + Sessions are labeled pre, regular, post and closed, and the closed + series is the weekend drift panel. The same methodology runs against + Robinhood Chain's tokenized equities on the companion bench, which + turns the two pages into the first public issuer head to head for + tokenized stocks. Fun fact this page makes measurable: HOODx, the + tokenized Robinhood stock, trades here on Solana, while Robinhood's + own chain does not tokenize HOOD at all. + +abstract: | + Every 60 seconds the harness quotes one share of each of 12 xStocks + through Jupiter's keyless lite API in both directions (sell to USDC, + buy back), takes the executable mid, corrects for the Token-2022 + scaled amount multiplier where present, and compares against the + Yahoo reference price with holiday-aware session labels. Deviation is + 10000 x |mid - reference| / reference in basis points. Same metric + contract as the Robinhood Chain bench with issuer="xstocks". + +methodology: + - "Onchain leg: Jupiter lite-api swap quotes, keyless, both directions per symbol (sell 1 share to USDC, buy the proceeds back), spaced 1.1s apart, ~26s per sweep for 12 symbols. Price = mid of the two implied prices: an executable aggregated price across Raydium, Orca and every routed venue, not a single pool spot." + - "Token-2022 correction: 7 of 12 xStocks mints carry a ScaledUiAmount multiplier (~1.0009 observed). One batched price/v3 call per tick supplies the live multiplier per mint (usdPrice over usdPricePrescaled), converting raw quote units to exactly one UI share. Ignoring it would bake a ~9bp systematic error into affected symbols." + - "Reference leg: Yahoo Finance spark batch, one keyless call for all 12 underliers per tick, 1 minute candles; session labels (pre, regular, post, closed) derived from Yahoo currentTradingPeriod, holiday aware. HOODx maps to HOOD, SPYx to SPY, QQQx to QQQ, COINx to COIN." + - "Deviation: 10000 x |executable mid - reference| / reference, quantiles over 24h via quantile_over_time. Headline pins market_state=\"regular\"; the closed series is the weekend drift panel." + - "Cohort: the 12 xStocks with verified Jupiter routes at under 2bp of 1-share price impact (2026-07-13): TSLAx, NVDAx, AAPLx, MSFTx, AMZNx, GOOGLx, METAx, HOODx, SPYx, QQQx, COINx, PLTRx. All mints 8 decimals, verified individually; counterfeit lookalike mints excluded by address allowlist." + - "Round-trip honesty: the sell/buy spread observed at verification was ~11bp on TSLAx. Using the mid rather than one side keeps the fee component out of the deviation number; the spread itself is executable cost, not tracking error." + - "Quote asset caveat: prices are in USDC against USD references. A USDC peg wobble would appear as a correlated deviation across all 12 symbols simultaneously." + - "Cross-issuer reading: this bench shares its metric contract with the Robinhood Chain tokenized stock bench (issuer label). Same equity, same reference, two issuers: the pages are directly comparable, with the depth difference disclosed (Solana xStocks pools run 10x to 100x deeper)." + +findings: + - "{{best_name}} tracks its reference tightest at {{best_p50}} (p50, 24h) across {{count}} measured xStocks." + - "{{name:tsla}} deviates {{p50:tsla}} (p50, 24h) against a Raydium pool holding about $2.1M, roughly 50x the depth of the equivalent Robinhood Chain pool, and the tighter tracking that buys is exactly what the cross-issuer comparison exists to show." + - "{{name:hood}} ({{p50:hood}}) is the page's inside joke made measurable: the tokenized Robinhood stock trades on Solana via a third-party issuer while Robinhood's own chain does not tokenize HOOD." + - "{{name:spy}} ({{p50:spy}}) and {{name:qqq}} ({{p50:qqq}}) give the bench its index rows, useful because index arbitrage against deep ETF markets should in theory be the tightest peg of all." + - "The number to watch on weekends: pools trade around the clock while the reference freezes at Friday close, and the closed-session drift measured here is the honest answer to what a 24/7 stock is worth on a Sunday." + +faq: + - q: "Do xStocks track their real stock price?" + a: "During regular hours the liquid names track within tens of basis points, and this page shows the live per-symbol number measured as an executable Jupiter mid, not a theoretical pool spot. Outside market hours the peg loosens by construction, and the closed-session series quantifies exactly how much." + - q: "How is this different from the Robinhood Chain tokenized stock bench?" + a: "Same methodology, different issuer and chain: Backed's xStocks on Solana measured through Jupiter, Robinhood's tokens on their own chain measured through Uniswap v4. The two benches share a metric contract, so the same equity can be compared across issuers directly, depth differences disclosed." + - q: "Why measure the Jupiter quote instead of a specific pool?" + a: "Because it is what a user actually gets: the aggregated executable route across every venue with the fee baked into the round trip. Taking the mid of both directions removes the fee component and leaves the tracking signal. A single pool spot can sit anywhere inside its fee band without being arbitrageable." + - q: "What is the ScaledUiAmount correction?" + a: "Seven xStocks mints use Token-2022 scaled amounts, where raw token units map to UI shares through a live multiplier (about 1.0009 at verification). The harness reads the multiplier every tick from Jupiter's price API and applies it, without which those symbols would carry a permanent ~9bp phantom deviation." + +source: https://github.com/ChainBench/OpenChainBench/tree/main/harnesses/xstocks-peg + +prometheus: + window: 24h + freshness_metric: tsp_deviation_bps + +providers: + - slug: tsla + name: TSLAx + tag: "Tesla xStock, deepest pool of the cohort (~$2.1M Raydium)" + formula: "p50 over 24h of the absolute deviation (bps) between the executable Jupiter mid for TSLAx on Solana and the Yahoo reference, regular market hours only." + queries: + p50: quantile_over_time(0.50, tsp_deviation_bps{issuer="xstocks", asset="tsla", market_state="regular"}[24h]) + p90: quantile_over_time(0.90, tsp_deviation_bps{issuer="xstocks", asset="tsla", market_state="regular"}[24h]) + p99: quantile_over_time(0.99, tsp_deviation_bps{issuer="xstocks", asset="tsla", market_state="regular"}[24h]) + mean: avg_over_time(tsp_deviation_bps{issuer="xstocks", asset="tsla", market_state="regular"}[24h]) + success: avg_over_time(tsp_health{asset="tsla"}[24h]) + series: tsp_deviation_bps{issuer="xstocks", asset="tsla", market_state="regular"} + - slug: nvda + name: NVDAx + tag: "Nvidia xStock, ~$2.7M Raydium pool" + formula: "p50 over 24h of the absolute deviation (bps) between the executable Jupiter mid for NVDAx on Solana and the Yahoo reference, regular market hours only." + queries: + p50: quantile_over_time(0.50, tsp_deviation_bps{issuer="xstocks", asset="nvda", market_state="regular"}[24h]) + p90: quantile_over_time(0.90, tsp_deviation_bps{issuer="xstocks", asset="nvda", market_state="regular"}[24h]) + p99: quantile_over_time(0.99, tsp_deviation_bps{issuer="xstocks", asset="nvda", market_state="regular"}[24h]) + mean: avg_over_time(tsp_deviation_bps{issuer="xstocks", asset="nvda", market_state="regular"}[24h]) + success: avg_over_time(tsp_health{asset="nvda"}[24h]) + series: tsp_deviation_bps{issuer="xstocks", asset="nvda", market_state="regular"} + - slug: aapl + name: AAPLx + tag: "Apple xStock" + formula: "p50 over 24h of the absolute deviation (bps) between the executable Jupiter mid for AAPLx on Solana and the Yahoo reference, regular market hours only." + queries: + p50: quantile_over_time(0.50, tsp_deviation_bps{issuer="xstocks", asset="aapl", market_state="regular"}[24h]) + p90: quantile_over_time(0.90, tsp_deviation_bps{issuer="xstocks", asset="aapl", market_state="regular"}[24h]) + p99: quantile_over_time(0.99, tsp_deviation_bps{issuer="xstocks", asset="aapl", market_state="regular"}[24h]) + mean: avg_over_time(tsp_deviation_bps{issuer="xstocks", asset="aapl", market_state="regular"}[24h]) + success: avg_over_time(tsp_health{asset="aapl"}[24h]) + series: tsp_deviation_bps{issuer="xstocks", asset="aapl", market_state="regular"} + - slug: msft + name: MSFTx + tag: "Microsoft xStock" + formula: "p50 over 24h of the absolute deviation (bps) between the executable Jupiter mid for MSFTx on Solana and the Yahoo reference, regular market hours only." + queries: + p50: quantile_over_time(0.50, tsp_deviation_bps{issuer="xstocks", asset="msft", market_state="regular"}[24h]) + p90: quantile_over_time(0.90, tsp_deviation_bps{issuer="xstocks", asset="msft", market_state="regular"}[24h]) + p99: quantile_over_time(0.99, tsp_deviation_bps{issuer="xstocks", asset="msft", market_state="regular"}[24h]) + mean: avg_over_time(tsp_deviation_bps{issuer="xstocks", asset="msft", market_state="regular"}[24h]) + success: avg_over_time(tsp_health{asset="msft"}[24h]) + series: tsp_deviation_bps{issuer="xstocks", asset="msft", market_state="regular"} + - slug: amzn + name: AMZNx + tag: "Amazon xStock" + formula: "p50 over 24h of the absolute deviation (bps) between the executable Jupiter mid for AMZNx on Solana and the Yahoo reference, regular market hours only." + queries: + p50: quantile_over_time(0.50, tsp_deviation_bps{issuer="xstocks", asset="amzn", market_state="regular"}[24h]) + p90: quantile_over_time(0.90, tsp_deviation_bps{issuer="xstocks", asset="amzn", market_state="regular"}[24h]) + p99: quantile_over_time(0.99, tsp_deviation_bps{issuer="xstocks", asset="amzn", market_state="regular"}[24h]) + mean: avg_over_time(tsp_deviation_bps{issuer="xstocks", asset="amzn", market_state="regular"}[24h]) + success: avg_over_time(tsp_health{asset="amzn"}[24h]) + series: tsp_deviation_bps{issuer="xstocks", asset="amzn", market_state="regular"} + - slug: googl + name: GOOGLx + tag: "Alphabet xStock" + formula: "p50 over 24h of the absolute deviation (bps) between the executable Jupiter mid for GOOGLx on Solana and the Yahoo reference, regular market hours only." + queries: + p50: quantile_over_time(0.50, tsp_deviation_bps{issuer="xstocks", asset="googl", market_state="regular"}[24h]) + p90: quantile_over_time(0.90, tsp_deviation_bps{issuer="xstocks", asset="googl", market_state="regular"}[24h]) + p99: quantile_over_time(0.99, tsp_deviation_bps{issuer="xstocks", asset="googl", market_state="regular"}[24h]) + mean: avg_over_time(tsp_deviation_bps{issuer="xstocks", asset="googl", market_state="regular"}[24h]) + success: avg_over_time(tsp_health{asset="googl"}[24h]) + series: tsp_deviation_bps{issuer="xstocks", asset="googl", market_state="regular"} + - slug: meta + name: METAx + tag: "Meta xStock" + formula: "p50 over 24h of the absolute deviation (bps) between the executable Jupiter mid for METAx on Solana and the Yahoo reference, regular market hours only." + queries: + p50: quantile_over_time(0.50, tsp_deviation_bps{issuer="xstocks", asset="meta", market_state="regular"}[24h]) + p90: quantile_over_time(0.90, tsp_deviation_bps{issuer="xstocks", asset="meta", market_state="regular"}[24h]) + p99: quantile_over_time(0.99, tsp_deviation_bps{issuer="xstocks", asset="meta", market_state="regular"}[24h]) + mean: avg_over_time(tsp_deviation_bps{issuer="xstocks", asset="meta", market_state="regular"}[24h]) + success: avg_over_time(tsp_health{asset="meta"}[24h]) + series: tsp_deviation_bps{issuer="xstocks", asset="meta", market_state="regular"} + - slug: hood + name: HOODx + tag: "Robinhood stock, tokenized by Backed, not by Robinhood" + formula: "p50 over 24h of the absolute deviation (bps) between the executable Jupiter mid for HOODx on Solana and the Yahoo reference, regular market hours only." + queries: + p50: quantile_over_time(0.50, tsp_deviation_bps{issuer="xstocks", asset="hood", market_state="regular"}[24h]) + p90: quantile_over_time(0.90, tsp_deviation_bps{issuer="xstocks", asset="hood", market_state="regular"}[24h]) + p99: quantile_over_time(0.99, tsp_deviation_bps{issuer="xstocks", asset="hood", market_state="regular"}[24h]) + mean: avg_over_time(tsp_deviation_bps{issuer="xstocks", asset="hood", market_state="regular"}[24h]) + success: avg_over_time(tsp_health{asset="hood"}[24h]) + series: tsp_deviation_bps{issuer="xstocks", asset="hood", market_state="regular"} + - slug: spy + name: SPYx + tag: "S&P 500 ETF xStock" + formula: "p50 over 24h of the absolute deviation (bps) between the executable Jupiter mid for SPYx on Solana and the Yahoo reference, regular market hours only." + queries: + p50: quantile_over_time(0.50, tsp_deviation_bps{issuer="xstocks", asset="spy", market_state="regular"}[24h]) + p90: quantile_over_time(0.90, tsp_deviation_bps{issuer="xstocks", asset="spy", market_state="regular"}[24h]) + p99: quantile_over_time(0.99, tsp_deviation_bps{issuer="xstocks", asset="spy", market_state="regular"}[24h]) + mean: avg_over_time(tsp_deviation_bps{issuer="xstocks", asset="spy", market_state="regular"}[24h]) + success: avg_over_time(tsp_health{asset="spy"}[24h]) + series: tsp_deviation_bps{issuer="xstocks", asset="spy", market_state="regular"} + - slug: qqq + name: QQQx + tag: "Nasdaq 100 ETF xStock" + formula: "p50 over 24h of the absolute deviation (bps) between the executable Jupiter mid for QQQx on Solana and the Yahoo reference, regular market hours only." + queries: + p50: quantile_over_time(0.50, tsp_deviation_bps{issuer="xstocks", asset="qqq", market_state="regular"}[24h]) + p90: quantile_over_time(0.90, tsp_deviation_bps{issuer="xstocks", asset="qqq", market_state="regular"}[24h]) + p99: quantile_over_time(0.99, tsp_deviation_bps{issuer="xstocks", asset="qqq", market_state="regular"}[24h]) + mean: avg_over_time(tsp_deviation_bps{issuer="xstocks", asset="qqq", market_state="regular"}[24h]) + success: avg_over_time(tsp_health{asset="qqq"}[24h]) + series: tsp_deviation_bps{issuer="xstocks", asset="qqq", market_state="regular"} + - slug: coin + name: COINx + tag: "Coinbase xStock" + formula: "p50 over 24h of the absolute deviation (bps) between the executable Jupiter mid for COINx on Solana and the Yahoo reference, regular market hours only." + queries: + p50: quantile_over_time(0.50, tsp_deviation_bps{issuer="xstocks", asset="coin", market_state="regular"}[24h]) + p90: quantile_over_time(0.90, tsp_deviation_bps{issuer="xstocks", asset="coin", market_state="regular"}[24h]) + p99: quantile_over_time(0.99, tsp_deviation_bps{issuer="xstocks", asset="coin", market_state="regular"}[24h]) + mean: avg_over_time(tsp_deviation_bps{issuer="xstocks", asset="coin", market_state="regular"}[24h]) + success: avg_over_time(tsp_health{asset="coin"}[24h]) + series: tsp_deviation_bps{issuer="xstocks", asset="coin", market_state="regular"} + - slug: pltr + name: PLTRx + tag: "Palantir xStock, thinnest route of the cohort (1.8bp impact)" + formula: "p50 over 24h of the absolute deviation (bps) between the executable Jupiter mid for PLTRx on Solana and the Yahoo reference, regular market hours only." + queries: + p50: quantile_over_time(0.50, tsp_deviation_bps{issuer="xstocks", asset="pltr", market_state="regular"}[24h]) + p90: quantile_over_time(0.90, tsp_deviation_bps{issuer="xstocks", asset="pltr", market_state="regular"}[24h]) + p99: quantile_over_time(0.99, tsp_deviation_bps{issuer="xstocks", asset="pltr", market_state="regular"}[24h]) + mean: avg_over_time(tsp_deviation_bps{issuer="xstocks", asset="pltr", market_state="regular"}[24h]) + success: avg_over_time(tsp_health{asset="pltr"}[24h]) + series: tsp_deviation_bps{issuer="xstocks", asset="pltr", market_state="regular"} diff --git a/harnesses/bridge-monitor/.gitignore b/harnesses/bridge-monitor/.gitignore index 091b86f3..ad59826f 100644 --- a/harnesses/bridge-monitor/.gitignore +++ b/harnesses/bridge-monitor/.gitignore @@ -1,5 +1,7 @@ .env bin/ +spend-state.json +spend-state.json.tmp *.log prometheus_data/ grafana_data/ diff --git a/harnesses/bridge-monitor/cmd/monitor/across_bridge.go b/harnesses/bridge-monitor/cmd/monitor/across_bridge.go index fa43f673..d7f6cb9b 100644 --- a/harnesses/bridge-monitor/cmd/monitor/across_bridge.go +++ b/harnesses/bridge-monitor/cmd/monitor/across_bridge.go @@ -188,7 +188,7 @@ func (a *AcrossBridge) TestRoute(route TestRoute, amount, amountUsd float64, raw // Latency is only meaningful for quotes that returned a usable route // (same rule as the other bridges). Fast 4xx rejections must not enter // the histogram or they skew the leaderboard. - bridgeQuoteLatency.WithLabelValues(labels...).Observe(float64(quoteLatency.Milliseconds())) + bridgeQuoteLatency.WithLabelValues(labels...).Observe(float64(quoteLatency.Nanoseconds()) / 1e6) bridgeQuoteSuccess.WithLabelValues(labels...).Set(1) // fees.total.amountUsd is Across's own USD valuation of input value minus diff --git a/harnesses/bridge-monitor/cmd/monitor/balance.go b/harnesses/bridge-monitor/cmd/monitor/balance.go index 570a38fa..bb9f5e1d 100644 --- a/harnesses/bridge-monitor/cmd/monitor/balance.go +++ b/harnesses/bridge-monitor/cmd/monitor/balance.go @@ -9,6 +9,9 @@ import ( "os" "strings" "time" + + "github.com/ethereum/go-ethereum/common" + "github.com/gagliardetto/solana-go" ) // BalanceChecker fetches wallet balances from Mobula API @@ -17,6 +20,18 @@ type BalanceChecker struct { apiKey string evmAddress string solanaAddress string + + // Fallback on-chain reader. When the Mobula portfolio API fails we read the + // triangle tokens directly from RPC instead of silently returning zeros, + // which used to make every tier look like "insufficient funds" during an + // API outage. Nil in quote-only mode (no TxExecutor). + onchain *TxExecutor +} + +// SetOnchainFallback wires the RPC-based balance readers. Called after the +// TxExecutor exists because the BalanceChecker is constructed first in main. +func (bc *BalanceChecker) SetOnchainFallback(tx *TxExecutor) { + bc.onchain = tx } // MobulaWalletResponse represents the Mobula wallet API response @@ -53,6 +68,17 @@ func NewBalanceChecker(apiKey, evmAddress, solanaAddress string) *BalanceChecker // GetAllBalances fetches balances for all configured wallets // Returns map[chain][token] = balance_usd func (bc *BalanceChecker) GetAllBalances() (map[string]map[string]float64, error) { + balances, _, err := bc.GetAllBalancesDetailed() + return balances, err +} + +// GetAllBalancesDetailed fetches balances for all configured wallets. +// Primary source is the Mobula portfolio API; on per-chain failure it falls +// back to direct RPC reads of the triangle tokens. The degraded flag is true +// when at least one chain could not be read by EITHER path: callers must treat +// degraded balances as unreadable (never "empty wallet") and must not trigger +// automatic rebalancing from them. +func (bc *BalanceChecker) GetAllBalancesDetailed() (map[string]map[string]float64, bool, error) { result := make(map[string]map[string]float64) // Initialize chains @@ -60,11 +86,17 @@ func (bc *BalanceChecker) GetAllBalances() (map[string]map[string]float64, error result["Base"] = make(map[string]float64) result["Arbitrum"] = make(map[string]float64) + degraded := false + // Fetch Solana balances if bc.solanaAddress != "" { solBalances, err := bc.fetchWalletBalance(bc.solanaAddress) if err != nil { log.Printf("⚠️ Failed to fetch Solana balances: %v", err) + if !bc.fillSolanaFromChain(result) { + log.Printf("⚠️ On-chain Solana fallback also failed: balances degraded") + degraded = true + } } else { indexAssets(result, solBalances, "Solana") } @@ -76,6 +108,10 @@ func (bc *BalanceChecker) GetAllBalances() (map[string]map[string]float64, error baseBalances, err := bc.fetchWalletBalanceByChain(bc.evmAddress, "base") if err != nil { log.Printf("⚠️ Failed to fetch Base balances: %v", err) + if !bc.fillEVMFromChain(result, "Base") { + log.Printf("⚠️ On-chain Base fallback also failed: balances degraded") + degraded = true + } } else { indexAssets(result, baseBalances, "Base") } @@ -84,12 +120,118 @@ func (bc *BalanceChecker) GetAllBalances() (map[string]map[string]float64, error arbBalances, err := bc.fetchWalletBalanceByChain(bc.evmAddress, "arbitrum") if err != nil { log.Printf("⚠️ Failed to fetch Arbitrum balances: %v", err) + if !bc.fillEVMFromChain(result, "Arbitrum") { + log.Printf("⚠️ On-chain Arbitrum fallback also failed: balances degraded") + degraded = true + } } else { indexAssets(result, arbBalances, "Arbitrum") } } - return result, nil + if degraded { + bridgeBalanceReadDegraded.Set(1) + } else { + bridgeBalanceReadDegraded.Set(0) + } + + return result, degraded, nil +} + +// Token addresses read by the on-chain fallback. Stablecoins are valued at $1 +// parity (good enough to gate executions); SOL and ETH go through the price +// cache with a static fallback because during a Mobula outage the pricer may +// be down too, and a stale gas estimate beats a zero. +const ( + solanaUSDCMint = "EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v" + baseUSDCAddr = "0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913" + arbUSDTAddr = "0xFd086bC7CD5C481DCC9C85ebE478A1C0b69FCbb9" + arbUSDCAddr = "0xaf88d065e77c8cc2239327c5edb3a432268e5831" +) + +// fillSolanaFromChain reads Solana USDC + native SOL via RPC. Returns true only +// when the triangle token (USDC) was read successfully: SOL alone is not enough +// to run the cycle simulation, so a USDC read failure keeps the chain degraded. +func (bc *BalanceChecker) fillSolanaFromChain(result map[string]map[string]float64) bool { + if bc.onchain == nil || bc.onchain.solanaClient == nil { + return false + } + owner, err := solana.PublicKeyFromBase58(bc.solanaAddress) + if err != nil { + return false + } + + mint := solana.MustPublicKeyFromBase58(solanaUSDCMint) + raw, err := bc.onchain.solanaSPLBalanceOf(owner, mint) + if err != nil { + return false + } + usd := rawToFloat(raw, 6) + result["Solana"]["USDC"] = usd + result["Solana"][strings.ToLower(solanaUSDCMint)] = usd + + if lamports, err := bc.onchain.solanaNativeBalance(owner); err == nil { + result["Solana"]["SOL"] = rawToFloat(lamports, 9) * TokenPriceUSD("SOL", 150) + } + log.Printf("✅ Solana balances recovered via on-chain fallback (USDC $%.2f)", usd) + return true +} + +// fillEVMFromChain reads the chain's triangle stablecoin + native ETH via RPC. +// Same success rule as Solana: the triangle token read must succeed. +func (bc *BalanceChecker) fillEVMFromChain(result map[string]map[string]float64, chain string) bool { + if bc.onchain == nil { + return false + } + owner := common.HexToAddress(bc.evmAddress) + + type tokenRead struct { + addr string + symbols []string + } + var reads []tokenRead + switch chain { + case "Base": + reads = []tokenRead{{baseUSDCAddr, []string{"USDC"}}} + case "Arbitrum": + // Both USDT0 and USDT symbols: cycle_sim reads "USDT0" (Mobula naming) + // while route helpers fall back to "USDT". + reads = []tokenRead{ + {arbUSDTAddr, []string{"USDT0", "USDT"}}, + {arbUSDCAddr, []string{"USDC"}}, + } + default: + return false + } + + ok := false + for i, r := range reads { + raw, err := bc.onchain.erc20BalanceOf(chain, common.HexToAddress(r.addr), owner) + if err != nil { + // Only the first entry is the triangle token; secondary reads are + // best-effort for the stranded-fund gauges. + if i == 0 { + return false + } + continue + } + usd := rawToFloat(raw, 6) + for _, sym := range r.symbols { + result[chain][sym] = usd + } + result[chain][strings.ToLower(r.addr)] = usd + if i == 0 { + ok = true + } + } + + if wei, err := bc.onchain.evmNativeBalance(chain, owner); err == nil { + result[chain]["ETH"] = rawToFloat(wei, 18) * TokenPriceUSD("ETH", 3000) + } + if ok { + log.Printf("✅ %s balances recovered via on-chain fallback", chain) + } + return ok } // indexAssets stores balances in the result map, keyed by BOTH symbol AND contract address (lowercase). @@ -124,7 +266,7 @@ func indexAssets(result map[string]map[string]float64, resp *MobulaWalletRespons // fetchWalletBalance fetches balance for a single wallet (all chains) func (bc *BalanceChecker) fetchWalletBalance(address string) (*MobulaWalletResponse, error) { - url := fmt.Sprintf("https://api.mobula.io/api/1/wallet/portfolio?wallet=%s", address) + url := fmt.Sprintf("https://demo-api.mobula.io/api/1/wallet/portfolio?wallet=%s", address) req, err := http.NewRequest("GET", url, nil) if err != nil { @@ -156,7 +298,7 @@ func (bc *BalanceChecker) fetchWalletBalance(address string) (*MobulaWalletRespo // fetchWalletBalanceByChain fetches balance for a specific chain func (bc *BalanceChecker) fetchWalletBalanceByChain(address, chain string) (*MobulaWalletResponse, error) { - url := fmt.Sprintf("https://api.mobula.io/api/1/wallet/portfolio?wallet=%s&blockchains=%s", address, chain) + url := fmt.Sprintf("https://demo-api.mobula.io/api/1/wallet/portfolio?wallet=%s&blockchains=%s", address, chain) req, err := http.NewRequest("GET", url, nil) if err != nil { @@ -268,12 +410,18 @@ func (bc *BalanceChecker) GetTotalBalanceUSD() (float64, error) { return total, nil } -// SimulateBalances returns fake balances for dry-run mode -func SimulateBalances() map[string]map[string]float64 { +// SimulateBalances returns fake balances for dry-run mode. Outside dry-run it +// always returns nil: simulated balances feeding a broadcast-capable process +// could green-light real transfers based on made-up numbers. +func SimulateBalances(mode ExecutionMode) map[string]map[string]float64 { // Check if we should simulate if os.Getenv("SIMULATE_BALANCES") != "true" { return nil } + if mode != ModeDryRun { + log.Printf("⚠️ SIMULATE_BALANCES=true ignored: EXECUTION_MODE is %q, simulated balances are only honored in dry-run", mode) + return nil + } log.Println("🧪 Using simulated balances (SIMULATE_BALANCES=true)") return map[string]map[string]float64{ diff --git a/harnesses/bridge-monitor/cmd/monitor/debridge_bridge.go b/harnesses/bridge-monitor/cmd/monitor/debridge_bridge.go index 2c15fd65..031eb0c3 100644 --- a/harnesses/bridge-monitor/cmd/monitor/debridge_bridge.go +++ b/harnesses/bridge-monitor/cmd/monitor/debridge_bridge.go @@ -121,7 +121,7 @@ func (d *DebridgeBridge) TestRoute(route TestRoute, amount, amountUsd float64, r // (the published methodology measures exactly that). Fast failures, e.g. // Cloudflare 403s answered in 30ms, must not enter the histogram: they // made deBridge look 15x faster the moment its API started rejecting us. - bridgeQuoteLatency.WithLabelValues(labels...).Observe(float64(quoteLatency.Milliseconds())) + bridgeQuoteLatency.WithLabelValues(labels...).Observe(float64(quoteLatency.Nanoseconds()) / 1e6) bridgeQuoteSuccess.WithLabelValues(labels...).Set(1) // Debridge returns input (with opEx ajusté) and output diff --git a/harnesses/bridge-monitor/cmd/monitor/executor.go b/harnesses/bridge-monitor/cmd/monitor/executor.go index 94ef212b..7428375f 100644 --- a/harnesses/bridge-monitor/cmd/monitor/executor.go +++ b/harnesses/bridge-monitor/cmd/monitor/executor.go @@ -1,6 +1,7 @@ package main import ( + "crypto/ecdsa" "encoding/json" "fmt" "log" @@ -18,16 +19,17 @@ const ( ModeProduction ExecutionMode = "production" // Full execution loop ) -// ExecutionConfig holds the execution loop configuration +// ExecutionConfig holds the execution loop configuration. +// Daily spend tracking lives in the Executor's SpendTracker (UTC-date keyed, +// persisted to disk), not here: a plain struct field never reset and was +// zeroed by every restart. type ExecutionConfig struct { - Mode ExecutionMode - Freq5USD time.Duration // How often to run $5 tests - Freq50USD time.Duration // How often to run $50 tests - Freq300USD time.Duration // How often to run $300 tests - EnableDebridge bool // Whether to execute Debridge (expensive) - MaxDailySpendUSD float64 // Safety cap on daily spending - DailySpentUSD float64 // Track daily spending - LastResetDay int // Day of month for daily reset + Mode ExecutionMode + Freq5USD time.Duration // How often to run $5 tests + Freq50USD time.Duration // How often to run $50 tests + Freq300USD time.Duration // How often to run $300 tests + EnableDebridge bool // Whether to execute Debridge (expensive) + MaxDailySpendUSD float64 // Safety cap on daily spending } // ExecutionResult holds the result of an execution test @@ -68,6 +70,36 @@ type Executor struct { debridge *DebridgeBridge region string slack *SlackNotifier + spend *SpendTracker +} + +// DailySpent returns today's consumed budget via the mutex-guarded tracker. +// All cap checks must go through here, never through a raw field, so the +// reaper goroutine and the scheduler loop cannot race on the counter. +func (e *Executor) DailySpent() float64 { + return e.spend.Spent() +} + +// AddDailySpend books usd against today's budget (UTC-date keyed, persisted). +func (e *Executor) AddDailySpend(usd float64) { + e.spend.Add(usd) +} + +// accountSpend books a broadcast's cost against the daily cap. A successful +// fill books the realized fee. Any broadcast that produced a TxHash but did +// not confirm as a success books a conservative flat estimate, because the +// deposit or approval TX most likely burned gas even without a fill. Results +// that never broadcast cost nothing. +func (e *Executor) accountSpend(result *ExecutionResult) { + if result == nil || result.DryRun { + return + } + switch { + case result.Success: + e.AddDailySpend(result.ActualFeeUSD) + case result.TxHash != "": + e.AddDailySpend(failedTxFeeEstimateUSD()) + } } // NewExecutor creates a new executor @@ -92,6 +124,7 @@ func NewExecutor( debridge: debridge, region: region, slack: slack, + spend: NewSpendTracker(spendStatePath(), time.Now), } // Initialize TxExecutor if we have private keys @@ -112,6 +145,11 @@ func NewExecutor( } else { e.txExecutor = txExec log.Println("✅ TxExecutor initialized") + // Give the balance checker an RPC fallback so a Mobula portfolio + // API outage no longer reads as an empty wallet. + if balanceCheck != nil { + balanceCheck.SetOnchainFallback(txExec) + } } } @@ -128,12 +166,21 @@ func (e *Executor) RunDryRun(route TestRoute, amountUSD float64) *ExecutionResul log.Printf("🧪 [DRY-RUN] Testing %s with $%.0f", route.Name, amountUSD) - // Step 1: Check balances + // Step 1: Check balances. Keyless dry-run has no balance checker, so fall + // back to the SIMULATE_BALANCES snapshot instead of crashing. log.Printf(" 📊 Checking balances...") - balances, err := e.balanceCheck.GetAllBalances() - if err != nil { - log.Printf(" ❌ Balance check failed: %v", err) - result.Error = err + var balances map[string]map[string]float64 + if e.balanceCheck != nil { + var err error + balances, err = e.balanceCheck.GetAllBalances() + if err != nil { + log.Printf(" ❌ Balance check failed: %v", err) + result.Error = err + return result + } + } else if balances = SimulateBalances(e.config.Mode); balances == nil { + log.Printf(" ❌ No balance checker and SIMULATE_BALANCES not set") + result.Error = fmt.Errorf("no balance source in dry-run") return result } @@ -198,8 +245,8 @@ func (e *Executor) RunReal(route TestRoute, amountUSD float64) []*ExecutionResul } // Check daily spending limit - if e.config.DailySpentUSD >= e.config.MaxDailySpendUSD { - msg := fmt.Sprintf("Daily spending limit reached ($%.2f / $%.2f)", e.config.DailySpentUSD, e.config.MaxDailySpendUSD) + if spent := e.DailySpent(); spent >= e.config.MaxDailySpendUSD { + msg := fmt.Sprintf("Daily spending limit reached ($%.2f / $%.2f)", spent, e.config.MaxDailySpendUSD) log.Printf("⚠️ %s", msg) if e.slack != nil { _ = e.slack.NotifyScheduledSkip(route.Name, route.FromChain, route.FromToken, amountUSD, msg) @@ -261,10 +308,9 @@ func (e *Executor) RunReal(route TestRoute, amountUSD float64) []*ExecutionResul } } - // Update daily spending - if result.Success { - e.config.DailySpentUSD += result.ActualFeeUSD - } + // Update daily spending (realized fee on success, flat gas + // estimate on a broadcast that never confirmed). + e.accountSpend(result) } // Wait between bridges to avoid rate limiting @@ -282,8 +328,8 @@ func (e *Executor) RunBridgeOnRoute(bridge string, route TestRoute, amountUSD fl if e.txExecutor == nil || !e.txExecutor.CanExecute() { return nil } - if e.config.DailySpentUSD >= e.config.MaxDailySpendUSD { - log.Printf("⚠️ Daily spending limit reached ($%.2f / $%.2f)", e.config.DailySpentUSD, e.config.MaxDailySpendUSD) + if spent := e.DailySpent(); spent >= e.config.MaxDailySpendUSD { + log.Printf("⚠️ Daily spending limit reached ($%.2f / $%.2f)", spent, e.config.MaxDailySpendUSD) return nil } @@ -306,9 +352,7 @@ func (e *Executor) RunBridgeOnRoute(bridge string, route TestRoute, amountUSD fl } } - if result.Success { - e.config.DailySpentUSD += result.ActualFeeUSD - } + e.accountSpend(result) return result } @@ -356,13 +400,17 @@ func (e *Executor) executeOnBridge(bridge string, route TestRoute, amount, amoun result.ToToken = route.ToToken result.AmountUSD = amountUSD + // Keep the TxHash even when the attempt errored out: callers use it to + // tell a pre-broadcast failure (safe to retry) from a broadcast whose + // final status is unknown (terminal, funds may still be in flight) and + // to account the gas a failed TX still burned. + result.TxHash = txHash + if err != nil { log.Printf(" ❌ Execution failed: %v", err) result.Error = err return result } - - result.TxHash = txHash // If the sub-function flagged a refund/revert, keep Success=false so Slack and // Prometheus correctly classify it (Reverted takes precedence over Success). result.Success = !result.Reverted @@ -427,12 +475,18 @@ func (e *Executor) executeMobula(route TestRoute, amount float64, quoteStart tim log.Printf(" [mobula] Getting quote: %s → %s, amount: %.4f, sender: %s", route.FromChain, route.ToChain, amount, senderAddress[:8]+"...") - // Get quote with TX - quote, _, err := e.mobula.GetQuote( + // Two-step quote: for EVM origins the first response's deposit is a + // placeholder; only the signed re-quote's deposit is executable. See + // mobula_bridge.go GetSignedQuote for the flow. + var evmKey *ecdsa.PrivateKey + if e.txExecutor != nil { + evmKey = e.txExecutor.EVMPrivateKey() + } + quote, _, err := e.mobula.GetSignedQuote( route.FromChainAPI, route.FromToken, route.ToChainAPI, route.ToToken, senderAddress, receiverAddress, - amount, + amount, evmKey, ) if err != nil { log.Printf(" [mobula] ❌ Quote error: %v", err) @@ -852,6 +906,12 @@ func (e *Executor) recordExecutionMetrics(result *ExecutionResult) { result.Route.ToToken, amountStr, e.region, + // chain dimension label, same convention as every quote path + // (mobula_bridge.go etc). This 8th label was missed when the + // metrics gained the chain dimension, and because execution was + // paused in prod the mismatch only surfaced at the first real + // single-test (panic: inconsistent label cardinality). + result.Route.ToChain, } // Record latencies @@ -1004,8 +1064,8 @@ func (e *Executor) testBridgeDryRun(bridge string, route TestRoute, amountUSD fl log.Printf(" 💰 Estimated cost: $%.4f", expectedCost) // Update daily spending tracker (even in dry-run for estimation) - e.config.DailySpentUSD += expectedCost - log.Printf(" 📈 Daily spend estimate: $%.2f / $%.2f max", e.config.DailySpentUSD, e.config.MaxDailySpendUSD) + e.AddDailySpend(expectedCost) + log.Printf(" 📈 Daily spend estimate: $%.2f / $%.2f max", e.DailySpent(), e.config.MaxDailySpendUSD) } // ValidateSetup checks that everything is configured correctly diff --git a/harnesses/bridge-monitor/cmd/monitor/gas_topup.go b/harnesses/bridge-monitor/cmd/monitor/gas_topup.go new file mode 100644 index 00000000..5e244ff6 --- /dev/null +++ b/harnesses/bridge-monitor/cmd/monitor/gas_topup.go @@ -0,0 +1,212 @@ +package main + +import ( + "fmt" + "log" + "os" + "strings" + "time" +) + +// Gas auto-top-up: an execution slot can fail for the dumbest possible reason, +// no native gas to sign the deposit TX. Pre-flight already fetches balances, +// so we check SOL / Base ETH / Arb ETH against a USD floor and, when allowed, +// swap wallet USDC into native gas on that same chain via a LI.FI same-chain +// swap (their /v1/quote accepts fromChain == toChain and returns broadcastable +// calldata, same shape as a bridge quote). +// +// SAFETY GATE: GAS_TOPUP_ENABLED defaults to false. The plumbing below reuses +// the proven LiFi execution path, but the same-chain variant has not been +// exercised with real funds from this harness yet. Until someone runs one +// supervised top-up per chain and flips the env var, low gas only alerts. + +// LiFi native-token markers. +const ( + lifiEVMNative = "0x0000000000000000000000000000000000000000" + lifiSolanaNative = "11111111111111111111111111111111" +) + +type gasChain struct { + Chain string + NativeSym string + USDCSource string +} + +var gasChains = []gasChain{ + {Chain: "Solana", NativeSym: "SOL", USDCSource: solanaUSDCMint}, + {Chain: "Base", NativeSym: "ETH", USDCSource: baseUSDCAddr}, + {Chain: "Arbitrum", NativeSym: "ETH", USDCSource: arbUSDCAddr}, +} + +type GasTopper struct { + executor *Executor + slack *SlackNotifier + + enabled bool + minUSD float64 + topupUSD float64 + + // Per-chain USD swapped today; resets on UTC day change. Caps a runaway + // price-feed glitch to one top-up per chain per day. + spentToday map[string]float64 + day int +} + +func NewGasTopper(executor *Executor, slack *SlackNotifier) *GasTopper { + if executor == nil { + return nil + } + return &GasTopper{ + executor: executor, + slack: slack, + enabled: strings.EqualFold(strings.TrimSpace(os.Getenv("GAS_TOPUP_ENABLED")), "true"), + minUSD: parseFloat(os.Getenv("GAS_MIN_USD"), 15), + topupUSD: parseFloat(os.Getenv("GAS_TOPUP_USD"), 25), + spentToday: make(map[string]float64), + day: time.Now().UTC().YearDay(), + } +} + +// CheckAndTopUp inspects native gas on each chain and tops up where needed. +// Called from tier pre-flight with balances the caller already validated as +// non-degraded: a phantom zero here would otherwise trigger a pointless swap. +func (g *GasTopper) CheckAndTopUp(balances map[string]map[string]float64) { + if g == nil { + return + } + + if today := time.Now().UTC().YearDay(); today != g.day { + g.day = today + g.spentToday = make(map[string]float64) + } + + for _, gc := range gasChains { + gasUSD := balances[gc.Chain][gc.NativeSym] + if gasUSD >= g.minUSD { + continue + } + log.Printf("⛽ Low gas on %s: $%.2f %s (floor $%.2f)", gc.Chain, gasUSD, gc.NativeSym, g.minUSD) + + if !g.enabled { + bridgeGasTopup.WithLabelValues(gc.Chain, "gated").Inc() + _ = g.slack.NotifyGasTopUp(gc.Chain, "gated", fmt.Sprintf( + "Native gas is $%.2f, below the $%.2f floor, but GAS_TOPUP_ENABLED is off. Top up manually or enable after a supervised test.", + gasUSD, g.minUSD)) + continue + } + if g.executor.txExecutor == nil || !g.executor.txExecutor.CanExecute() { + // Dry-run or missing keys: detection is still useful in logs, but + // there is nothing safe to broadcast. + continue + } + if g.spentToday[gc.Chain] >= g.topupUSD { + bridgeGasTopup.WithLabelValues(gc.Chain, "capped").Inc() + _ = g.slack.NotifyGasTopUp(gc.Chain, "capped", fmt.Sprintf( + "Native gas is $%.2f but today's top-up budget ($%.2f) is already spent.", gasUSD, g.topupUSD)) + continue + } + // Same daily-cap pre-check the executor applies before any broadcast: + // a top-up is still spend and must not blow past MaxDailySpendUSD. + if spent := g.executor.DailySpent(); spent >= g.executor.config.MaxDailySpendUSD { + bridgeGasTopup.WithLabelValues(gc.Chain, "capped").Inc() + _ = g.slack.NotifyGasTopUp(gc.Chain, "capped", fmt.Sprintf( + "Native gas is $%.2f but the daily spend limit is reached ($%.2f / $%.2f). No top-up attempted.", + gasUSD, spent, g.executor.config.MaxDailySpendUSD)) + continue + } + + amount := g.topupUSD - g.spentToday[gc.Chain] + if amount > g.topupUSD { + amount = g.topupUSD + } + g.topUpChain(gc, amount, gasUSD) + } +} + +// topUpChain swaps `amountUSD` of USDC into native gas on one chain. +func (g *GasTopper) topUpChain(gc gasChain, amountUSD, gasUSD float64) { + bridgeGasTopup.WithLabelValues(gc.Chain, "attempted").Inc() + + toToken := lifiEVMNative + sender := g.executor.walletManager.EVMAddress + if gc.Chain == "Solana" { + toToken = lifiSolanaNative + sender = g.executor.walletManager.SolanaAddress + } + + route := TestRoute{ + Name: fmt.Sprintf("GAS_TOPUP_%s", strings.ToUpper(gc.Chain)), + FromChain: gc.Chain, + FromChainAPI: chainAPIFor(gc.Chain), + FromToken: gc.USDCSource, + ToChain: gc.Chain, + ToChainAPI: chainAPIFor(gc.Chain), + ToToken: toToken, + IsSolanaSrc: gc.Chain == "Solana", + } + rawUnits := toRawUnits(amountUSD) + + quote, _, err := g.executor.lifi.GetQuote(route, rawUnits, sender, sender) + if err != nil { + bridgeGasTopup.WithLabelValues(gc.Chain, "failed").Inc() + _ = g.slack.NotifyGasTopUp(gc.Chain, "failed", fmt.Sprintf( + "LI.FI same-chain swap quote failed (gas $%.2f, wanted $%.2f USDC to %s): %v", gasUSD, amountUSD, gc.NativeSym, err)) + return + } + + // EVM swaps of ERC-20 input need the router approved first, same as bridges. + if quote.Estimate.ApprovalAddress != "" && gc.Chain != "Solana" { + approvalHash, err := g.executor.txExecutor.ApproveERC20(gc.Chain, quote.Action.FromToken.Address, quote.Estimate.ApprovalAddress, quote.Action.FromAmount) + if err != nil { + bridgeGasTopup.WithLabelValues(gc.Chain, "failed").Inc() + _ = g.slack.NotifyGasTopUp(gc.Chain, "failed", fmt.Sprintf("USDC approval failed: %v", err)) + return + } + time.Sleep(5 * time.Second) + if ok, err := g.executor.txExecutor.CheckEVMTxStatus(gc.Chain, approvalHash); err != nil || !ok { + bridgeGasTopup.WithLabelValues(gc.Chain, "failed").Inc() + // The approval TX was broadcast and likely paid gas even though + // it never confirmed: book the conservative estimate. + g.executor.AddDailySpend(failedTxFeeEstimateUSD()) + _ = g.slack.NotifyGasTopUp(gc.Chain, "failed", "USDC approval not confirmed") + return + } + } + + var txHash string + if gc.Chain == "Solana" { + txHash, err = g.executor.txExecutor.ExecuteSolanaTransaction(quote.TransactionRequest.Data) + } else { + txHash, err = g.executor.txExecutor.ExecuteEVMTransaction(gc.Chain, quote.TransactionRequest.To, quote.TransactionRequest.Data, quote.TransactionRequest.Value) + } + if err != nil { + bridgeGasTopup.WithLabelValues(gc.Chain, "failed").Inc() + _ = g.slack.NotifyGasTopUp(gc.Chain, "failed", fmt.Sprintf("Broadcast failed: %v", err)) + return + } + + // Same-chain swaps settle in one TX: a confirmed receipt is a fill, no + // bridge status polling needed. Solana broadcast acceptance is our signal. + if gc.Chain != "Solana" { + time.Sleep(8 * time.Second) + if ok, err := g.executor.txExecutor.CheckEVMTxStatus(gc.Chain, txHash); err != nil || !ok { + bridgeGasTopup.WithLabelValues(gc.Chain, "failed").Inc() + // Broadcast happened: even a reverted or unconfirmed swap TX + // bled gas, so it counts toward the daily cap. + g.executor.AddDailySpend(failedTxFeeEstimateUSD()) + _ = g.slack.NotifyGasTopUp(gc.Chain, "failed", fmt.Sprintf("Swap TX not confirmed or reverted: %s", txHash)) + return + } + } + + g.spentToday[gc.Chain] += amountUSD + fee := 0.0 + for _, f := range quote.Estimate.FeeCosts { + fee += parseFloatOrZero(f.AmountUSD) + } + g.executor.AddDailySpend(fee) + bridgeGasTopup.WithLabelValues(gc.Chain, "succeeded").Inc() + _ = g.slack.NotifyGasTopUp(gc.Chain, "succeeded", fmt.Sprintf( + "Swapped $%.2f USDC to %s (was $%.2f, fee $%.4f, tx %s). Daily budget used: $%.2f / $%.2f.", + amountUSD, gc.NativeSym, gasUSD, fee, txHash, g.spentToday[gc.Chain], g.topupUSD)) +} diff --git a/harnesses/bridge-monitor/cmd/monitor/lifi_bridge.go b/harnesses/bridge-monitor/cmd/monitor/lifi_bridge.go index 82c928e1..ba7b9a58 100644 --- a/harnesses/bridge-monitor/cmd/monitor/lifi_bridge.go +++ b/harnesses/bridge-monitor/cmd/monitor/lifi_bridge.go @@ -151,7 +151,7 @@ func (l *LiFiBridge) TestRoute(route TestRoute, amount, amountUsd float64, rawUn // (the published methodology measures exactly that). Fast failures, e.g. // Cloudflare 403s answered in 30ms, must not enter the histogram: they // made deBridge look 15x faster the moment its API started rejecting us. - bridgeQuoteLatency.WithLabelValues(labels...).Observe(float64(quoteLatency.Milliseconds())) + bridgeQuoteLatency.WithLabelValues(labels...).Observe(float64(quoteLatency.Nanoseconds()) / 1e6) bridgeQuoteSuccess.WithLabelValues(labels...).Set(1) inUsd, _ := strconv.ParseFloat(quote.Estimate.FromAmountUSD, 64) diff --git a/harnesses/bridge-monitor/cmd/monitor/main.go b/harnesses/bridge-monitor/cmd/monitor/main.go index 5f867994..a8fe5f4e 100644 --- a/harnesses/bridge-monitor/cmd/monitor/main.go +++ b/harnesses/bridge-monitor/cmd/monitor/main.go @@ -31,6 +31,13 @@ func main() { // Log configuration config.LogConfig() + // SIMULATE_BALANCES is a dry-run testing aid only. In any mode that can + // broadcast, phantom balances could green-light real transfers, so the + // flag is ignored everywhere outside dry-run (see SimulateBalances). + if config.SimulateBalances && config.ExecutionMode != string(ModeDryRun) { + log.Printf("⚠️ SIMULATE_BALANCES=true is set but EXECUTION_MODE is %q: simulated balances are IGNORED outside dry-run", config.ExecutionMode) + } + // Initialize bridges var mobulaBridge *MobulaBridge if config.MobulaAPIKey != "" { @@ -139,6 +146,9 @@ func main() { } } + // Pre-seed the self-healing series so alerting rules match from startup. + initSelfHealingMetrics() + // Start Prometheus metrics endpoint. Railway / most PaaS inject $PORT // and route external traffic to whatever value they chose. If we bind // to the wrong port the edge proxy returns 502 with x-railway-fallback. @@ -244,6 +254,7 @@ func main() { log.Println("⚠️ This will execute REAL transactions!") bridges := []string{"mobula", "relay", "lifi"} + execMu.Lock() for _, bridge := range bridges { log.Printf("\n━━━ Bridge: %s ━━━", bridge) for _, route := range triangleRoutes { @@ -252,6 +263,7 @@ func main() { time.Sleep(2 * time.Second) } } + execMu.Unlock() log.Println("\n✅ Single-test complete! Exiting.") return @@ -336,6 +348,20 @@ func main() { log.Println("🚀 Production mode: fixed-time scheduler started") } + // Auto-rebalancer shares the executor's quote and broadcast plumbing. Inert + // unless the executor can actually broadcast (production mode, keys present), + // so it costs nothing in dry-run or while paused. + var rebalancer *Rebalancer + var gasTopper *GasTopper + if executor != nil { + rebalancer = NewRebalancer(executor, slackNotifier) + gasTopper = NewGasTopper(executor, slackNotifier) + } + + // Stuck-fund reaper: gauges in every mode (alerting must survive a pause), + // corrective transfers only in production and unpaused. + StartReaper(balanceChecker, rebalancer, slackNotifier, config.ExecutionMode, paused) + // Track last meme execution day to run weekly lastMemeDay := -1 @@ -345,65 +371,145 @@ func main() { case <-getSchedulerChan(scheduler, "$5"): // $5 execution loop - daily at 10:00 UTC if executor != nil && config.ExecutionMode == "production" { - runTierIfViable(executor, balanceChecker, slackNotifier, GetTriangleRoutes(), 5.0, "daily") + runTierIfViable(executor, balanceChecker, slackNotifier, rebalancer, gasTopper, GetTriangleRoutes(), 5.0, "daily") // Meme routes use independent capital (TRUMP) — always attempt, // the per-route RunReal check catches insufficient TRUMP. now := time.Now().UTC() if now.Weekday() == time.Monday && now.YearDay() != lastMemeDay { log.Println("💸 Running $5 meme execution tests (weekly)...") + execMu.Lock() for _, route := range GetMemeRoutes() { executor.RunReal(route, 5.0) } + execMu.Unlock() lastMemeDay = now.YearDay() } } case <-getSchedulerChan(scheduler, "$50"): if executor != nil && config.ExecutionMode == "production" { - runTierIfViable(executor, balanceChecker, slackNotifier, GetTriangleRoutes(), 50.0, "Mon+Thu") + runTierIfViable(executor, balanceChecker, slackNotifier, rebalancer, gasTopper, GetTriangleRoutes(), 50.0, "Mon+Thu") } case <-getSchedulerChan(scheduler, "$300"): if executor != nil && config.ExecutionMode == "production" { - runTierIfViable(executor, balanceChecker, slackNotifier, GetTriangleRoutes(), 300.0, "Mon weekly") + runTierIfViable(executor, balanceChecker, slackNotifier, rebalancer, gasTopper, GetTriangleRoutes(), 300.0, "Mon weekly") } } } } +// downgradeLadder returns the tier amounts to try, largest first, starting at +// the scheduled tier. Partial data beats none: if $300 cannot run we still +// want the $50 or $5 datapoint from the same slot. +func downgradeLadder(tier float64) []float64 { + all := []float64{300, 50, 5} + var out []float64 + for _, t := range all { + if t <= tier { + out = append(out, t) + } + } + if len(out) == 0 { + out = []float64{tier} + } + return out +} + // runTierIfViable pre-flights the full R1→R2→R3 cycle at the given tier. If the -// simulation says the cycle cannot complete, emit ONE Slack "couldn't run" message -// and skip — next scheduler tick will retry. Returns true if the tier actually ran. +// simulation says the cycle cannot complete, it first lets the auto-rebalancer +// try to unblock the tier, then walks the downgrade ladder to a smaller amount. +// Only if nothing on the ladder is viable does it emit ONE Slack "couldn't run" +// message and skip, so the next scheduler tick retries. Returns true if any +// amount actually ran. func runTierIfViable(executor *Executor, bc *BalanceChecker, slack *SlackNotifier, - routes []TestRoute, tier float64, tierLabel string, + rebalancer *Rebalancer, gasTopper *GasTopper, routes []TestRoute, tier float64, tierLabel string, ) bool { if bc == nil { log.Printf("⚠️ No balance checker — skipping tier $%.0f pre-flight", tier) return false } - balances, err := bc.GetAllBalances() - if err != nil { - log.Printf("⚠️ Pre-flight balance fetch failed for $%.0f tier: %v", tier, err) - if slack != nil { - _ = slack.NotifyTierSkipped(tier, tierLabel, fmt.Sprintf("Balance API error: %v", err)) + // Single-flight: this slot owns the wallets end to end (gas top-up, + // corrective rebalances, triangle runs). The reaper TryLocks and skips + // its corrective action while this is held. + execMu.Lock() + defer execMu.Unlock() + + ladder := downgradeLadder(tier) + blockedReason := "" + + // ONE corrective-transfer budget for the whole scheduler slot, shared by + // every ladder rung. Without sharing, each rung consumed its own attempt + // counter and a single slot could broadcast up to six transfers. + budget := newSlotBudget() + + for i, amount := range ladder { + balances, degraded, err := bc.GetAllBalancesDetailed() + if err != nil || degraded { + // Unreadable is not the same as empty: refuse to act (and above all + // refuse to rebalance) on balances we cannot trust. + log.Printf("⚠️ Pre-flight balances unreadable for $%.0f tier (degraded=%v err=%v)", tier, degraded, err) + if slack != nil { + _ = slack.NotifyTierSkipped(tier, tierLabel, + fmt.Sprintf("Balances unreadable (degraded=%v, err=%v). Refusing to run or rebalance blind.", degraded, err)) + } + return false } - return false - } - sim := SimulateTriangleCycle(balances, tier) - if !sim.Viable { - log.Printf("⏭️ Tier $%.0f skipped: %s", tier, sim.Reason) - if slack != nil { - _ = slack.NotifyTierSkipped(tier, tierLabel, sim.Reason) + // Gas check once per slot: the same balances snapshot already carries + // SOL / ETH, and an execution without gas fails later anyway. + if i == 0 { + gasTopper.CheckAndTopUp(balances) } - return false + + sim := SimulateTriangleCycle(balances, amount) + if !sim.Viable && rebalancer != nil { + sim, _ = rebalancer.TryUnblockTier(sim, balances, tierLabel, budget) + } + if !sim.Viable { + if blockedReason == "" { + blockedReason = sim.Reason + } + log.Printf("⏭️ Tier $%.0f not viable at $%.0f: %s", tier, amount, sim.Reason) + continue + } + + if i > 0 { + log.Printf("⬇️ Tier $%.0f downgraded to $%.0f: %s", tier, amount, blockedReason) + bridgeTierDowngraded.WithLabelValues(tierAmountLabel(tier), tierAmountLabel(amount)).Set(1) + if slack != nil { + _ = slack.NotifyTierDowngrade(tier, amount, blockedReason) + } + } else { + // The scheduled tier runs at full size again: clear its downgrade + // gauges so the alert stops firing. + for _, smaller := range ladder[1:] { + bridgeTierDowngraded.WithLabelValues(tierAmountLabel(tier), tierAmountLabel(smaller)).Set(0) + } + } + + runTriangle(executor, routes, amount) + return true + } + + log.Printf("⏭️ Tier $%.0f skipped entirely: %s", tier, blockedReason) + if slack != nil { + _ = slack.NotifyTierSkipped(tier, tierLabel, blockedReason) } + return false +} + +func tierAmountLabel(tier float64) string { + return strconv.FormatFloat(tier, 'f', 0, 64) +} - // Sequential per-bridge orchestration: each bridge runs a full R1→R2→R3 triangle - // before the next bridge starts. Halves the peak capital need per leg, gives a - // clean round-trip cost per provider, and unlocks larger tiers with less capital. +// runTriangle runs the sequential per-bridge orchestration: each bridge runs a +// full R1→R2→R3 triangle before the next bridge starts. Halves the peak capital +// need per leg, gives a clean round-trip cost per provider, and unlocks larger +// tiers with less capital. +func runTriangle(executor *Executor, routes []TestRoute, tier float64) { log.Printf("💸 Running $%.0f triangle (sequential per-bridge)...", tier) bridges := []string{"mobula", "relay", "lifi"} for _, bridge := range bridges { @@ -423,7 +529,6 @@ func runTierIfViable(executor *Executor, bc *BalanceChecker, slack *SlackNotifie } } } - return true } // getSchedulerChan returns the appropriate scheduler channel or nil diff --git a/harnesses/bridge-monitor/cmd/monitor/metrics.go b/harnesses/bridge-monitor/cmd/monitor/metrics.go index 0ac368ae..5f2fa0f3 100644 --- a/harnesses/bridge-monitor/cmd/monitor/metrics.go +++ b/harnesses/bridge-monitor/cmd/monitor/metrics.go @@ -6,11 +6,17 @@ import ( ) var ( - // Quote latency (time to get quote response) + // Quote latency (time to get bridge quote in ms). Sub-50ms buckets + // (10, 25) added 2026-07-13 because Near Intents on solver-cached + // corridors (HyperCore in particular) returns in <50ms via the 1Click + // coordinator's cached-price fast path, and the previous [50, ...] + // scheme lumped every one of those observations into the (0, 50] + // bucket. histogram_quantile then linearly interpolated to ~25ms + // regardless of the true value, hiding the real bimodality. bridgeQuoteLatency = promauto.NewHistogramVec(prometheus.HistogramOpts{ Name: "bridge_quote_latency_ms", Help: "Latency to get bridge quote in milliseconds", - Buckets: []float64{50, 100, 200, 500, 1000, 2000, 5000, 10000}, + Buckets: []float64{10, 25, 50, 100, 200, 500, 1000, 2000, 5000, 10000}, }, []string{"bridge", "from_chain", "to_chain", "from_token", "to_token", "amount_usd", "region", "chain"}) // Execution latency (broadcast to funds received) @@ -128,4 +134,63 @@ var ( Name: "bridge_consecutive_failures", Help: "Number of consecutive execution failures for a bridge (resets on success)", }, []string{"bridge", "region"}) + + // 1 when at least one chain's balances could not be read by either the + // Mobula API or the on-chain fallback. Lets alerting distinguish "wallet is + // empty" from "we cannot see the wallet". + bridgeBalanceReadDegraded = promauto.NewGauge(prometheus.GaugeOpts{ + Name: "bridge_balance_read_degraded", + Help: "1 if wallet balances are currently unreadable via both API and RPC, 0 otherwise", + }) + + // Auto-rebalance attempts by outcome: attempted, succeeded, failed, + // capped, in_flight (broadcast whose bridge status never resolved). + bridgeRebalanceAttempts = promauto.NewCounterVec(prometheus.CounterOpts{ + Name: "bridge_rebalance_attempts_total", + Help: "Total automatic rebalance attempts by outcome (attempted, succeeded, failed, capped, in_flight)", + }, []string{"outcome"}) + + // Set to 1 when a scheduled tier was downgraded to a smaller amount because + // the requested tier was not viable even after rebalancing. Reset to 0 when + // the original tier runs at full size again. + bridgeTierDowngraded = promauto.NewGaugeVec(prometheus.GaugeOpts{ + Name: "bridge_tier_downgraded", + Help: "1 when the last run of tier `from` was downgraded to tier `to`, 0 otherwise", + }, []string{"from", "to"}) + + // Hours funds have been sitting off their home triangle leg. 0 when the + // wallet only holds expected inventory. Exported continuously (also while + // paused) so Prometheus alerting can fire without any execution enabled. + bridgeStrandedHours = promauto.NewGaugeVec(prometheus.GaugeOpts{ + Name: "bridge_stranded_hours", + Help: "Hours a balance has been stranded off its home triangle leg (0 when home)", + }, []string{"chain", "token"}) + + // Gas top-up attempts per chain by outcome: attempted, succeeded, failed, + // capped, gated (needed but GAS_TOPUP_ENABLED is off). + bridgeGasTopup = promauto.NewCounterVec(prometheus.CounterOpts{ + Name: "bridge_gas_topup_total", + Help: "Total gas top-up attempts by chain and outcome", + }, []string{"chain", "outcome"}) ) + +// initSelfHealingMetrics pre-seeds the label combinations the alerting rules +// query, so the series exist on /metrics from process start instead of only +// after the first event. +func initSelfHealingMetrics() { + for _, outcome := range []string{"attempted", "succeeded", "failed", "capped", "in_flight"} { + bridgeRebalanceAttempts.WithLabelValues(outcome).Add(0) + } + bridgeTierDowngraded.WithLabelValues("300", "50").Set(0) + bridgeTierDowngraded.WithLabelValues("300", "5").Set(0) + bridgeTierDowngraded.WithLabelValues("50", "5").Set(0) + bridgeStrandedHours.WithLabelValues("Solana", "USDC").Set(0) + bridgeStrandedHours.WithLabelValues("Base", "USDC").Set(0) + bridgeStrandedHours.WithLabelValues("Arbitrum", "USDT0").Set(0) + for _, chain := range []string{"Solana", "Base", "Arbitrum"} { + for _, outcome := range []string{"attempted", "succeeded", "failed", "capped", "gated"} { + bridgeGasTopup.WithLabelValues(chain, outcome).Add(0) + } + } + bridgeBalanceReadDegraded.Set(0) +} diff --git a/harnesses/bridge-monitor/cmd/monitor/mobula_bridge.go b/harnesses/bridge-monitor/cmd/monitor/mobula_bridge.go index 73e2a41d..706c294a 100644 --- a/harnesses/bridge-monitor/cmd/monitor/mobula_bridge.go +++ b/harnesses/bridge-monitor/cmd/monitor/mobula_bridge.go @@ -1,6 +1,8 @@ package main import ( + "crypto/ecdsa" + "encoding/hex" "encoding/json" "fmt" "io" @@ -8,6 +10,10 @@ import ( "net/http" "strconv" "time" + + ethmath "github.com/ethereum/go-ethereum/common/math" + "github.com/ethereum/go-ethereum/crypto" + "github.com/ethereum/go-ethereum/signer/core/apitypes" ) type MobulaBridge struct { @@ -17,12 +23,22 @@ type MobulaBridge struct { type MobulaQuoteResponse struct { Data struct { + // Two-step flow fields: EVM origins return SignatureRequired=true + // and require a re-quote with the EIP-712 signature over TypedData + // before the deposit becomes executable (unsigned deposit stays + // unregistered server-side and the solver never picks it up). + IntentId string `json:"intentId"` + Deadline int64 `json:"deadline"` + SignatureRequired bool `json:"signatureRequired"` + TypedData json.RawMessage `json:"typedData"` + EstimatedAmountOut string `json:"estimatedAmountOut"` EstimatedAmountOutUsd string `json:"estimatedAmountOutUsd"` EstimatedTimeMs int64 `json:"estimatedTimeMs"` MaxTradeUsd int64 `json:"maxTradeUsd"` Fees struct { BridgeFeeBps int `json:"bridgeFeeBps"` + BridgeFeeUsd string `json:"bridgeFeeUsd"` GasFeeUsd string `json:"gasFeeUsd"` TotalFeeUsd string `json:"totalFeeUsd"` } `json:"fees"` @@ -87,11 +103,120 @@ func (m *MobulaBridge) APIKey() string { return m.apiKey } +// mobulaTypedData mirrors the typedData block of the quote response for +// EIP-712 signing. +type mobulaTypedData struct { + Domain struct { + Name string `json:"name"` + Version string `json:"version"` + ChainId int64 `json:"chainId"` + } `json:"domain"` + Types apitypes.Types `json:"types"` + PrimaryType string `json:"primaryType"` + Message apitypes.TypedDataMessage `json:"message"` +} + +// GetQuote fetches an unsigned quote. For EVM origins the returned deposit is +// a placeholder that the solver will not fill: use GetSignedQuote for real +// executions. func (m *MobulaBridge) GetQuote(originChain, originToken, destChain, destToken, senderAddress, walletAddress string, amount float64) (*MobulaQuoteResponse, time.Duration, error) { + return m.quote(originChain, originToken, destChain, destToken, senderAddress, walletAddress, amount, "") +} + +// GetSignedQuote runs the full two-step flow: unsigned quote → EIP-712 sign +// the returned typedData → re-quote echoing signature/intentId/deadline/ +// minAmountOut. Only the second response's deposit is executable. Solana +// origins skip the second call. evmKey==nil falls back to the unsigned quote +// (used by the read-only quote loop where no key is configured). +func (m *MobulaBridge) GetSignedQuote(originChain, originToken, destChain, destToken, senderAddress, walletAddress string, amount float64, evmKey *ecdsa.PrivateKey) (*MobulaQuoteResponse, time.Duration, error) { + quote, latency, err := m.GetQuote(originChain, originToken, destChain, destToken, senderAddress, walletAddress, amount) + if err != nil { + return nil, latency, err + } + if !quote.Data.SignatureRequired { + return quote, latency, nil + } + if evmKey == nil { + log.Printf(" [mobula] ⚠️ origin %s requires an EIP-712 signature but no EVM key is configured — returning unsigned quote (NOT executable)", originChain) + return quote, latency, nil + } + if len(quote.Data.TypedData) == 0 || quote.Data.IntentId == "" || quote.Data.Deadline == 0 { + return nil, latency, fmt.Errorf("signatureRequired but quote is missing typedData/intentId/deadline") + } + signature, minAmountOut, err := signBridgeIntent(quote.Data.TypedData, evmKey) + if err != nil { + return nil, latency, fmt.Errorf("failed to sign bridge intent: %w", err) + } + if minAmountOut == "" { + return nil, latency, fmt.Errorf("typedData message has no minAmountOut") + } + log.Printf(" [mobula] 🔏 Signed intent %s (deadline %d), fetching signed quote...", quote.Data.IntentId, quote.Data.Deadline) + extra := fmt.Sprintf("&signature=%s&intentId=%s&deadline=%d&minAmountOut=%s", signature, quote.Data.IntentId, quote.Data.Deadline, minAmountOut) + signed, signedLatency, err := m.quote(originChain, originToken, destChain, destToken, senderAddress, walletAddress, amount, extra) + if err != nil { + return nil, latency + signedLatency, fmt.Errorf("signed quote failed: %w", err) + } + return signed, latency + signedLatency, nil +} + +// signBridgeIntent hashes the EIP-712 typedData v4 and signs with evmKey. +// Returns the 0x-prefixed 65-byte signature and the minAmountOut extracted +// from the message (echoed back to the confirm endpoint). +func signBridgeIntent(typedDataJSON json.RawMessage, key *ecdsa.PrivateKey) (string, string, error) { + var raw mobulaTypedData + if err := json.Unmarshal(typedDataJSON, &raw); err != nil { + return "", "", fmt.Errorf("parse typedData: %w", err) + } + // Mobula's typedData omits the EIP712Domain type definition (only + // BridgeIntent is listed under types). go-ethereum requires it to hash + // the domain, so we synthesize one from the domain fields we actually + // received (name/version/chainId). + if _, ok := raw.Types["EIP712Domain"]; !ok { + raw.Types["EIP712Domain"] = []apitypes.Type{ + {Name: "name", Type: "string"}, + {Name: "version", Type: "string"}, + {Name: "chainId", Type: "uint256"}, + } + } + td := apitypes.TypedData{ + Types: raw.Types, + PrimaryType: raw.PrimaryType, + Domain: apitypes.TypedDataDomain{ + Name: raw.Domain.Name, + Version: raw.Domain.Version, + ChainId: ethmath.NewHexOrDecimal256(raw.Domain.ChainId), + }, + Message: raw.Message, + } + minAmountOut := "" + if v, ok := raw.Message["minAmountOut"]; ok { + minAmountOut = fmt.Sprintf("%v", v) + } + domainSep, err := td.HashStruct("EIP712Domain", td.Domain.Map()) + if err != nil { + return "", "", fmt.Errorf("hash domain: %w", err) + } + msgHash, err := td.HashStruct(td.PrimaryType, td.Message) + if err != nil { + return "", "", fmt.Errorf("hash message: %w", err) + } + digest := crypto.Keccak256([]byte("\x19\x01"), domainSep, msgHash) + sig, err := crypto.Sign(digest, key) + if err != nil { + return "", "", fmt.Errorf("sign digest: %w", err) + } + // canonical v is 27 or 28 (crypto.Sign returns 0/1) + if sig[64] < 27 { + sig[64] += 27 + } + return "0x" + hex.EncodeToString(sig), minAmountOut, nil +} + +func (m *MobulaBridge) quote(originChain, originToken, destChain, destToken, senderAddress, walletAddress string, amount float64, extraParams string) (*MobulaQuoteResponse, time.Duration, error) { start := time.Now() url := fmt.Sprintf( - "https://api.mobula.io/api/2/bridge/quote?originChainId=%s&originToken=%s&destinationChainId=%s&destinationToken=%s&amount=%s&walletAddress=%s&apiKey=%s", + "https://demo-api.mobula.io/api/2/bridge/quote?originChainId=%s&originToken=%s&destinationChainId=%s&destinationToken=%s&amount=%s&walletAddress=%s&apiKey=%s", originChain, originToken, destChain, destToken, strconv.FormatFloat(amount, 'f', -1, 64), walletAddress, m.apiKey, @@ -99,6 +224,7 @@ func (m *MobulaBridge) GetQuote(originChain, originToken, destChain, destToken, if senderAddress != "" { url += "&senderAddress=" + senderAddress } + url += extraParams req, err := http.NewRequest("GET", url, nil) if err != nil { @@ -135,7 +261,7 @@ func (m *MobulaBridge) GetQuote(originChain, originToken, destChain, destToken, } func (m *MobulaBridge) GetStatus(txHash string) (*MobulaStatusResponse, error) { - url := fmt.Sprintf("https://api.mobula.io/api/2/bridge/status/%s", txHash) + url := fmt.Sprintf("https://demo-api.mobula.io/api/2/bridge/status/%s", txHash) req, err := http.NewRequest("GET", url, nil) if err != nil { @@ -164,7 +290,7 @@ func (m *MobulaBridge) GetStatus(txHash string) (*MobulaStatusResponse, error) { } func (m *MobulaBridge) VerifyRoutes() (*MobulaRoutesResponse, error) { - url := "https://api.mobula.io/api/2/bridge/routes" + url := "https://demo-api.mobula.io/api/2/bridge/routes" req, err := http.NewRequest("GET", url, nil) if err != nil { @@ -363,7 +489,7 @@ func (m *MobulaBridge) TestRoute(route TestRoute, amount, amountUsd float64, reg } // Success-only: see debridge_bridge.go, failures must not enter the histogram. - bridgeQuoteLatency.WithLabelValues(labels...).Observe(float64(quoteLatency.Milliseconds())) + bridgeQuoteLatency.WithLabelValues(labels...).Observe(float64(quoteLatency.Nanoseconds()) / 1e6) bridgeQuoteSuccess.WithLabelValues(labels...).Set(1) diff --git a/harnesses/bridge-monitor/cmd/monitor/nearintents_bridge.go b/harnesses/bridge-monitor/cmd/monitor/nearintents_bridge.go index 645f2f0c..2958a8e6 100644 --- a/harnesses/bridge-monitor/cmd/monitor/nearintents_bridge.go +++ b/harnesses/bridge-monitor/cmd/monitor/nearintents_bridge.go @@ -219,7 +219,7 @@ func (n *NearIntentsBridge) TestRoute(route TestRoute, amount, amountUsd float64 // Latency is only meaningful for quotes that returned a usable response // (same rule as the other bridges). Fast 4xx rejections must not enter // the histogram or they skew the leader board. - bridgeQuoteLatency.WithLabelValues(labels...).Observe(float64(quoteLatency.Milliseconds())) + bridgeQuoteLatency.WithLabelValues(labels...).Observe(float64(quoteLatency.Nanoseconds()) / 1e6) bridgeQuoteSuccess.WithLabelValues(labels...).Set(1) inUsd, _ := strconv.ParseFloat(quote.Quote.AmountInUsd, 64) diff --git a/harnesses/bridge-monitor/cmd/monitor/onchain_balance.go b/harnesses/bridge-monitor/cmd/monitor/onchain_balance.go index ac450ef1..c6aac96d 100644 --- a/harnesses/bridge-monitor/cmd/monitor/onchain_balance.go +++ b/harnesses/bridge-monitor/cmd/monitor/onchain_balance.go @@ -12,6 +12,7 @@ import ( "github.com/ethereum/go-ethereum" "github.com/ethereum/go-ethereum/common" "github.com/ethereum/go-ethereum/common/hexutil" + "github.com/ethereum/go-ethereum/ethclient" "github.com/gagliardetto/solana-go" "github.com/gagliardetto/solana-go/rpc" ) @@ -49,6 +50,27 @@ func (tx *TxExecutor) evmClientFor(chain string) interface{ CallContract(context return nil } +// evmNativeBalance returns the native (ETH) balance of owner in wei. Needed by +// the balance fallback path: the Mobula portfolio API is the primary source, but +// when it is down we still need gas balances to gate executions safely. +func (tx *TxExecutor) evmNativeBalance(chain string, owner common.Address) (*big.Int, error) { + var client *ethclient.Client + switch strings.ToLower(chain) { + case "base": + client = tx.baseClient + case "arbitrum": + client = tx.arbitrumClient + default: + return nil, fmt.Errorf("unknown EVM chain: %s", chain) + } + if client == nil { + return nil, fmt.Errorf("no RPC client for %s", chain) + } + ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) + defer cancel() + return client.BalanceAt(ctx, owner, nil) +} + // solanaSPLBalanceOf returns the SPL token balance for `owner` and `mint`. If // the associated token account does not yet exist (never received this token), // returns 0 — that's a valid pre-execution state. diff --git a/harnesses/bridge-monitor/cmd/monitor/pricer.go b/harnesses/bridge-monitor/cmd/monitor/pricer.go index 024aac08..4a677987 100644 --- a/harnesses/bridge-monitor/cmd/monitor/pricer.go +++ b/harnesses/bridge-monitor/cmd/monitor/pricer.go @@ -52,7 +52,7 @@ func TokenPriceUSD(symbol string, fallback float64) float64 { func fetchPriceUSD(symbol string) float64 { apiKey := os.Getenv("MOBULA_API_KEY") - url := fmt.Sprintf("https://api.mobula.io/api/1/market/data?symbol=%s", symbol) + url := fmt.Sprintf("https://demo-api.mobula.io/api/1/market/data?symbol=%s", symbol) req, err := http.NewRequest("GET", url, nil) if err != nil { return 0 diff --git a/harnesses/bridge-monitor/cmd/monitor/reaper.go b/harnesses/bridge-monitor/cmd/monitor/reaper.go new file mode 100644 index 00000000..a4387451 --- /dev/null +++ b/harnesses/bridge-monitor/cmd/monitor/reaper.go @@ -0,0 +1,273 @@ +package main + +import ( + "fmt" + "log" + "os" + "strings" + "time" +) + +// Stuck-fund reaper: the triangle expects inventory on exactly three legs +// (Sol USDC, Base USDC, Arb USDT) plus native gas. Refunds and partial fills +// sometimes leave capital elsewhere (typically USDC on Arbitrum after an R3 +// refund), where no scheduled route will ever pick it up again. The reaper +// tracks how long such balances sit stranded, exports that as a gauge for +// alerting, and, when execution is enabled, sends ONE corrective transfer per +// hourly tick back to the neediest home leg. + +// Ignore balances below this: bridge dust and rounding leftovers are not worth +// a corrective transfer's fees. +const strandedDustUSD = 1.0 + +// Reaper transfers share the rebalancer's spirit of "never move more than a +// tier plus buffer": one capped hop per tick instead of one big blind sweep. +const reaperMaxTransferUSD = 300 * rebalanceBufferFactor + +// Tokens that are SUPPOSED to sit on each chain. Everything else above the +// dust floor counts as stranded. TRUMP and BRETT are meme-route inventory, +// native SOL and ETH are gas. +var homeTokens = map[string]map[string]bool{ + "Solana": {"USDC": true, "SOL": true, "TRUMP": true}, + "Base": {"USDC": true, "ETH": true, "BRETT": true}, + "Arbitrum": {"USDT0": true, "USDT": true, "ETH": true}, +} + +// Stranded tokens we know how to route home. Anything else is alert-only: +// building a bridge TX for an unknown asset from inside a repair loop is how +// funds get burned, so we only ever move assets we have addresses for. +var movableStranded = map[string]map[string]string{ + "Solana": {"USDT": "Es9vMFrzaCERmJfrF4H2FYD4KCoNkY11McCe8BenwNYB"}, + "Base": {"USDT": "0xfde4C96c8593536E31F229EA8f37b2ADa2699bb2"}, + "Arbitrum": {"USDC": arbUSDCAddr}, +} + +type strandKey struct { + Chain string + Token string +} + +// StrandedTracker remembers when each off-home balance was first seen so the +// stranded duration survives across ticks (but not restarts: a restart resets +// the clock, which only delays the reaper, never makes it over-eager). +type StrandedTracker struct { + firstSeen map[strandKey]time.Time +} + +func NewStrandedTracker() *StrandedTracker { + return &StrandedTracker{firstSeen: make(map[strandKey]time.Time)} +} + +// Update computes stranded hours for every off-home balance in the snapshot +// and returns them, including explicit zeros for keys that just came home so +// the exported gauge resets. Pure given (balances, now), unit-tested. +func (t *StrandedTracker) Update(balances map[string]map[string]float64, now time.Time) map[strandKey]float64 { + hours := make(map[strandKey]float64) + current := make(map[strandKey]bool) + + for chain, tokens := range balances { + home := homeTokens[chain] + for token, usd := range tokens { + // Contract-address keys duplicate the symbol entries. + if strings.HasPrefix(token, "0x") || len(token) > 12 { + continue + } + if usd <= strandedDustUSD || (home != nil && home[token]) { + continue + } + k := strandKey{chain, token} + current[k] = true + first, seen := t.firstSeen[k] + if !seen { + first = now + t.firstSeen[k] = first + } + hours[k] = now.Sub(first).Hours() + } + } + + // Keys that just came home report an explicit 0 (so the exported gauge + // resets) and lose their firstSeen entry (so a later re-strand restarts + // the clock instead of inheriting the old one). + for k := range t.firstSeen { + if !current[k] { + hours[k] = 0 + delete(t.firstSeen, k) + } + } + return hours +} + +// StartReaper launches the hourly stuck-fund loop. The gauge side runs in +// every mode, including while BENCHMARK_PAUSED, because alerting must keep +// working during a pause. The corrective transfer only fires in production +// mode, unpaused, with broadcast-capable keys. +func StartReaper(bc *BalanceChecker, rebalancer *Rebalancer, slack *SlackNotifier, mode string, paused bool) { + // Balance source: real checker in normal operation, the SIMULATE_BALANCES + // snapshot in keyless dry-run so the gauge path stays testable locally. + // SimulateBalances itself refuses to return anything outside dry-run, so + // a production process with keys can never act on made-up numbers. + execMode := ExecutionMode(mode) + var fetch func() (map[string]map[string]float64, bool, error) + switch { + case bc != nil: + fetch = bc.GetAllBalancesDetailed + case SimulateBalances(execMode) != nil: + fetch = func() (map[string]map[string]float64, bool, error) { + return SimulateBalances(execMode), false, nil + } + default: + log.Println("🧹 Reaper disabled: no balance checker") + return + } + + strandedHours := parseFloat(os.Getenv("REAPER_STRANDED_HOURS"), 6) + tracker := NewStrandedTracker() + + canAct := mode == "production" && !paused && rebalancer.canAct() + log.Printf("🧹 Stuck-fund reaper started (threshold %.0fh, acting=%v)", strandedHours, canAct) + + go func() { + // First evaluation immediately so gauges exist right after boot, then hourly. + reaperTick(fetch, rebalancer, slack, tracker, strandedHours, canAct) + ticker := time.NewTicker(1 * time.Hour) + defer ticker.Stop() + for range ticker.C { + reaperTick(fetch, rebalancer, slack, tracker, strandedHours, canAct) + } + }() +} + +func reaperTick(fetch func() (map[string]map[string]float64, bool, error), rebalancer *Rebalancer, slack *SlackNotifier, + tracker *StrandedTracker, thresholdHours float64, canAct bool, +) { + balances, degraded, err := fetch() + if err != nil || degraded { + // Without trustworthy balances we cannot tell stranded from home, so + // neither the gauge nor a transfer would mean anything this tick. + log.Printf("🧹 Reaper tick skipped: balances unreadable (degraded=%v err=%v)", degraded, err) + return + } + + hours := tracker.Update(balances, time.Now().UTC()) + + // Home legs always report 0 so alert expressions have a baseline series. + for _, leg := range triangleLegs { + sym := leg.Token + if leg.Chain == "Arbitrum" { + sym = "USDT0" + } + bridgeStrandedHours.WithLabelValues(leg.Chain, sym).Set(0) + } + for k, h := range hours { + bridgeStrandedHours.WithLabelValues(k.Chain, k.Token).Set(h) + } + + // Pick the single worst offender above threshold; one transfer per tick + // keeps the blast radius of a bad tick to one capped TX. + var worst strandKey + worstHours := 0.0 + for k, h := range hours { + if h >= thresholdHours && h > worstHours { + worst, worstHours = k, h + } + } + if worstHours == 0 { + log.Printf("🧹 Reaper tick: no funds stranded beyond %.0fh", thresholdHours) + return + } + + amount := balances[worst.Chain][worst.Token] + log.Printf("🧹 Stranded funds: $%.2f %s on %s for %.1fh (threshold %.0fh)", + amount, worst.Token, worst.Chain, worstHours, thresholdHours) + + if !canAct { + // Read-only mode: the gauge above is the alert path, a human handles it. + return + } + + srcAddr, known := movableStranded[worst.Chain][worst.Token] + if !known { + _ = slack.NotifyReaper(fmt.Sprintf( + "$%.2f of %s stranded on %s for %.1fh, but I have no safe route for that asset. Manual rebalance needed (cmd/rebalance).", + amount, worst.Token, worst.Chain, worstHours)) + return + } + + if spent := rebalancer.executor.DailySpent(); spent >= rebalancer.executor.config.MaxDailySpendUSD { + _ = slack.NotifyReaper(fmt.Sprintf( + "$%.2f of %s stranded on %s for %.1fh, but daily spend limit is reached ($%.2f / $%.2f). Will retry next tick.", + amount, worst.Token, worst.Chain, worstHours, + spent, rebalancer.executor.config.MaxDailySpendUSD)) + return + } + + // Single-flight: never broadcast while a scheduler slot (triangle run, + // rebalance, gas top-up) holds the execution lock. Skip instead of + // queueing: waiting here could fire a stale corrective transfer right + // after the slot already rebalanced the same leg. + if !execMu.TryLock() { + log.Printf("🧹 Reaper: execution lock busy (another actor is broadcasting), skipping corrective transfer this tick") + return + } + defer execMu.Unlock() + + dest := neediestHomeLeg(balances) + if amount > reaperMaxTransferUSD { + amount = reaperMaxTransferUSD + } + + route := TestRoute{ + Name: fmt.Sprintf("REAPER_%s_%s", strings.ToUpper(worst.Chain), strings.ToUpper(dest.Chain)), + FromChain: worst.Chain, + FromChainAPI: chainAPIFor(worst.Chain), + FromToken: srcAddr, + ToChain: dest.Chain, + ToChainAPI: dest.ChainAPI, + ToToken: dest.TokenAddr, + IsSolanaSrc: worst.Chain == "Solana", + } + + _ = slack.NotifyReaper(fmt.Sprintf( + "Moving $%.2f of stranded %s from %s (stuck %.1fh) home to %s %s via cheapest bridge.", + amount, worst.Token, worst.Chain, worstHours, dest.Chain, dest.Token)) + + // ExecuteCheapest books the spend (realized fee on success, flat estimate + // for a broadcast that never confirmed), so no accounting here. + result := rebalancer.ExecuteCheapest(route, amount) + switch classifyCorrectiveResult(result) { + case outcomeSuccess: + _ = slack.NotifyReaper(fmt.Sprintf( + "Corrective transfer succeeded via %s (fee $%.4f, tx %s).", result.Bridge, result.ActualFeeUSD, result.TxHash)) + case outcomeInFlight: + _ = slack.NotifyReaper(fmt.Sprintf( + "Corrective transfer broadcast (tx %s) but its bridge status did not resolve. Funds may still be in flight; standing down, next tick re-evaluates fresh balances.", + result.TxHash)) + default: + detail := "no bridge produced a usable quote" + if result != nil && result.Error != nil { + detail = result.Error.Error() + } + _ = slack.NotifyReaper(fmt.Sprintf("Corrective transfer FAILED before broadcast: %s. Will retry next tick.", detail)) + } +} + +// neediestHomeLeg returns the triangle leg with the lowest balance: stranded +// funds should land where they unblock the next scheduled cycle soonest. +func neediestHomeLeg(balances map[string]map[string]float64) legSpec { + best := triangleLegs[0] + bestBal := legBalanceUSD(balances, best) + for _, leg := range triangleLegs[1:] { + if b := legBalanceUSD(balances, leg); b < bestBal { + best, bestBal = leg, b + } + } + return best +} + +func chainAPIFor(chain string) string { + if leg, ok := findLeg(chain); ok { + return leg.ChainAPI + } + return "" +} diff --git a/harnesses/bridge-monitor/cmd/monitor/rebalancer.go b/harnesses/bridge-monitor/cmd/monitor/rebalancer.go new file mode 100644 index 00000000..9ead71dc --- /dev/null +++ b/harnesses/bridge-monitor/cmd/monitor/rebalancer.go @@ -0,0 +1,401 @@ +package main + +import ( + "fmt" + "log" + "strings" + "time" +) + +// Auto-rebalancer: when the pre-flight cycle simulation says a tier cannot run +// because one leg is short, move the shortfall from the leg holding excess +// inventory instead of waiting for a human. Reuses the executor's quote and +// broadcast plumbing so a rebalance transfer is measured and notified exactly +// like a benchmark execution. cmd/rebalance stays as the manual fallback. + +const ( + // Two corrective transfers per SCHEDULER SLOT, shared across every rung + // of the downgrade ladder. One transfer plus one retry covers transient + // pre-broadcast failures without letting a broken bridge drain the daily + // budget. Before this was per rung, a single $300 slot could broadcast + // up to six transfers back to back. + maxCorrectiveTransfersPerSlot = 2 + + // Move 10 percent more than the computed shortfall so bridge fees and + // slippage on the transfer itself do not leave the leg short again. + rebalanceBufferFactor = 1.10 +) + +// slotBudget caps corrective transfers for one scheduler slot. A single +// instance is created per slot and threaded through every ladder rung. +type slotBudget struct { + remaining int + // inFlight marks that a corrective transfer broadcast a TX whose bridge + // status never resolved. The funds are most likely still moving (legit + // fills can take 2-5 minutes, longer than our status polling), so any + // rung that needs the same destination leg must stand down for the rest + // of the slot instead of double-sending. + inFlight bool + inFlightChain string +} + +func newSlotBudget() *slotBudget { + return &slotBudget{remaining: maxCorrectiveTransfersPerSlot} +} + +// blockedByInFlight reports whether a rung needing refillChain must stand +// down because an earlier transfer to that leg is still unresolved. +func (b *slotBudget) blockedByInFlight(refillChain string) bool { + return b != nil && b.inFlight && strings.EqualFold(b.inFlightChain, refillChain) +} + +// correctiveOutcome classifies a corrective transfer attempt for retry logic. +type correctiveOutcome int + +const ( + // outcomePreBroadcast: nothing left the wallet (quote failed, approval + // failed to broadcast, insufficient funds). Safe to consume another + // attempt from the slot budget. + outcomePreBroadcast correctiveOutcome = iota + // outcomeInFlight: a TX was broadcast but the attempt did not confirm as + // a success. The deposit may well have confirmed with the bridge fill + // still pending, so the funds must be assumed to be in flight. TERMINAL + // for the slot: retrying here is exactly how a double-send happens. + outcomeInFlight + // outcomeSuccess: the transfer confirmed end to end. + outcomeSuccess +) + +// classifyCorrectiveResult decides whether a corrective transfer attempt may +// be retried. Anything that produced a TxHash moved, or may have moved, real +// funds and is terminal. Only failures that provably never broadcast are +// safe to retry. +func classifyCorrectiveResult(result *ExecutionResult) correctiveOutcome { + if result == nil { + return outcomePreBroadcast + } + if result.Success { + return outcomeSuccess + } + if result.TxHash != "" { + return outcomeInFlight + } + return outcomePreBroadcast +} + +// legSpec describes one home leg of the USDC triangle. +type legSpec struct { + Chain string + ChainAPI string + Token string + TokenAddr string +} + +// The triangle's expected inventory distribution. Order matches R1, R2, R3. +var triangleLegs = []legSpec{ + {Chain: "Solana", ChainAPI: "solana:solana", Token: "USDC", TokenAddr: solanaUSDCMint}, + {Chain: "Base", ChainAPI: "evm:8453", Token: "USDC", TokenAddr: baseUSDCAddr}, + {Chain: "Arbitrum", ChainAPI: "evm:42161", Token: "USDT", TokenAddr: arbUSDTAddr}, +} + +func findLeg(chain string) (legSpec, bool) { + for _, leg := range triangleLegs { + if strings.EqualFold(leg.Chain, chain) { + return leg, true + } + } + return legSpec{}, false +} + +// legBalanceUSD looks up a leg's balance by contract address first (robust +// against symbol drift like USDT vs USDT0 in Mobula's DB), then by symbol. +func legBalanceUSD(balances map[string]map[string]float64, leg legSpec) float64 { + chain := balances[leg.Chain] + if chain == nil { + return 0 + } + if v, ok := chain[strings.ToLower(leg.TokenAddr)]; ok { + return v + } + if v, ok := chain[leg.Token]; ok { + return v + } + if leg.Token == "USDT" { + if v, ok := chain["USDT0"]; ok { + return v + } + } + return 0 +} + +// BuildRefillRoute turns a failed CycleSimulation into a corrective TestRoute. +// The source leg is the triangle leg holding the largest surplus above its own +// 1x tier requirement: taking from anywhere else would just move the blockage. +// The amount is the shortfall plus the fee buffer, never more, so a rebalance +// cannot silently drain a healthy leg. Pure function, unit-tested. +func BuildRefillRoute(sim CycleSimulation, balances map[string]map[string]float64) (TestRoute, float64, error) { + if sim.Viable { + return TestRoute{}, 0, fmt.Errorf("cycle already viable, nothing to rebalance") + } + if sim.RefillUSD <= 0 { + return TestRoute{}, 0, fmt.Errorf("simulation has no refill amount") + } + dest, ok := findLeg(sim.RefillChain) + if !ok { + return TestRoute{}, 0, fmt.Errorf("unknown refill chain %q", sim.RefillChain) + } + + var source legSpec + bestSurplus := 0.0 + for _, leg := range triangleLegs { + if leg.Chain == dest.Chain { + continue + } + surplus := legBalanceUSD(balances, leg) - sim.Tier + if surplus > bestSurplus { + source = leg + bestSurplus = surplus + } + } + if bestSurplus <= 0 { + return TestRoute{}, 0, fmt.Errorf("no leg holds excess inventory above its own $%.0f tier need", sim.Tier) + } + if bestSurplus < sim.RefillUSD { + return TestRoute{}, 0, fmt.Errorf("no leg holds enough excess: need $%.2f, best surplus is $%.2f on %s", + sim.RefillUSD, bestSurplus, source.Chain) + } + + amount := sim.RefillUSD * rebalanceBufferFactor + if amount > bestSurplus { + amount = bestSurplus + } + + route := TestRoute{ + Name: fmt.Sprintf("REBALANCE_%s_%s", strings.ToUpper(source.Chain), strings.ToUpper(dest.Chain)), + FromChain: source.Chain, + FromChainAPI: source.ChainAPI, + FromToken: source.TokenAddr, + ToChain: dest.Chain, + ToChainAPI: dest.ChainAPI, + ToToken: dest.TokenAddr, + IsSolanaSrc: source.Chain == "Solana", + } + return route, amount, nil +} + +// Rebalancer executes corrective transfers via the cheapest live bridge quote. +type Rebalancer struct { + executor *Executor + slack *SlackNotifier +} + +func NewRebalancer(executor *Executor, slack *SlackNotifier) *Rebalancer { + if executor == nil { + return nil + } + return &Rebalancer{executor: executor, slack: slack} +} + +// canAct verifies keys exist and we are in a mode allowed to broadcast. Keeps +// the rebalancer inert in dry-run and while the benchmark is paused. +func (r *Rebalancer) canAct() bool { + return r != nil && r.executor != nil && r.executor.txExecutor != nil && r.executor.txExecutor.CanExecute() +} + +// TryUnblockTier attempts corrective transfers to make the given failed +// simulation viable, consuming from the slot-wide budget shared across all +// ladder rungs. Returns the latest simulation and whether the tier can now +// run. Callers must only pass non-degraded balances: rebalancing on +// unreadable balances could move real funds based on phantom zeros. +func (r *Rebalancer) TryUnblockTier(sim CycleSimulation, balances map[string]map[string]float64, tierLabel string, budget *slotBudget) (CycleSimulation, bool) { + if !r.canAct() { + return sim, false + } + if budget == nil { + budget = newSlotBudget() + } + + for { + if budget.blockedByInFlight(sim.RefillChain) { + log.Printf("🔧 Tier $%.0f needs %s but a corrective transfer to that leg is already in flight, standing down for this slot", + sim.Tier, sim.RefillChain) + return sim, false + } + if budget.remaining <= 0 { + bridgeRebalanceAttempts.WithLabelValues("capped").Inc() + _ = r.slack.NotifyRebalance("capped", fmt.Sprintf( + "Tier $%.0f (%s) still blocked (%s) but this slot's corrective transfer budget (%d) is spent. Standing down until next slot.", + sim.Tier, tierLabel, sim.Reason, maxCorrectiveTransfersPerSlot)) + return sim, false + } + if spent := r.executor.DailySpent(); spent >= r.executor.config.MaxDailySpendUSD { + bridgeRebalanceAttempts.WithLabelValues("capped").Inc() + _ = r.slack.NotifyRebalance("capped", fmt.Sprintf( + "Tier $%.0f (%s) blocked (%s) but daily spend limit is reached ($%.2f / $%.2f). No rebalance attempted.", + sim.Tier, tierLabel, sim.Reason, spent, r.executor.config.MaxDailySpendUSD)) + return sim, false + } + + route, amountUSD, err := BuildRefillRoute(sim, balances) + if err != nil { + bridgeRebalanceAttempts.WithLabelValues("failed").Inc() + log.Printf("🔧 Rebalance not possible for tier $%.0f: %v", sim.Tier, err) + _ = r.slack.NotifyRebalance("failed", fmt.Sprintf( + "Tier $%.0f (%s) blocked (%s) and no corrective route could be built: %v", + sim.Tier, tierLabel, sim.Reason, err)) + return sim, false + } + + budget.remaining-- + attempt := maxCorrectiveTransfersPerSlot - budget.remaining + bridgeRebalanceAttempts.WithLabelValues("attempted").Inc() + log.Printf("🔧 Rebalance attempt %d/%d (slot budget): $%.2f %s -> %s (unblocks tier $%.0f)", + attempt, maxCorrectiveTransfersPerSlot, amountUSD, route.FromChain, route.ToChain, sim.Tier) + _ = r.slack.NotifyRebalance("attempted", fmt.Sprintf( + "Tier $%.0f (%s) blocked: %s\nMoving $%.2f from %s %s to %s %s (slot attempt %d/%d).", + sim.Tier, tierLabel, sim.Reason, amountUSD, route.FromChain, sourceSymbol(route), route.ToChain, sim.RefillToken, + attempt, maxCorrectiveTransfersPerSlot)) + + result := r.ExecuteCheapest(route, amountUSD) + switch classifyCorrectiveResult(result) { + case outcomeInFlight: + // The deposit TX exists on-chain but the bridge status never + // resolved. Funds are most likely still moving toward the short + // leg: any further transfer to that leg this slot would be a + // double send. Terminal for the whole scheduler slot. + budget.inFlight = true + budget.inFlightChain = sim.RefillChain + bridgeRebalanceAttempts.WithLabelValues("in_flight").Inc() + _ = r.slack.NotifyRebalance("in_flight", fmt.Sprintf( + "Corrective transfer for tier $%.0f broadcast (tx %s) but its bridge status did not resolve. Funds are likely still in flight: standing down until next slot to avoid a double send.", + sim.Tier, result.TxHash)) + return sim, false + + case outcomePreBroadcast: + bridgeRebalanceAttempts.WithLabelValues("failed").Inc() + detail := "no bridge produced a usable quote" + if result != nil && result.Error != nil { + detail = result.Error.Error() + } + _ = r.slack.NotifyRebalance("failed", fmt.Sprintf( + "Rebalance transfer for tier $%.0f failed before broadcast: %s", sim.Tier, detail)) + // Nothing left the wallet, so another attempt is safe if the + // slot budget still allows one. + continue + + case outcomeSuccess: + bridgeRebalanceAttempts.WithLabelValues("succeeded").Inc() + _ = r.slack.NotifyRebalance("succeeded", fmt.Sprintf( + "Moved $%.2f from %s to %s via %s (fee $%.4f, tx %s). Re-checking tier $%.0f viability.", + amountUSD, route.FromChain, route.ToChain, result.Bridge, result.ActualFeeUSD, result.TxHash, sim.Tier)) + } + + // Give the destination credit a moment to be indexed by the portfolio + // API before re-simulating, otherwise we would re-read the old state. + time.Sleep(15 * time.Second) + + fresh, degraded, err := r.executor.balanceCheck.GetAllBalancesDetailed() + if err != nil || degraded { + log.Printf("🔧 Post-rebalance balances unreadable (degraded=%v err=%v), stopping", degraded, err) + return sim, false + } + balances = fresh + sim = SimulateTriangleCycle(balances, sim.Tier) + if sim.Viable { + return sim, true + } + } +} + +// ExecuteCheapest quotes all three executing bridges and broadcasts through +// the cheapest one that returned a live quote. Shared by the tier rebalancer +// and the stuck-fund reaper so both follow the same cost discipline. +func (r *Rebalancer) ExecuteCheapest(route TestRoute, amountUSD float64) *ExecutionResult { + bridge, fee, err := r.quoteCheapest(route, amountUSD) + if err != nil { + log.Printf("🔧 Cheapest-quote selection failed for %s: %v", route.Name, err) + return nil + } + log.Printf("🔧 Cheapest bridge for %s: %s (quoted fee $%.4f)", route.Name, bridge, fee) + + rawUnits := toRawUnits(amountUSD) + result := r.executor.executeOnBridge(bridge, route, amountUSD, amountUSD, rawUnits) + // Book the cost here so both callers (tier rebalancer and reaper) share + // the same accounting, including the flat estimate for broadcasts that + // never confirmed. + r.executor.accountSpend(result) + return result +} + +// quoteCheapest fetches quotes from Mobula, Relay and LI.FI and returns the +// bridge with the lowest total quoted fee among those that answered. +func (r *Rebalancer) quoteCheapest(route TestRoute, amountUSD float64) (string, float64, error) { + e := r.executor + sender := e.walletManager.EVMAddress + receiver := e.walletManager.EVMAddress + if route.FromChain == "Solana" { + sender = e.walletManager.SolanaAddress + } + if route.ToChain == "Solana" { + receiver = e.walletManager.SolanaAddress + } + rawUnits := toRawUnits(amountUSD) + + bestBridge := "" + bestFee := 0.0 + + consider := func(bridge string, fee float64, err error) { + if err != nil { + log.Printf("🔧 [%s] rebalance quote failed: %v", bridge, err) + return + } + if bestBridge == "" || fee < bestFee { + bestBridge, bestFee = bridge, fee + } + } + + if e.mobula != nil { + quote, _, err := e.mobula.GetQuote(route.FromChainAPI, route.FromToken, route.ToChainAPI, route.ToToken, sender, receiver, amountUSD) + if err != nil { + consider("mobula", 0, err) + } else { + consider("mobula", parseFloatOrZero(quote.Data.Fees.TotalFeeUsd)+parseFloatOrZero(quote.Data.Fees.GasFeeUsd), nil) + } + } + + if quote, _, err := e.relay.GetQuote(route, rawUnits, sender, receiver); err != nil { + consider("relay", 0, err) + } else { + consider("relay", parseFloatOrZero(quote.Fees.RelayerService.AmountUsd)+parseFloatOrZero(quote.Fees.RelayerGas.AmountUsd), nil) + } + + if quote, _, err := e.lifi.GetQuote(route, rawUnits, sender, receiver); err != nil { + consider("lifi", 0, err) + } else { + fee := 0.0 + for _, f := range quote.Estimate.FeeCosts { + fee += parseFloatOrZero(f.AmountUSD) + } + consider("lifi", fee, nil) + } + + if bestBridge == "" { + return "", 0, fmt.Errorf("all bridges failed to quote %s", route.Name) + } + return bestBridge, bestFee, nil +} + +func parseFloatOrZero(s string) float64 { + return parseFloat(s, 0) +} + +// sourceSymbol maps the route's source token address back to a human symbol +// for Slack messages. +func sourceSymbol(route TestRoute) string { + for _, leg := range triangleLegs { + if strings.EqualFold(leg.TokenAddr, route.FromToken) { + return leg.Token + } + } + return route.FromToken +} diff --git a/harnesses/bridge-monitor/cmd/monitor/relay_bridge.go b/harnesses/bridge-monitor/cmd/monitor/relay_bridge.go index 708aa052..7350d132 100644 --- a/harnesses/bridge-monitor/cmd/monitor/relay_bridge.go +++ b/harnesses/bridge-monitor/cmd/monitor/relay_bridge.go @@ -188,7 +188,7 @@ func (r *RelayBridge) TestRoute(route TestRoute, amount, amountUsd float64, rawU // (the published methodology measures exactly that). Fast failures, e.g. // Cloudflare 403s answered in 30ms, must not enter the histogram: they // made deBridge look 15x faster the moment its API started rejecting us. - bridgeQuoteLatency.WithLabelValues(labels...).Observe(float64(quoteLatency.Milliseconds())) + bridgeQuoteLatency.WithLabelValues(labels...).Observe(float64(quoteLatency.Nanoseconds()) / 1e6) bridgeQuoteSuccess.WithLabelValues(labels...).Set(1) inUsd, _ := strconv.ParseFloat(quote.Details.CurrencyIn.AmountUsd, 64) diff --git a/harnesses/bridge-monitor/cmd/monitor/selfheal_test.go b/harnesses/bridge-monitor/cmd/monitor/selfheal_test.go new file mode 100644 index 00000000..7eb6d013 --- /dev/null +++ b/harnesses/bridge-monitor/cmd/monitor/selfheal_test.go @@ -0,0 +1,269 @@ +package main + +import ( + "fmt" + "math" + "testing" + "time" +) + +func balancesSnapshot(sol, base, arb float64) map[string]map[string]float64 { + return map[string]map[string]float64{ + "Solana": {"USDC": sol, "SOL": 30}, + "Base": {"USDC": base, "ETH": 40}, + "Arbitrum": {"USDT0": arb, "ETH": 25}, + } +} + +func TestBuildRefillRouteSourceSelection(t *testing.T) { + // Solana leg is short for a $50 tier; Base holds the biggest surplus so it + // must be picked as the source, not Arbitrum. + balances := balancesSnapshot(10, 400, 120) + sim := SimulateTriangleCycle(balances, 50) + if sim.Viable { + t.Fatal("expected $50 cycle to be blocked on Solana") + } + if sim.RefillChain != "Solana" { + t.Fatalf("expected refill chain Solana, got %s", sim.RefillChain) + } + + route, amount, err := BuildRefillRoute(sim, balances) + if err != nil { + t.Fatalf("BuildRefillRoute failed: %v", err) + } + if route.FromChain != "Base" { + t.Errorf("expected source Base (largest surplus), got %s", route.FromChain) + } + if route.ToChain != "Solana" { + t.Errorf("expected destination Solana, got %s", route.ToChain) + } + if route.FromToken != baseUSDCAddr { + t.Errorf("expected Base USDC source token, got %s", route.FromToken) + } + if route.ToToken != solanaUSDCMint { + t.Errorf("expected Solana USDC destination token, got %s", route.ToToken) + } + if !route.IsSolanaSrc == (route.FromChain == "Solana") { + t.Error("IsSolanaSrc inconsistent with FromChain") + } + + // Shortfall is 50 - 10 = 40, buffered by 10 percent. + want := sim.RefillUSD * rebalanceBufferFactor + if math.Abs(amount-want) > 0.01 { + t.Errorf("expected amount %.2f (need + 10%% buffer), got %.2f", want, amount) + } + if amount > 400-50 { + t.Errorf("amount %.2f exceeds Base surplus", amount) + } +} + +func TestBuildRefillRouteCapsAtSurplus(t *testing.T) { + // Surplus barely covers the shortfall: the buffered amount must be clamped + // to the surplus instead of overdrawing the source leg. + balances := balancesSnapshot(0, 55.5, 5) + sim := SimulateTriangleCycle(balances, 5) + if sim.Viable { + t.Fatal("expected $5 cycle to be blocked on Solana") + } + route, amount, err := BuildRefillRoute(sim, balances) + if err != nil { + t.Fatalf("BuildRefillRoute failed: %v", err) + } + if route.FromChain != "Base" { + t.Fatalf("expected source Base, got %s", route.FromChain) + } + surplus := 55.5 - 5 + if amount > surplus+0.001 { + t.Errorf("amount %.2f exceeds surplus %.2f", amount, surplus) + } +} + +func TestBuildRefillRouteNoExcess(t *testing.T) { + // Every leg is broke: refusing is the only safe answer. + balances := balancesSnapshot(1, 2, 3) + sim := SimulateTriangleCycle(balances, 50) + if _, _, err := BuildRefillRoute(sim, balances); err == nil { + t.Fatal("expected error when no leg holds excess inventory") + } +} + +func TestBuildRefillRouteViableRejected(t *testing.T) { + balances := balancesSnapshot(500, 500, 500) + sim := SimulateTriangleCycle(balances, 50) + if !sim.Viable { + t.Fatal("expected viable cycle") + } + if _, _, err := BuildRefillRoute(sim, balances); err == nil { + t.Fatal("expected error for viable simulation") + } +} + +func TestDowngradeLadder(t *testing.T) { + cases := []struct { + tier float64 + want []float64 + }{ + {300, []float64{300, 50, 5}}, + {50, []float64{50, 5}}, + {5, []float64{5}}, + } + for _, c := range cases { + got := downgradeLadder(c.tier) + if len(got) != len(c.want) { + t.Fatalf("tier %.0f: got %v want %v", c.tier, got, c.want) + } + for i := range got { + if got[i] != c.want[i] { + t.Errorf("tier %.0f: got %v want %v", c.tier, got, c.want) + break + } + } + } +} + +func TestClassifyCorrectiveResult(t *testing.T) { + if got := classifyCorrectiveResult(nil); got != outcomePreBroadcast { + t.Errorf("nil result: got %v, want pre-broadcast", got) + } + if got := classifyCorrectiveResult(&ExecutionResult{Success: true, TxHash: "0xabc"}); got != outcomeSuccess { + t.Errorf("confirmed fill: got %v, want success", got) + } + // A broadcast whose status never resolved is TERMINAL: the deposit may + // have confirmed with the fill still pending, so a retry double-sends. + if got := classifyCorrectiveResult(&ExecutionResult{Success: false, TxHash: "0xabc", Error: fmt.Errorf("status poll failed: timeout")}); got != outcomeInFlight { + t.Errorf("status timeout with TxHash: got %v, want in-flight", got) + } + // A revert or refund also carries a hash and stays terminal for the slot. + if got := classifyCorrectiveResult(&ExecutionResult{Reverted: true, TxHash: "0xabc"}); got != outcomeInFlight { + t.Errorf("reverted with TxHash: got %v, want in-flight", got) + } + // Pre-broadcast failures (quote failed, approval failed to broadcast, + // insufficient funds) have no hash and may consume another attempt. + if got := classifyCorrectiveResult(&ExecutionResult{Error: fmt.Errorf("quote failed")}); got != outcomePreBroadcast { + t.Errorf("quote failure: got %v, want pre-broadcast", got) + } +} + +func TestSlotBudgetSharedAcrossRungs(t *testing.T) { + if maxCorrectiveTransfersPerSlot != 2 { + t.Fatalf("slot budget must be 2 corrective transfers, got %d", maxCorrectiveTransfersPerSlot) + } + b := newSlotBudget() + if b.remaining != 2 { + t.Fatalf("fresh budget: got %d remaining, want 2", b.remaining) + } + + // Rung 1 ($300) consumes one attempt, rung 2 ($50) consumes the second: + // the SAME budget instance is threaded through the ladder, so rung 3 + // ($5) has nothing left. Per-rung counters allowed up to 6 transfers. + b.remaining-- + b.remaining-- + if b.remaining > 0 { + t.Fatalf("after two attempts across rungs the slot budget must be exhausted, got %d", b.remaining) + } +} + +func TestSlotBudgetInFlightBlocksSameLeg(t *testing.T) { + b := newSlotBudget() + if b.blockedByInFlight("Solana") { + t.Fatal("fresh budget must not block any leg") + } + + // A transfer to Solana broadcast but never resolved: every later rung + // needing Solana stands down, regardless of remaining budget. + b.inFlight = true + b.inFlightChain = "Solana" + if !b.blockedByInFlight("Solana") { + t.Error("rung needing the in-flight leg must stand down") + } + if !b.blockedByInFlight("solana") { + t.Error("leg matching must be case-insensitive") + } + if b.blockedByInFlight("Base") { + t.Error("a rung needing a different leg is not blocked by the in-flight transfer") + } + + var nilBudget *slotBudget + if nilBudget.blockedByInFlight("Solana") { + t.Error("nil budget must not block") + } +} + +func TestSimulateBalancesOnlyInDryRun(t *testing.T) { + t.Setenv("SIMULATE_BALANCES", "true") + if SimulateBalances(ModeDryRun) == nil { + t.Fatal("dry-run with SIMULATE_BALANCES=true must return the snapshot") + } + if SimulateBalances(ModeProduction) != nil { + t.Fatal("production mode must ignore SIMULATE_BALANCES") + } + if SimulateBalances(ModeSingleTest) != nil { + t.Fatal("single-test mode must ignore SIMULATE_BALANCES") + } + + t.Setenv("SIMULATE_BALANCES", "false") + if SimulateBalances(ModeDryRun) != nil { + t.Fatal("no snapshot when SIMULATE_BALANCES is off") + } +} + +func TestStrandedHoursComputation(t *testing.T) { + tracker := NewStrandedTracker() + t0 := time.Date(2026, 7, 12, 10, 0, 0, 0, time.UTC) + + // USDC on Arbitrum is off-home (triangle expects USDT there). + balances := balancesSnapshot(100, 100, 100) + balances["Arbitrum"]["USDC"] = 42 + + hours := tracker.Update(balances, t0) + k := strandKey{"Arbitrum", "USDC"} + if hours[k] != 0 { + t.Errorf("first sighting should report 0 hours, got %.2f", hours[k]) + } + // Home legs never appear as stranded. + if _, ok := hours[strandKey{"Solana", "USDC"}]; ok { + t.Error("home leg Solana/USDC reported as stranded") + } + if _, ok := hours[strandKey{"Arbitrum", "USDT0"}]; ok { + t.Error("home leg Arbitrum/USDT0 reported as stranded") + } + + // 7 hours later the same balance is still there: above the 6h default. + hours = tracker.Update(balances, t0.Add(7*time.Hour)) + if math.Abs(hours[k]-7) > 0.001 { + t.Errorf("expected 7 stranded hours, got %.2f", hours[k]) + } + + // Funds moved home: the key must report an explicit 0 so the gauge resets, + // and the clock must restart if it strands again later. + delete(balances["Arbitrum"], "USDC") + hours = tracker.Update(balances, t0.Add(8*time.Hour)) + if hours[k] != 0 { + t.Errorf("expected explicit 0 after funds came home, got %.2f", hours[k]) + } + + balances["Arbitrum"]["USDC"] = 42 + hours = tracker.Update(balances, t0.Add(20*time.Hour)) + if hours[k] != 0 { + t.Errorf("re-stranding must restart the clock at 0, got %.2f", hours[k]) + } +} + +func TestStrandedIgnoresDustAndAddressKeys(t *testing.T) { + tracker := NewStrandedTracker() + now := time.Now().UTC() + + balances := balancesSnapshot(100, 100, 100) + balances["Arbitrum"]["USDC"] = 0.5 + balances["Arbitrum"][arbUSDCAddr] = 5000 + + hours := tracker.Update(balances, now) + if _, ok := hours[strandKey{"Arbitrum", "USDC"}]; ok { + t.Error("dust below $1 must not count as stranded") + } + for k := range hours { + if len(k.Token) > 12 { + t.Errorf("contract-address key leaked into stranded set: %v", k) + } + } +} diff --git a/harnesses/bridge-monitor/cmd/monitor/slack.go b/harnesses/bridge-monitor/cmd/monitor/slack.go index 7b9422f1..579f514b 100644 --- a/harnesses/bridge-monitor/cmd/monitor/slack.go +++ b/harnesses/bridge-monitor/cmd/monitor/slack.go @@ -5,6 +5,7 @@ import ( "encoding/json" "fmt" "net/http" + "strings" "time" ) @@ -175,7 +176,7 @@ func (s *SlackNotifier) getBalanceSummary() string { // fetchBalances calls Mobula API to get wallet balances func (s *SlackNotifier) fetchBalances(wallet, blockchain string) string { - url := fmt.Sprintf("https://api.mobula.io/api/1/wallet/portfolio?wallet=%s&blockchains=%s", wallet, blockchain) + url := fmt.Sprintf("https://demo-api.mobula.io/api/1/wallet/portfolio?wallet=%s&blockchains=%s", wallet, blockchain) req, err := http.NewRequest("GET", url, nil) if err != nil { @@ -278,6 +279,67 @@ Waiting for next scheduled slot. Top up the blocked leg to unblock. return s.send(message) } +// NotifyRebalance reports every auto-rebalance event (attempted, succeeded, +// failed, capped) so the owner can audit what the self-healing loop moved. +func (s *SlackNotifier) NotifyRebalance(outcome, detail string) error { + if s == nil { + return nil + } + emoji := map[string]string{ + "attempted": "🔧", + "succeeded": "✅", + "failed": "❌", + "capped": "🛑", + }[outcome] + if emoji == "" { + emoji = "🔧" + } + message := fmt.Sprintf(`%s *Auto-Rebalance %s* + +%s`, emoji, strings.ToUpper(outcome), detail) + return s.send(message) +} + +// NotifyTierDowngrade reports that a scheduled tier ran at a smaller amount +// because the requested size was not viable. Partial data beats none, but the +// owner should know the wallet needs attention. +func (s *SlackNotifier) NotifyTierDowngrade(from, to float64, reason string) error { + if s == nil { + return nil + } + message := fmt.Sprintf(`⬇️ *Tier DOWNGRADED: $%.0f to $%.0f* + +*Reason $%.0f was blocked:* %s + +Running the smaller tier so the benchmark keeps producing data.`, + from, to, from, reason) + return s.send(message) +} + +// NotifyReaper reports stuck-fund reaper actions (corrective transfer of funds +// stranded off their home triangle leg). +func (s *SlackNotifier) NotifyReaper(detail string) error { + if s == nil { + return nil + } + message := fmt.Sprintf(`🧹 *Stuck-Fund Reaper* + +%s`, detail) + return s.send(message) +} + +// NotifyGasTopUp reports gas top-up events (low native gas detected, swap +// attempted, succeeded, failed, capped or gated). +func (s *SlackNotifier) NotifyGasTopUp(chain, outcome, detail string) error { + if s == nil { + return nil + } + message := fmt.Sprintf(`⛽ *Gas Top-Up %s on %s* + +%s`, strings.ToUpper(outcome), chain, detail) + return s.send(message) +} + // NotifyStartup sends a startup notification func (s *SlackNotifier) NotifyStartup(mode string) error { if s == nil { diff --git a/harnesses/bridge-monitor/cmd/monitor/spend_tracker.go b/harnesses/bridge-monitor/cmd/monitor/spend_tracker.go new file mode 100644 index 00000000..93865dd9 --- /dev/null +++ b/harnesses/bridge-monitor/cmd/monitor/spend_tracker.go @@ -0,0 +1,129 @@ +package main + +import ( + "encoding/json" + "log" + "os" + "strings" + "sync" + "time" +) + +// execMu is the package-level single-flight lock for every code path that can +// broadcast a transaction: scheduled triangle runs, the auto-rebalancer, the +// stuck-fund reaper and gas top-ups. All of these actors sign with the same +// two wallets, so two concurrent broadcasts race on PendingNonceAt and can +// reuse a nonce. Scheduler slots take the lock for the whole slot; the reaper +// only TryLocks and skips its corrective action when the lock is busy, it +// never queues behind a running slot. +var execMu sync.Mutex + +// SpendTracker accounts spending against the daily cap. The counter is keyed +// by UTC date so the budget resets exactly at midnight UTC, and the +// (date, spent) pair is persisted to disk so a process restart within the +// same UTC day keeps the consumed budget instead of minting a fresh one +// (a crash-looping process used to reset the counter on every start). +type SpendTracker struct { + mu sync.Mutex + path string + now func() time.Time + date string + spent float64 +} + +type spendState struct { + Date string `json:"date"` + SpentUSD float64 `json:"spent_usd"` +} + +// NewSpendTracker loads persisted state from path. A nil clock uses time.Now; +// tests inject a fake clock to drive the date rollover. State recorded on a +// different UTC date is discarded on load. +func NewSpendTracker(path string, now func() time.Time) *SpendTracker { + if now == nil { + now = time.Now + } + t := &SpendTracker{path: path, now: now} + t.date = t.utcDate() + if data, err := os.ReadFile(path); err == nil { + var s spendState + if json.Unmarshal(data, &s) == nil && s.Date == t.date && s.SpentUSD > 0 { + t.spent = s.SpentUSD + log.Printf("💾 Loaded daily spend state: $%.2f already spent on %s (%s)", t.spent, t.date, path) + } + } + return t +} + +func (t *SpendTracker) utcDate() string { + return t.now().UTC().Format("2006-01-02") +} + +// rolloverLocked resets the counter when the UTC date has changed since the +// last access. Callers must hold t.mu. +func (t *SpendTracker) rolloverLocked() { + if d := t.utcDate(); d != t.date { + log.Printf("💾 Daily spend reset: %s -> %s (was $%.2f)", t.date, d, t.spent) + t.date = d + t.spent = 0 + t.saveLocked() + } +} + +// Add books usd against today's budget and persists the new total. +func (t *SpendTracker) Add(usd float64) { + if usd <= 0 { + return + } + t.mu.Lock() + defer t.mu.Unlock() + t.rolloverLocked() + t.spent += usd + t.saveLocked() +} + +// Spent returns today's consumed budget, applying the UTC rollover first. +func (t *SpendTracker) Spent() float64 { + t.mu.Lock() + defer t.mu.Unlock() + t.rolloverLocked() + return t.spent +} + +// saveLocked persists atomically (write temp, rename). Callers hold t.mu. +func (t *SpendTracker) saveLocked() { + data, err := json.Marshal(spendState{Date: t.date, SpentUSD: t.spent}) + if err != nil { + return + } + tmp := t.path + ".tmp" + if err := os.WriteFile(tmp, data, 0o644); err != nil { + log.Printf("⚠️ Failed to persist spend state to %s: %v", tmp, err) + return + } + if err := os.Rename(tmp, t.path); err != nil { + log.Printf("⚠️ Failed to persist spend state to %s: %v", t.path, err) + } +} + +// spendStatePath resolves where the (date, spent) pair lives. Defaults to a +// path relative to the working directory (NOT /tmp, which some distros wipe +// on a timer) so restarts on the same host see the same file. +func spendStatePath() string { + if p := strings.TrimSpace(os.Getenv("SPEND_STATE_PATH")); p != "" { + return p + } + // Dry-run gets its own default file so simulated spend never eats the + // production budget when both run on the same host (review pass 2). + if strings.EqualFold(strings.TrimSpace(os.Getenv("EXECUTION_MODE")), "production") { + return "./spend-state.json" + } + return "./spend-state.dryrun.json" +} + +// failedTxFeeEstimateUSD is booked against the daily cap for any broadcast +// that produced a TxHash but never confirmed as a success: the deposit or +// approval TX most likely paid gas even though the bridge never filled. +func failedTxFeeEstimateUSD() float64 { + return parseFloat(os.Getenv("FAILED_TX_FEE_ESTIMATE_USD"), 0.50) +} diff --git a/harnesses/bridge-monitor/cmd/monitor/spend_tracker_test.go b/harnesses/bridge-monitor/cmd/monitor/spend_tracker_test.go new file mode 100644 index 00000000..f7637d8e --- /dev/null +++ b/harnesses/bridge-monitor/cmd/monitor/spend_tracker_test.go @@ -0,0 +1,79 @@ +package main + +import ( + "math" + "os" + "path/filepath" + "testing" + "time" +) + +func TestSpendTrackerUTCRollover(t *testing.T) { + path := filepath.Join(t.TempDir(), "spend-state.json") + now := time.Date(2026, 7, 12, 23, 0, 0, 0, time.UTC) + clock := func() time.Time { return now } + + tr := NewSpendTracker(path, clock) + tr.Add(3.5) + if got := tr.Spent(); math.Abs(got-3.5) > 1e-9 { + t.Fatalf("expected 3.5 spent, got %v", got) + } + + // A restart within the same UTC day must keep the consumed budget: a + // crash-loop must not mint a fresh budget. + tr2 := NewSpendTracker(path, clock) + if got := tr2.Spent(); math.Abs(got-3.5) > 1e-9 { + t.Fatalf("restart lost same-day spend: got %v, want 3.5", got) + } + + // Crossing midnight UTC resets the counter without a restart. + now = time.Date(2026, 7, 13, 1, 0, 0, 0, time.UTC) + if got := tr2.Spent(); got != 0 { + t.Fatalf("expected 0 after UTC date rollover, got %v", got) + } + tr2.Add(1.25) + + // A restart on the new day loads the new day's spend. + tr3 := NewSpendTracker(path, clock) + if got := tr3.Spent(); math.Abs(got-1.25) > 1e-9 { + t.Fatalf("restart on new day: got %v, want 1.25", got) + } +} + +func TestSpendTrackerDiscardsStaleState(t *testing.T) { + path := filepath.Join(t.TempDir(), "spend-state.json") + if err := os.WriteFile(path, []byte(`{"date":"2026-07-11","spent_usd":9.99}`), 0o644); err != nil { + t.Fatal(err) + } + clock := func() time.Time { return time.Date(2026, 7, 12, 8, 0, 0, 0, time.UTC) } + tr := NewSpendTracker(path, clock) + if got := tr.Spent(); got != 0 { + t.Fatalf("yesterday's spend must not carry over, got %v", got) + } +} + +func TestSpendTrackerIgnoresCorruptState(t *testing.T) { + path := filepath.Join(t.TempDir(), "spend-state.json") + if err := os.WriteFile(path, []byte("not json"), 0o644); err != nil { + t.Fatal(err) + } + tr := NewSpendTracker(path, nil) + if got := tr.Spent(); got != 0 { + t.Fatalf("corrupt state must read as 0, got %v", got) + } + tr.Add(2) + if got := tr.Spent(); math.Abs(got-2) > 1e-9 { + t.Fatalf("expected 2 after Add, got %v", got) + } +} + +func TestFailedTxFeeEstimateDefault(t *testing.T) { + t.Setenv("FAILED_TX_FEE_ESTIMATE_USD", "") + if got := failedTxFeeEstimateUSD(); math.Abs(got-0.50) > 1e-9 { + t.Fatalf("default estimate must be $0.50, got %v", got) + } + t.Setenv("FAILED_TX_FEE_ESTIMATE_USD", "1.25") + if got := failedTxFeeEstimateUSD(); math.Abs(got-1.25) > 1e-9 { + t.Fatalf("env override not honored, got %v", got) + } +} diff --git a/harnesses/bridge-monitor/cmd/monitor/tx_executor.go b/harnesses/bridge-monitor/cmd/monitor/tx_executor.go index 0e51c1bf..a9d08380 100644 --- a/harnesses/bridge-monitor/cmd/monitor/tx_executor.go +++ b/harnesses/bridge-monitor/cmd/monitor/tx_executor.go @@ -120,6 +120,13 @@ func (tx *TxExecutor) CanExecute() bool { return tx.solanaPrivateKey != nil && tx.evmPrivateKey != nil && !tx.dryRun } +// EVMPrivateKey exposes the loaded EVM key so the Mobula client can EIP-712 +// sign the bridge intent on the confirm step. Returns nil in quote-only mode +// (no key configured), which GetSignedQuote handles by degrading to unsigned. +func (tx *TxExecutor) EVMPrivateKey() *ecdsa.PrivateKey { + return tx.evmPrivateKey +} + // ExecuteSolanaTransaction signs and broadcasts a Solana transaction func (tx *TxExecutor) ExecuteSolanaTransaction(serializedTxBase64 string) (string, error) { if tx.dryRun { @@ -158,9 +165,14 @@ func (tx *TxExecutor) ExecuteSolanaTransaction(serializedTxBase64 string) (strin return "", fmt.Errorf("failed to sign tx: %w", err) } - // Send transaction + // Send transaction. Same send-ambiguity rule as the EVM path: a send + // error can follow node acceptance, so surface the signature when the + // client returns one and let callers treat it as in-flight. sig, err := tx.solanaClient.SendTransaction(ctx, transaction) if err != nil { + if sig != (solana.Signature{}) { + return sig.String(), fmt.Errorf("failed to send tx (may be accepted, sig %s): %w", sig.String(), err) + } return "", fmt.Errorf("failed to send tx: %w", err) } @@ -235,9 +247,14 @@ func (tx *TxExecutor) ExecuteSolanaFromInstructions(instructions []RelaySolanaIn return "", fmt.Errorf("failed to sign tx: %w", err) } - // Send transaction + // Send transaction. Same send-ambiguity rule as the EVM path: a send + // error can follow node acceptance, so surface the signature when the + // client returns one and let callers treat it as in-flight. sig, err := tx.solanaClient.SendTransaction(ctx, transaction) if err != nil { + if sig != (solana.Signature{}) { + return sig.String(), fmt.Errorf("failed to send tx (may be accepted, sig %s): %w", sig.String(), err) + } return "", fmt.Errorf("failed to send tx: %w", err) } @@ -314,13 +331,16 @@ func (tx *TxExecutor) ExecuteEVMTransaction(chain string, to string, data string return "", fmt.Errorf("failed to sign tx: %w", err) } - // Send transaction + // Send transaction. On error the node may STILL have accepted the tx + // (RPC timeout after acceptance): return the locally computed hash so + // callers classify this as in-flight, never as retry-safe. Discarding + // it caused the residual double-send hole found in review pass 2. + txHash := signedTx.Hash().Hex() err = client.SendTransaction(ctx, signedTx) if err != nil { - return "", fmt.Errorf("failed to send tx: %w", err) + return txHash, fmt.Errorf("failed to send tx (may be accepted, hash %s): %w", txHash, err) } - txHash := signedTx.Hash().Hex() log.Printf("📤 %s TX sent: %s", chain, txHash) return txHash, nil } @@ -407,7 +427,7 @@ func (tx *TxExecutor) PollMobulaStatus(txHash string, timeout time.Duration) (*B } func (tx *TxExecutor) getMobulaStatus(txHash string) (*BridgeStatus, error) { - url := fmt.Sprintf("https://api.mobula.io/api/2/bridge/status/%s", txHash) + url := fmt.Sprintf("https://demo-api.mobula.io/api/2/bridge/status/%s", txHash) req, err := http.NewRequest("GET", url, nil) if err != nil { return nil, err diff --git a/harnesses/chain-kpis/cmd/script/registry.go b/harnesses/chain-kpis/cmd/script/registry.go index eb084453..b68a25eb 100644 --- a/harnesses/chain-kpis/cmd/script/registry.go +++ b/harnesses/chain-kpis/cmd/script/registry.go @@ -52,6 +52,7 @@ var Registry = []Chain{ {Slug: "arbitrum", DefiLlama: "Arbitrum", Mobula: "Arbitrum", NativeSymbol: "ETH"}, {Slug: "optimism", DefiLlama: "Optimism", Mobula: "Optimistic", NativeSymbol: "ETH"}, {Slug: "base", DefiLlama: "Base", Mobula: "Base", NativeSymbol: "ETH"}, + {Slug: "robinhood", DefiLlama: "Robinhood Chain", Mobula: "Robinhood Chain", NativeSymbol: "ETH"}, {Slug: "zksync", DefiLlama: "ZKsync Era", Mobula: "ZkSync", NativeSymbol: "ETH"}, {Slug: "linea", DefiLlama: "Linea", Mobula: "Linea", NativeSymbol: "ETH"}, {Slug: "scroll", DefiLlama: "Scroll", Mobula: "Scroll", NativeSymbol: "ETH"}, diff --git a/harnesses/evm-block-builders/Dockerfile b/harnesses/evm-block-builders/Dockerfile new file mode 100644 index 00000000..7a6bc559 --- /dev/null +++ b/harnesses/evm-block-builders/Dockerfile @@ -0,0 +1,22 @@ +FROM golang:1.24-alpine AS builder + +WORKDIR /app +RUN apk add --no-cache git + +COPY go.mod go.sum ./ +RUN go mod download + +COPY . . + +RUN CGO_ENABLED=0 GOOS=linux go build -o /app/evm-block-builders ./cmd/script + +FROM debian:bookworm-slim + +WORKDIR /app +RUN apt-get update && apt-get install -y ca-certificates && rm -rf /var/lib/apt/lists/* + +COPY --from=builder /app/evm-block-builders /app/evm-block-builders + +EXPOSE 2112 + +CMD ["/app/evm-block-builders"] diff --git a/harnesses/evm-block-builders/cmd/script/arb_feed.go b/harnesses/evm-block-builders/cmd/script/arb_feed.go new file mode 100644 index 00000000..48784d96 --- /dev/null +++ b/harnesses/evm-block-builders/cmd/script/arb_feed.go @@ -0,0 +1,285 @@ +package main + +import ( + "context" + "encoding/json" + "fmt" + "net/http" + "sync" + "sync/atomic" + "time" + + "github.com/gorilla/websocket" +) + +// Arbitrum sequencer feed soft-confirmation lag. +// +// The sequencer feed (wss://arb1.arbitrum.io/feed) streams every L2 +// message the sequencer accepts, seconds before the block is queryable +// on a public RPC. That gap IS the soft-confirmation advantage: trading +// systems reading the feed act on state that RPC users haven't seen +// yet. We hold the feed open, timestamp each message's arrival keyed by +// sequence number, poll the public RPC head, and emit the feed→RPC +// visibility lag as a histogram. +// +// Feed sequenceNumber → L2 block number is a FIXED offset (genesis +// alignment; 22207817 at inception) but we derive it at runtime instead +// of hardcoding. Per calibration poll we sample cand = rpcHead - +// latestFeedSeq. While the feed is live, cand is trueOffset minus the +// 0-1 blocks in flight between feed and RPC (lag ~0.1s vs 0.25s block +// time), so trueOffset is the LARGEST frequently-seen candidate. A +// plain max is NOT safe: on connect the feed replays a short backlog, +// latestSeq briefly trails the chain, and cand overshoots (observed +// +3 → every lag inflated by ~750ms). We therefore ignore the first +// polls after the feed goes live and pick the highest candidate with +// meaningful support. The derived value is logged and exported +// (ebb_arb_derived_offset) so drift would be visible. + +const ( + arbDialTimeout = 15 * time.Second + arbReadDeadline = 60 * time.Second + arbWatchdog = 30 * time.Second + arbMinBackoff = 2 * time.Second + arbMaxBackoff = 60 * time.Second + arbCalibSamples = 60 // ~18s of head polls before freezing the offset + arbCalibWarmup = 10 // polls to discard while the feed drains its connect backlog + arbMaxLagSeconds = 30 // sanity bound: drop reconnect-gap outliers + arrivalsMax = 4000 +) + +type arbFeedState struct { + mu sync.Mutex + arrivals map[int64]time.Time // seq → first feed arrival + order []int64 // insertion order, for bounded pruning + latestSeq int64 +} + +func newArbFeedState() *arbFeedState { + return &arbFeedState{arrivals: make(map[int64]time.Time, arrivalsMax)} +} + +func (s *arbFeedState) record(seq int64, at time.Time) { + s.mu.Lock() + defer s.mu.Unlock() + if _, dup := s.arrivals[seq]; dup { + return // only first arrival counts + } + s.arrivals[seq] = at + s.order = append(s.order, seq) + if seq > s.latestSeq { + s.latestSeq = seq + } + for len(s.order) > arrivalsMax { + delete(s.arrivals, s.order[0]) + s.order = s.order[1:] + } +} + +func (s *arbFeedState) lookup(seq int64) (time.Time, bool) { + s.mu.Lock() + defer s.mu.Unlock() + t, ok := s.arrivals[seq] + return t, ok +} + +func (s *arbFeedState) latest() int64 { + s.mu.Lock() + defer s.mu.Unlock() + return s.latestSeq +} + +// arbFeedMsg is the minimal shape of a broadcast frame: +// +// {"version":1,"messages":[{"sequenceNumber":296300000,"message":{...}}]} +type arbFeedMsg struct { + Messages []struct { + SequenceNumber int64 `json:"sequenceNumber"` + } `json:"messages"` +} + +// runArbFeed holds the sequencer feed WS open (reconnect loop with +// backoff + silence watchdog, same pattern as l2-block-time). +func runArbFeed(ctx context.Context, state *arbFeedState) { + url := arbFeedURL() + fmt.Printf("[arb-feed] connecting %s\n", url) + backoff := arbMinBackoff + for ctx.Err() == nil { + connStart := time.Now() + err := readArbFeed(ctx, url, state) + if time.Since(connStart) > 5*time.Minute { + backoff = arbMinBackoff // stable session: don't carry stale backoff + } + arbFeedHealth.Set(0) + streamReconnects.WithLabelValues("arb-feed").Inc() + if err != nil { + fmt.Printf("[arb-feed] error: %v (reconnecting in %v)\n", err, backoff) + } + select { + case <-ctx.Done(): + return + case <-time.After(backoff): + } + if backoff < arbMaxBackoff { + backoff *= 2 + if backoff > arbMaxBackoff { + backoff = arbMaxBackoff + } + } + } +} + +func readArbFeed(ctx context.Context, url string, state *arbFeedState) error { + dialer := *websocket.DefaultDialer + dialer.HandshakeTimeout = arbDialTimeout + hdr := http.Header{"User-Agent": []string{harnessUserAgent}} + conn, _, err := dialer.Dial(url, hdr) + if err != nil { + return fmt.Errorf("dial: %w", err) + } + defer conn.Close() + arbFeedHealth.Set(1) + fmt.Println("[arb-feed] connected") + + // Silence watchdog: Arbitrum produces ~4 blocks/s, so 30s without a + // frame is unambiguously a wedged subscription. Atomic keeps the + // watchdog lock-free (same pattern as l2-block-time's head watchdog). + done := make(chan struct{}) + defer close(done) + var lastMsgUnixNano atomic.Int64 + lastMsgUnixNano.Store(time.Now().UnixNano()) + go func() { + t := time.NewTicker(10 * time.Second) + defer t.Stop() + for { + select { + case <-done: + return + case <-ctx.Done(): + _ = conn.Close() + return + case now := <-t.C: + silent := now.Sub(time.Unix(0, lastMsgUnixNano.Load())) + if silent > arbWatchdog { + fmt.Printf("[arb-feed] watchdog: silent %s, forcing reconnect\n", silent.Round(time.Second)) + _ = conn.Close() + return + } + } + } + }() + + for { + _ = conn.SetReadDeadline(time.Now().Add(arbReadDeadline)) + _, raw, err := conn.ReadMessage() + if err != nil { + return fmt.Errorf("read: %w", err) + } + now := time.Now() + lastMsgUnixNano.Store(now.UnixNano()) + + var msg arbFeedMsg + if err := json.Unmarshal(raw, &msg); err != nil { + continue // keepalive / non-broadcast frame + } + for _, m := range msg.Messages { + if m.SequenceNumber > 0 { + state.record(m.SequenceNumber, now) + arbFeedMessages.Inc() + } + } + } +} + +// runArbHeadPoller polls the Arbitrum RPC head, derives the +// seq→blockNumber offset during calibration, then emits the feed→RPC +// soft-confirmation lag for every head advance. +func runArbHeadPoller(ctx context.Context, state *arbFeedState) { + url := arbRPCURL() + fmt.Printf("[arb-rpc] head poll: %s every %s\n", url, l2HeadPollInterval) + + var ( + offset int64 + warmup int + candidates = map[int64]int{} + calibCount int + calibrated bool + prevHead int64 + ) + t := time.NewTicker(l2HeadPollInterval) + defer t.Stop() + for { + select { + case <-ctx.Done(): + return + case <-t.C: + } + + head, err := blockNumber(url) + if err != nil { + arbRPCErrors.Inc() + continue + } + pollAt := time.Now() + latestSeq := state.latest() + if latestSeq == 0 { + continue // feed not live yet + } + + if !calibrated { + // Discard the first polls after the feed goes live: the connect + // backlog replay makes latestSeq briefly trail the chain, which + // would overshoot the offset (and inflate every lag sample). + if warmup < arbCalibWarmup { + warmup++ + continue + } + candidates[head-latestSeq]++ + calibCount++ + if calibCount >= arbCalibSamples { + // Highest candidate with real support (>= 20% of samples). + // Live-feed candidates cluster on trueOffset and + // trueOffset-1 (0-1 blocks in flight); transient overshoots + // have negligible counts and are rejected here. + minSupport := calibCount / 5 + for cand, n := range candidates { + if n >= minSupport && cand > offset { + offset = cand + } + } + if offset <= 0 { + // No candidate reached support (feed stalled mid-window, + // so cand drifted and the counts spread out). Locking + // offset=0 would silence the lag histogram forever; + // discard the window and re-calibrate instead. + fmt.Printf("[arb-rpc] calibration failed (no candidate with >=%d support over %d samples), retrying\n", minSupport, calibCount) + candidates = map[int64]int{} + calibCount = 0 + warmup = 0 + continue + } + calibrated = true + prevHead = head + arbDerivedOffset.Set(float64(offset)) + fmt.Printf("[arb-rpc] calibrated seq→block offset=%d over %d samples (expected ≈22207817)\n", offset, calibCount) + } + continue + } + + if head <= prevHead { + continue + } + for b := prevHead + 1; b <= head; b++ { + arrival, ok := state.lookup(b - offset) + if !ok { + continue + } + lagMs := float64(pollAt.Sub(arrival).Milliseconds()) + if lagMs >= 0 && lagMs < arbMaxLagSeconds*1000 { + arbSoftConfLag.Observe(lagMs) + arbLagSamples.Inc() + fmt.Printf("[arb] block=%d softconf lag=%.0fms\n", b, lagMs) + } + } + prevHead = head + } +} diff --git a/harnesses/evm-block-builders/cmd/script/base_flashblocks.go b/harnesses/evm-block-builders/cmd/script/base_flashblocks.go new file mode 100644 index 00000000..c4b37f74 --- /dev/null +++ b/harnesses/evm-block-builders/cmd/script/base_flashblocks.go @@ -0,0 +1,261 @@ +package main + +import ( + "bytes" + "context" + "encoding/json" + "fmt" + "io" + "net/http" + "sync" + "sync/atomic" + "time" + + "github.com/andybalholm/brotli" + "github.com/gorilla/websocket" +) + +// Base flashblocks (preconfirmation) stream. +// +// wss://mainnet.flashblocks.base.org/ws pushes one frame per flashblock +// (~every 200-250ms; ~10 flashblocks per 2s Base block). Frames are +// binary, brotli-compressed JSON of the shape: +// +// {"payload_id":"0x..","index":3,"base":{...},"diff":{...}, +// "metadata":{"block_number":31234567,"receipts":{...},...}} +// +// Two measurements: +// 1. inter-frame cadence (ebb_base_flashblock_interval_milliseconds): +// is the advertised 200ms preconfirmation rhythm actually held? +// 2. soft-confirmation lag (ebb_base_softconf_lag_milliseconds): time +// from the FIRST flashblock of block N arriving on the stream to +// block N being visible on the public RPC head. This is the head +// start a flashblock consumer gets over an RPC poller. +// +// If a frame fails brotli+JSON parsing we still record cadence (the +// frame arrived) and count a parse failure; cadence-only degradation is +// acceptable and disclosed. + +const ( + baseDialTimeout = 15 * time.Second + baseReadDeadline = 60 * time.Second + baseWatchdog = 20 * time.Second + baseMinBackoff = 2 * time.Second + baseMaxBackoff = 60 * time.Second + baseMaxLagMs = 30 * 1000 + // A reconnect gap must not pollute the cadence histogram. + baseMaxIntervalMs = 10 * 1000 + baseArrivalsMax = 200 +) + +type baseFeedState struct { + mu sync.Mutex + arrivals map[int64]time.Time // block number → first flashblock arrival + order []int64 +} + +func newBaseFeedState() *baseFeedState { + return &baseFeedState{arrivals: make(map[int64]time.Time, baseArrivalsMax)} +} + +func (s *baseFeedState) recordFirst(block int64, at time.Time) bool { + s.mu.Lock() + defer s.mu.Unlock() + if _, dup := s.arrivals[block]; dup { + return false + } + s.arrivals[block] = at + s.order = append(s.order, block) + for len(s.order) > baseArrivalsMax { + delete(s.arrivals, s.order[0]) + s.order = s.order[1:] + } + return true +} + +func (s *baseFeedState) lookup(block int64) (time.Time, bool) { + s.mu.Lock() + defer s.mu.Unlock() + t, ok := s.arrivals[block] + return t, ok +} + +type flashblockFrame struct { + Index *int64 `json:"index"` + Metadata struct { + BlockNumber int64 `json:"block_number"` + } `json:"metadata"` +} + +// runBaseFlashblocks holds the flashblocks WS open with reconnect +// backoff + silence watchdog and feeds cadence + arrival state. +func runBaseFlashblocks(ctx context.Context, state *baseFeedState) { + url := baseFlashblocksURL() + fmt.Printf("[base-fb] connecting %s\n", url) + backoff := baseMinBackoff + for ctx.Err() == nil { + connStart := time.Now() + err := readBaseFlashblocks(ctx, url, state) + if time.Since(connStart) > 5*time.Minute { + backoff = baseMinBackoff // stable session: don't carry stale backoff + } + baseFeedHealth.Set(0) + streamReconnects.WithLabelValues("base-flashblocks").Inc() + if err != nil { + fmt.Printf("[base-fb] error: %v (reconnecting in %v)\n", err, backoff) + } + select { + case <-ctx.Done(): + return + case <-time.After(backoff): + } + if backoff < baseMaxBackoff { + backoff *= 2 + if backoff > baseMaxBackoff { + backoff = baseMaxBackoff + } + } + } +} + +func readBaseFlashblocks(ctx context.Context, url string, state *baseFeedState) error { + dialer := *websocket.DefaultDialer + dialer.HandshakeTimeout = baseDialTimeout + hdr := http.Header{"User-Agent": []string{harnessUserAgent}} + conn, _, err := dialer.Dial(url, hdr) + if err != nil { + return fmt.Errorf("dial: %w", err) + } + defer conn.Close() + baseFeedHealth.Set(1) + fmt.Println("[base-fb] connected") + + done := make(chan struct{}) + defer close(done) + var lastFrameUnixNano atomic.Int64 + lastFrameUnixNano.Store(time.Now().UnixNano()) + go func() { + t := time.NewTicker(5 * time.Second) + defer t.Stop() + for { + select { + case <-done: + return + case <-ctx.Done(): + _ = conn.Close() + return + case now := <-t.C: + silent := now.Sub(time.Unix(0, lastFrameUnixNano.Load())) + if silent > baseWatchdog { + fmt.Printf("[base-fb] watchdog: silent %s, forcing reconnect\n", silent.Round(time.Second)) + _ = conn.Close() + return + } + } + } + }() + + var lastFrame time.Time + logEvery := 0 + for { + _ = conn.SetReadDeadline(time.Now().Add(baseReadDeadline)) + _, raw, err := conn.ReadMessage() + if err != nil { + return fmt.Errorf("read: %w", err) + } + now := time.Now() + lastFrameUnixNano.Store(now.UnixNano()) + + // Cadence: frame arrival rhythm, parse success or not. + if !lastFrame.IsZero() { + ivMs := float64(now.Sub(lastFrame).Milliseconds()) + if ivMs >= 0 && ivMs < baseMaxIntervalMs { + baseFlashblockInterval.Observe(ivMs) + } + } + lastFrame = now + baseFrames.Inc() + + frame, err := decodeFlashblockFrame(raw) + if err != nil { + baseParseFailures.Inc() + continue + } + if frame.Metadata.BlockNumber > 0 { + if state.recordFirst(frame.Metadata.BlockNumber, now) { + logEvery++ + if logEvery%10 == 1 { // ~1 log line per 20s, not per 200ms frame + idx := int64(-1) + if frame.Index != nil { + idx = *frame.Index + } + fmt.Printf("[base-fb] first flashblock block=%d index=%d\n", frame.Metadata.BlockNumber, idx) + } + } + } + } +} + +// decodeFlashblockFrame handles both brotli-compressed and plain-JSON +// frames (the stream compressed at inception; tolerate both so a +// server-side change doesn't blind the bench). +func decodeFlashblockFrame(raw []byte) (*flashblockFrame, error) { + var frame flashblockFrame + if len(raw) > 0 && (raw[0] == '{' || raw[0] == '[') { + if err := json.Unmarshal(raw, &frame); err == nil { + return &frame, nil + } + } + dec, err := io.ReadAll(io.LimitReader(brotli.NewReader(bytes.NewReader(raw)), 16<<20)) + if err != nil { + return nil, fmt.Errorf("brotli: %w", err) + } + if err := json.Unmarshal(dec, &frame); err != nil { + return nil, fmt.Errorf("json: %w", err) + } + return &frame, nil +} + +// runBaseHeadPoller polls the Base RPC head and emits the +// flashblock→canonical-RPC soft-confirmation lag. +func runBaseHeadPoller(ctx context.Context, state *baseFeedState) { + url := baseRPCURL() + fmt.Printf("[base-rpc] head poll: %s every %s\n", url, l2HeadPollInterval) + + var prevHead int64 + t := time.NewTicker(l2HeadPollInterval) + defer t.Stop() + for { + select { + case <-ctx.Done(): + return + case <-t.C: + } + + head, err := blockNumber(url) + if err != nil { + baseRPCErrors.Inc() + continue + } + pollAt := time.Now() + if prevHead == 0 { + prevHead = head + continue + } + for b := prevHead + 1; b <= head; b++ { + arrival, ok := state.lookup(b) + if !ok { + continue + } + lagMs := float64(pollAt.Sub(arrival).Milliseconds()) + if lagMs >= 0 && lagMs < baseMaxLagMs { + baseSoftConfLag.Observe(lagMs) + baseLagSamples.Inc() + fmt.Printf("[base] block=%d softconf lag=%.0fms\n", b, lagMs) + } + } + if head > prevHead { + prevHead = head + } + } +} diff --git a/harnesses/evm-block-builders/cmd/script/builders.go b/harnesses/evm-block-builders/cmd/script/builders.go new file mode 100644 index 00000000..0156ee66 --- /dev/null +++ b/harnesses/evm-block-builders/cmd/script/builders.go @@ -0,0 +1,192 @@ +package main + +import ( + "context" + "encoding/hex" + "fmt" + "strings" + "time" +) + +// Ethereum builder attribution via block extraData. +// +// Same philosophy as bench 016 (solana-tx-landing): a curated table of +// known-entity tags, plus explicit logging of every string we could NOT +// attribute so the table grows from operator logs instead of silently +// under-counting. Builders self-label their blocks in extraData +// ("Titan (titanbuilder.xyz)", "beaverbuild.org", ...); vanilla +// (locally-built) blocks carry the execution client's default tag +// (geth/reth/besu/nethermind version strings) or are empty. +// +// extraData is self-reported: a builder could change or strip its tag +// at any time. The methodology in the spec discloses this; the relay +// bidtrace cross-check (relays.go) provides an independent signal. + +// builderRule maps a lowercase substring of the decoded extraData to a +// canonical builder slug. Order matters: first match wins, so put the +// most specific substrings first. +type builderRule struct { + substr string + slug string +} + +// builderTable is the curated attribution table. Slugs are the Prom +// label values the spec YAML queries. Verified against live mainnet +// blocks at bench inception (50-block sample: Titan 19, Quasar 16, +// Eureka 6, BuilderNet 2, beaverbuild 1, Builder+ 1, vanilla 1). +var builderTable = []builderRule{ + // Specific multi-word tags first. + {"bob-the-builder", "bobthebuilder"}, + {"bobthebuilder", "bobthebuilder"}, + {"builder0x69", "builder0x69"}, + {"penguinbuild", "penguinbuild"}, + {"beaverbuild", "beaverbuild"}, + {"buildernet", "buildernet"}, + {"builder+", "btcs"}, // BTCS "Builder+" + {"btcs", "btcs"}, + {"titan", "titan"}, + {"quasar", "quasar"}, + {"eureka", "eureka"}, + {"rsync", "rsync"}, + // Flashbots historically tags "Illuminate Dmocratize Dstribute". + {"flashbots", "flashbots"}, + {"illuminate", "flashbots"}, + {"dmocratize", "flashbots"}, + {"bloxroute", "bloxroute"}, + {"gambit", "gambit"}, + {"antbuilder", "antbuilder"}, + {"manifold", "manifold"}, + {"blockbeelder", "blockbeelder"}, + {"jetbldr", "jetbuilder"}, + {"payload", "payload"}, + {"boba-builder", "boba"}, +} + +// vanillaTags identify execution-client default extraData: the proposer +// built the block locally instead of outsourcing to a builder. These +// count as "vanilla", NOT as unattributed - a locally-built block is a +// positive decentralization signal, not a table gap. +var vanillaTags = []string{ + "geth", "go1.", "nethermind", "besu", "reth", "erigon", "linux", + "darwin", "windows", "ubuntu", +} + +// slugOther collects blocks whose extraData carries a tag we don't +// recognize. Every occurrence is logged with the raw string so the +// table can be grown (016 pattern). +const ( + slugOther = "other" + slugVanilla = "vanilla" +) + +// decodeExtraData turns the hex extraData field into a printable string +// for substring matching. Non-printable bytes (client tags like geth +// pack RLP fragments around the ASCII) are dropped. +func decodeExtraData(hexStr string) string { + raw, err := hex.DecodeString(strings.TrimPrefix(hexStr, "0x")) + if err != nil { + return "" + } + var b strings.Builder + for _, c := range raw { + if c >= 0x20 && c <= 0x7e { + b.WriteByte(c) + } + } + return b.String() +} + +// attributeBuilder maps decoded extraData to a builder slug. +// Returns (slug, attributed). attributed=false means the tag was +// non-empty and unrecognized (goes to "other" + unattributed counter). +func attributeBuilder(decoded string) (string, bool) { + lower := strings.ToLower(decoded) + if strings.TrimSpace(lower) == "" { + return slugVanilla, true + } + for _, r := range builderTable { + if strings.Contains(lower, r.substr) { + return r.slug, true + } + } + for _, t := range vanillaTags { + if strings.Contains(lower, t) { + return slugVanilla, true + } + } + return slugOther, false +} + +type ethBlock struct { + Number string `json:"number"` + ExtraData string `json:"extraData"` +} + +// runBuilderPoll polls the ETH head every slot (12s), attributes each +// new block's extraData and backfills small gaps by number so the +// market-share counters see every block, not just the ones that happen +// to be head at poll time. +func runBuilderPoll(ctx context.Context) { + url := ethRPCURL() + fmt.Printf("[eth] builder attribution poll: %s every %s\n", url, ethPollInterval) + + var lastSeen int64 + t := time.NewTicker(ethPollInterval) + defer t.Stop() + for { + var head ethBlock + err := jsonRPCCall(url, "eth_getBlockByNumber", []any{"latest", false}, &head) + if err != nil { + ethPollHealth.Set(0) + ethPollErrors.Inc() + fmt.Printf("[eth] poll error: %v\n", err) + } else { + ethPollHealth.Set(1) + headNum := parseHexInt64(head.Number) + if headNum > lastSeen { + // Backfill missed blocks (cap 10: a longer gap means we were + // down; counting a burst of stale blocks then would skew the + // 5m rate series without changing 24h share meaningfully). + start := headNum + if lastSeen > 0 { + start = lastSeen + 1 + if headNum-lastSeen > 10 { + start = headNum - 10 + 1 + } + } + for n := start; n < headNum; n++ { + var blk ethBlock + if err := jsonRPCCall(url, "eth_getBlockByNumber", + []any{fmt.Sprintf("0x%x", n), false}, &blk); err != nil { + fmt.Printf("[eth] backfill %d error: %v\n", n, err) + continue + } + recordBlock(n, blk.ExtraData) + } + recordBlock(headNum, head.ExtraData) + lastSeen = headNum + ethLastBlock.Set(float64(headNum)) + } + } + + select { + case <-ctx.Done(): + return + case <-t.C: + } + } +} + +func recordBlock(num int64, extraHex string) { + decoded := decodeExtraData(extraHex) + slug, attributed := attributeBuilder(decoded) + blocksTotal.WithLabelValues(slug).Inc() + if !attributed { + unattributedTotal.Inc() + // The whole point of the log line: grow the table from operator + // logs, exactly like 016 logs unattributable tip patterns. + fmt.Printf("[eth] UNATTRIBUTED block=%d extraData=%q raw=%s\n", num, decoded, extraHex) + } else { + fmt.Printf("[eth] block=%d builder=%s extraData=%q\n", num, slug, decoded) + } +} diff --git a/harnesses/evm-block-builders/cmd/script/config.go b/harnesses/evm-block-builders/cmd/script/config.go new file mode 100644 index 00000000..d5e08c65 --- /dev/null +++ b/harnesses/evm-block-builders/cmd/script/config.go @@ -0,0 +1,153 @@ +package main + +import ( + "bytes" + "encoding/json" + "fmt" + "io" + "net/http" + "os" + "strconv" + "strings" + "time" +) + +// Endpoints. All public, keyless, verified live at bench inception +// (2026-07-14). Every URL is env-overridable without a rebuild so a +// flaky public endpoint can be swapped from the deploy env. +// +// publicnode 403s default Go/Python User-Agents (bench 067 lesson), so +// EVERY outbound HTTP request and WS dial in this harness sends +// harnessUserAgent - see newRequest and the WS dial headers in +// arb_feed.go / base_flashblocks.go. + +const ( + defaultEthRPC = "https://ethereum-rpc.publicnode.com" + defaultArbRPC = "https://arbitrum-one-rpc.publicnode.com" + defaultBaseRPC = "https://base-rpc.publicnode.com" + defaultArbFeedURL = "wss://arb1.arbitrum.io/feed" + defaultBaseFlashblocks = "wss://mainnet.flashblocks.base.org/ws" + + // Browser-prefixed but honest: identifies the harness while passing + // UA-based bot filters (publicnode 403s bare Go-http-client UAs). + harnessUserAgent = "Mozilla/5.0 (compatible; OpenChainBench-harness/1.0; +https://openchainbench.com)" + + // ETH mainnet slot time. One poll per slot samples ~every block; the + // gap-backfill in builders.go covers the occasional missed one. + ethPollInterval = 12 * time.Second + + // Relay bidtrace APIs are rate-limited and slow-moving (1 payload per + // 12s slot at most); 5 min polls with limit=50 never miss a slot. + relayPollInterval = 5 * time.Minute + + // L2 head pollers. 300ms against publicnode is comfortably inside the + // public rate limits (verified live) and fine-grained enough to + // resolve a ~100ms soft-conf lag. + l2HeadPollInterval = 300 * time.Millisecond +) + +func envDefault(key, def string) string { + if v := strings.TrimSpace(os.Getenv(key)); v != "" { + return v + } + return def +} + +func ethRPCURL() string { return envDefault("ETH_RPC_URL", defaultEthRPC) } +func arbRPCURL() string { return envDefault("ARB_RPC_URL", defaultArbRPC) } +func baseRPCURL() string { return envDefault("BASE_RPC_URL", defaultBaseRPC) } +func arbFeedURL() string { + return envDefault("ARB_FEED_URL", defaultArbFeedURL) +} +func baseFlashblocksURL() string { + return envDefault("BASE_FLASHBLOCKS_URL", defaultBaseFlashblocks) +} + +// listenAddr defaults to :2112 - OCB convention, scraped by the shared +// Prometheus. We deliberately ignore $PORT so a platform-injected public +// port doesn't move the listener away from the address Prom expects. +// METRICS_ADDR exists for local dev where another harness holds 2112. +func listenAddr() string { return envDefault("METRICS_ADDR", ":2112") } + +var httpClient = &http.Client{Timeout: 15 * time.Second} + +// newRequest builds an *http.Request with the harness User-Agent set. +func newRequest(method, url string, body []byte) (*http.Request, error) { + var rdr io.Reader + if body != nil { + rdr = bytes.NewReader(body) + } + req, err := http.NewRequest(method, url, rdr) + if err != nil { + return nil, err + } + req.Header.Set("User-Agent", harnessUserAgent) + req.Header.Set("Accept", "application/json") + if body != nil { + req.Header.Set("Content-Type", "application/json") + } + return req, nil +} + +// jsonRPCCall does a single JSON-RPC 2.0 request and unmarshals the +// `result` field into out. +func jsonRPCCall(url, method string, params []any, out any) error { + payload, err := json.Marshal(map[string]any{ + "jsonrpc": "2.0", "id": 1, "method": method, "params": params, + }) + if err != nil { + return err + } + req, err := newRequest(http.MethodPost, url, payload) + if err != nil { + return err + } + resp, err := httpClient.Do(req) + if err != nil { + return err + } + defer resp.Body.Close() + if resp.StatusCode != http.StatusOK { + body, _ := io.ReadAll(io.LimitReader(resp.Body, 200)) + return fmt.Errorf("%s: HTTP %d: %s", method, resp.StatusCode, string(body)) + } + var envl struct { + Result json.RawMessage `json:"result"` + Error *struct { + Code int `json:"code"` + Message string `json:"message"` + } `json:"error"` + } + if err := json.NewDecoder(resp.Body).Decode(&envl); err != nil { + return fmt.Errorf("%s: decode: %w", method, err) + } + if envl.Error != nil { + return fmt.Errorf("%s: rpc error %d: %s", method, envl.Error.Code, envl.Error.Message) + } + if envl.Result == nil { + return fmt.Errorf("%s: empty result", method) + } + return json.Unmarshal(envl.Result, out) +} + +// blockNumber fetches the current head number via eth_blockNumber. +func blockNumber(url string) (int64, error) { + var hexNum string + if err := jsonRPCCall(url, "eth_blockNumber", []any{}, &hexNum); err != nil { + return 0, err + } + n := parseHexInt64(hexNum) + if n == 0 { + return 0, fmt.Errorf("eth_blockNumber: unparseable %q", hexNum) + } + return n, nil +} + +func parseHexInt64(s string) int64 { + s = strings.TrimPrefix(s, "0x") + if s == "" { + return 0 + } + n, _ := strconv.ParseInt(s, 16, 64) + return n +} diff --git a/harnesses/evm-block-builders/cmd/script/loghub.go b/harnesses/evm-block-builders/cmd/script/loghub.go new file mode 100644 index 00000000..9dd74488 --- /dev/null +++ b/harnesses/evm-block-builders/cmd/script/loghub.go @@ -0,0 +1,114 @@ +package main + +import ( + "bufio" + "fmt" + "io" + "net/http" + "os" + "strconv" + "sync" + "time" +) + +// Auto-generated by the loghub inline pattern. Captures stdout/stderr into a +// bounded ring buffer and exposes GET /logs?tail=N protected by X-Logs-Token +// matching the LOGS_TOKEN env var. +// +// Keep in sync across miniapps (was previously the shared/loghub package; we +// inline because Railway's per-harness Docker build context can't reach a +// sibling shared module via go.mod replace). + +const logRingMax = 5000 + +type logRing struct { + mu sync.Mutex + lines []string + max int +} + +var globalLogRing = &logRing{max: logRingMax} + +func (b *logRing) push(line string) { + entry := time.Now().UTC().Format("2006-01-02T15:04:05.000Z") + " " + line + b.mu.Lock() + if len(b.lines) >= b.max { + b.lines = append(b.lines[1:], entry) + } else { + b.lines = append(b.lines, entry) + } + b.mu.Unlock() +} + +func (b *logRing) snapshot(tail int) []string { + b.mu.Lock() + defer b.mu.Unlock() + if tail <= 0 || tail >= len(b.lines) { + out := make([]string, len(b.lines)) + copy(out, b.lines) + return out + } + start := len(b.lines) - tail + out := make([]string, tail) + copy(out, b.lines[start:]) + return out +} + +var logSetupOnce sync.Once + +// installLogCapture replaces os.Stdout (and os.Stderr) with the write-end of a +// pipe, then spawns a goroutine that fan-outs every line to the original +// stdout AND the in-memory ring buffer. Call exactly once, very early in +// main(). +func installLogCapture() { logSetupOnce.Do(doInstallLogCapture) } + +func doInstallLogCapture() { + originalStdout := os.Stdout + originalStderr := os.Stderr + r, w, err := os.Pipe() + if err != nil { + fmt.Fprintf(originalStdout, "[loghub] pipe failed: %v (/logs will be empty)\n", err) + return + } + os.Stdout = w + os.Stderr = w + + go func() { + scanner := bufio.NewScanner(r) + buf := make([]byte, 0, 1024*1024) + scanner.Buffer(buf, 1024*1024) + for scanner.Scan() { + line := scanner.Text() + fmt.Fprintln(originalStdout, line) + globalLogRing.push(line) + } + _, _ = io.Copy(originalStdout, r) + _ = originalStderr + }() +} + +// logsHandler returns an http.Handler for GET /logs?tail=N. Requires header +// X-Logs-Token to match the LOGS_TOKEN env var. Returns 403 if env unset. +func logsHandler() http.Handler { + return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + expected := os.Getenv("LOGS_TOKEN") + if expected == "" { + http.Error(w, "logs disabled: LOGS_TOKEN unset", http.StatusForbidden) + return + } + if r.Header.Get("X-Logs-Token") != expected { + http.Error(w, "forbidden", http.StatusForbidden) + return + } + tail := 500 + if t := r.URL.Query().Get("tail"); t != "" { + if n, err := strconv.Atoi(t); err == nil && n > 0 { + tail = n + } + } + w.Header().Set("Content-Type", "text/plain; charset=utf-8") + for _, l := range globalLogRing.snapshot(tail) { + fmt.Fprintln(w, l) + } + }) +} diff --git a/harnesses/evm-block-builders/cmd/script/main.go b/harnesses/evm-block-builders/cmd/script/main.go new file mode 100644 index 00000000..244d38ff --- /dev/null +++ b/harnesses/evm-block-builders/cmd/script/main.go @@ -0,0 +1,76 @@ +package main + +import ( + "context" + "fmt" + "os" + "os/signal" + "syscall" + "time" +) + +// Bench 085 "evm-block-builders": who builds Ethereum, plus L2 +// soft-confirmation lag. +// +// Four independent measurement loops share one process: +// 1. builders.go - ETH head poll, extraData → builder attribution +// 2. relays.go - MEV-Boost relay bidtrace pollers (cross-check) +// 3. arb_feed.go - Arbitrum sequencer feed → RPC visibility lag +// 4. base_flashblocks.go - Base flashblock cadence + → RPC lag +// +// Metrics server listens on :2112 - OCB convention; $PORT is ignored +// so Prometheus can scrape the expected address. + +func main() { + installLogCapture() // capture stdout into /logs ring buffer + fmt.Println("=== EVM Block Builders Harness (bench 085) ===") + fmt.Println("OpenChainBench: builder market share via extraData + L2 soft-confirmation lag.") + fmt.Println() + fmt.Printf("ETH RPC: %s (poll %s)\n", ethRPCURL(), ethPollInterval) + fmt.Printf("Relays: %d bidtrace APIs (poll %s)\n", len(relays()), relayPollInterval) + fmt.Printf("Arb feed: %s\n", arbFeedURL()) + fmt.Printf("Arb RPC: %s (poll %s)\n", arbRPCURL(), l2HeadPollInterval) + fmt.Printf("Base flashblocks: %s\n", baseFlashblocksURL()) + fmt.Printf("Base RPC: %s (poll %s)\n", baseRPCURL(), l2HeadPollInterval) + fmt.Println() + fmt.Println("Metrics server: :2112/metrics") + fmt.Println() + + go func() { + if err := StartMetricsServer(listenAddr()); err != nil { + fmt.Printf("[fatal] metrics server: %v\n", err) + // Give the loghub forwarding goroutine a beat to flush the + // fatal line to the real stdout before we exit. + time.Sleep(200 * time.Millisecond) + os.Exit(1) + } + }() + + ctx, cancel := context.WithCancel(context.Background()) + defer cancel() + + // Initialise health gauges to 0; each loop flips its own to 1. + ethPollHealth.Set(0) + arbFeedHealth.Set(0) + baseFeedHealth.Set(0) + for _, r := range relays() { + relayHealth.WithLabelValues(r.Slug).Set(0) + } + + go runBuilderPoll(ctx) + runRelayPolls(ctx) + + arbState := newArbFeedState() + go runArbFeed(ctx, arbState) + go runArbHeadPoller(ctx, arbState) + + baseState := newBaseFeedState() + go runBaseFlashblocks(ctx, baseState) + go runBaseHeadPoller(ctx, baseState) + + sig := make(chan os.Signal, 1) + signal.Notify(sig, os.Interrupt, syscall.SIGTERM) + s := <-sig + fmt.Printf("\n[shutdown] received %v\n", s) + cancel() +} diff --git a/harnesses/evm-block-builders/cmd/script/metrics.go b/harnesses/evm-block-builders/cmd/script/metrics.go new file mode 100644 index 00000000..93cfb14e --- /dev/null +++ b/harnesses/evm-block-builders/cmd/script/metrics.go @@ -0,0 +1,223 @@ +package main + +import ( + "net/http" + + "github.com/prometheus/client_golang/prometheus" + "github.com/prometheus/client_golang/prometheus/promauto" + "github.com/prometheus/client_golang/prometheus/promhttp" +) + +// Prom metrics for bench 085 (evm-block-builders). Prefix `ebb_`. +// The spec YAML (benchmarks/evm-block-builders.yml) queries these +// directly via PromQL. +var ( + /* ---- Ethereum builder attribution (headline) ---- */ + + // Per-builder block counter, attributed via extraData substring + // table. Includes the synthetic slugs "vanilla" (client-default + // extraData → locally built) and "other" (unrecognized tag). + blocksTotal = promauto.NewCounterVec( + prometheus.CounterOpts{ + Name: "ebb_blocks_total", + Help: "Ethereum mainnet blocks attributed to each builder via the block's extraData self-label. builder=vanilla means client-default extraData (locally built); builder=other means an unrecognized tag (also logged raw for table growth).", + }, + []string{"builder"}, + ) + + // Blocks whose extraData carried an unrecognized tag. The raw string + // is logged so the attribution table can grow (016 pattern). + unattributedTotal = promauto.NewCounter( + prometheus.CounterOpts{ + Name: "ebb_unattributed_total", + Help: "Blocks whose extraData tag was non-empty but not in the curated builder table. Every occurrence logs the raw extraData for later table growth.", + }, + ) + + ethPollHealth = promauto.NewGauge( + prometheus.GaugeOpts{ + Name: "ebb_eth_poll_health", + Help: "1 when the Ethereum head poll is succeeding, 0 after the last poll errored.", + }, + ) + + ethPollErrors = promauto.NewCounter( + prometheus.CounterOpts{ + Name: "ebb_eth_poll_errors_total", + Help: "Ethereum RPC poll errors since process start.", + }, + ) + + ethLastBlock = promauto.NewGauge( + prometheus.GaugeOpts{ + Name: "ebb_eth_last_block", + Help: "Most recent Ethereum block number attributed by the harness. Freshness probe target.", + }, + ) + + /* ---- Relay bidtrace cross-check ---- */ + + relayPayloadsTotal = promauto.NewCounterVec( + prometheus.CounterOpts{ + Name: "ebb_relay_payloads_total", + Help: "Delivered payloads (proposer_payload_delivered bidtraces) counted per MEV-Boost relay, deduped per relay via a slot high-water-mark. The same slot on multiple relays is normal (multi-homed bids), so this measures relay share, not a partition of blocks.", + }, + []string{"relay"}, + ) + + relayPollErrors = promauto.NewCounterVec( + prometheus.CounterOpts{ + Name: "ebb_relay_poll_errors_total", + Help: "Bidtrace poll errors per relay since process start.", + }, + []string{"relay"}, + ) + + relayHealth = promauto.NewGaugeVec( + prometheus.GaugeOpts{ + Name: "ebb_relay_health", + Help: "1 when the last bidtrace poll for this relay succeeded, 0 otherwise.", + }, + []string{"relay"}, + ) + + relayLastSlot = promauto.NewGaugeVec( + prometheus.GaugeOpts{ + Name: "ebb_relay_last_slot", + Help: "Highest slot seen in bidtraces per relay.", + }, + []string{"relay"}, + ) + + /* ---- Arbitrum sequencer feed soft-confirmation lag ---- */ + + arbSoftConfLag = promauto.NewHistogram( + prometheus.HistogramOpts{ + Name: "ebb_arb_softconf_lag_milliseconds", + Help: "Lag from a block's sequencer-feed arrival to the same block being visible on the public Arbitrum RPC head. Quantized by the ~300ms RPC poll interval.", + Buckets: []float64{ + 25, 50, 75, 100, 150, 200, 300, 400, 500, 750, + 1000, 1500, 2000, 3000, 5000, 10000, + }, + }, + ) + + arbLagSamples = promauto.NewCounter( + prometheus.CounterOpts{ + Name: "ebb_arb_lag_samples_total", + Help: "Arbitrum soft-confirmation lag samples recorded.", + }, + ) + + arbFeedHealth = promauto.NewGauge( + prometheus.GaugeOpts{ + Name: "ebb_arb_feed_health", + Help: "1 while the Arbitrum sequencer feed WebSocket is connected, 0 otherwise.", + }, + ) + + arbFeedMessages = promauto.NewCounter( + prometheus.CounterOpts{ + Name: "ebb_arb_feed_messages_total", + Help: "Sequencer feed messages received.", + }, + ) + + arbDerivedOffset = promauto.NewGauge( + prometheus.GaugeOpts{ + Name: "ebb_arb_derived_offset", + Help: "Runtime-derived sequenceNumber→blockNumber offset (expected ≈22207817). Drift signals a feed format change.", + }, + ) + + arbRPCErrors = promauto.NewCounter( + prometheus.CounterOpts{ + Name: "ebb_arb_rpc_errors_total", + Help: "Arbitrum RPC head-poll errors since process start.", + }, + ) + + /* ---- Base flashblocks ---- */ + + baseFlashblockInterval = promauto.NewHistogram( + prometheus.HistogramOpts{ + Name: "ebb_base_flashblock_interval_milliseconds", + Help: "Wall-clock interval between consecutive flashblock frames on the Base flashblocks WebSocket (advertised rhythm: ~200ms).", + Buckets: []float64{ + 50, 100, 150, 200, 250, 300, 400, 500, 750, + 1000, 2000, 5000, + }, + }, + ) + + baseSoftConfLag = promauto.NewHistogram( + prometheus.HistogramOpts{ + Name: "ebb_base_softconf_lag_milliseconds", + Help: "Lag from the first flashblock of a Base block arriving on the stream to that block being visible on the public Base RPC head. Quantized by the ~300ms RPC poll interval.", + Buckets: []float64{ + 100, 250, 500, 750, 1000, 1250, 1500, 1750, 2000, + 2500, 3000, 5000, 10000, + }, + }, + ) + + baseLagSamples = promauto.NewCounter( + prometheus.CounterOpts{ + Name: "ebb_base_lag_samples_total", + Help: "Base soft-confirmation lag samples recorded.", + }, + ) + + baseFeedHealth = promauto.NewGauge( + prometheus.GaugeOpts{ + Name: "ebb_base_feed_health", + Help: "1 while the Base flashblocks WebSocket is connected, 0 otherwise.", + }, + ) + + baseFrames = promauto.NewCounter( + prometheus.CounterOpts{ + Name: "ebb_base_frames_total", + Help: "Flashblock frames received.", + }, + ) + + baseParseFailures = promauto.NewCounter( + prometheus.CounterOpts{ + Name: "ebb_base_parse_failures_total", + Help: "Flashblock frames that failed brotli+JSON decoding (cadence still recorded).", + }, + ) + + baseRPCErrors = promauto.NewCounter( + prometheus.CounterOpts{ + Name: "ebb_base_rpc_errors_total", + Help: "Base RPC head-poll errors since process start.", + }, + ) + + /* ---- Shared ---- */ + + streamReconnects = promauto.NewCounterVec( + prometheus.CounterOpts{ + Name: "ebb_stream_reconnects_total", + Help: "WebSocket reconnects per stream (arb-feed, base-flashblocks) since process start.", + }, + []string{"stream"}, + ) +) + +// StartMetricsServer binds /metrics + /health + /logs on addr. Blocking +// call, run in its own goroutine. +func StartMetricsServer(addr string) error { + mux := http.NewServeMux() + mux.Handle("/metrics", promhttp.Handler()) + mux.Handle("/logs", logsHandler()) + mux.HandleFunc("/health", func(w http.ResponseWriter, r *http.Request) { + _, _ = w.Write([]byte("ok")) + }) + mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) { + _, _ = w.Write([]byte("evm-block-builders harness · OpenChainBench")) + }) + return http.ListenAndServe(addr, mux) +} diff --git a/harnesses/evm-block-builders/cmd/script/relays.go b/harnesses/evm-block-builders/cmd/script/relays.go new file mode 100644 index 00000000..91c7f922 --- /dev/null +++ b/harnesses/evm-block-builders/cmd/script/relays.go @@ -0,0 +1,145 @@ +package main + +import ( + "context" + "encoding/json" + "fmt" + "io" + "net/http" + "sort" + "strconv" + "time" +) + +// MEV-Boost relay bidtrace pollers. +// +// Every major relay exposes the standard Relay Data API keyless: +// GET /relay/v1/data/bidtraces/proposer_payload_delivered?limit=N +// One entry per delivered payload (= one relayed block). We poll each +// relay every 5 min and count NEW slots per relay via a high-water-mark +// so restarts / overlapping windows never double-count within a relay. +// +// The same slot appearing on multiple relays is NORMAL (builders +// multi-home their bids; the winning payload is reported by every relay +// that carried it), so relay counters are a market-share signal for +// relays, not a partition of blocks. This is the independent +// cross-check on the extraData attribution: relay-side builder pubkeys +// don't depend on the builder's self-chosen extraData tag. + +type relayEndpoint struct { + Slug string + Base string +} + +// relays verified answering keyless at bench inception (2026-07-14). +func relays() []relayEndpoint { + return []relayEndpoint{ + {"flashbots", "https://boost-relay.flashbots.net"}, + {"ultrasound", "https://relay.ultrasound.money"}, + {"agnostic", "https://agnostic-relay.net"}, + {"bloxroute-max-profit", "https://bloxroute.max-profit.blxrbdn.com"}, + {"bloxroute-regulated", "https://bloxroute.regulated.blxrbdn.com"}, + {"titan", "https://titanrelay.xyz"}, + {"aestus", "https://aestus.live"}, + } +} + +type bidTrace struct { + Slot string `json:"slot"` +} + +// runRelayPolls launches one poller goroutine per relay. +func runRelayPolls(ctx context.Context) { + for _, r := range relays() { + r := r + go pollRelay(ctx, r) + } +} + +func pollRelay(ctx context.Context, r relayEndpoint) { + url := r.Base + "/relay/v1/data/bidtraces/proposer_payload_delivered?limit=50" + fmt.Printf("[relay:%s] polling %s every %s\n", r.Slug, url, relayPollInterval) + + var maxSeenSlot int64 + primed := false // first successful fetch seeds the high-water-mark only + t := time.NewTicker(relayPollInterval) + defer t.Stop() + for { + slots, err := fetchDeliveredSlots(url) + if err != nil { + relayPollErrors.WithLabelValues(r.Slug).Inc() + relayHealth.WithLabelValues(r.Slug).Set(0) + fmt.Printf("[relay:%s] poll error: %v\n", r.Slug, err) + } else { + relayHealth.WithLabelValues(r.Slug).Set(1) + fresh := 0 + for _, s := range slots { + if s > maxSeenSlot { + fresh++ + } + } + if len(slots) > 0 { + sort.Slice(slots, func(i, j int) bool { return slots[i] > slots[j] }) + if slots[0] > maxSeenSlot { + maxSeenSlot = slots[0] + } + } + if !primed { + // First fetch after (re)start returns up to 50 historical + // payloads that were already counted before the restart. + // Seed the high-water-mark from them WITHOUT incrementing: + // Prom increase() tolerates counter resets, but re-counting + // old slots would inflate the series after every restart. + primed = true + fmt.Printf("[relay:%s] primed high-water-mark maxSlot=%d (skipped %d historical)\n", r.Slug, maxSeenSlot, fresh) + } else { + relayPayloadsTotal.WithLabelValues(r.Slug).Add(float64(fresh)) + fmt.Printf("[relay:%s] delivered=%d fresh=%d maxSlot=%d\n", r.Slug, len(slots), fresh, maxSeenSlot) + } + relayLastSlot.WithLabelValues(r.Slug).Set(float64(maxSeenSlot)) + } + + select { + case <-ctx.Done(): + return + case <-t.C: + } + } +} + +// fetchDeliveredSlots GETs the bidtraces endpoint and returns the slot +// numbers, deduped within the response (defensive: the same slot can +// appear twice on a relay after a reorged proposal). +func fetchDeliveredSlots(url string) ([]int64, error) { + req, err := newRequest(http.MethodGet, url, nil) + if err != nil { + return nil, err + } + resp, err := httpClient.Do(req) + if err != nil { + return nil, err + } + defer resp.Body.Close() + if resp.StatusCode != http.StatusOK { + body, _ := io.ReadAll(io.LimitReader(resp.Body, 200)) + return nil, fmt.Errorf("HTTP %d: %s", resp.StatusCode, string(body)) + } + var traces []bidTrace + if err := json.NewDecoder(resp.Body).Decode(&traces); err != nil { + return nil, fmt.Errorf("decode: %w", err) + } + seen := make(map[int64]struct{}, len(traces)) + out := make([]int64, 0, len(traces)) + for _, tr := range traces { + n, err := strconv.ParseInt(tr.Slot, 10, 64) + if err != nil || n == 0 { + continue + } + if _, dup := seen[n]; dup { + continue + } + seen[n] = struct{}{} + out = append(out, n) + } + return out, nil +} diff --git a/harnesses/evm-block-builders/go.mod b/harnesses/evm-block-builders/go.mod new file mode 100644 index 00000000..6b4d23fc --- /dev/null +++ b/harnesses/evm-block-builders/go.mod @@ -0,0 +1,22 @@ +module evm-block-builders + +go 1.24.0 + +require ( + github.com/andybalholm/brotli v1.1.1 + github.com/gorilla/websocket v1.5.3 + github.com/prometheus/client_golang v1.23.2 +) + +require ( + github.com/beorn7/perks v1.0.1 // indirect + github.com/cespare/xxhash/v2 v2.3.0 // indirect + github.com/kr/text v0.2.0 // indirect + github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect + github.com/prometheus/client_model v0.6.2 // indirect + github.com/prometheus/common v0.66.1 // indirect + github.com/prometheus/procfs v0.16.1 // indirect + go.yaml.in/yaml/v2 v2.4.2 // indirect + golang.org/x/sys v0.35.0 // indirect + google.golang.org/protobuf v1.36.8 // indirect +) diff --git a/harnesses/evm-block-builders/go.sum b/harnesses/evm-block-builders/go.sum new file mode 100644 index 00000000..91fbc2f0 --- /dev/null +++ b/harnesses/evm-block-builders/go.sum @@ -0,0 +1,52 @@ +github.com/andybalholm/brotli v1.1.1 h1:PR2pgnyFznKEugtsUo0xLdDop5SKXd5Qf5ysW+7XdTA= +github.com/andybalholm/brotli v1.1.1/go.mod h1:05ib4cKhjx3OQYUY22hTVd34Bc8upXjOLL2rKwwZBoA= +github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= +github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= +github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs= +github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= +github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= +github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= +github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= +github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU= +github.com/gorilla/websocket v1.5.3 h1:saDtZ6Pbx/0u+bgYQ3q96pZgCzfhKXGPqt7kZ72aNNg= +github.com/gorilla/websocket v1.5.3/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE= +github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo= +github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ= +github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= +github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= +github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= +github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= +github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc= +github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw= +github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA= +github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ= +github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= +github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/prometheus/client_golang v1.23.2 h1:Je96obch5RDVy3FDMndoUsjAhG5Edi49h0RJWRi/o0o= +github.com/prometheus/client_golang v1.23.2/go.mod h1:Tb1a6LWHB3/SPIzCoaDXI4I8UHKeFTEQ1YCr+0Gyqmg= +github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk= +github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE= +github.com/prometheus/common v0.66.1 h1:h5E0h5/Y8niHc5DlaLlWLArTQI7tMrsfQjHV+d9ZoGs= +github.com/prometheus/common v0.66.1/go.mod h1:gcaUsgf3KfRSwHY4dIMXLPV0K/Wg1oZ8+SbZk/HH/dA= +github.com/prometheus/procfs v0.16.1 h1:hZ15bTNuirocR6u0JZ6BAHHmwS1p8B4P6MRqxtzMyRg= +github.com/prometheus/procfs v0.16.1/go.mod h1:teAbpZRB1iIAJYREa1LsoWUXykVXA1KlTmWl8x/U+Is= +github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjRBZyWFQ= +github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog= +github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U= +github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U= +github.com/xyproto/randomstring v1.0.5 h1:YtlWPoRdgMu3NZtP45drfy1GKoojuR7hmRcnhZqKjWU= +github.com/xyproto/randomstring v1.0.5/go.mod h1:rgmS5DeNXLivK7YprL0pY+lTuhNQW3iGxZ18UQApw/E= +go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto= +go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE= +go.yaml.in/yaml/v2 v2.4.2 h1:DzmwEr2rDGHl7lsFgAHxmNz/1NlQ7xLIrlN2h5d1eGI= +go.yaml.in/yaml/v2 v2.4.2/go.mod h1:081UH+NErpNdqlCXm3TtEran0rJZGxAYx9hb/ELlsPU= +golang.org/x/sys v0.35.0 h1:vz1N37gP5bs89s7He8XuIYXpyY0+QlsKmzipCbUtyxI= +golang.org/x/sys v0.35.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k= +google.golang.org/protobuf v1.36.8 h1:xHScyCOEuuwZEc6UtSOvPbAT4zRh0xcNRYekJwfqyMc= +google.golang.org/protobuf v1.36.8/go.mod h1:fuxRtAxBytpl4zzqUh6/eyUujkJdNiuEkXntxiD/uRU= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= +gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= +gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= diff --git a/harnesses/indexer-latency/Dockerfile b/harnesses/indexer-latency/Dockerfile new file mode 100644 index 00000000..63108cfc --- /dev/null +++ b/harnesses/indexer-latency/Dockerfile @@ -0,0 +1,22 @@ +FROM golang:1.24-alpine AS builder + +WORKDIR /app +RUN apk add --no-cache git + +COPY go.mod go.sum ./ +RUN go mod download + +COPY . . + +RUN CGO_ENABLED=0 GOOS=linux go build -o /app/monitor ./cmd/script + +FROM debian:bookworm-slim + +WORKDIR /app +RUN apt-get update && apt-get install -y ca-certificates && rm -rf /var/lib/apt/lists/* + +COPY --from=builder /app/monitor /app/monitor + +EXPOSE 2112 + +CMD ["/app/monitor"] diff --git a/harnesses/indexer-latency/cmd/script/alchemy.go b/harnesses/indexer-latency/cmd/script/alchemy.go new file mode 100644 index 00000000..9e46a8a7 --- /dev/null +++ b/harnesses/indexer-latency/cmd/script/alchemy.go @@ -0,0 +1,40 @@ +package main + +import ( + "fmt" + "net/http" + "strings" +) + +// Alchemy poller. +// +// Probed entity: the exact USDC Transfer (matched by tx hash) via +// alchemy_getAssetTransfers with fromBlock=toBlock=N and the USDC +// contract filter. This is Alchemy's indexed transfers pipeline (not a +// raw eth_getLogs proxy), which is what apps built on Alchemy's data +// APIs actually consume. ~150 CU/call on the 30M CU/month free tier. +// +// The key rides in the URL path per Alchemy convention; sanitize() +// strips URLs from any logged error so it can never leak. + +func checkAlchemy(ev probeEvent) (bool, error) { + bn, txHash := ev.Block, ev.TxHash + apiCalls.WithLabelValues("alchemy").Inc() + blk := fmt.Sprintf("0x%x", bn) + body := fmt.Sprintf( + `{"jsonrpc":"2.0","id":1,"method":"alchemy_getAssetTransfers","params":[{"fromBlock":"%s","toBlock":"%s","contractAddresses":["%s"],"category":["erc20"],"maxCount":"0x3e8","withMetadata":false,"excludeZeroValue":false}]}`, + blk, blk, usdcContract, + ) + u := "https://eth-mainnet.g.alchemy.com/v2/" + alchemyAPIKey() + req, err := http.NewRequest("POST", u, strings.NewReader(body)) + if err != nil { + return false, err + } + req.Header.Set("Content-Type", "application/json") + req.Header.Set("User-Agent", "OpenChainBench/1.0 (+https://openchainbench.com)") + resp, err := httpClient.Do(req) + if err != nil { + return false, err + } + return bodyContains(resp, txHash) +} diff --git a/harnesses/indexer-latency/cmd/script/config.go b/harnesses/indexer-latency/cmd/script/config.go new file mode 100644 index 00000000..fc34331b --- /dev/null +++ b/harnesses/indexer-latency/cmd/script/config.go @@ -0,0 +1,109 @@ +package main + +import ( + "os" + "strconv" + "strings" + "time" +) + +// indexer-latency — bench №084. +// +// Cohort: indexing pipelines reachable at $0 recurring cost (HyperSync, +// The Graph, Mobula). Bench 070 (indexing-freshness) races wallet-data +// APIs on Base native transfers; Mobula appears in both cohorts but on +// a different chain and event class (Ethereum ERC-20 event +// queryability here), so the two benches stay complementary. Alchemy +// was dropped from the cohort: the team's Alchemy account is a paid +// plan, and this bench only runs providers that cost nothing to probe. +// (checkAlchemy stays in the tree for a future dedicated free-tier +// key.) + +// Provider is one indexing pipeline we race. Enabled reports whether +// the required credential is present; a provider without one runs in +// the "disabled" state (counted, health=0) so the harness degrades +// gracefully instead of exiting. +// +// EveryN throttles participation for quota-tight free tiers: the +// provider only joins every Nth probe event. Budget is the monthly +// API-call budget for the guard (90% cutoff, calendar-month reset). +type Provider struct { + Slug string + EveryN int + Budget int64 + Enabled func() bool + // Check asks the provider whether the probe event (USDC Transfer + // tx in block bn) is queryable yet. The probed entity differs per + // provider (raw log vs subgraph block head vs wallet history) — + // disclosed in the bench methodology; the question is identical: + // "is block N data queryable at time T?". + Check func(ev probeEvent) (bool, error) +} + +// probeEvent is one sampled USDC Transfer: block, tx hash, and the +// recipient wallet (topics[2] of the Transfer log), which +// wallet-scoped pipelines like Mobula need to form their query. +type probeEvent struct { + Block uint64 + TxHash string + Wallet string +} + +var cohort = []Provider{ + // HyperSync free tier (token required since Nov 2025) is sized in + // the hundreds of millions of rows/month; our row volume is tiny, + // so the guard is request-count based with wide headroom. + {Slug: "hypersync", EveryN: 1, Budget: 400_000, Enabled: func() bool { return hypersyncToken() != "" }, Check: checkHyperSync}, + // The Graph free plan: 100k gateway queries/month. EveryN=2 on the + // 5-min event cadence = 1 probe event / 10 min → 144 events/day × + // 14 polls ≈ 60k queries/month worst case, inside the 90k budget. + {Slug: "thegraph", EveryN: 2, Budget: 90_000, Enabled: func() bool { return graphAPIKey() != "" }, Check: checkTheGraph}, + // Mobula: operated by the same team as OpenChainBench (disclosed in + // the spec); free for this project. Budget mirrors bench 070's + // Mobula guard so the two harnesses stay inside one key's comfort. + {Slug: "mobula", EveryN: 1, Budget: 250_000, Enabled: func() bool { return mobulaAPIKey() != "" }, Check: checkMobula}, +} + +const chainSlug = "ethereum" + +// USDC on Ethereum mainnet + the ERC-20 Transfer topic — the natural +// high-frequency event stream we piggyback on (no canary contract, $0). +const ( + usdcContract = "0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48" + transferTopic = "0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef" +) + +func hypersyncToken() string { return strings.TrimSpace(os.Getenv("HYPERSYNC_TOKEN")) } +func graphAPIKey() string { return strings.TrimSpace(os.Getenv("GRAPH_API_KEY")) } +func alchemyAPIKey() string { return strings.TrimSpace(os.Getenv("ALCHEMY_API_KEY")) } +func mobulaAPIKey() string { return strings.TrimSpace(os.Getenv("MOBULA_API_KEY")) } + +// rpcHTTP: OUR keyless reference RPC. T0 = the instant it first shows +// the block containing the probe event. publicnode by default; never a +// cohort member's RPC (that would gift them a head start). +func rpcHTTP() string { + if v := strings.TrimSpace(os.Getenv("INDEXER_RPC_HTTP")); v != "" { + return v + } + return "https://ethereum-rpc.publicnode.com" +} + +// eventInterval: one probe event (fresh USDC Transfer picked from a new +// block) per interval. 5 min default → 288 events/day, which keeps +// every provider inside its monthly free quota given the poll schedule +// (The Graph additionally joins only every 2nd event). +func eventInterval() time.Duration { + if v := strings.TrimSpace(os.Getenv("IDX_EVENT_SECONDS")); v != "" { + if n, err := strconv.Atoi(v); err == nil && n >= 30 { + return time.Duration(n) * time.Second + } + } + return 5 * time.Minute +} + +// pollSchedule: seconds after T0 at which each provider is polled. +// Front-loaded because HyperSync claims sub-second freshness while +// subgraph pipelines can lag minutes; capped at 240s (< event interval) +// after which the event counts as "timeout". Measured lag is an upper +// bound with resolution equal to the gap between consecutive polls. +var pollSchedule = []int{1, 2, 3, 5, 8, 12, 20, 30, 45, 60, 90, 120, 180, 240} diff --git a/harnesses/indexer-latency/cmd/script/hypersync.go b/harnesses/indexer-latency/cmd/script/hypersync.go new file mode 100644 index 00000000..6116ec9c --- /dev/null +++ b/harnesses/indexer-latency/cmd/script/hypersync.go @@ -0,0 +1,70 @@ +package main + +import ( + "encoding/json" + "fmt" + "io" + "net/http" + "strings" +) + +// Envio HyperSync poller. +// +// Probed entity: the exact USDC Transfer log (matched by tx hash) via +// POST /query scoped to [bn, bn+1). While HyperSync's archive height is +// below bn the query legitimately returns no logs (next_block <= bn), +// so substring detection on the tx hash is a correct "queryable yet?" +// signal. /query requires a free bearer token since Nov 2025 +// (HYPERSYNC_TOKEN); without it the provider runs disabled. + +const hypersyncBase = "https://eth.hypersync.xyz" + +func checkHyperSync(ev probeEvent) (bool, error) { + bn, txHash := ev.Block, ev.TxHash + apiCalls.WithLabelValues("hypersync").Inc() + body := fmt.Sprintf( + `{"from_block":%d,"to_block":%d,"logs":[{"address":["%s"],"topics":[["%s"]]}],"field_selection":{"log":["transaction_hash","block_number","log_index"]}}`, + bn, bn+1, usdcContract, transferTopic, + ) + req, err := http.NewRequest("POST", hypersyncBase+"/query", strings.NewReader(body)) + if err != nil { + return false, err + } + req.Header.Set("Authorization", "Bearer "+hypersyncToken()) + req.Header.Set("Content-Type", "application/json") + req.Header.Set("User-Agent", "OpenChainBench/1.0 (+https://openchainbench.com)") + resp, err := httpClient.Do(req) + if err != nil { + return false, err + } + return bodyContains(resp, txHash) +} + +// hypersyncHeight hits the keyless /height endpoint. Used at startup as +// a reachability proof (and logged) even when HYPERSYNC_TOKEN is unset. +func hypersyncHeight() (uint64, error) { + req, err := http.NewRequest("GET", hypersyncBase+"/height", nil) + if err != nil { + return 0, err + } + req.Header.Set("User-Agent", "OpenChainBench/1.0 (+https://openchainbench.com)") + resp, err := httpClient.Do(req) + if err != nil { + return 0, err + } + defer resp.Body.Close() + raw, err := io.ReadAll(io.LimitReader(resp.Body, 1<<20)) + if err != nil { + return 0, err + } + if resp.StatusCode != 200 { + return 0, fmt.Errorf("status %d", resp.StatusCode) + } + var out struct { + Height uint64 `json:"height"` + } + if err := json.Unmarshal(raw, &out); err != nil { + return 0, err + } + return out.Height, nil +} diff --git a/harnesses/indexer-latency/cmd/script/loghub.go b/harnesses/indexer-latency/cmd/script/loghub.go new file mode 100644 index 00000000..9dd74488 --- /dev/null +++ b/harnesses/indexer-latency/cmd/script/loghub.go @@ -0,0 +1,114 @@ +package main + +import ( + "bufio" + "fmt" + "io" + "net/http" + "os" + "strconv" + "sync" + "time" +) + +// Auto-generated by the loghub inline pattern. Captures stdout/stderr into a +// bounded ring buffer and exposes GET /logs?tail=N protected by X-Logs-Token +// matching the LOGS_TOKEN env var. +// +// Keep in sync across miniapps (was previously the shared/loghub package; we +// inline because Railway's per-harness Docker build context can't reach a +// sibling shared module via go.mod replace). + +const logRingMax = 5000 + +type logRing struct { + mu sync.Mutex + lines []string + max int +} + +var globalLogRing = &logRing{max: logRingMax} + +func (b *logRing) push(line string) { + entry := time.Now().UTC().Format("2006-01-02T15:04:05.000Z") + " " + line + b.mu.Lock() + if len(b.lines) >= b.max { + b.lines = append(b.lines[1:], entry) + } else { + b.lines = append(b.lines, entry) + } + b.mu.Unlock() +} + +func (b *logRing) snapshot(tail int) []string { + b.mu.Lock() + defer b.mu.Unlock() + if tail <= 0 || tail >= len(b.lines) { + out := make([]string, len(b.lines)) + copy(out, b.lines) + return out + } + start := len(b.lines) - tail + out := make([]string, tail) + copy(out, b.lines[start:]) + return out +} + +var logSetupOnce sync.Once + +// installLogCapture replaces os.Stdout (and os.Stderr) with the write-end of a +// pipe, then spawns a goroutine that fan-outs every line to the original +// stdout AND the in-memory ring buffer. Call exactly once, very early in +// main(). +func installLogCapture() { logSetupOnce.Do(doInstallLogCapture) } + +func doInstallLogCapture() { + originalStdout := os.Stdout + originalStderr := os.Stderr + r, w, err := os.Pipe() + if err != nil { + fmt.Fprintf(originalStdout, "[loghub] pipe failed: %v (/logs will be empty)\n", err) + return + } + os.Stdout = w + os.Stderr = w + + go func() { + scanner := bufio.NewScanner(r) + buf := make([]byte, 0, 1024*1024) + scanner.Buffer(buf, 1024*1024) + for scanner.Scan() { + line := scanner.Text() + fmt.Fprintln(originalStdout, line) + globalLogRing.push(line) + } + _, _ = io.Copy(originalStdout, r) + _ = originalStderr + }() +} + +// logsHandler returns an http.Handler for GET /logs?tail=N. Requires header +// X-Logs-Token to match the LOGS_TOKEN env var. Returns 403 if env unset. +func logsHandler() http.Handler { + return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + expected := os.Getenv("LOGS_TOKEN") + if expected == "" { + http.Error(w, "logs disabled: LOGS_TOKEN unset", http.StatusForbidden) + return + } + if r.Header.Get("X-Logs-Token") != expected { + http.Error(w, "forbidden", http.StatusForbidden) + return + } + tail := 500 + if t := r.URL.Query().Get("tail"); t != "" { + if n, err := strconv.Atoi(t); err == nil && n > 0 { + tail = n + } + } + w.Header().Set("Content-Type", "text/plain; charset=utf-8") + for _, l := range globalLogRing.snapshot(tail) { + fmt.Fprintln(w, l) + } + }) +} diff --git a/harnesses/indexer-latency/cmd/script/main.go b/harnesses/indexer-latency/cmd/script/main.go new file mode 100644 index 00000000..7f72c892 --- /dev/null +++ b/harnesses/indexer-latency/cmd/script/main.go @@ -0,0 +1,361 @@ +package main + +import ( + "bytes" + "context" + "encoding/json" + "fmt" + "io" + "math/rand" + "net/http" + "os" + "os/signal" + "strconv" + "strings" + "sync" + "syscall" + "time" +) + +// indexer-latency — bench №084. +// +// How fast does indexing infrastructure make new onchain events +// queryable? One probe event per interval: pick a fresh USDC Transfer +// from the newest Ethereum block via our keyless reference RPC +// (T0 = the instant WE first see the block), then poll each indexing +// pipeline on a front-loaded schedule until block N's data is +// queryable (lag = poll time − T0) or 240s passes (timeout). +// +// $0 methodology: no canary contract, we piggyback on the natural +// high-frequency USDC stream. Ground truth is a random organic +// transfer, different every event, publicly re-verifiable by tx hash. + +func main() { + installLogCapture() // capture stdout into /logs ring buffer + + fmt.Println("=== Indexer Latency Harness (bench 084) ===") + fmt.Println("OpenChainBench — onchain event → indexer time-to-queryable.") + fmt.Printf("Reference RPC: %s\n", rpcHTTP()) + + // Keyless reachability proof: HyperSync /height vs our RPC head. + if h, err := hypersyncHeight(); err == nil { + fmt.Printf("HyperSync /height (keyless): %d\n", h) + } else { + fmt.Printf("HyperSync /height (keyless): unreachable (%s)\n", sanitize(err)) + } + + enabled := 0 + fmt.Printf("Chain: %s | event interval: %s | poll cap: %ds\n", chainSlug, eventInterval(), pollSchedule[len(pollSchedule)-1]) + for _, p := range cohort { + state := "disabled (credential missing)" + if p.Enabled() { + state = fmt.Sprintf("enabled everyN=%d budget=%d calls/mo", p.EveryN, p.Budget) + enabled++ + } + fmt.Printf(" - %-10s %s\n", p.Slug, state) + } + if enabled == 0 { + fmt.Println("[warn] no provider credentials set (HYPERSYNC_TOKEN / GRAPH_API_KEY / MOBULA_API_KEY)") + fmt.Println("[warn] running in degraded mode: events sampled, all providers report result=disabled") + } + + addr := ":2112" + if v := strings.TrimSpace(os.Getenv("METRICS_ADDR")); v != "" { + addr = v + } + fmt.Printf("Metrics server: %s/metrics\n\n", addr) + go func() { + if err := StartMetricsServer(addr); err != nil { + fmt.Printf("[fatal] metrics server: %v\n", err) + os.Exit(1) + } + }() + + ctx, cancel := context.WithCancel(context.Background()) + defer cancel() + go runEvents(ctx) + + sig := make(chan os.Signal, 1) + signal.Notify(sig, os.Interrupt, syscall.SIGTERM) + s := <-sig + fmt.Printf("\n[shutdown] %v\n", s) + cancel() +} + +func runEvents(ctx context.Context) { + t := time.NewTicker(eventInterval()) + defer t.Stop() + var lastSeen uint64 + eventN := 0 + for { + eventN++ + ev, t0 := waitFreshEvent(ctx, lastSeen) + if ev.Block == 0 { + return // ctx cancelled + } + lastSeen = ev.Block + fmt.Printf("[event %d] block=%d usdc-transfer tx=%s\n", eventN, ev.Block, ev.TxHash[:14]+"…") + + var wg sync.WaitGroup + for _, p := range cohort { + if eventN%p.EveryN != 0 { + continue + } + if !p.Enabled() { + probeTotal.WithLabelValues(p.Slug, "disabled").Inc() + health.WithLabelValues(p.Slug).Set(0) + fmt.Printf(" %-10s disabled (credential missing)\n", p.Slug) + continue + } + if !quota.allow(p) { + probeTotal.WithLabelValues(p.Slug, "quota_paused").Inc() + fmt.Printf(" %-10s quota guard tripped — paused until month rollover\n", p.Slug) + continue + } + wg.Add(1) + go func(p Provider) { + defer wg.Done() + raceProvider(ctx, p, ev, t0) + }(p) + } + wg.Wait() + + select { + case <-ctx.Done(): + return + case <-t.C: + } + } +} + +// raceProvider polls one indexing pipeline on the shared front-loaded +// schedule until block bn's probe event is queryable, or the schedule +// is exhausted (timeout), or every poll errored (error). +func raceProvider(ctx context.Context, p Provider, ev probeEvent, t0 time.Time) { + var lastErr error + polls, errs := 0, 0 + for _, after := range pollSchedule { + wait := time.Until(t0.Add(time.Duration(after) * time.Second)) + if wait > 0 { + select { + case <-ctx.Done(): + return + case <-time.After(wait): + } + } + polls++ + found, err := p.Check(ev) + if err != nil { + errs++ + lastErr = err + continue + } + if found { + lag := time.Since(t0).Seconds() + latencyHist.WithLabelValues(p.Slug).Observe(lag) + latencyLast.WithLabelValues(p.Slug).Set(lag) + probeTotal.WithLabelValues(p.Slug, "found").Inc() + health.WithLabelValues(p.Slug).Set(1) + fmt.Printf(" %-10s queryable in %.1fs\n", p.Slug, lag) + return + } + } + health.WithLabelValues(p.Slug).Set(0) + // "error" only when EVERY poll of the event errored (matches the spec's + // classification). An event with at least one clean "not yet" answer is a + // timeout: reclassifying it as error would drop it from the success-rate + // denominator and let transient blips inflate a provider's score. + if errs == polls && lastErr != nil { + probeTotal.WithLabelValues(p.Slug, "error").Inc() + fmt.Printf(" %-10s error: %s\n", p.Slug, sanitize(lastErr)) + return + } + probeTotal.WithLabelValues(p.Slug, "timeout").Inc() + if lastErr != nil { + fmt.Printf(" %-10s TIMEOUT (not queryable within %ds; %d/%d polls errored, last: %s)\n", p.Slug, pollSchedule[len(pollSchedule)-1], errs, polls, sanitize(lastErr)) + return + } + fmt.Printf(" %-10s TIMEOUT (not queryable within %ds)\n", p.Slug, pollSchedule[len(pollSchedule)-1]) +} + +// --------------------------------------------------------------------------- +// Reference sampler: T0 from OUR keyless RPC (publicnode), never a +// cohort member's endpoint. 500ms head polling bounds the T0 error at +// +500ms, identical for every provider. +// --------------------------------------------------------------------------- + +var httpClient = &http.Client{Timeout: 8 * time.Second} + +func rpcCall(method, params string) (json.RawMessage, error) { + body := fmt.Sprintf(`{"jsonrpc":"2.0","method":"%s","params":%s,"id":%d}`, method, params, time.Now().UnixNano()) + req, err := http.NewRequest("POST", rpcHTTP(), strings.NewReader(body)) + if err != nil { + return nil, err + } + req.Header.Set("Content-Type", "application/json") + req.Header.Set("User-Agent", "OpenChainBench/1.0 (+https://openchainbench.com)") + resp, err := httpClient.Do(req) + if err != nil { + return nil, err + } + defer resp.Body.Close() + raw, err := io.ReadAll(io.LimitReader(resp.Body, 8<<20)) + if err != nil { + return nil, err + } + var env struct { + Result json.RawMessage `json:"result"` + Error *struct { + Message string `json:"message"` + } `json:"error"` + } + if err := json.Unmarshal(raw, &env); err != nil { + return nil, err + } + if env.Error != nil { + return nil, fmt.Errorf("rpc: %s", env.Error.Message) + } + return env.Result, nil +} + +func headBlock() (uint64, error) { + res, err := rpcCall("eth_blockNumber", "[]") + if err != nil { + return 0, err + } + var hexN string + if err := json.Unmarshal(res, &hexN); err != nil { + return 0, err + } + return strconv.ParseUint(strings.TrimPrefix(hexN, "0x"), 16, 64) +} + +type rpcLog struct { + TransactionHash string `json:"transactionHash"` + Topics []string `json:"topics"` +} + +// recipient extracts the ERC-20 Transfer `to` address from topics[2] +// (32-byte left-padded). Empty string when the log shape is unexpected. +func recipient(l rpcLog) string { + if len(l.Topics) < 3 || len(l.Topics[2]) != 66 { + return "" + } + return "0x" + l.Topics[2][26:] +} + +// usdcTransfersInBlock returns the tx hashes of USDC Transfer logs in +// block bn, via eth_getLogs on the reference RPC. +func usdcTransfersInBlock(bn uint64) ([]rpcLog, error) { + params := fmt.Sprintf(`[{"fromBlock":"0x%x","toBlock":"0x%x","address":"%s","topics":["%s"]}]`, bn, bn, usdcContract, transferTopic) + res, err := rpcCall("eth_getLogs", params) + if err != nil { + return nil, err + } + var logs []rpcLog + if err := json.Unmarshal(res, &logs); err != nil { + return nil, err + } + return logs, nil +} + +// waitFreshEvent polls the reference RPC head until a block newer than +// lastSeen containing at least one USDC Transfer shows up. T0 is the +// instant we first observed the new head. Returns (0, "", zero) only +// when ctx is cancelled. +func waitFreshEvent(ctx context.Context, lastSeen uint64) (probeEvent, time.Time) { + for { + select { + case <-ctx.Done(): + return probeEvent{}, time.Time{} + default: + } + bn, err := headBlock() + if err != nil { + fmt.Printf("[sampler] head poll error: %s\n", sanitize(err)) + time.Sleep(2 * time.Second) + continue + } + if bn > lastSeen { + t0 := time.Now() + logs, err := usdcTransfersInBlock(bn) + if err != nil { + fmt.Printf("[sampler] getLogs error: %s\n", sanitize(err)) + time.Sleep(time.Second) + continue + } + if len(logs) > 0 { + pick := logs[rand.Intn(len(logs))] + if strings.HasPrefix(pick.TransactionHash, "0x") && len(pick.TransactionHash) == 66 && recipient(pick) != "" { + return probeEvent{Block: bn, TxHash: strings.ToLower(pick.TransactionHash), Wallet: strings.ToLower(recipient(pick))}, t0 + } + } + // Rare on ETH mainnet: block without a USDC transfer. Skip it. + lastSeen = bn + } + time.Sleep(500 * time.Millisecond) + } +} + +// --------------------------------------------------------------------------- +// Shared helpers. +// --------------------------------------------------------------------------- + +// bodyContains — deliberately parser-free detection: lowercase the raw +// response body and look for the tx hash substring. Immune to +// per-provider schema churn; no response shape is favoured. +func bodyContains(resp *http.Response, txHash string) (bool, error) { + defer resp.Body.Close() + raw, err := io.ReadAll(io.LimitReader(resp.Body, 4<<20)) + if err != nil { + return false, err + } + if resp.StatusCode != 200 { + return false, fmt.Errorf("status %d", resp.StatusCode) + } + return bytes.Contains(bytes.ToLower(raw), []byte(strings.ToLower(txHash))), nil +} + +// sanitize strips anything URL-ish from provider errors before logging +// (the Alchemy key rides in its URL; never let it reach stdout/loghub). +func sanitize(err error) string { + if err == nil { + return "" + } + s := err.Error() + if i := strings.Index(s, "http"); i >= 0 { + return s[:i] + "" + } + return s +} + +// --------------------------------------------------------------------------- +// Monthly quota guard (same design as indexing-freshness / rpc-keyed-latency). +// --------------------------------------------------------------------------- + +type quotaGuard struct { + mu sync.Mutex + month string + counts map[string]int64 +} + +var quota = "aGuard{counts: make(map[string]int64)} + +func (q *quotaGuard) allow(p Provider) bool { + q.mu.Lock() + defer q.mu.Unlock() + m := time.Now().UTC().Format("2006-01") + if m != q.month { + q.month = m + q.counts = make(map[string]int64) + } + used := q.counts[p.Slug] + ratio := float64(used) / float64(p.Budget) + quotaUsedRatio.WithLabelValues(p.Slug).Set(ratio) + if ratio >= 0.90 { + return false + } + // Reserve the worst case for one event (full poll schedule). + q.counts[p.Slug] = used + int64(len(pollSchedule)) + return true +} diff --git a/harnesses/indexer-latency/cmd/script/metrics.go b/harnesses/indexer-latency/cmd/script/metrics.go new file mode 100644 index 00000000..4ee048d7 --- /dev/null +++ b/harnesses/indexer-latency/cmd/script/metrics.go @@ -0,0 +1,71 @@ +package main + +import ( + "net/http" + + "github.com/prometheus/client_golang/prometheus" + "github.com/prometheus/client_golang/prometheus/promauto" + "github.com/prometheus/client_golang/prometheus/promhttp" +) + +var ( + latencyHist = promauto.NewHistogramVec( + prometheus.HistogramOpts{ + Name: "idx_latency_seconds", + Help: "Histogram of time-to-queryable lags: seconds between our reference RPC first showing block N and the provider first returning block N's probe event.", + Buckets: []float64{1, 2, 3, 5, 8, 12, 20, 30, 45, 60, 90, 120, 180, 240, 360, 600}, + }, + []string{"provider"}, + ) + + latencyLast = promauto.NewGaugeVec( + prometheus.GaugeOpts{ + Name: "idx_latency_last_seconds", + Help: "Latest observed time-to-queryable lag per provider (drives the bench page time series).", + }, + []string{"provider"}, + ) + + probeTotal = promauto.NewCounterVec( + prometheus.CounterOpts{ + Name: "idx_probe_total", + Help: "Probe outcomes per provider: found | timeout (not queryable within 240s) | error | quota_paused | disabled (credential missing).", + }, + []string{"provider", "result"}, + ) + + quotaUsedRatio = promauto.NewGaugeVec( + prometheus.GaugeOpts{ + Name: "idx_quota_used_ratio", + Help: "Fraction of the provider's monthly call budget consumed (probing pauses at 0.90, calendar-month reset).", + }, + []string{"provider"}, + ) + + health = promauto.NewGaugeVec( + prometheus.GaugeOpts{ + Name: "idx_health", + Help: "1 when the provider's last probe found the event, 0 on timeout/error/disabled. Doubles as the harness freshness signal.", + }, + []string{"provider"}, + ) + + apiCalls = promauto.NewCounterVec( + prometheus.CounterOpts{ + Name: "idx_api_calls_total", + Help: "API calls issued per provider (feeds the monthly quota guard).", + }, + []string{"provider"}, + ) +) + +func StartMetricsServer(addr string) error { + mux := http.NewServeMux() + mux.Handle("/metrics", promhttp.Handler()) + mux.Handle("/logs", logsHandler()) + mux.HandleFunc("/health", func(w http.ResponseWriter, _ *http.Request) { + w.WriteHeader(http.StatusOK) + _, _ = w.Write([]byte("ok")) + }) + return http.ListenAndServe(addr, mux) +} diff --git a/harnesses/indexer-latency/cmd/script/mobula.go b/harnesses/indexer-latency/cmd/script/mobula.go new file mode 100644 index 00000000..6eb83fad --- /dev/null +++ b/harnesses/indexer-latency/cmd/script/mobula.go @@ -0,0 +1,42 @@ +package main + +import ( + "fmt" + "net/http" +) + +// Mobula poller. +// +// Probed entity: the exact USDC Transfer (matched by tx hash) via the +// wallet-transactions endpoint of the RECIPIENT address captured from +// the Transfer log's topics. This is Mobula's indexed wallet-history +// pipeline, the same product surface bench 070 races on Base native +// transfers; here it answers the 084 question (Ethereum ERC-20 event +// time-to-queryable) so the two benches stay complementary, not +// duplicated. +// +// Disclosure (also in the spec methodology): Mobula operates +// OpenChainBench. The poller runs the exact probe/quota rules as every +// other cohort member and the query is keyless-reproducible modulo the +// free API key. +// +// limit=100: probe recipients are arbitrary USDC receivers, sometimes +// exchange hot wallets busy enough to bury a minutes-old tx below a +// short first page. 100 rows keeps burial (a false "not yet") rare; +// residual risk is disclosed as upper-bound semantics. + +func checkMobula(ev probeEvent) (bool, error) { + apiCalls.WithLabelValues("mobula").Inc() + u := fmt.Sprintf("https://api.mobula.io/api/1/wallet/transactions?wallet=%s&limit=100", ev.Wallet) + req, err := http.NewRequest("GET", u, nil) + if err != nil { + return false, err + } + req.Header.Set("Authorization", mobulaAPIKey()) + req.Header.Set("User-Agent", "OpenChainBench/1.0 (+https://openchainbench.com)") + resp, err := httpClient.Do(req) + if err != nil { + return false, err + } + return bodyContains(resp, ev.TxHash) +} diff --git a/harnesses/indexer-latency/cmd/script/thegraph.go b/harnesses/indexer-latency/cmd/script/thegraph.go new file mode 100644 index 00000000..e5a1efb9 --- /dev/null +++ b/harnesses/indexer-latency/cmd/script/thegraph.go @@ -0,0 +1,70 @@ +package main + +import ( + "encoding/json" + "fmt" + "io" + "net/http" + "strings" +) + +// The Graph poller. +// +// The Graph only serves what subgraphs index, so we cannot ask it for a +// raw USDC Transfer log. Probed entity instead: the canonical Uniswap +// V3 Ethereum subgraph queried WITH a block constraint pinned to the +// reference block N. graph-node rejects block-constrained queries until +// the subgraph has indexed block N ("only indexed up to block ..."), so +// a successful data response is exactly "block N data queryable". This +// per-provider entity asymmetry is disclosed in the bench methodology; +// the measured question (block N queryable at T?) is identical. +// +// Gateway free plan (100k queries/month) requires GRAPH_API_KEY; the +// key travels in the Authorization header, never in the URL. + +const uniswapV3SubgraphID = "5zvR82QoaXYFyDEKLZ9t6v9adgnptxYpKpSbxtgVENFV" + +func checkTheGraph(ev probeEvent) (bool, error) { + bn := ev.Block + apiCalls.WithLabelValues("thegraph").Inc() + q := fmt.Sprintf(`{"query":"{ pools(block: {number: %d}, first: 1) { id } }"}`, bn) + u := "https://gateway.thegraph.com/api/subgraphs/id/" + uniswapV3SubgraphID + req, err := http.NewRequest("POST", u, strings.NewReader(q)) + if err != nil { + return false, err + } + req.Header.Set("Authorization", "Bearer "+graphAPIKey()) + req.Header.Set("Content-Type", "application/json") + req.Header.Set("User-Agent", "OpenChainBench/1.0 (+https://openchainbench.com)") + resp, err := httpClient.Do(req) + if err != nil { + return false, err + } + defer resp.Body.Close() + raw, err := io.ReadAll(io.LimitReader(resp.Body, 4<<20)) + if err != nil { + return false, err + } + if resp.StatusCode != 200 { + return false, fmt.Errorf("status %d", resp.StatusCode) + } + var out struct { + Data json.RawMessage `json:"data"` + Errors []struct { + Message string `json:"message"` + } `json:"errors"` + } + if err := json.Unmarshal(raw, &out); err != nil { + return false, err + } + if len(out.Errors) > 0 { + msg := strings.ToLower(out.Errors[0].Message) + // graph-node's "block N not yet indexed" family of errors is the + // expected pre-sync answer, not an API failure. + if strings.Contains(msg, "indexed up to") || strings.Contains(msg, "has only indexed") || strings.Contains(msg, "missing block") || strings.Contains(msg, "not yet") { + return false, nil + } + return false, fmt.Errorf("graphql: %s", out.Errors[0].Message) + } + return len(out.Data) > 0 && string(out.Data) != "null", nil +} diff --git a/harnesses/indexer-latency/go.mod b/harnesses/indexer-latency/go.mod new file mode 100644 index 00000000..c07b9b31 --- /dev/null +++ b/harnesses/indexer-latency/go.mod @@ -0,0 +1,18 @@ +module indexer-latency + +go 1.24.0 + +require github.com/prometheus/client_golang v1.23.2 + +require ( + github.com/beorn7/perks v1.0.1 // indirect + github.com/cespare/xxhash/v2 v2.3.0 // indirect + github.com/kr/text v0.2.0 // indirect + github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect + github.com/prometheus/client_model v0.6.2 // indirect + github.com/prometheus/common v0.66.1 // indirect + github.com/prometheus/procfs v0.16.1 // indirect + go.yaml.in/yaml/v2 v2.4.2 // indirect + golang.org/x/sys v0.35.0 // indirect + google.golang.org/protobuf v1.36.8 // indirect +) diff --git a/harnesses/indexer-latency/go.sum b/harnesses/indexer-latency/go.sum new file mode 100644 index 00000000..d6b8ca98 --- /dev/null +++ b/harnesses/indexer-latency/go.sum @@ -0,0 +1,46 @@ +github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= +github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= +github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs= +github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= +github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= +github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= +github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= +github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU= +github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo= +github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ= +github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= +github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= +github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= +github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= +github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc= +github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw= +github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA= +github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ= +github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= +github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/prometheus/client_golang v1.23.2 h1:Je96obch5RDVy3FDMndoUsjAhG5Edi49h0RJWRi/o0o= +github.com/prometheus/client_golang v1.23.2/go.mod h1:Tb1a6LWHB3/SPIzCoaDXI4I8UHKeFTEQ1YCr+0Gyqmg= +github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk= +github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE= +github.com/prometheus/common v0.66.1 h1:h5E0h5/Y8niHc5DlaLlWLArTQI7tMrsfQjHV+d9ZoGs= +github.com/prometheus/common v0.66.1/go.mod h1:gcaUsgf3KfRSwHY4dIMXLPV0K/Wg1oZ8+SbZk/HH/dA= +github.com/prometheus/procfs v0.16.1 h1:hZ15bTNuirocR6u0JZ6BAHHmwS1p8B4P6MRqxtzMyRg= +github.com/prometheus/procfs v0.16.1/go.mod h1:teAbpZRB1iIAJYREa1LsoWUXykVXA1KlTmWl8x/U+Is= +github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjRBZyWFQ= +github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog= +github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U= +github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U= +go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto= +go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE= +go.yaml.in/yaml/v2 v2.4.2 h1:DzmwEr2rDGHl7lsFgAHxmNz/1NlQ7xLIrlN2h5d1eGI= +go.yaml.in/yaml/v2 v2.4.2/go.mod h1:081UH+NErpNdqlCXm3TtEran0rJZGxAYx9hb/ELlsPU= +golang.org/x/sys v0.35.0 h1:vz1N37gP5bs89s7He8XuIYXpyY0+QlsKmzipCbUtyxI= +golang.org/x/sys v0.35.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k= +google.golang.org/protobuf v1.36.8 h1:xHScyCOEuuwZEc6UtSOvPbAT4zRh0xcNRYekJwfqyMc= +google.golang.org/protobuf v1.36.8/go.mod h1:fuxRtAxBytpl4zzqUh6/eyUujkJdNiuEkXntxiD/uRU= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= +gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= +gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= diff --git a/harnesses/l2-block-time/cmd/script/config.go b/harnesses/l2-block-time/cmd/script/config.go index 98bdb9c4..0879337f 100644 --- a/harnesses/l2-block-time/cmd/script/config.go +++ b/harnesses/l2-block-time/cmd/script/config.go @@ -13,6 +13,12 @@ type L2Chain struct { Slug string Name string URL string + // Poll switches the chain to HTTP block-number polling instead of a + // WebSocket newHeads subscription, for sequencers that expose no + // public WSS (Robinhood Chain). Block time is then derived from + // block-number deltas over each poll window; per-block jitter is + // not captured and the methodology discloses it. + Poll bool } // l2Chains is the source of truth for which Layer-2s appear in the @@ -22,6 +28,12 @@ type L2Chain struct { // rebuild — useful if a public endpoint goes flaky. func l2Chains() []L2Chain { return []L2Chain{ + { + Slug: "robinhood", + Name: "Robinhood Chain", + URL: envDefault("RPC_HTTP_ROBINHOOD", "https://rpc.mainnet.chain.robinhood.com"), + Poll: true, + }, { Slug: "arbitrum", Name: "Arbitrum One", diff --git a/harnesses/l2-block-time/cmd/script/sequencer_poll.go b/harnesses/l2-block-time/cmd/script/sequencer_poll.go new file mode 100644 index 00000000..30a03d10 --- /dev/null +++ b/harnesses/l2-block-time/cmd/script/sequencer_poll.go @@ -0,0 +1,106 @@ +package main + +import ( + "bytes" + "encoding/json" + "fmt" + "io" + "net/http" + "time" +) + +// HTTP polling fallback for sequencers with no public WebSocket +// (Robinhood Chain). Every pollInterval we read eth_blockNumber; a +// window that advanced by N blocks contributes N histogram samples of +// (elapsed / N) ms. This yields an accurate average block time while +// keeping the "one sample per block" weighting of the WS path; what it +// cannot capture is per-block jitter, which the bench methodology +// discloses for polled chains. +const ( + pollInterval = 2 * time.Second + pollTimeout = 10 * time.Second + maxWindowBlend = 600 // safety cap on samples per window (reconnect gaps) +) + +func runPollChain(ch L2Chain) error { + client := &http.Client{Timeout: pollTimeout} + var lastNum int64 + var lastAt time.Time + fails := 0 + + t := time.NewTicker(pollInterval) + defer t.Stop() + fmt.Printf("[%s] HTTP polling mode, interval=%s\n", ch.Slug, pollInterval) + + for now := range t.C { + num, err := fetchBlockNumber(client, ch.URL) + if err != nil || num == 0 { + fails++ + if fails >= 5 { + blockTimeHealth.WithLabelValues(ch.Slug).Set(0) + } + if fails == 5 || fails%150 == 0 { + fmt.Printf("[%s] poll error x%d: %v\n", ch.Slug, fails, err) + } + continue + } + fails = 0 + blockTimeHealth.WithLabelValues(ch.Slug).Set(1) + + if lastNum > 0 && num > lastNum && !lastAt.IsZero() { + delta := num - lastNum + elapsedMs := float64(now.Sub(lastAt).Milliseconds()) + perBlock := elapsedMs / float64(delta) + if perBlock > 0 && perBlock < maxSampleMs { + blockTimeGauge.WithLabelValues(ch.Slug).Set(perBlock) + n := int(delta) + if n > maxWindowBlend { + n = maxWindowBlend + } + for i := 0; i < n; i++ { + blockTimeHistogram.WithLabelValues(ch.Slug).Observe(perBlock) + } + blockTimeSampleCtr.WithLabelValues(ch.Slug).Add(float64(n)) + } + } + lastNum = num + lastAt = now + } + return nil +} + +func fetchBlockNumber(client *http.Client, url string) (int64, error) { + body := []byte(fmt.Sprintf( + `{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":%d}`, + time.Now().UnixNano(), + )) + req, err := http.NewRequest("POST", url, bytes.NewReader(body)) + if err != nil { + return 0, err + } + req.Header.Set("Content-Type", "application/json") + req.Header.Set("User-Agent", "OpenChainBench/1.0 (+https://openchainbench.com)") + resp, err := client.Do(req) + if err != nil { + return 0, err + } + defer resp.Body.Close() + raw, err := io.ReadAll(io.LimitReader(resp.Body, 1<<16)) + if err != nil { + return 0, err + } + if resp.StatusCode != 200 { + return 0, fmt.Errorf("status %d", resp.StatusCode) + } + var env struct { + Result string `json:"result"` + } + if err := json.Unmarshal(raw, &env); err != nil { + return 0, err + } + n := parseHexInt64(env.Result) + if n == 0 { + return 0, fmt.Errorf("empty block number") + } + return n, nil +} diff --git a/harnesses/l2-block-time/cmd/script/sequencer_ws.go b/harnesses/l2-block-time/cmd/script/sequencer_ws.go index 26d59f69..e9327aaf 100644 --- a/harnesses/l2-block-time/cmd/script/sequencer_ws.go +++ b/harnesses/l2-block-time/cmd/script/sequencer_ws.go @@ -75,7 +75,12 @@ func StartSequencerWS() { go func() { backoff := minBackoff for { - err := runChain(ch) + var err error + if ch.Poll { + err = runPollChain(ch) + } else { + err = runChain(ch) + } blockTimeHealth.WithLabelValues(ch.Slug).Set(0) blockTimeReconnectCtr.WithLabelValues(ch.Slug).Inc() if err != nil { diff --git a/harnesses/nft-metadata-coverage/cmd/script/monitor.go b/harnesses/nft-metadata-coverage/cmd/script/monitor.go index 26f14653..5fd4d0da 100644 --- a/harnesses/nft-metadata-coverage/cmd/script/monitor.go +++ b/harnesses/nft-metadata-coverage/cmd/script/monitor.go @@ -25,7 +25,7 @@ func truncate(s string, n int) string { // providerOrder is the column order in the condensed line, log table, and // any debug dump. -var providerOrder = []string{"moralis", "alchemy", "opensea", "rarible"} +var providerOrder = []string{"moralis", "alchemy", "opensea"} // fieldLetter maps each field to a single-letter code (NIDFE) used in the // condensed per-collection log line. @@ -228,10 +228,11 @@ func runCheckAllProviders(cfg *Config) { results["opensea"] = osRes openSeaWaitTurn() // accounts for the second internal call's spacing - // Rarible is serial @ 1 req/sec. - raribleWaitTurn() - raRes := checkRarible(coll, cfg.RaribleAPIKey, cfg.MonitorRegion) - results["rarible"] = raRes + // Rarible delisted 2026-07-13: the free tier caps at 100 requests + // per MONTH (one 50-collection cycle burns half of it), below any + // measurable cadence. The probe code stays in rarible.go for a + // sponsored or paid key revival; it was never a spec provider, so + // nothing user-facing changes. for _, p := range providerOrder { applyResult(results[p], cfg.MonitorRegion) diff --git a/harnesses/nft-metadata-coverage/cmd/script/opensea.go b/harnesses/nft-metadata-coverage/cmd/script/opensea.go index 944ae839..0869c7d7 100644 --- a/harnesses/nft-metadata-coverage/cmd/script/opensea.go +++ b/harnesses/nft-metadata-coverage/cmd/script/opensea.go @@ -43,7 +43,12 @@ var openSeaClient = &http.Client{Timeout: 10 * time.Second} func resolveOpenSeaSlug(contract, apiKey, region string) (string, error) { url := fmt.Sprintf("https://api.opensea.io/api/v2/chain/ethereum/contract/%s", contract) req, _ := http.NewRequest("GET", url, nil) - req.Header.Set("x-api-key", apiKey) + // OpenSea v2 serves these endpoints keyless (verified 2026-07-13), + // but an EMPTY x-api-key header gets a 401. Only set it when a key + // is actually configured. + if apiKey != "" { + req.Header.Set("x-api-key", apiKey) + } req.Header.Set("Accept", "application/json") req.Header.Set("User-Agent", userAgent) @@ -73,12 +78,9 @@ func resolveOpenSeaSlug(contract, apiKey, region string) (string, error) { func checkOpenSea(coll NFTCollection, apiKey, region string) NFTResult { res := newResult("opensea", coll.Name) - if apiKey == "" { - res.Error = "missing_api_key" - res.ErrorType = "config" - recordError("opensea", region, "config") - return res - } + // Keyless mode is supported: OpenSea v2 collections/stats/contract + // endpoints answer without a key (verified 2026-07-13, incl. burst). + // A configured key only raises the rate-limit headroom. if coll.OpenSeaSlug == "" { res.Error = "missing_slug" res.ErrorType = "slug_resolve_error" @@ -91,7 +93,9 @@ func checkOpenSea(coll NFTCollection, apiKey, region string) NFTResult { // Call 1: collection metadata url1 := fmt.Sprintf("https://api.opensea.io/api/v2/collections/%s", coll.OpenSeaSlug) req1, _ := http.NewRequest("GET", url1, nil) - req1.Header.Set("x-api-key", apiKey) + if apiKey != "" { + req1.Header.Set("x-api-key", apiKey) + } req1.Header.Set("Accept", "application/json") req1.Header.Set("User-Agent", userAgent) @@ -130,7 +134,9 @@ func checkOpenSea(coll NFTCollection, apiKey, region string) NFTResult { // Call 2: stats (floor price). Non-fatal if it fails — we still score 4/5. url2 := fmt.Sprintf("https://api.opensea.io/api/v2/collections/%s/stats", coll.OpenSeaSlug) req2, _ := http.NewRequest("GET", url2, nil) - req2.Header.Set("x-api-key", apiKey) + if apiKey != "" { + req2.Header.Set("x-api-key", apiKey) + } req2.Header.Set("Accept", "application/json") req2.Header.Set("User-Agent", userAgent) diff --git a/harnesses/oracle-deviation/README.md b/harnesses/oracle-deviation/README.md index 1d41b04e..107e80fe 100644 --- a/harnesses/oracle-deviation/README.md +++ b/harnesses/oracle-deviation/README.md @@ -1,7 +1,11 @@ -# oracle-deviation — OpenChainBench № 025 +# oracle-deviation — OpenChainBench № 025 + № 082 Compares four price oracles in real time across ten USD pairs and -publishes the per-pair deviation as Prometheus gauges. +publishes the per-pair deviation as Prometheus gauges (bench № 025). +The same process also tracks oracle freshness (bench № 082): +staleness in seconds derived from each oracle's own last-update +timestamp, for Chainlink on three chains, Pyth via Hermes and +RedStone via its public per-symbol API. ## Sources @@ -12,6 +16,14 @@ publishes the per-pair deviation as Prometheus gauges. | Binance | REST `/api/v3/ticker/price` | none | 30 s | | Coinbase | REST `/products/

/ticker` | none | 30 s | +Freshness-only sources (№ 082, never enter the № 025 deviation matrix): + +| Source | Transport | Auth | Cadence | +| ------------------- | ------------------------------------------------ | ---- | ------- | +| Chainlink Arbitrum | `eth_call` on AggregatorV3 (ETH/USD) | none | 30 s | +| Chainlink Base | `eth_call` on AggregatorV3 (ETH/USD) | none | 30 s | +| RedStone | REST `/prices?symbol=&provider=redstone` | none | 30 s | + ## Pairs BTC, ETH, SOL, BNB, XRP, ADA, DOGE, AVAX, LINK, MATIC — all quoted @@ -48,6 +60,25 @@ ocb_oracle_scrape_errors_total{source, pair} counter ocb_oracle_last_round_age_seconds{source="chainlink", pair} gauge ``` +Bench № 082 (freshness) families. New label set (`oracle`, `pair`, +`chain`) instead of adding a `chain` label to the № 025 families, +which would have split their series and changed the live bench's +numbers: + +``` +ocb_oracle_staleness_seconds{oracle, pair, chain} gauge (s) +ocb_oracle_update_events_total{oracle, pair, chain} counter +ocb_oracle_stale_but_moved{oracle, pair, chain} gauge (0/1) +ocb_oracle_freshness_scrape_errors_total{oracle, pair, chain} counter +``` + +`chain` is where the freshness timestamp lives: `ethereum` / +`arbitrum` / `base` for Chainlink deployments, `hermes` for Pyth +(publish_time), `gateway` for RedStone (signed data-package +timestamp). `stale_but_moved` is 1 when staleness > 300 s AND the +Binance/Coinbase reference moved > 0.5 % since the update was +observed (thresholds documented in `freshness.go`). + The headline ranking signal for the OCB leaderboard is `ocb_oracle_max_deviation_pct` — a per-pair quantile over time (`quantile_over_time(0.95, ocb_oracle_max_deviation_pct[24h])`) @@ -76,3 +107,7 @@ curl -s :2112/metrics | grep ocb_oracle_max_deviation_pct | ------------------------ | ------------------------------------------- | | `ORACLE_RPC_PRIMARY` | `https://ethereum-rpc.publicnode.com` | | `ORACLE_RPC_FALLBACK` | `https://eth.llamarpc.com` | +| `ORACLE_RPC_ARBITRUM` | `https://arbitrum-one-rpc.publicnode.com` | +| `ORACLE_RPC_BASE` | `https://base-rpc.publicnode.com` | +| `ORACLE_REDSTONE_URL` | `https://api.redstone.finance/prices` | +| `ORACLE_METRICS_ADDR` | `:2112` (local-run override only) | diff --git a/harnesses/oracle-deviation/cmd/script/chainlink.go b/harnesses/oracle-deviation/cmd/script/chainlink.go index fdd292de..f8648564 100644 --- a/harnesses/oracle-deviation/cmd/script/chainlink.go +++ b/harnesses/oracle-deviation/cmd/script/chainlink.go @@ -66,9 +66,11 @@ type rpcResp struct { type chainlinkPoller struct { httpClient *http.Client - // rotated[i] is the index of the next RPC to try; on consecutive - // errors the poller flips to the fallback. Cheap atomic instead - // of a mutex since the field is only read/written by this poller. + // rpcs is the ordered endpoint list for the chain this poller + // reads from. rpcIdx rotates through it on transient errors. + // Cheap atomic instead of a mutex since the field is only + // read/written by this poller. + rpcs []string rpcIdx atomic.Int32 // decimals cache (feed → decimals). Populated lazily on first // successful read. @@ -80,9 +82,10 @@ type chainlinkPoller struct { failureCount map[string]int } -func newChainlinkPoller() *chainlinkPoller { +func newChainlinkPoller(rpcs []string) *chainlinkPoller { return &chainlinkPoller{ httpClient: &http.Client{Timeout: httpTimeout}, + rpcs: rpcs, decimalsCache: make(map[string]int), unsupported: make(map[string]bool), failureCount: make(map[string]int), @@ -90,10 +93,11 @@ func newChainlinkPoller() *chainlinkPoller { } func (c *chainlinkPoller) rpcURL() string { - if c.rpcIdx.Load()%2 == 0 { - return rpcEndpoint() + idx := int(c.rpcIdx.Load()) % len(c.rpcs) + if idx < 0 { + idx = -idx } - return rpcEndpointFallback() + return c.rpcs[idx] } // runChainlink loops every pair every pollInterval. The 10 pairs are @@ -101,7 +105,7 @@ func (c *chainlinkPoller) rpcURL() string { // (one batched request would be ideal — public RPCs often refuse // JSON-RPC batches, so we stick to one call per pair). func runChainlink(ctx context.Context, specs []PairSpec) { - c := newChainlinkPoller() + c := newChainlinkPoller([]string{rpcEndpoint(), rpcEndpointFallback()}) t := time.NewTicker(pollInterval) defer t.Stop() @@ -118,8 +122,12 @@ func runChainlink(ctx context.Context, specs []PairSpec) { } if c.unsupported[s.ChainlinkFeed] { // Mark error every cycle so the deviation calc skips - // stale values, but skip the network round-trip. + // stale values, but skip the network round-trip. Also + // signal the freshness error counter (mirrors + // runChainlinkChains) so a dead feed's climbing + // staleness gauge is distinguishable from a quiet feed. oracleScrapeErrors.WithLabelValues(string(SourceChainlink), string(s.Pair)).Inc() + freshnessScrapeErrors.WithLabelValues(string(SourceChainlink), string(s.Pair), ChainEthereum).Inc() continue } pollCtx, cancel := context.WithTimeout(ctx, httpTimeout*2) @@ -127,6 +135,7 @@ func runChainlink(ctx context.Context, specs []PairSpec) { cancel() if err != nil { oracleScrapeErrors.WithLabelValues(string(SourceChainlink), string(s.Pair)).Inc() + freshnessScrapeErrors.WithLabelValues(string(SourceChainlink), string(s.Pair), ChainEthereum).Inc() c.failureCount[s.ChainlinkFeed]++ // Distinguish reverts (permanent — bad address / // non-AggregatorV3 contract) from transient HTTP @@ -150,6 +159,10 @@ func runChainlink(ctx context.Context, specs []PairSpec) { // values (which would tag Chainlink's heartbeat lag as // "deviation" — that artifact is exactly what Fix C kills). recordPriceAt(SourceChainlink, s.Pair, price, time.Unix(updatedAt, 0)) + // Feed the № 082 freshness tracker with the same on-chain + // updatedAt (chain=ethereum). Freshness for other chains + // comes from runChainlinkChains. + recordFreshness(string(SourceChainlink), s.Pair, ChainEthereum, time.Unix(updatedAt, 0)) ageS := time.Since(time.Unix(updatedAt, 0)).Seconds() if ageS < 0 { ageS = 0 @@ -169,6 +182,62 @@ func runChainlink(ctx context.Context, specs []PairSpec) { } } +// runChainlinkChains polls the non-Ethereum Chainlink deployments +// (bench № 082, freshness only). Same read path as the mainnet +// poller, one dedicated chainlinkPoller per chain so the RPC rotation +// and decimals cache stay chain-local. Successes feed ONLY the +// freshness tracker: these prices never enter the 025 deviation store +// (see ChainFeed doc in config.go). Errors go to the freshness error +// counter, never to ocb_oracle_scrape_errors_total. +func runChainlinkChains(ctx context.Context, feeds []ChainFeed) { + pollers := make(map[string]*chainlinkPoller) + for _, f := range feeds { + if _, ok := pollers[f.Chain]; !ok { + pollers[f.Chain] = newChainlinkPoller(f.RPCs) + } + } + t := time.NewTicker(pollInterval) + defer t.Stop() + + tick := func() { + for _, f := range feeds { + c := pollers[f.Chain] + if c.unsupported[f.Feed] { + freshnessScrapeErrors.WithLabelValues(string(SourceChainlink), string(f.Pair), f.Chain).Inc() + continue + } + pollCtx, cancel := context.WithTimeout(ctx, httpTimeout*2) + price, updatedAt, err := c.read(pollCtx, f.Feed) + cancel() + if err != nil || price <= 0 { + freshnessScrapeErrors.WithLabelValues(string(SourceChainlink), string(f.Pair), f.Chain).Inc() + c.failureCount[f.Feed]++ + if isRevert(err) && c.failureCount[f.Feed] >= 3 { + c.unsupported[f.Feed] = true + fmt.Printf("[chainlink-%s/%s] feed %s marked unsupported after 3 reverts\n", f.Chain, f.Pair, f.Feed) + } + if err != nil && !isRevert(err) { + c.rpcIdx.Add(1) + } + fmt.Printf("[chainlink-%s/%s] err: %v (price=%v)\n", f.Chain, f.Pair, err, price) + continue + } + c.failureCount[f.Feed] = 0 + recordFreshness(string(SourceChainlink), f.Pair, f.Chain, time.Unix(updatedAt, 0)) + } + } + + tick() + for { + select { + case <-ctx.Done(): + return + case <-t.C: + tick() + } + } +} + // read returns (price USD, updatedAt unix seconds, err). func (c *chainlinkPoller) read(ctx context.Context, feed string) (float64, int64, error) { // Fetch decimals first (cached). diff --git a/harnesses/oracle-deviation/cmd/script/config.go b/harnesses/oracle-deviation/cmd/script/config.go index db0f23b6..628e5b6a 100644 --- a/harnesses/oracle-deviation/cmd/script/config.go +++ b/harnesses/oracle-deviation/cmd/script/config.go @@ -83,6 +83,71 @@ func pairs() []PairSpec { } } +// ChainFeed wires one Chainlink AggregatorV3 deployment on a non- +// Ethereum chain to the freshness tracker (bench № 082). These feeds +// are freshness-only: they never enter the 025 deviation store (the +// price store is keyed by (pair, source), so an Arbitrum ETH/USD +// sample would silently overwrite the Ethereum one and change 025's +// deviation numbers). +type ChainFeed struct { + Pair Pair + Chain string // freshness `chain` label: arbitrum | base + Feed string // AggregatorV3 contract on that chain + RPCs []string +} + +// extraChainlinkFeeds lists the non-Ethereum Chainlink deployments the +// freshness bench tracks. Ethereum mainnet feeds are covered by the +// existing 025 poller (which now also feeds recordFreshness). +func extraChainlinkFeeds() []ChainFeed { + return []ChainFeed{ + { + Pair: "ETH/USD", + Chain: ChainArbitrum, + Feed: "0x639Fe6ab55C921f74e7fac1ee960C0B6293ba612", + RPCs: []string{ + envDefault("ORACLE_RPC_ARBITRUM", "https://arbitrum-one-rpc.publicnode.com"), + "https://arb1.arbitrum.io/rpc", + }, + }, + { + Pair: "ETH/USD", + Chain: ChainBase, + Feed: "0x71041dddad3595F9CEd3DcCFBe3D1F4b0a16Bb70", + RPCs: []string{ + envDefault("ORACLE_RPC_BASE", "https://base-rpc.publicnode.com"), + "https://mainnet.base.org", + }, + }, + } +} + +// RedstoneFeed maps a canonical OCB pair to RedStone's symbol on the +// public per-symbol price API. +type RedstoneFeed struct { + Pair Pair + Symbol string +} + +// redstoneFeeds is the freshness cohort for RedStone. ETH + BTC only +// in v1: the per-symbol endpoint costs one request per symbol per +// tick, and the bench's headline is ETH/USD. +func redstoneFeeds() []RedstoneFeed { + return []RedstoneFeed{ + {Pair: "ETH/USD", Symbol: "ETH"}, + {Pair: "BTC/USD", Symbol: "BTC"}, + } +} + +// redstoneBaseURL is RedStone's public per-symbol price API. The +// data-packages gateway payload for the whole redstone-primary-prod +// service is ~1.7 MB per request; the per-symbol endpoint returns a +// single small object with the signed package's value + timestamp, +// so we poll per symbol instead. +func redstoneBaseURL() string { + return envDefault("ORACLE_REDSTONE_URL", "https://api.redstone.finance/prices") +} + // rpcEndpoint is the Ethereum mainnet public RPC used by the // Chainlink poller to call latestRoundData() on the AggregatorV3 // contracts. Two free public endpoints are kept; the second is a diff --git a/harnesses/oracle-deviation/cmd/script/freshness.go b/harnesses/oracle-deviation/cmd/script/freshness.go new file mode 100644 index 00000000..1713f026 --- /dev/null +++ b/harnesses/oracle-deviation/cmd/script/freshness.go @@ -0,0 +1,176 @@ +package main + +import ( + "context" + "math" + "sync" + "time" +) + +// Freshness tracking for OCB bench № 082 (oracle-freshness). +// +// Every freshness-capable poller calls recordFreshness with the +// source-declared update timestamp: +// +// - Chainlink: latestRoundData().updatedAt, per chain (ethereum, +// arbitrum, base). Push oracle: staleness in the multiple hundreds +// of seconds is EXPECTED in calm markets (deviation trigger + +// heartbeat mechanics), not a failure. +// - Pyth: Hermes publish_time. Pull oracle: Hermes IS the price +// source integrators pull from, so publish_time is the honest +// freshness measure (typically 1-2s). +// - RedStone: the signed data-package timestamp from the public +// per-symbol API (typically 15-25s; packages are produced on a +// ~10s cadence). +// +// Because "old" is not the same as "wrong" for a push oracle, the +// bench pairs raw staleness with a second signal: stale_but_moved, +// which only fires when the feed is old AND the market has left it +// behind. Thresholds: +// +// staleThresholdSeconds (300s): half of Chainlink's typical 1h +// heartbeat divided by 6 is arbitrary; 300s was picked because +// every observed healthy update gap on the measured feeds +// (deviation-triggered) lands under it EXCEPT genuinely quiet +// periods, and because 5 minutes of price lag is where perp/ +// lending liquidation math starts to hurt. +// staleMoveThresholdPct (0.5%): matches the widest deviation +// trigger configured on the measured Chainlink feeds. If the CEX +// print moved more than the feed's own trigger and the feed still +// hasn't updated, the feed is late by its own standard. +const ( + staleThresholdSeconds = 300.0 + staleMoveThresholdPct = 0.5 + // freshnessTickInterval refreshes the staleness gauges between + // polls so the gauge grows monotonically instead of stair-stepping + // on the 30s poll cadence. + freshnessTickInterval = 5 * time.Second +) + +// chain label values. For Chainlink the label is the chain the +// aggregator contract lives on. Pyth and RedStone are not read from a +// chain at all (pull-model oracles); their freshness source is named +// instead. +const ( + ChainEthereum = "ethereum" + ChainArbitrum = "arbitrum" + ChainBase = "base" + ChainHermes = "hermes" // Pyth Hermes publish_time + ChainGateway = "gateway" // RedStone public data gateway +) + +type freshKey struct { + oracle string + pair Pair + chain string +} + +type freshState struct { + // lastUpdate is the source-declared timestamp of the feed's most + // recent update (NOT our fetch time). + lastUpdate time.Time + // cexAtUpdate is the CEX reference price snapshotted when we first + // observed this update. 0 when no fresh CEX sample was available. + // Cold-start caveat: for a feed whose current round predates the + // harness boot, the snapshot is taken at boot, not at the round's + // true landing time; it converges on the first real update event. + cexAtUpdate float64 +} + +var ( + freshMu sync.Mutex + fresh = make(map[freshKey]freshState) +) + +// recordFreshness ingests one observation of a feed's own update +// timestamp. Increments the update-events counter when the timestamp +// moved forward vs the previous observation, snapshots the CEX +// reference at that moment, then republishes the gauges. +func recordFreshness(oracle string, pair Pair, chain string, sourceTS time.Time) { + if sourceTS.Unix() <= 0 { + // A zero/absent timestamp would read as ~56 years of staleness. + freshnessScrapeErrors.WithLabelValues(oracle, string(pair), chain).Inc() + return + } + key := freshKey{oracle: oracle, pair: pair, chain: chain} + freshMu.Lock() + st, seen := fresh[key] + if !seen || sourceTS.After(st.lastUpdate) { + if seen { + oracleUpdateEvents.WithLabelValues(oracle, string(pair), chain).Inc() + } + ref, _ := cexRefPrice(pair) + st = freshState{lastUpdate: sourceTS, cexAtUpdate: ref} + fresh[key] = st + } + freshMu.Unlock() + publishFreshness(key, st) +} + +// publishFreshness sets the staleness + stale_but_moved gauges for one +// (oracle, pair, chain) from its stored state. +func publishFreshness(key freshKey, st freshState) { + stale := time.Since(st.lastUpdate).Seconds() + if stale < 0 { + // Source clock marginally ahead of ours (Hermes publish_time + // can lead by sub-second). Clamp instead of publishing a + // negative age. + stale = 0 + } + oracleStalenessSeconds.WithLabelValues(key.oracle, string(key.pair), key.chain).Set(stale) + + moved := 0.0 + if stale > staleThresholdSeconds && st.cexAtUpdate > 0 { + if cur, ok := cexRefPrice(key.pair); ok { + movePct := math.Abs(cur-st.cexAtUpdate) / st.cexAtUpdate * 100 + if movePct > staleMoveThresholdPct { + moved = 1 + } + } + } + oracleStaleButMoved.WithLabelValues(key.oracle, string(key.pair), key.chain).Set(moved) +} + +// cexRefPrice returns the freshest CEX print for a pair from the +// existing 025 price store: Binance first, Coinbase as fallback, both +// subject to the same 2x-poll-interval staleness guard the deviation +// calc uses. ok=false when neither has a fresh sample. +func cexRefPrice(pair Pair) (float64, bool) { + storeMu.RLock() + defer storeMu.RUnlock() + srcMap := store[pair] + if srcMap == nil { + return 0, false + } + for _, src := range []Source{SourceBinance, SourceCoinbase} { + if p, ok := srcMap[src]; ok && time.Since(p.TS) <= 2*pollInterval { + return p.Value, true + } + } + return 0, false +} + +// runFreshnessUpdater republishes every tracked freshness gauge on a +// short cadence so staleness keeps climbing between the 30s polls (a +// Prometheus scrape landing mid-window sees the true age, not the age +// as of the last poll). +func runFreshnessUpdater(ctx context.Context) { + t := time.NewTicker(freshnessTickInterval) + defer t.Stop() + for { + select { + case <-ctx.Done(): + return + case <-t.C: + freshMu.Lock() + snapshot := make(map[freshKey]freshState, len(fresh)) + for k, v := range fresh { + snapshot[k] = v + } + freshMu.Unlock() + for k, v := range snapshot { + publishFreshness(k, v) + } + } + } +} diff --git a/harnesses/oracle-deviation/cmd/script/main.go b/harnesses/oracle-deviation/cmd/script/main.go index c7b96cec..f3e9869c 100644 --- a/harnesses/oracle-deviation/cmd/script/main.go +++ b/harnesses/oracle-deviation/cmd/script/main.go @@ -320,6 +320,7 @@ func main() { installLogCapture() // capture stdout into /logs ring buffer fmt.Println("=== Oracle Deviation Harness ===") fmt.Println("OpenChainBench № 025 — 4 oracles × 10 pairs, max deviation gauge.") + fmt.Println("OpenChainBench № 082 — oracle freshness (chainlink eth/arb/base, pyth hermes, redstone gateway).") fmt.Println() specs := pairs() @@ -331,12 +332,16 @@ func main() { fmt.Println() fmt.Printf("RPC primary: %s\n", rpcEndpoint()) fmt.Printf("RPC fallback: %s\n", rpcEndpointFallback()) + // :2112 is the OCB convention (see top-of-file comment). The env + // override exists for LOCAL runs only, e.g. when another process + // already holds 2112 on a dev machine. Never set it in production. + metricsAddr := envDefault("ORACLE_METRICS_ADDR", ":2112") fmt.Printf("Poll cadence: %s per source per pair\n", pollInterval) - fmt.Println("Metrics server: :2112/metrics") + fmt.Printf("Metrics server: %s/metrics\n", metricsAddr) fmt.Println() go func() { - if err := StartMetricsServer(":2112"); err != nil { + if err := StartMetricsServer(metricsAddr); err != nil { fmt.Printf("[fatal] metrics server: %v\n", err) os.Exit(1) } @@ -351,6 +356,12 @@ func main() { go runCoinbase(ctx, specs) go runLatencyUpdater(ctx) + // № 082 freshness-only pollers. Kept out of the deviation store by + // construction (see ChainFeed / runRedstone docs). + go runChainlinkChains(ctx, extraChainlinkFeeds()) + go runRedstone(ctx, redstoneFeeds()) + go runFreshnessUpdater(ctx) + sig := make(chan os.Signal, 1) signal.Notify(sig, os.Interrupt, syscall.SIGTERM) s := <-sig diff --git a/harnesses/oracle-deviation/cmd/script/metrics.go b/harnesses/oracle-deviation/cmd/script/metrics.go index 9b47ca1b..b56f4ff3 100644 --- a/harnesses/oracle-deviation/cmd/script/metrics.go +++ b/harnesses/oracle-deviation/cmd/script/metrics.go @@ -106,6 +106,59 @@ var ( }, []string{"source", "pair"}, ) + + // --- Bench № 082 (oracle-freshness) metric families ------------- + // + // Deliberately NEW families instead of adding a `chain` label to + // the existing ones: bench 025's PromQL surface selects on + // {source, pair} and aggregates with count by (source); injecting + // a chain label there would split its series and silently change + // its numbers. Freshness gets its own label set: + // + // oracle: chainlink | pyth | redstone + // pair: canonical OCB pair (ETH/USD, BTC/USD, ...) + // chain: where the freshness timestamp lives. For Chainlink + // this is the chain the aggregator contract is deployed + // on (ethereum | arbitrum | base). Pyth is pull-based so + // its authoritative timestamp is the Hermes publish_time + // (chain="hermes"); RedStone's is the signed data-package + // timestamp from its public gateway (chain="gateway"). + + oracleStalenessSeconds = promauto.NewGaugeVec( + prometheus.GaugeOpts{ + Name: "ocb_oracle_staleness_seconds", + Help: "Seconds between now and the oracle's own last-update timestamp (Chainlink latestRoundData.updatedAt per chain, Pyth Hermes publish_time, RedStone data-package timestamp). Refreshed every 5s between polls so staleness grows monotonically until the next observed update.", + }, + []string{"oracle", "pair", "chain"}, + ) + + oracleUpdateEvents = promauto.NewCounterVec( + prometheus.CounterOpts{ + Name: "ocb_oracle_update_events_total", + Help: "Count of observed oracle updates: increments each time the source-declared update timestamp moves forward. Bounded below by reality and above by the 30s poll cadence (two updates inside one poll window count once).", + }, + []string{"oracle", "pair", "chain"}, + ) + + oracleStaleButMoved = promauto.NewGaugeVec( + prometheus.GaugeOpts{ + Name: "ocb_oracle_stale_but_moved", + Help: "1 when the feed is older than 300s AND the CEX reference price (Binance, Coinbase fallback) has moved more than 0.5% since the update was observed, else 0. Staleness alone is normal for deviation-triggered feeds; stale AND moved is the dangerous state.", + }, + []string{"oracle", "pair", "chain"}, + ) + + // Separate error counter for the freshness pollers. NOT merged into + // ocb_oracle_scrape_errors_total because bench 025's success queries + // sum that counter across all sources per pair; redstone / L2 + // chainlink failures must not depress 025's success rate. + freshnessScrapeErrors = promauto.NewCounterVec( + prometheus.CounterOpts{ + Name: "ocb_oracle_freshness_scrape_errors_total", + Help: "Failed freshness scrape attempts per (oracle, pair, chain). Kept separate from ocb_oracle_scrape_errors_total so bench 025's error-rate queries are unaffected by freshness-only sources.", + }, + []string{"oracle", "pair", "chain"}, + ) ) // StartMetricsServer binds /metrics + /health on addr. Blocking call — diff --git a/harnesses/oracle-deviation/cmd/script/pyth.go b/harnesses/oracle-deviation/cmd/script/pyth.go index ac4fd163..20e27206 100644 --- a/harnesses/oracle-deviation/cmd/script/pyth.go +++ b/harnesses/oracle-deviation/cmd/script/pyth.go @@ -56,6 +56,7 @@ func runPyth(ctx context.Context, specs []PairSpec) { if err != nil { for _, s := range specs { oracleScrapeErrors.WithLabelValues(string(SourcePyth), string(s.Pair)).Inc() + freshnessScrapeErrors.WithLabelValues(string(SourcePyth), string(s.Pair), ChainHermes).Inc() } fmt.Printf("[pyth] http err: %v\n", err) return @@ -65,6 +66,7 @@ func runPyth(ctx context.Context, specs []PairSpec) { if resp.StatusCode != 200 { for _, s := range specs { oracleScrapeErrors.WithLabelValues(string(SourcePyth), string(s.Pair)).Inc() + freshnessScrapeErrors.WithLabelValues(string(SourcePyth), string(s.Pair), ChainHermes).Inc() } fmt.Printf("[pyth] http %d: %s\n", resp.StatusCode, string(body)) return @@ -73,6 +75,7 @@ func runPyth(ctx context.Context, specs []PairSpec) { if err := json.Unmarshal(body, &entries); err != nil { for _, s := range specs { oracleScrapeErrors.WithLabelValues(string(SourcePyth), string(s.Pair)).Inc() + freshnessScrapeErrors.WithLabelValues(string(SourcePyth), string(s.Pair), ChainHermes).Inc() } fmt.Printf("[pyth] decode err: %v\n", err) return @@ -86,9 +89,24 @@ func runPyth(ctx context.Context, specs []PairSpec) { price, err := pythToFloat(e.Price.Price, e.Price.Expo) if err != nil { oracleScrapeErrors.WithLabelValues(string(SourcePyth), string(pair)).Inc() + freshnessScrapeErrors.WithLabelValues(string(SourcePyth), string(pair), ChainHermes).Inc() continue } - recordPrice(SourcePyth, pair, price) + // Bug fix: Hermes attaches publish_time to every price and + // we used to discard it (recordPrice stamped fetch time as + // SourceTS). Pyth is a pull oracle, so publish_time IS the + // authoritative freshness of the price integrators pull. + // Use it as SourceTS (typically 1-2s behind fetch, well + // inside the 10s alignment tolerance, so the 025 aligned + // deviation is unaffected in practice) and feed the № 082 + // freshness tracker from it. + if e.Price.PublishTime > 0 { + srcTS := time.Unix(e.Price.PublishTime, 0) + recordPriceAt(SourcePyth, pair, price, srcTS) + recordFreshness(string(SourcePyth), pair, ChainHermes, srcTS) + } else { + recordPrice(SourcePyth, pair, price) + } got[pair] = true } // Mark missing pairs as errored so the deviation map doesn't @@ -96,6 +114,7 @@ func runPyth(ctx context.Context, specs []PairSpec) { for _, s := range specs { if !got[s.Pair] { oracleScrapeErrors.WithLabelValues(string(SourcePyth), string(s.Pair)).Inc() + freshnessScrapeErrors.WithLabelValues(string(SourcePyth), string(s.Pair), ChainHermes).Inc() } } } diff --git a/harnesses/oracle-deviation/cmd/script/redstone.go b/harnesses/oracle-deviation/cmd/script/redstone.go new file mode 100644 index 00000000..a91930fb --- /dev/null +++ b/harnesses/oracle-deviation/cmd/script/redstone.go @@ -0,0 +1,106 @@ +package main + +import ( + "context" + "encoding/json" + "fmt" + "io" + "net/http" + "net/url" + "time" +) + +// RedStone freshness poller (bench № 082 only). RedStone is a +// modular oracle: signed data packages are produced off-chain on a +// ~10s cadence and delivered on-chain either by relayers (push +// model) or attached to transactions by the integrator (pull model, +// "core"). The freshest public view of the primary production +// service is the keyless per-symbol price API, which returns the +// latest signed package's value and timestamp. We deliberately avoid +// the data-packages gateway's full-service payload (~1.7 MB per +// request for redstone-primary-prod) and poll one small per-symbol +// object per pair instead. +// +// RedStone samples feed ONLY the freshness metrics. They never enter +// the 025 deviation store: bench 025 is a fixed 4-source matrix +// (chainlink/pyth/binance/coinbase) and its queries count sources per +// pair, so adding a 5th source would silently change its numbers. + +type redstonePrice struct { + Symbol string `json:"symbol"` + Value float64 `json:"value"` + Timestamp int64 `json:"timestamp"` // unix milliseconds +} + +func runRedstone(ctx context.Context, feeds []RedstoneFeed) { + client := &http.Client{Timeout: httpTimeout} + t := time.NewTicker(pollInterval) + defer t.Stop() + + tick := func() { + for _, f := range feeds { + pollCtx, cancel := context.WithTimeout(ctx, httpTimeout) + ts, err := fetchRedstone(pollCtx, client, f.Symbol) + cancel() + if err != nil { + freshnessScrapeErrors.WithLabelValues("redstone", string(f.Pair), ChainGateway).Inc() + fmt.Printf("[redstone/%s] err: %v\n", f.Pair, err) + continue + } + recordFreshness("redstone", f.Pair, ChainGateway, ts) + } + } + + tick() + for { + select { + case <-ctx.Done(): + return + case <-t.C: + tick() + } + } +} + +// fetchRedstone returns the timestamp of the latest signed data +// package for one symbol. GET /prices?symbol=&provider=redstone&limit=1 +// answers with a one-element array; timestamp is unix ms. +func fetchRedstone(ctx context.Context, client *http.Client, symbol string) (time.Time, error) { + u := redstoneBaseURL() + "?symbol=" + url.QueryEscape(symbol) + "&provider=redstone&limit=1" + req, err := http.NewRequestWithContext(ctx, "GET", u, nil) + if err != nil { + return time.Time{}, err + } + req.Header.Set("User-Agent", "openchainbench-oracle-freshness/1.0") + resp, err := client.Do(req) + if err != nil { + return time.Time{}, err + } + defer resp.Body.Close() + body, _ := io.ReadAll(resp.Body) + if resp.StatusCode != 200 { + return time.Time{}, fmt.Errorf("http %d: %s", resp.StatusCode, truncate(string(body), 200)) + } + var arr []redstonePrice + if err := json.Unmarshal(body, &arr); err != nil { + return time.Time{}, fmt.Errorf("decode: %w", err) + } + if len(arr) == 0 { + return time.Time{}, fmt.Errorf("empty result for %s", symbol) + } + p := arr[0] + if p.Value <= 0 { + return time.Time{}, fmt.Errorf("non-positive value %v for %s", p.Value, symbol) + } + if p.Timestamp <= 0 { + return time.Time{}, fmt.Errorf("missing timestamp for %s", symbol) + } + return time.UnixMilli(p.Timestamp), nil +} + +func truncate(s string, n int) string { + if len(s) <= n { + return s + } + return s[:n] + "…" +} diff --git a/harnesses/robinhood-subsidy/Dockerfile b/harnesses/robinhood-subsidy/Dockerfile new file mode 100644 index 00000000..63108cfc --- /dev/null +++ b/harnesses/robinhood-subsidy/Dockerfile @@ -0,0 +1,22 @@ +FROM golang:1.24-alpine AS builder + +WORKDIR /app +RUN apk add --no-cache git + +COPY go.mod go.sum ./ +RUN go mod download + +COPY . . + +RUN CGO_ENABLED=0 GOOS=linux go build -o /app/monitor ./cmd/script + +FROM debian:bookworm-slim + +WORKDIR /app +RUN apt-get update && apt-get install -y ca-certificates && rm -rf /var/lib/apt/lists/* + +COPY --from=builder /app/monitor /app/monitor + +EXPOSE 2112 + +CMD ["/app/monitor"] diff --git a/harnesses/robinhood-subsidy/cmd/script/loghub.go b/harnesses/robinhood-subsidy/cmd/script/loghub.go new file mode 100644 index 00000000..9dd74488 --- /dev/null +++ b/harnesses/robinhood-subsidy/cmd/script/loghub.go @@ -0,0 +1,114 @@ +package main + +import ( + "bufio" + "fmt" + "io" + "net/http" + "os" + "strconv" + "sync" + "time" +) + +// Auto-generated by the loghub inline pattern. Captures stdout/stderr into a +// bounded ring buffer and exposes GET /logs?tail=N protected by X-Logs-Token +// matching the LOGS_TOKEN env var. +// +// Keep in sync across miniapps (was previously the shared/loghub package; we +// inline because Railway's per-harness Docker build context can't reach a +// sibling shared module via go.mod replace). + +const logRingMax = 5000 + +type logRing struct { + mu sync.Mutex + lines []string + max int +} + +var globalLogRing = &logRing{max: logRingMax} + +func (b *logRing) push(line string) { + entry := time.Now().UTC().Format("2006-01-02T15:04:05.000Z") + " " + line + b.mu.Lock() + if len(b.lines) >= b.max { + b.lines = append(b.lines[1:], entry) + } else { + b.lines = append(b.lines, entry) + } + b.mu.Unlock() +} + +func (b *logRing) snapshot(tail int) []string { + b.mu.Lock() + defer b.mu.Unlock() + if tail <= 0 || tail >= len(b.lines) { + out := make([]string, len(b.lines)) + copy(out, b.lines) + return out + } + start := len(b.lines) - tail + out := make([]string, tail) + copy(out, b.lines[start:]) + return out +} + +var logSetupOnce sync.Once + +// installLogCapture replaces os.Stdout (and os.Stderr) with the write-end of a +// pipe, then spawns a goroutine that fan-outs every line to the original +// stdout AND the in-memory ring buffer. Call exactly once, very early in +// main(). +func installLogCapture() { logSetupOnce.Do(doInstallLogCapture) } + +func doInstallLogCapture() { + originalStdout := os.Stdout + originalStderr := os.Stderr + r, w, err := os.Pipe() + if err != nil { + fmt.Fprintf(originalStdout, "[loghub] pipe failed: %v (/logs will be empty)\n", err) + return + } + os.Stdout = w + os.Stderr = w + + go func() { + scanner := bufio.NewScanner(r) + buf := make([]byte, 0, 1024*1024) + scanner.Buffer(buf, 1024*1024) + for scanner.Scan() { + line := scanner.Text() + fmt.Fprintln(originalStdout, line) + globalLogRing.push(line) + } + _, _ = io.Copy(originalStdout, r) + _ = originalStderr + }() +} + +// logsHandler returns an http.Handler for GET /logs?tail=N. Requires header +// X-Logs-Token to match the LOGS_TOKEN env var. Returns 403 if env unset. +func logsHandler() http.Handler { + return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + expected := os.Getenv("LOGS_TOKEN") + if expected == "" { + http.Error(w, "logs disabled: LOGS_TOKEN unset", http.StatusForbidden) + return + } + if r.Header.Get("X-Logs-Token") != expected { + http.Error(w, "forbidden", http.StatusForbidden) + return + } + tail := 500 + if t := r.URL.Query().Get("tail"); t != "" { + if n, err := strconv.Atoi(t); err == nil && n > 0 { + tail = n + } + } + w.Header().Set("Content-Type", "text/plain; charset=utf-8") + for _, l := range globalLogRing.snapshot(tail) { + fmt.Fprintln(w, l) + } + }) +} diff --git a/harnesses/robinhood-subsidy/cmd/script/main.go b/harnesses/robinhood-subsidy/cmd/script/main.go new file mode 100644 index 00000000..6169fbdb --- /dev/null +++ b/harnesses/robinhood-subsidy/cmd/script/main.go @@ -0,0 +1,315 @@ +package main + +import ( + "encoding/json" + "fmt" + "io" + "math" + "net/http" + "os" + "strconv" + "strings" + "time" + + "github.com/prometheus/client_golang/prometheus" + "github.com/prometheus/client_golang/prometheus/promauto" + "github.com/prometheus/client_golang/prometheus/promhttp" +) + +// Robinhood Chain gas subsidy tracker. +// +// Robinhood Chain launched July 1, 2026 with a 90-day promotion where +// users pay $0 in gas because Robinhood covers the sequencer cost. +// The window closes September 29, 2026 (public wording: "cover gas +// fees for eligible users for the first 90 days"; math from mainnet +// launch confirms end-of-Q3). +// +// The bench publishes two things live: +// +// 1. Countdown: days remaining until the subsidy ends, and the +// fraction of the 90-day window already elapsed. +// 2. Estimated chain-side gas cost that Robinhood is paying to the +// sequencer since launch. Deliberately labeled "chain-side" and +// not "user fees": growthepie's fees_paid_by_users is ~$0 +// during the subsidy (because users literally pay $0), so a +// number in the $millions would contradict them if mislabeled. +// This bench measures what Robinhood, not the user, is paying. +// +// Data leg: Blockscout's /api/v2/stats/charts/transactions (daily +// tx count from launch, keyless) + /api/v2/stats (today's gas_used, +// current gas_prices tier, ETH price). The historical gas-used chart +// doesn't exist on this Blockscout instance (verified 400 on +// /api/v2/stats/charts/gas-used), so per-day gas is estimated from +// daily tx count multiplied by the current per-tx gas draw +// (gas_used_today / transactions_today), a stable ratio on an Orbit +// with a fixed-cost transaction mix. + +const ( + launchDate = "2026-07-01T00:00:00Z" + subsidyEndDate = "2026-09-29T23:59:59Z" + + pollInterval = 1 * time.Hour + httpTimeout = 20 * time.Second + + blockscoutBase = "https://robinhoodchain.blockscout.com" + robinhoodRPC = "https://rpc.mainnet.chain.robinhood.com" +) + +var ( + daysRemaining = promauto.NewGauge(prometheus.GaugeOpts{ + Name: "robinhood_subsidy_days_remaining", + Help: "Days until the 90-day gas subsidy window ends on 2026-09-29 (negative once expired).", + }) + + windowFractionElapsed = promauto.NewGauge(prometheus.GaugeOpts{ + Name: "robinhood_subsidy_window_fraction_elapsed", + Help: "Share of the 90-day subsidy window elapsed (0 to 1).", + }) + + chainSideCostTotalUSD = promauto.NewGauge(prometheus.GaugeOpts{ + Name: "robinhood_subsidy_chainside_cost_total_usd", + Help: "Estimated cumulative USD chain-side gas cost since launch, i.e. what Robinhood is paying the sequencer while users pay $0. NOT growthepie's fees_paid_by_users (~$0 during the subsidy by design).", + }) + + chainSideCostTodayUSD = promauto.NewGauge(prometheus.GaugeOpts{ + Name: "robinhood_subsidy_chainside_cost_today_usd", + Help: "Today's estimated chain-side gas cost (gas_used_today x average base fee x ETH price).", + }) + + projectedTotalUSD = promauto.NewGauge(prometheus.GaugeOpts{ + Name: "robinhood_subsidy_projected_total_usd", + Help: "Linear projection: cumulative_cost x (90 / days_elapsed). Where chain-side gas cost lands if today's daily rate holds through Sep 29.", + }) + + dailyTxCountGauge = promauto.NewGaugeVec(prometheus.GaugeOpts{ + Name: "robinhood_subsidy_daily_tx_count", + Help: "Daily transaction count on Robinhood Chain since launch (Blockscout charts/transactions).", + }, []string{"date"}) + + baseFeeGweiGauge = promauto.NewGauge(prometheus.GaugeOpts{ + Name: "robinhood_subsidy_base_fee_gwei", + Help: "Current base fee on Robinhood Chain in gwei (eth_gasPrice / 1e9).", + }) + + ethPriceGauge = promauto.NewGauge(prometheus.GaugeOpts{ + Name: "robinhood_subsidy_eth_price_usd", + Help: "ETH USD spot as reported by the chain's Blockscout instance.", + }) + + sourceCall = promauto.NewCounterVec(prometheus.CounterOpts{ + Name: "robinhood_subsidy_source_call_total", + Help: "Fetch outcomes per source.", + }, []string{"source", "result"}) +) + +func envDefault(k, def string) string { + if v := strings.TrimSpace(os.Getenv(k)); v != "" { + return v + } + return def +} + +func main() { + installLogCapture() + fmt.Println("=== Robinhood Chain Gas Subsidy Tracker ===") + fmt.Printf("Launch: %s | Subsidy ends: %s\n", launchDate, subsidyEndDate) + + go func() { + mux := http.NewServeMux() + mux.Handle("/metrics", promhttp.Handler()) + mux.Handle("/logs", logsHandler()) + mux.HandleFunc("/health", func(w http.ResponseWriter, _ *http.Request) { + w.WriteHeader(http.StatusOK) + _, _ = w.Write([]byte("ok")) + }) + if err := http.ListenAndServe(envDefault("LISTEN_ADDR", ":2112"), mux); err != nil { + fmt.Printf("[fatal] metrics server: %v\n", err) + } + }() + + client := &http.Client{Timeout: httpTimeout} + tick := func() { + now := time.Now().UTC() + launch, _ := time.Parse(time.RFC3339, launchDate) + end, _ := time.Parse(time.RFC3339, subsidyEndDate) + + daysLeft := end.Sub(now).Hours() / 24 + daysRemaining.Set(daysLeft) + totalWindow := end.Sub(launch).Hours() / 24 + elapsed := now.Sub(launch).Hours() / 24 + frac := math.Max(0, math.Min(1, elapsed/totalWindow)) + windowFractionElapsed.Set(frac) + + baseFeeGwei, ok := fetchBaseFeeGwei(client) + if !ok { + // Orbit floor observed since launch: ~0.05 gwei. Safe fallback + // so a transient RPC blip doesn't null the burn number. + baseFeeGwei = 0.05 + } + baseFeeGweiGauge.Set(baseFeeGwei) + + stats, ok := fetchBlockscoutStats(client) + if !ok || stats.CoinPrice <= 0 { + return + } + ethPriceGauge.Set(stats.CoinPrice) + + daily := fetchTransactionsChart(client) + if daily == nil { + return + } + + // Per-tx gas draw derived from today's Blockscout stats. The + // mix of tokenized-stock swaps + native transfers is stable + // on this chain, so gas_used_today / transactions_today is a + // clean proxy for historical days. + perTxGas := 0.0 + if stats.TxsToday > 0 { + perTxGas = float64(stats.GasUsedToday) / float64(stats.TxsToday) + } + if perTxGas <= 0 { + perTxGas = 150_000 // conservative fallback if stats missed + } + + // Cumulative cost: sum(daily_tx × per_tx_gas × base_fee × ETH). + total := 0.0 + for _, d := range daily { + gasUnits := float64(d.value) * perTxGas + eth := gasUnits * baseFeeGwei / 1e9 + total += eth * stats.CoinPrice + dailyTxCountGauge.WithLabelValues(d.date).Set(float64(d.value)) + } + chainSideCostTotalUSD.Set(total) + + todayCost := float64(stats.GasUsedToday) * baseFeeGwei / 1e9 * stats.CoinPrice + chainSideCostTodayUSD.Set(todayCost) + + if elapsed > 0.5 { + projectedTotalUSD.Set(total * (totalWindow / elapsed)) + } + + fmt.Printf("[tick] days_left=%.1f cost_total=$%.0f today=$%.0f eth=$%.2f base_fee=%.4fgwei per_tx_gas=%.0f\n", + daysLeft, total, todayCost, stats.CoinPrice, baseFeeGwei, perTxGas) + } + + tick() + t := time.NewTicker(pollInterval) + defer t.Stop() + for range t.C { + tick() + } +} + +// fetchBaseFeeGwei calls eth_gasPrice on the Robinhood RPC. +func fetchBaseFeeGwei(client *http.Client) (float64, bool) { + body := strings.NewReader(`{"jsonrpc":"2.0","id":1,"method":"eth_gasPrice","params":[]}`) + req, _ := http.NewRequest("POST", robinhoodRPC, body) + req.Header.Set("Content-Type", "application/json") + req.Header.Set("User-Agent", "OpenChainBench/1.0 (+https://openchainbench.com)") + resp, err := client.Do(req) + if err != nil { + sourceCall.WithLabelValues("rpc", "network").Inc() + return 0, false + } + defer resp.Body.Close() + raw, _ := io.ReadAll(io.LimitReader(resp.Body, 1<<16)) + if resp.StatusCode != 200 { + sourceCall.WithLabelValues("rpc", fmt.Sprintf("http_%d", resp.StatusCode)).Inc() + return 0, false + } + var env struct { + Result string `json:"result"` + } + if err := json.Unmarshal(raw, &env); err != nil || env.Result == "" { + sourceCall.WithLabelValues("rpc", "parse").Inc() + return 0, false + } + n, err := strconv.ParseUint(strings.TrimPrefix(env.Result, "0x"), 16, 64) + if err != nil { + sourceCall.WithLabelValues("rpc", "decode").Inc() + return 0, false + } + sourceCall.WithLabelValues("rpc", "ok").Inc() + return float64(n) / 1e9, true +} + +type stats struct { + CoinPrice float64 + GasUsedToday uint64 + TxsToday uint64 +} + +func fetchBlockscoutStats(client *http.Client) (stats, bool) { + raw, ok := getKeyless(client, blockscoutBase+"/api/v2/stats", "blockscout_stats") + if !ok { + return stats{}, false + } + var env struct { + GasUsedToday string `json:"gas_used_today"` + CoinPrice string `json:"coin_price"` + TransactionsToday string `json:"transactions_today"` + } + if err := json.Unmarshal(raw, &env); err != nil { + sourceCall.WithLabelValues("blockscout_stats", "parse").Inc() + return stats{}, false + } + price, _ := strconv.ParseFloat(env.CoinPrice, 64) + gas, _ := strconv.ParseUint(env.GasUsedToday, 10, 64) + txs, _ := strconv.ParseUint(env.TransactionsToday, 10, 64) + return stats{CoinPrice: price, GasUsedToday: gas, TxsToday: txs}, true +} + +type dailyPoint struct { + date string + value uint64 +} + +// fetchTransactionsChart is Blockscout's chart of daily tx count. +// Verified 2026-07-14 shape: {"chart_data":[{"date":"YYYY-MM-DD","transactions_count":N},...]} +func fetchTransactionsChart(client *http.Client) []dailyPoint { + raw, ok := getKeyless(client, blockscoutBase+"/api/v2/stats/charts/transactions", "blockscout_chart") + if !ok { + return nil + } + var env struct { + ChartData []struct { + Date string `json:"date"` + Count uint64 `json:"transactions_count"` + } `json:"chart_data"` + } + if err := json.Unmarshal(raw, &env); err != nil || len(env.ChartData) == 0 { + sourceCall.WithLabelValues("blockscout_chart", "parse").Inc() + return nil + } + out := make([]dailyPoint, 0, len(env.ChartData)) + for _, p := range env.ChartData { + out = append(out, dailyPoint{date: p.Date, value: p.Count}) + } + return out +} + +func getKeyless(client *http.Client, url, tag string) ([]byte, bool) { + req, err := http.NewRequest("GET", url, nil) + if err != nil { + return nil, false + } + req.Header.Set("User-Agent", "OpenChainBench/1.0 (+https://openchainbench.com)") + req.Header.Set("Accept", "application/json") + resp, err := client.Do(req) + if err != nil { + sourceCall.WithLabelValues(tag, "network").Inc() + return nil, false + } + defer resp.Body.Close() + raw, err := io.ReadAll(io.LimitReader(resp.Body, 1<<22)) + if err != nil { + sourceCall.WithLabelValues(tag, "read").Inc() + return nil, false + } + if resp.StatusCode != 200 { + sourceCall.WithLabelValues(tag, fmt.Sprintf("http_%d", resp.StatusCode)).Inc() + return nil, false + } + sourceCall.WithLabelValues(tag, "ok").Inc() + return raw, true +} diff --git a/harnesses/robinhood-subsidy/go.mod b/harnesses/robinhood-subsidy/go.mod new file mode 100644 index 00000000..0fe5c37c --- /dev/null +++ b/harnesses/robinhood-subsidy/go.mod @@ -0,0 +1,18 @@ +module robinhood-subsidy + +go 1.24.0 + +require github.com/prometheus/client_golang v1.23.2 + +require ( + github.com/beorn7/perks v1.0.1 // indirect + github.com/cespare/xxhash/v2 v2.3.0 // indirect + github.com/kr/text v0.2.0 // indirect + github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect + github.com/prometheus/client_model v0.6.2 // indirect + github.com/prometheus/common v0.66.1 // indirect + github.com/prometheus/procfs v0.16.1 // indirect + go.yaml.in/yaml/v2 v2.4.2 // indirect + golang.org/x/sys v0.35.0 // indirect + google.golang.org/protobuf v1.36.8 // indirect +) diff --git a/harnesses/robinhood-subsidy/go.sum b/harnesses/robinhood-subsidy/go.sum new file mode 100644 index 00000000..d6b8ca98 --- /dev/null +++ b/harnesses/robinhood-subsidy/go.sum @@ -0,0 +1,46 @@ +github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= +github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= +github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs= +github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= +github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= +github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= +github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= +github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU= +github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo= +github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ= +github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= +github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= +github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= +github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= +github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc= +github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw= +github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA= +github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ= +github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= +github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/prometheus/client_golang v1.23.2 h1:Je96obch5RDVy3FDMndoUsjAhG5Edi49h0RJWRi/o0o= +github.com/prometheus/client_golang v1.23.2/go.mod h1:Tb1a6LWHB3/SPIzCoaDXI4I8UHKeFTEQ1YCr+0Gyqmg= +github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk= +github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE= +github.com/prometheus/common v0.66.1 h1:h5E0h5/Y8niHc5DlaLlWLArTQI7tMrsfQjHV+d9ZoGs= +github.com/prometheus/common v0.66.1/go.mod h1:gcaUsgf3KfRSwHY4dIMXLPV0K/Wg1oZ8+SbZk/HH/dA= +github.com/prometheus/procfs v0.16.1 h1:hZ15bTNuirocR6u0JZ6BAHHmwS1p8B4P6MRqxtzMyRg= +github.com/prometheus/procfs v0.16.1/go.mod h1:teAbpZRB1iIAJYREa1LsoWUXykVXA1KlTmWl8x/U+Is= +github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjRBZyWFQ= +github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog= +github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U= +github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U= +go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto= +go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE= +go.yaml.in/yaml/v2 v2.4.2 h1:DzmwEr2rDGHl7lsFgAHxmNz/1NlQ7xLIrlN2h5d1eGI= +go.yaml.in/yaml/v2 v2.4.2/go.mod h1:081UH+NErpNdqlCXm3TtEran0rJZGxAYx9hb/ELlsPU= +golang.org/x/sys v0.35.0 h1:vz1N37gP5bs89s7He8XuIYXpyY0+QlsKmzipCbUtyxI= +golang.org/x/sys v0.35.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k= +google.golang.org/protobuf v1.36.8 h1:xHScyCOEuuwZEc6UtSOvPbAT4zRh0xcNRYekJwfqyMc= +google.golang.org/protobuf v1.36.8/go.mod h1:fuxRtAxBytpl4zzqUh6/eyUujkJdNiuEkXntxiD/uRU= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= +gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= +gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= diff --git a/harnesses/rpc-capabilities/cmd/script/archive.go b/harnesses/rpc-capabilities/cmd/script/archive.go index 8362f56b..1c9e74e6 100644 --- a/harnesses/rpc-capabilities/cmd/script/archive.go +++ b/harnesses/rpc-capabilities/cmd/script/archive.go @@ -43,9 +43,12 @@ var depthBuckets = []uint64{300, 7200, 216000, 1296000, 5000000} func StartArchiveLoop(ctx context.Context) { for _, c := range chains() { c := c - if c.Kind == "solana" { - // eth_getBalance at historical heights has no Solana - // equivalent on public endpoints; skip the archive loop. + if c.Kind == "solana" || c.Kind == "polkadot" { + // eth_getBalance at historical heights has no Solana or + // Substrate equivalent on public endpoints (Polkadot state + // is accessed via state_getStorage keyed by a Blake2 + // hashed storage key, no chain-agnostic depth analog); + // skip the archive loop. continue } for _, p := range c.Providers { diff --git a/harnesses/rpc-capabilities/cmd/script/config.go b/harnesses/rpc-capabilities/cmd/script/config.go index c759b0b5..d9c769ca 100644 --- a/harnesses/rpc-capabilities/cmd/script/config.go +++ b/harnesses/rpc-capabilities/cmd/script/config.go @@ -29,9 +29,17 @@ type Chain struct { Slug string Name string Providers []Provider - // Kind selects the probe path: "" (EVM, eth_getBlockByNumber) or - // "solana" (getSlot at processed commitment, slot-based staleness, - // no archive-depth loop). + // Kind selects the probe path: + // "" or "evm": eth_getBlockByNumber("latest", false), block-based + // staleness, archive-depth loop. + // "solana": getSlot at processed commitment, slot-based staleness, + // no archive-depth loop. + // "polkadot": chain_getHeader against Substrate JSON-RPC, + // block-based staleness (Polkadot relay produces one block every + // ~6 s so staleBlockGap needs a Polkadot-specific override, see + // polkadotStaleBlockGap), no archive-depth loop (Polkadot's + // state model does not map onto the eth_getBalance-by-depth + // probe cleanly). Kind string } @@ -57,8 +65,34 @@ type Chain struct { // (drpc caches eth_blockNumber → only 2 clean providers), opBNB // (1rpc 429s at probe cadence, only 3 solid providers), Mode // (3 providers), Zora / Abstract / HyperEVM (≤2 keyless providers). +// +// Local-run filter: OCB_CHAINS=ethereum,base restricts the matrix to +// the listed slugs (unset or unmatched = full matrix). Used for local +// smoke runs; never set in production. func chains() []Chain { - return []Chain{ + all := []Chain{ + // ─── Polkadot relay chain — first non-EVM, non-Solana chain + // added to the cohort. Substrate JSON-RPC via chain_getHeader + // (returns hex block number, staleness by relay-block gap). + // Providers verified keyless with 4 consecutive probes on + // 2026-07-15: rpc.polkadot.io (Parity official), OnFinality + // public gateway, and PublicNode (Allnodes). Excluded by that + // sweep: 1RPC (chain_getHeader filtered as "Not Allowed" on + // their privacy relay), Dwellir (503 Service Unavailable + // during audit, retry when their gateway stabilises), Ankr + // (paid Polkadot tier only), Chainstack (key-gated), + // RadiumBlock (empty response without referral header), Grove/ + // Thirdweb (invalid-chain or key-gated). + { + Slug: "polkadot", + Name: "Polkadot", + Kind: "polkadot", + Providers: []Provider{ + {Slug: "polkadot-official", Name: "Parity", URL: envDefault("RPC_URL_POLKADOT_OFFICIAL", "https://rpc.polkadot.io")}, + {Slug: "onfinality", Name: "OnFinality", URL: envDefault("RPC_URL_POLKADOT_ONFINALITY", "https://polkadot.api.onfinality.io/public")}, + {Slug: "publicnode", Name: "PublicNode", URL: envDefault("RPC_URL_POLKADOT_PUBLICNODE", "https://polkadot-rpc.publicnode.com")}, + }, + }, // ─── Solana mainnet — added 2026-07-12, all 5 endpoints keyless // and live-verified (getSlot + getLatestBlockhash + getVersion, // mutually consistent advancing slots). Excluded by that sweep: @@ -364,6 +398,25 @@ func chains() []Chain { }, }, } + + filter := strings.TrimSpace(os.Getenv("OCB_CHAINS")) + if filter == "" { + return all + } + keep := make(map[string]bool) + for _, s := range strings.Split(filter, ",") { + keep[strings.TrimSpace(s)] = true + } + var out []Chain + for _, c := range all { + if keep[c.Slug] { + out = append(out, c) + } + } + if len(out) == 0 { + return all + } + return out } func envDefault(key, def string) string { diff --git a/harnesses/rpc-capabilities/cmd/script/consensus.go b/harnesses/rpc-capabilities/cmd/script/consensus.go new file mode 100644 index 00000000..695567e5 --- /dev/null +++ b/harnesses/rpc-capabilities/cmd/script/consensus.go @@ -0,0 +1,154 @@ +package main + +import "sync" + +// consensus.go — bench 083 rpc-reliability, cross-provider head +// agreement. The latency probe already fetches the full latest header +// (anti-cache design) and used to throw the hash away; here every +// successful probe feeds a per-chain height→hash quorum map so we can +// publish two things the latency bench cannot see: +// +// rpc_consensus_lag_blocks — how far this provider's head sits +// behind the highest head any probed provider reported for the +// same chain (the chainTips rolling max). Zero for a provider at +// the shared tip; 1-2 for a gateway one block behind (observed +// live on 1rpc/tenderly); tens for something wedged. +// +// rpc_hash_mismatch_total — the provider's hash at height H +// disagrees with the hash a >=2-provider strict plurality agreed +// on at H. Same-height hash divergence is the serious signal +// (serving a non-canonical or fabricated block), so it is a +// counter, incremented at most once per (provider, height). +// +// Reorg honesty: during a real reorg two providers can legitimately +// sit on different hashes at the same height. quorumHash requires a +// strict plurality (>=2 votes AND strictly more than any competing +// hash), so a 2-2 split counts nobody. Only a provider outvoted by an +// established majority is flagged. + +const ( + // quorumMinProviders: minimum providers agreeing on one hash + // before that hash is treated as canonical at its height. + quorumMinProviders = 2 + // consensusPruneDepth: heights this far below the chain tip are + // dropped from the vote map so memory stays bounded (~2 blocks/s + // chains would otherwise grow forever). + consensusPruneDepth uint64 = 128 +) + +type heightVotes struct { + hash map[string]string // provider -> reported hash + flagged map[string]bool // providers already counted at this height +} + +type chainConsensus struct { + heights map[uint64]*heightVotes +} + +type consensusTracker struct { + mu sync.Mutex + chains map[string]*chainConsensus +} + +func newConsensusTracker() *consensusTracker { + return &consensusTracker{chains: make(map[string]*chainConsensus)} +} + +var consensus = newConsensusTracker() + +// observe records one valid head probe. Called from probeOne for +// results classified ok or stale (a stale block is still a valid +// (height, hash) observation — its lag is exactly the point). hash is +// "" on the Solana path (getSlot carries no hash): lag is emitted, +// quorum voting is skipped. +func (t *consensusTracker) observe(chain, provider string, height uint64, hash string) { + // Lag against the cross-provider rolling max the staleness check + // already maintains. The caller updates tips before observing, so + // a provider that IS the tip reads 0. Noise note: tips is updated + // asynchronously by per-provider goroutines staggered across the + // probe interval, so ±1 block of jitter on fast chains is expected + // and averages out in the 24h quantiles. + tip := tips.get(chain) + lag := 0.0 + if tip > height { + lag = float64(tip - height) + } + rpcConsensusLag.WithLabelValues(provider, chain, currentRegion).Set(lag) + + if hash == "" { + return + } + + t.mu.Lock() + defer t.mu.Unlock() + + cc := t.chains[chain] + if cc == nil { + cc = &chainConsensus{heights: make(map[uint64]*heightVotes)} + t.chains[chain] = cc + } + hv := cc.heights[height] + if hv == nil { + hv = &heightVotes{hash: make(map[string]string), flagged: make(map[string]bool)} + cc.heights[height] = hv + } + hv.hash[provider] = hash + + if q := quorumHash(hv.hash); q != "" { + for p, h := range hv.hash { + if h != q && !hv.flagged[p] { + hv.flagged[p] = true + rpcHashMismatch.WithLabelValues(p, chain, currentRegion).Inc() + } + } + } + + for h := range cc.heights { + if h+consensusPruneDepth < tip { + delete(cc.heights, h) + } + } +} + +// quorumHash returns the hash backed by >= quorumMinProviders votes +// AND strictly more votes than any competing hash. Ties (the 2-2 reorg +// split) return "" so no side is punished without a real majority. +func quorumHash(votes map[string]string) string { + counts := make(map[string]int, len(votes)) + for _, h := range votes { + counts[h]++ + } + best, bestN, secondN := "", 0, 0 + for h, n := range counts { + if n > bestN { + best, secondN, bestN = h, bestN, n + } else if n > secondN { + secondN = n + } + } + if bestN >= quorumMinProviders && bestN > secondN { + return best + } + return "" +} + +// initConsensusMetrics zero-initializes the mismatch counters for the +// full (provider × chain) matrix so `increase()` in the bench's +// incident queries resolves to 0 instead of an absent series for +// providers that never misbehave (which is, hopefully, most of them). +func initConsensusMetrics() { + for _, c := range chains() { + if c.Kind == "solana" || c.Kind == "polkadot" { + // solana: getSlot returns a number only, no hash to vote on. + // polkadot: chain_getHeader returns parentHash (N-1) but no + // current block hash; deriving it needs Blake2 over the + // SCALE-encoded header, which is not worth wiring for a + // bench 083 cross-provider quorum check that already has + // value on 20+ EVM chains. + continue + } + for _, p := range c.Providers { + rpcHashMismatch.WithLabelValues(p.Slug, c.Slug, currentRegion).Add(0) + } + } +} diff --git a/harnesses/rpc-capabilities/cmd/script/consensus_test.go b/harnesses/rpc-capabilities/cmd/script/consensus_test.go new file mode 100644 index 00000000..27e36d65 --- /dev/null +++ b/harnesses/rpc-capabilities/cmd/script/consensus_test.go @@ -0,0 +1,144 @@ +package main + +import ( + "testing" + + "github.com/prometheus/client_golang/prometheus/testutil" +) + +func TestQuorumHash(t *testing.T) { + cases := []struct { + name string + votes map[string]string + want string + }{ + {"empty", map[string]string{}, ""}, + {"single vote no quorum", map[string]string{"a": "0xaa"}, ""}, + {"two agree", map[string]string{"a": "0xaa", "b": "0xaa"}, "0xaa"}, + {"majority beats minority", map[string]string{"a": "0xaa", "b": "0xaa", "c": "0xbb"}, "0xaa"}, + {"reorg 2-2 split counts nobody", map[string]string{"a": "0xaa", "b": "0xaa", "c": "0xbb", "d": "0xbb"}, ""}, + {"3-2 split resolves", map[string]string{"a": "0xaa", "b": "0xaa", "c": "0xaa", "d": "0xbb", "e": "0xbb"}, "0xaa"}, + {"all distinct", map[string]string{"a": "0xaa", "b": "0xbb", "c": "0xcc"}, ""}, + } + for _, tc := range cases { + if got := quorumHash(tc.votes); got != tc.want { + t.Errorf("%s: quorumHash = %q, want %q", tc.name, got, tc.want) + } + } +} + +func TestObserveFlagsMinorityOnce(t *testing.T) { + tr := newConsensusTracker() + chain := "testchain-mismatch" + tips.update(chain, 100) + + mismatches := func(p string) float64 { + return testutil.ToFloat64(rpcHashMismatch.WithLabelValues(p, chain, currentRegion)) + } + + // First vote: no quorum yet, nobody flagged. + tr.observe(chain, "alpha", 100, "0xaa") + if got := mismatches("alpha"); got != 0 { + t.Fatalf("alpha flagged before any quorum existed: %v", got) + } + + // Quorum forms on 0xaa; gamma disagrees and is flagged exactly once. + tr.observe(chain, "beta", 100, "0xaa") + tr.observe(chain, "gamma", 100, "0xbb") + if got := mismatches("gamma"); got != 1 { + t.Fatalf("gamma mismatch count = %v, want 1", got) + } + if mismatches("alpha") != 0 || mismatches("beta") != 0 { + t.Fatalf("quorum members were flagged") + } + + // Re-reporting the same bad hash at the same height must not + // double-count: 60s probes revisit heights on slow chains. + tr.observe(chain, "gamma", 100, "0xbb") + if got := mismatches("gamma"); got != 1 { + t.Fatalf("gamma double-counted at same height: %v", got) + } + + // A different height is a new incident. + tr.observe(chain, "alpha", 101, "0xcc") + tr.observe(chain, "beta", 101, "0xcc") + tr.observe(chain, "gamma", 101, "0xdd") + if got := mismatches("gamma"); got != 2 { + t.Fatalf("gamma mismatch count after 2nd height = %v, want 2", got) + } +} + +func TestObserveReorgSplitCountsNobody(t *testing.T) { + tr := newConsensusTracker() + chain := "testchain-reorg" + tips.update(chain, 50) + + tr.observe(chain, "a", 50, "0xaa") + tr.observe(chain, "b", 50, "0xaa") + tr.observe(chain, "c", 50, "0xbb") + tr.observe(chain, "d", 50, "0xbb") + // c was flagged while 0xaa held a 2-1 plurality; d's vote made it + // 2-2, and from that point no NEW flags may appear. + before := testutil.ToFloat64(rpcHashMismatch.WithLabelValues("d", chain, currentRegion)) + if before != 0 { + t.Fatalf("d flagged on a 2-2 split: %v", before) + } +} + +func TestObserveConsensusLagGauge(t *testing.T) { + tr := newConsensusTracker() + chain := "testchain-lag" + tips.update(chain, 200) + + tr.observe(chain, "laggy", 197, "0xaa") + got := testutil.ToFloat64(rpcConsensusLag.WithLabelValues("laggy", chain, currentRegion)) + if got != 3 { + t.Fatalf("lag gauge = %v, want 3", got) + } + + // Provider at (or ahead of) the recorded tip reads 0. + tr.observe(chain, "fresh", 200, "0xaa") + if got := testutil.ToFloat64(rpcConsensusLag.WithLabelValues("fresh", chain, currentRegion)); got != 0 { + t.Fatalf("fresh lag gauge = %v, want 0", got) + } + + // Solana path: empty hash emits lag but never votes. + tr.observe(chain, "solananode", 195, "") + if got := testutil.ToFloat64(rpcConsensusLag.WithLabelValues("solananode", chain, currentRegion)); got != 5 { + t.Fatalf("solana lag gauge = %v, want 5", got) + } + if got := testutil.ToFloat64(rpcHashMismatch.WithLabelValues("solananode", chain, currentRegion)); got != 0 { + t.Fatalf("hashless observation voted: %v", got) + } +} + +func TestConsensusPruning(t *testing.T) { + tr := newConsensusTracker() + chain := "testchain-prune" + tips.update(chain, 10) + + tr.observe(chain, "a", 10, "0xaa") + tips.update(chain, 10+consensusPruneDepth+5) + tr.observe(chain, "a", 10+consensusPruneDepth+5, "0xbb") + + tr.mu.Lock() + defer tr.mu.Unlock() + if _, ok := tr.chains[chain].heights[10]; ok { + t.Fatalf("height 10 not pruned at tip %d", 10+consensusPruneDepth+5) + } +} + +func TestMajority(t *testing.T) { + if v, ok := majority(map[int]int{1371: 3, 1370: 1}); !ok || v != 1371 { + t.Fatalf("majority(3-1) = %v %v", v, ok) + } + if _, ok := majority(map[int]int{1371: 2, 1370: 2}); ok { + t.Fatalf("tie must not resolve") + } + if _, ok := majority(map[int]int{1371: 1}); ok { + t.Fatalf("single answer must not resolve") + } + if v, ok := majority(map[string]int{"0x1": 2}); !ok || v != "0x1" { + t.Fatalf("majority(2-0) = %v %v", v, ok) + } +} diff --git a/harnesses/rpc-capabilities/cmd/script/integrity.go b/harnesses/rpc-capabilities/cmd/script/integrity.go new file mode 100644 index 00000000..e8663ed4 --- /dev/null +++ b/harnesses/rpc-capabilities/cmd/script/integrity.go @@ -0,0 +1,304 @@ +package main + +import ( + "bytes" + "context" + "encoding/json" + "fmt" + "io" + "net/http" + "time" +) + +// integrity.go — bench 083 rpc-reliability, fixed-vector correctness +// checks. Every 5 minutes ONE chain (rotating through the set below) +// gets the same two questions asked of every provider serving it: +// +// logs — `eth_getLogs` for the chain's canonical USDC contract +// over a 10-block window ending at tip-N. Completeness: +// a provider silently dropping logs (or blocking the +// method, or gating the depth behind a paid tier) shows +// up against the cross-provider majority count. +// balance — `eth_getBalance` of a fixed well-known address at the +// same tip-N block. Consistency: the hex answer must be +// byte-identical across providers; the comparison works +// whether or not the balance is non-zero. +// +// Anti-gaming: N rotates daily over {20, 30, 40, 50, 60} blocks so a +// provider cannot special-case a fixed range, and every request id +// rotates (same edge-cache defeat as the latency probe). All depths +// stay far inside non-archive territory so pruned-but-honest nodes +// are never penalized; a provider that gates even 60-blocks-deep data +// behind a key (observed live: publicnode -32602 "Archive requests +// require a personal token") is emitting exactly the signal this +// bench exists to record. +// +// Errors ARE signal: result="error" on rpc_integrity_check_total is +// counted as an incident by the spec, because "method blocked" and +// "depth gated" are reliability failures from the caller's seat. + +const ( + integrityInterval = 5 * time.Minute + integrityTimeout = 15 * time.Second + // integrityCallSpacing: sequential per-provider spacing; meowrpc + // 429s on rapid bursts (same lesson as archive.go). + integrityCallSpacing = 1500 * time.Millisecond + // integrityLogsSpan: width of the getLogs window, in blocks. + integrityLogsSpan uint64 = 10 + // integrityBalanceAddr: same well-known address the archive loop + // uses (Vitalik). The check compares answers across providers + // byte-for-byte, so it is divergence we measure, not the value. + integrityBalanceAddr = archiveTestAddr +) + +// integrityVectors maps chain slug -> canonical USDC contract used as +// the fixed eth_getLogs vector. Only chains with a heavily traded +// canonical USDC participate (a quiet contract would return 0 logs +// everywhere and the completeness check would be vacuous). Native +// Circle deployments except BNB (Binance-peg, still the busiest +// USDC-family contract there). +var integrityVectors = map[string]string{ + "ethereum": "0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48", + "arbitrum": "0xaf88d065e77c8cC2239327C5EDb3A432268e5831", + "optimism": "0x0b2C639c533813f4Aa9D7837CAf62653d097Ff85", + "base": "0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913", + "polygon": "0x3c499c542cEF5E3811e1192ce70d8cC03d5c3359", + "bnb": "0x8AC76a51cc950d9822D68b83fE1Ad97B32Cd580d", + "avalanche": "0xB97EF9Ef8734C71904D8002F8b6Bc66Dd9c48a6E", +} + +// integrityOffset returns today's tip-N base depth: 20 + 10*(day%5), +// i.e. {20, 30, 40, 50, 60}, rotating at UTC midnight. +func integrityOffset() uint64 { + day := uint64(time.Now().UTC().Unix() / 86400) + return 20 + 10*(day%5) +} + +// StartIntegrityLoop rotates through the vector-equipped chains, one +// chain per 5-minute tick. Non-blocking; spawns its own goroutine. +func StartIntegrityLoop(ctx context.Context) { + var targets []Chain + for _, c := range chains() { + if c.Kind == "" && integrityVectors[c.Slug] != "" { + targets = append(targets, c) + } + } + if len(targets) == 0 { + return + } + initIntegrityMetrics(targets) + + go func() { + // Let the latency loop populate chainTips first: the vector + // block range is derived from the cross-provider tip. + select { + case <-ctx.Done(): + return + case <-time.After(90 * time.Second): + } + i := 0 + integrityTick(ctx, targets[i%len(targets)]) + i++ + t := time.NewTicker(integrityInterval) + defer t.Stop() + for { + select { + case <-ctx.Done(): + return + case <-t.C: + integrityTick(ctx, targets[i%len(targets)]) + i++ + } + } + }() +} + +type integrityObs struct { + p Provider + logsN int + logsErr error + balHex string + balErr error +} + +func integrityTick(ctx context.Context, c Chain) { + tip := tips.get(c.Slug) + off := integrityOffset() + if tip <= off+integrityLogsSpan { + fmt.Printf("[integrity/%s] no tip yet (tip=%d), skipping round\n", c.Slug, tip) + return + } + to := tip - off + from := to - (integrityLogsSpan - 1) + + obs := make([]integrityObs, 0, len(c.Providers)) + for _, p := range c.Providers { + o := integrityObs{p: p} + o.logsN, o.logsErr = fetchLogsCount(ctx, p.URL, integrityVectors[c.Slug], from, to) + if !integritySleep(ctx) { + return + } + o.balHex, o.balErr = fetchBalanceHex(ctx, p.URL, integrityBalanceAddr, to) + obs = append(obs, o) + if !integritySleep(ctx) { + return + } + } + + // Logs completeness: strict majority of the successful counts. + logsCounts := make(map[int]int) + for _, o := range obs { + if o.logsErr == nil { + logsCounts[o.logsN]++ + } + } + majLogs, logsQuorum := majority(logsCounts) + for _, o := range obs { + switch { + case o.logsErr != nil: + rpcLogsCount.DeleteLabelValues(o.p.Slug, c.Slug, currentRegion) + rpcIntegrityCheck.WithLabelValues(o.p.Slug, c.Slug, currentRegion, "logs", "error").Inc() + fmt.Printf("[integrity/%s/%s] logs range=%d-%d err=%v\n", c.Slug, o.p.Slug, from, to, o.logsErr) + case logsQuorum && o.logsN != majLogs: + rpcLogsCount.WithLabelValues(o.p.Slug, c.Slug, currentRegion).Set(float64(o.logsN)) + rpcLogsDisagreement.WithLabelValues(o.p.Slug, c.Slug, currentRegion).Inc() + rpcIntegrityCheck.WithLabelValues(o.p.Slug, c.Slug, currentRegion, "logs", "disagree").Inc() + fmt.Printf("[integrity/%s/%s] logs DISAGREE got=%d majority=%d range=%d-%d\n", c.Slug, o.p.Slug, o.logsN, majLogs, from, to) + default: + // No >=2 quorum this round (too few successes) also lands + // here: a lone answer is unverifiable, not wrong. + rpcLogsCount.WithLabelValues(o.p.Slug, c.Slug, currentRegion).Set(float64(o.logsN)) + rpcIntegrityCheck.WithLabelValues(o.p.Slug, c.Slug, currentRegion, "logs", "ok").Inc() + fmt.Printf("[integrity/%s/%s] logs ok count=%d range=%d-%d\n", c.Slug, o.p.Slug, o.logsN, from, to) + } + } + + // State consistency: strict majority of the raw hex answers. + balCounts := make(map[string]int) + for _, o := range obs { + if o.balErr == nil { + balCounts[o.balHex]++ + } + } + majBal, balQuorum := majority(balCounts) + for _, o := range obs { + switch { + case o.balErr != nil: + rpcIntegrityCheck.WithLabelValues(o.p.Slug, c.Slug, currentRegion, "balance", "error").Inc() + fmt.Printf("[integrity/%s/%s] balance block=%d err=%v\n", c.Slug, o.p.Slug, to, o.balErr) + case balQuorum && o.balHex != majBal: + rpcStateDisagreement.WithLabelValues(o.p.Slug, c.Slug, currentRegion).Inc() + rpcIntegrityCheck.WithLabelValues(o.p.Slug, c.Slug, currentRegion, "balance", "disagree").Inc() + fmt.Printf("[integrity/%s/%s] balance DISAGREE got=%s majority=%s block=%d\n", c.Slug, o.p.Slug, o.balHex, majBal, to) + default: + rpcIntegrityCheck.WithLabelValues(o.p.Slug, c.Slug, currentRegion, "balance", "ok").Inc() + fmt.Printf("[integrity/%s/%s] balance ok block=%d\n", c.Slug, o.p.Slug, to) + } + } +} + +func integritySleep(ctx context.Context) bool { + select { + case <-ctx.Done(): + return false + case <-time.After(integrityCallSpacing): + return true + } +} + +// majority returns the value backed by >=2 votes and strictly more +// than any competing value; ok=false when no such strict majority +// exists (all-distinct answers, or a tie). +func majority[T comparable](counts map[T]int) (T, bool) { + var best T + bestN, secondN := 0, 0 + for v, n := range counts { + if n > bestN { + best, secondN, bestN = v, bestN, n + } else if n > secondN { + secondN = n + } + } + return best, bestN >= 2 && bestN > secondN +} + +func fetchLogsCount(ctx context.Context, url, addr string, from, to uint64) (int, error) { + body := fmt.Sprintf( + `{"jsonrpc":"2.0","method":"eth_getLogs","params":[{"fromBlock":"0x%x","toBlock":"0x%x","address":"%s"}],"id":%d}`, + from, to, addr, time.Now().UnixNano(), + ) + raw, err := integrityPost(ctx, url, body) + if err != nil { + return 0, err + } + var r rpcBlockEnvelope + if err := json.Unmarshal(raw, &r); err != nil { + return 0, err + } + if r.Error != nil { + return 0, fmt.Errorf("rpc %d: %s", r.Error.Code, r.Error.Message) + } + var logs []json.RawMessage + if err := json.Unmarshal(r.Result, &logs); err != nil { + return 0, fmt.Errorf("non-array result") + } + return len(logs), nil +} + +func fetchBalanceHex(ctx context.Context, url, addr string, block uint64) (string, error) { + body := fmt.Sprintf( + `{"jsonrpc":"2.0","method":"eth_getBalance","params":["%s","0x%x"],"id":%d}`, + addr, block, time.Now().UnixNano(), + ) + raw, err := integrityPost(ctx, url, body) + if err != nil { + return "", err + } + var r rpcEnvelope + if err := json.Unmarshal(raw, &r); err != nil { + return "", err + } + if r.Error != nil { + return "", fmt.Errorf("rpc %d: %s", r.Error.Code, r.Error.Message) + } + if r.Result == "" { + return "", fmt.Errorf("empty result") + } + return r.Result, nil +} + +func integrityPost(ctx context.Context, url, body string) ([]byte, error) { + c, cancel := context.WithTimeout(ctx, integrityTimeout) + defer cancel() + req, _ := http.NewRequestWithContext(c, "POST", url, bytes.NewReader([]byte(body))) + req.Header.Set("Content-Type", "application/json") + req.Header.Set("User-Agent", "OpenChainBench/1.0 (+https://openchainbench.com)") + client := &http.Client{Timeout: integrityTimeout} + resp, err := client.Do(req) + if err != nil { + return nil, err + } + defer resp.Body.Close() + if resp.StatusCode != 200 { + _, _ = io.Copy(io.Discard, resp.Body) + return nil, fmt.Errorf("http %d", resp.StatusCode) + } + return io.ReadAll(resp.Body) +} + +// initIntegrityMetrics zero-initializes every counter in the vector +// matrix so the bench's `increase()` queries read 0, not absent, for +// clean providers. +func initIntegrityMetrics(targets []Chain) { + for _, c := range targets { + for _, p := range c.Providers { + for _, check := range []string{"logs", "balance"} { + for _, result := range []string{"ok", "error", "disagree"} { + rpcIntegrityCheck.WithLabelValues(p.Slug, c.Slug, currentRegion, check, result).Add(0) + } + } + rpcLogsDisagreement.WithLabelValues(p.Slug, c.Slug, currentRegion).Add(0) + rpcStateDisagreement.WithLabelValues(p.Slug, c.Slug, currentRegion).Add(0) + } + } +} diff --git a/harnesses/rpc-capabilities/cmd/script/main.go b/harnesses/rpc-capabilities/cmd/script/main.go index 9bea8647..9b18b764 100644 --- a/harnesses/rpc-capabilities/cmd/script/main.go +++ b/harnesses/rpc-capabilities/cmd/script/main.go @@ -73,11 +73,16 @@ func main() { } } fmt.Println() - fmt.Println("Metrics server: :2112/metrics") + + // OCB_METRICS_ADDR: explicit local-run override only (e.g. a dev + // box where :2112 is taken). We still ignore Railway's injected + // $PORT on purpose; Prometheus scrapes :2112 in prod. + addr := envDefault("OCB_METRICS_ADDR", ":2112") + fmt.Printf("Metrics server: %s/metrics\n", addr) fmt.Println() go func() { - if err := StartMetricsServer(":2112"); err != nil { + if err := StartMetricsServer(addr); err != nil { fmt.Printf("[fatal] metrics server: %v\n", err) os.Exit(1) } @@ -88,6 +93,7 @@ func main() { StartProbeLoop(ctx) StartArchiveLoop(ctx) + StartIntegrityLoop(ctx) sig := make(chan os.Signal, 1) signal.Notify(sig, os.Interrupt, syscall.SIGTERM) diff --git a/harnesses/rpc-capabilities/cmd/script/metrics.go b/harnesses/rpc-capabilities/cmd/script/metrics.go index df15637f..304c860e 100644 --- a/harnesses/rpc-capabilities/cmd/script/metrics.go +++ b/harnesses/rpc-capabilities/cmd/script/metrics.go @@ -57,6 +57,57 @@ var ( }, []string{"provider", "chain", "region", "depth"}, ) + + // ─── Bench 083 rpc-reliability: correctness / integrity metrics ──── + // Emitted from the same probe matrix; consensus.go + integrity.go. + + rpcConsensusLag = promauto.NewGaugeVec( + prometheus.GaugeOpts{ + Name: "rpc_consensus_lag_blocks", + Help: "Blocks (slots on Solana) between this provider's reported head and the highest head any probed provider reported for the same chain. Set on every valid head probe (ok or stale); deleted on failure so a dead endpoint ages out instead of freezing at its last lag.", + }, + []string{"provider", "chain", "region"}, + ) + + rpcHashMismatch = promauto.NewCounterVec( + prometheus.CounterOpts{ + Name: "rpc_hash_mismatch_total", + Help: "Times a provider reported a block hash at height H that disagrees with the hash at least 2 other-or-same providers agreed on at H (strict plurality). Incremented at most once per (provider, height); a 2-2 reorg split yields no quorum and nobody is counted.", + }, + []string{"provider", "chain", "region"}, + ) + + rpcLogsCount = promauto.NewGaugeVec( + prometheus.GaugeOpts{ + Name: "rpc_logs_count", + Help: "Number of logs the provider returned for the fixed-vector `eth_getLogs` check (canonical USDC contract, 10-block range at a daily-rotating depth behind tip). Deleted when the call errors so a blocked method doesn't freeze a stale count.", + }, + []string{"provider", "chain", "region"}, + ) + + rpcLogsDisagreement = promauto.NewCounterVec( + prometheus.CounterOpts{ + Name: "rpc_logs_disagreement_total", + Help: "Times a provider's `eth_getLogs` count for the fixed vector deviated from the strict cross-provider majority count in the same round.", + }, + []string{"provider", "chain", "region"}, + ) + + rpcStateDisagreement = promauto.NewCounterVec( + prometheus.CounterOpts{ + Name: "rpc_state_disagreement_total", + Help: "Times a provider's `eth_getBalance` hex at the fixed recent block was not byte-identical to the strict cross-provider majority answer in the same round.", + }, + []string{"provider", "chain", "region"}, + ) + + rpcIntegrityCheck = promauto.NewCounterVec( + prometheus.CounterOpts{ + Name: "rpc_integrity_check_total", + Help: "Fixed-vector integrity check outcomes. check is `logs` (eth_getLogs USDC 10-block window) or `balance` (eth_getBalance at a fixed recent block). result is ok (matches majority), error (method blocked, archive gated, transport failure: errors are signal), or disagree (diverged from the >=2-provider majority).", + }, + []string{"provider", "chain", "region", "check", "result"}, + ) ) // StartMetricsServer binds /metrics + /health on addr. Blocking call — diff --git a/harnesses/rpc-capabilities/cmd/script/probe.go b/harnesses/rpc-capabilities/cmd/script/probe.go index d19f1925..a8b92ddb 100644 --- a/harnesses/rpc-capabilities/cmd/script/probe.go +++ b/harnesses/rpc-capabilities/cmd/script/probe.go @@ -30,6 +30,11 @@ const ( // ~2 minutes behind the cross-provider tip - the same order of // tolerance the EVM gap gives a 12s-block chain. solanaStaleSlotGap uint64 = 300 + // polkadotStaleBlockGap: Polkadot relay produces one block every + // ~6 s, so 40 blocks ≈ 4 min. Same order of tolerance the EVM + // gap gives a 12s-block chain, scaled for the faster relay + // cadence. + polkadotStaleBlockGap uint64 = 40 ) // chainTips tracks the highest block seen for each chain across all @@ -83,6 +88,7 @@ type rpcBlockEnvelope struct { type blockHeader struct { Number string `json:"number"` + Hash string `json:"hash"` } // callLatestBlock issues `eth_getBlockByNumber("latest", false)` and @@ -95,8 +101,9 @@ type blockHeader struct { // node, which let cache-fronted gateways top the latency leaderboard // on cache hits rather than real RPC work. Fetching the full latest // header with a rotating request id defeats body-keyed edge caches; -// the header's `number` field keeps the staleness check intact. -func callLatestBlock(ctx context.Context, url string) (block uint64, result string, latencyMs float64, err error) { +// the header's `number` field keeps the staleness check intact and its +// `hash` feeds the bench-083 cross-provider quorum map (consensus.go). +func callLatestBlock(ctx context.Context, url string) (block uint64, hash string, result string, latencyMs float64, err error) { body := []byte(fmt.Sprintf( `{"jsonrpc":"2.0","method":"eth_getBlockByNumber","params":["latest",false],"id":%d}`, time.Now().UnixNano(), @@ -112,48 +119,49 @@ func callLatestBlock(ctx context.Context, url string) (block uint64, result stri if err != nil { if ctx.Err() != nil || strings.Contains(err.Error(), "deadline exceeded") || strings.Contains(err.Error(), "Timeout") { - return 0, "timeout", latencyMs, err + return 0, "", "timeout", latencyMs, err } - return 0, "http_err", latencyMs, err + return 0, "", "http_err", latencyMs, err } defer resp.Body.Close() if resp.StatusCode != 200 { _, _ = io.Copy(io.Discard, resp.Body) - return 0, "http_err", latencyMs, fmt.Errorf("status %d", resp.StatusCode) + return 0, "", "http_err", latencyMs, fmt.Errorf("status %d", resp.StatusCode) } raw, err := io.ReadAll(resp.Body) if err != nil { - return 0, "http_err", latencyMs, err + return 0, "", "http_err", latencyMs, err } var r rpcBlockEnvelope if err := json.Unmarshal(raw, &r); err != nil { - return 0, "http_err", latencyMs, err + return 0, "", "http_err", latencyMs, err } if r.Error != nil { - return 0, "jsonrpc_err", latencyMs, fmt.Errorf("rpc -%d: %s", r.Error.Code, r.Error.Message) + return 0, "", "jsonrpc_err", latencyMs, fmt.Errorf("rpc -%d: %s", r.Error.Code, r.Error.Message) } if len(r.Result) == 0 || string(r.Result) == "null" { - return 0, "jsonrpc_err", latencyMs, fmt.Errorf("empty result") + return 0, "", "jsonrpc_err", latencyMs, fmt.Errorf("empty result") } var hdr blockHeader if err := json.Unmarshal(r.Result, &hdr); err != nil { - return 0, "jsonrpc_err", latencyMs, err + return 0, "", "jsonrpc_err", latencyMs, err } if hdr.Number == "" { - return 0, "jsonrpc_err", latencyMs, fmt.Errorf("header missing number") + return 0, "", "jsonrpc_err", latencyMs, fmt.Errorf("header missing number") } n, err := strconv.ParseUint(strings.TrimPrefix(hdr.Number, "0x"), 16, 64) if err != nil { - return 0, "jsonrpc_err", latencyMs, err + return 0, "", "jsonrpc_err", latencyMs, err } - return n, "ok", latencyMs, nil + return n, hdr.Hash, "ok", latencyMs, nil } // StartProbeLoop spawns one goroutine per (chain × provider). Each // goroutine runs forever, ticking every probeInterval. func StartProbeLoop(ctx context.Context) { + initConsensusMetrics() for _, c := range chains() { c := c for _, p := range c.Providers { @@ -180,26 +188,50 @@ func probeOne(ctx context.Context, c Chain, p Provider) { probeCtx, cancel := context.WithTimeout(ctx, probeTimeout) defer cancel() var block uint64 + var hash string var result string var latency float64 var err error - if c.Kind == "solana" { + switch c.Kind { + case "solana": block, result, latency, err = callLatestSlot(probeCtx, p.URL) - } else { - block, result, latency, err = callLatestBlock(probeCtx, p.URL) + case "polkadot": + block, hash, result, latency, err = callSubstrateHeader(probeCtx, p.URL) + default: + block, hash, result, latency, err = callLatestBlock(probeCtx, p.URL) } if result == "ok" { tips.update(c.Slug, block) tip := tips.get(c.Slug) gap := staleBlockGap - if c.Kind == "solana" { + switch c.Kind { + case "solana": gap = solanaStaleSlotGap + case "polkadot": + gap = polkadotStaleBlockGap } if tip > 0 && block+gap < tip { result = "stale" } } + // Bench 083: valid observations (fresh or stale, both carry a + // real height + hash) feed the consensus lag gauge and the + // height→hash quorum map; anything else deletes the lag series + // so a dead endpoint ages out instead of freezing. Skipped on + // chains whose probe cannot return the current-block hash + // (Solana: getSlot returns a number only; Polkadot: header + // carries parentHash, not the current block hash). + switch c.Kind { + case "solana", "polkadot": + // no consensus participation + default: + if result == "ok" || result == "stale" { + consensus.observe(c.Slug, p.Slug, block, hash) + } else { + rpcConsensusLag.DeleteLabelValues(p.Slug, c.Slug, currentRegion) + } + } rpcCallTotal.WithLabelValues(p.Slug, c.Slug, currentRegion, result).Inc() if result == "ok" { // Latency is recorded ONLY for fresh, valid responses. Error @@ -296,3 +328,76 @@ func callLatestSlot(ctx context.Context, url string) (slot uint64, result string } return n, "ok", latencyMs, nil } + +// substrateHeader is the shape of the `chain_getHeader` result on +// Polkadot / Kusama and every Substrate-based relay chain. The block +// number is hex-encoded (`0x` prefix) matching the EVM header convention, +// so the parse path reuses the same strconv rule. `parentHash` is here +// so the consensus.go quorum map can key on it identically to EVM +// (bench 083 cross-provider height-hash agreement). +type substrateHeader struct { + Number string `json:"number"` + ParentHash string `json:"parentHash"` +} + +// callSubstrateHeader is the Polkadot probe path: chain_getHeader with a +// rotating request id (same anti-cache rule as the EVM header fetch). +// Returns the block number and parent hash so the caller can plug the +// probe result into the same tips / consensus machinery the EVM chain +// path uses. The returned "block" is the relay-chain block height, not +// a chain-agnostic slot: staleness classification uses +// polkadotStaleBlockGap in probeOne. +func callSubstrateHeader(ctx context.Context, url string) (block uint64, hash string, result string, latencyMs float64, err error) { + body := []byte(fmt.Sprintf( + `{"jsonrpc":"2.0","method":"chain_getHeader","params":[],"id":%d}`, + time.Now().UnixNano(), + )) + req, _ := http.NewRequestWithContext(ctx, "POST", url, bytes.NewReader(body)) + req.Header.Set("Content-Type", "application/json") + req.Header.Set("User-Agent", "OpenChainBench/1.0 (+https://openchainbench.com)") + client := &http.Client{Timeout: probeTimeout} + + start := time.Now() + resp, err := client.Do(req) + latencyMs = float64(time.Since(start).Milliseconds()) + + if err != nil { + if ctx.Err() != nil || strings.Contains(err.Error(), "deadline exceeded") || strings.Contains(err.Error(), "Timeout") { + return 0, "", "timeout", latencyMs, err + } + return 0, "", "http_err", latencyMs, err + } + defer resp.Body.Close() + + if resp.StatusCode != 200 { + _, _ = io.Copy(io.Discard, resp.Body) + return 0, "", "http_err", latencyMs, fmt.Errorf("status %d", resp.StatusCode) + } + + raw, err := io.ReadAll(resp.Body) + if err != nil { + return 0, "", "http_err", latencyMs, err + } + var r rpcBlockEnvelope + if err := json.Unmarshal(raw, &r); err != nil { + return 0, "", "http_err", latencyMs, err + } + if r.Error != nil { + return 0, "", "jsonrpc_err", latencyMs, fmt.Errorf("rpc -%d: %s", r.Error.Code, r.Error.Message) + } + if len(r.Result) == 0 || string(r.Result) == "null" { + return 0, "", "jsonrpc_err", latencyMs, fmt.Errorf("empty result") + } + var hdr substrateHeader + if err := json.Unmarshal(r.Result, &hdr); err != nil { + return 0, "", "jsonrpc_err", latencyMs, err + } + if hdr.Number == "" { + return 0, "", "jsonrpc_err", latencyMs, fmt.Errorf("substrate header missing number") + } + n, err := strconv.ParseUint(strings.TrimPrefix(hdr.Number, "0x"), 16, 64) + if err != nil { + return 0, "", "jsonrpc_err", latencyMs, err + } + return n, hdr.ParentHash, "ok", latencyMs, nil +} diff --git a/harnesses/rpc-capabilities/go.mod b/harnesses/rpc-capabilities/go.mod index 74e2e1fb..3c5d672c 100644 --- a/harnesses/rpc-capabilities/go.mod +++ b/harnesses/rpc-capabilities/go.mod @@ -8,6 +8,7 @@ require ( github.com/beorn7/perks v1.0.1 // indirect github.com/cespare/xxhash/v2 v2.3.0 // indirect github.com/kr/text v0.2.0 // indirect + github.com/kylelemons/godebug v1.1.0 // indirect github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect github.com/prometheus/client_model v0.6.2 // indirect github.com/prometheus/common v0.66.1 // indirect diff --git a/harnesses/token-deployment-cost/cmd/script/cosmos_cardano_stellar.go b/harnesses/token-deployment-cost/cmd/script/cosmos_cardano_stellar.go index 1aa7c5f6..061ae2fa 100644 --- a/harnesses/token-deployment-cost/cmd/script/cosmos_cardano_stellar.go +++ b/harnesses/token-deployment-cost/cmd/script/cosmos_cardano_stellar.go @@ -175,7 +175,18 @@ type koiosEpochParams []struct { CoinsPerUtxoSize json.Number `json:"coins_per_utxo_size"` } +// Cached for an hour: epoch params move once per 5-day epoch and the +// shared koios public tier (5k/day per IP) is also consumed by the +// transaction-fee harness on this host. +var cardanoCostCache struct { + sample Sample + fetchedAt time.Time +} + func (s *cardanoSampler) Sample(ch ChainConfig) (Sample, error) { + if time.Since(cardanoCostCache.fetchedAt) < time.Hour && cardanoCostCache.sample.CostNative > 0 { + return cardanoCostCache.sample, nil + } resp, err := s.http.Get(ch.RPCURL + "/epoch_params") if err != nil { return Sample{}, err @@ -200,13 +211,16 @@ func (s *cardanoSampler) Sample(ch ChainConfig) (Sample, error) { } utxoSize := float64(cardanoEntryOverheadBytes + cardanoBundleBytes) lovelace := coins*utxoSize + cardanoMintTxFeeLovelace - return Sample{CostNative: lovelace, NativeUnit: "lovelace", GasUnits: math.NaN()}, nil + out := Sample{CostNative: lovelace, NativeUnit: "lovelace", GasUnits: math.NaN()} + cardanoCostCache.sample = out + cardanoCostCache.fetchedAt = time.Now() + return out, nil } // ----- Stellar ---------------------------------------------------------- -// Stellar custom asset cost is 3 base reserves locked (issuer account + -// distribution account + trustline on distribution) + 2 tx fees +// Stellar custom asset cost is 5 base reserves locked (2 per new +// account for issuer + distribution, 1 for the trustline) + 2 tx fees // (create_account + change_trust). base_reserve_in_stroops and // base_fee_in_stroops are in /ledgers — we use the latest ledger. // Reusing the issuer as the distribution account is an anti-pattern @@ -240,8 +254,11 @@ func (s *stellarSampler) Sample(ch ChainConfig) (Sample, error) { return Sample{}, fmt.Errorf("empty ledgers response") } rec := l.Embedded.Records[0] - // 3 × base_reserve (issuer + distribution + trustline) + 2 × base_fee - // (create_account + change_trust). - stroops := 3*rec.BaseReserve + 2*rec.BaseFee + // 5 × base_reserve + 2 × base_fee (create_account + change_trust). + // Protocol minimums: every new account locks 2 base reserves and + // each trustline adds 1, so issuer (2) + distribution (2) + + // trustline on distribution (1) = 5. The previous formula counted + // 3 and understated the locked capital by a full XLM. + stroops := 5*rec.BaseReserve + 2*rec.BaseFee return Sample{CostNative: float64(stroops), NativeUnit: "stroop", GasUnits: math.NaN()}, nil } diff --git a/harnesses/tokenized-stock-peg/Dockerfile b/harnesses/tokenized-stock-peg/Dockerfile new file mode 100644 index 00000000..63108cfc --- /dev/null +++ b/harnesses/tokenized-stock-peg/Dockerfile @@ -0,0 +1,22 @@ +FROM golang:1.24-alpine AS builder + +WORKDIR /app +RUN apk add --no-cache git + +COPY go.mod go.sum ./ +RUN go mod download + +COPY . . + +RUN CGO_ENABLED=0 GOOS=linux go build -o /app/monitor ./cmd/script + +FROM debian:bookworm-slim + +WORKDIR /app +RUN apt-get update && apt-get install -y ca-certificates && rm -rf /var/lib/apt/lists/* + +COPY --from=builder /app/monitor /app/monitor + +EXPOSE 2112 + +CMD ["/app/monitor"] diff --git a/harnesses/tokenized-stock-peg/cmd/script/arb_latency.go b/harnesses/tokenized-stock-peg/cmd/script/arb_latency.go new file mode 100644 index 00000000..48244f60 --- /dev/null +++ b/harnesses/tokenized-stock-peg/cmd/script/arb_latency.go @@ -0,0 +1,150 @@ +package main + +import ( + "math" + "sync" + "time" + + "github.com/prometheus/client_golang/prometheus" + "github.com/prometheus/client_golang/prometheus/promauto" +) + +// Arb latency: how many seconds it takes for the Uniswap v4 pool to +// come back within convergedBps of the Yahoo reference after a +// reference move of triggerBps or more in a single one-minute window. +// Fires only during market_state="regular" — outside regular hours +// there is no fresh reference to arbitrage against. +// +// Design: the per-symbol event tracker holds the last reference price +// seen, the "in flight" arb event if one is active, and its start +// time. On each tick: +// - update the trailing 1-minute reference move +// - if not in flight and the move exceeds triggerBps, start a new +// event, arm a timer +// - if in flight and the pool is now within convergedBps, close the +// event, observe the elapsed seconds into the histogram, tag with +// the trigger magnitude bucket for filtering +// - if in flight but not yet converged, keep waiting (no cap: the +// "unresolved" case is the story we want to tell too) + +const ( + arbTriggerBps = 50.0 + arbConvergedBps = 20.0 + arbMoveWindow = 90 * time.Second // ~1 minute plus one poll jitter +) + +var ( + tspArbLatencySeconds = promauto.NewHistogramVec(prometheus.HistogramOpts{ + Name: "tsp_arb_latency_seconds", + Help: "Seconds until the pool price converged to within 20 bps of the reference after a 50 bps ref move (regular hours only).", + // 15s to 30min: 15, 30, 60, 120, 240, 480, 960, 1920 seconds + Buckets: prometheus.ExponentialBuckets(15, 2, 8), + }, []string{"asset", "issuer"}) + + tspArbEventTotal = promauto.NewCounterVec(prometheus.CounterOpts{ + Name: "tsp_arb_event_total", + Help: "Count of arb events per outcome: converged / still_open at rollover.", + }, []string{"asset", "issuer", "outcome"}) + + tspArbOpenAgeSeconds = promauto.NewGaugeVec(prometheus.GaugeOpts{ + Name: "tsp_arb_open_age_seconds", + Help: "Age of the currently-open arb event, if any (0 when no event is in flight).", + }, []string{"asset", "issuer"}) +) + +type refSample struct { + price float64 + at time.Time +} + +type arbTracker struct { + mu sync.Mutex + // Ring of recent reference samples per asset, used to derive the + // trailing move without leaning on Prom's delta(). + refs map[string][]refSample + // In-flight events per (asset). + open map[string]arbEvent +} + +type arbEvent struct { + startedAt time.Time + triggerBps float64 + refAtTrigger float64 + poolAtTrigger float64 +} + +func newArbTracker() *arbTracker { + return &arbTracker{ + refs: make(map[string][]refSample), + open: make(map[string]arbEvent), + } +} + +// observe consumes one (ref, pool) reading for a symbol; called once +// per tick per asset during regular hours only. +func (t *arbTracker) observe(asset, issuer string, now time.Time, ref, pool float64) { + if ref <= 0 || pool <= 0 { + return + } + t.mu.Lock() + defer t.mu.Unlock() + + // Roll the ref window. + buf := t.refs[asset] + buf = append(buf, refSample{price: ref, at: now}) + cut := now.Add(-arbMoveWindow) + for len(buf) > 0 && buf[0].at.Before(cut) { + buf = buf[1:] + } + t.refs[asset] = buf + + // Trailing 1-minute reference move in bps. + oldest := buf[0] + moveBps := 0.0 + if oldest.price > 0 { + moveBps = math.Abs(ref-oldest.price) / oldest.price * 10000 + } + devBps := math.Abs(pool-ref) / ref * 10000 + + ev, inFlight := t.open[asset] + tspArbOpenAgeSeconds.WithLabelValues(asset, issuer).Set(0) + + if !inFlight { + // Start a new event only if the reference actually moved and + // the pool is currently out of the convergence band. If the + // pool was already within band during the move, arbs closed + // it faster than one tick, credit as "sub-poll". + if moveBps >= arbTriggerBps && devBps > arbConvergedBps { + t.open[asset] = arbEvent{ + startedAt: now, + triggerBps: moveBps, + refAtTrigger: ref, + poolAtTrigger: pool, + } + tspArbOpenAgeSeconds.WithLabelValues(asset, issuer).Set(0) + } + return + } + + // In flight: report age, close on convergence. + age := now.Sub(ev.startedAt).Seconds() + tspArbOpenAgeSeconds.WithLabelValues(asset, issuer).Set(age) + if devBps <= arbConvergedBps { + tspArbLatencySeconds.WithLabelValues(asset, issuer).Observe(age) + tspArbEventTotal.WithLabelValues(asset, issuer, "converged").Inc() + delete(t.open, asset) + } +} + +// closeStaleOnStateChange is called when the market session flips out +// of "regular": any in-flight event is unresolved by market close, so +// we count it as "still_open" without polluting the latency histogram. +func (t *arbTracker) closeStaleOnStateChange(issuer string) { + t.mu.Lock() + defer t.mu.Unlock() + for asset := range t.open { + tspArbEventTotal.WithLabelValues(asset, issuer, "still_open").Inc() + tspArbOpenAgeSeconds.WithLabelValues(asset, issuer).Set(0) + delete(t.open, asset) + } +} diff --git a/harnesses/tokenized-stock-peg/cmd/script/config.go b/harnesses/tokenized-stock-peg/cmd/script/config.go new file mode 100644 index 00000000..0a91141b --- /dev/null +++ b/harnesses/tokenized-stock-peg/cmd/script/config.go @@ -0,0 +1,72 @@ +package main + +import ( + "os" + "strings" + "time" +) + +// Tokenized-stock-peg harness: onchain price of Robinhood Chain +// tokenized equities (Uniswap v4 pools vs USDG) against the real +// Nasdaq/NYSE price from Yahoo Finance, deviation in basis points, +// labeled by market session state. +// +// Cohort: the 11 official " • Robinhood Token" equities whose +// USDG pool has real liquidity and swap activity (verified 2026-07-13 +// via Blockscout + PoolManager extsload sweep). Excluded and why: +// SPCX (SpaceX is not listed, no reference price exists), SNDK + QQQ +// (pool depth under $2k, pure noise), CRCL (pool has zero liquidity), +// the ~80 other official tokens (issuer-seeded placeholder pools at +// 90-95% fee, zero swaps), HOOD (never issued onchain, only spam). +// +// Orientation: Uniswap v4 orders currencies by address; USDGIsC0 says +// whether USDG (6 decimals) is currency0 in that pool. Stocks are 18 +// decimals, so the raw sqrtPriceX96 price converts with a 1e12 factor +// whose direction depends on the ordering. + +const ( + rpcDefault = "https://rpc.mainnet.chain.robinhood.com" + stateView = "0xF3334192D15450CdD385c8B70e03f9A6bD9E673b" + getSlot0Sel = "0xc815641c" // StateView.getSlot0(bytes32) — live-verified + pollInterval = 60 * time.Second + httpTimeout = 15 * time.Second + // A tokenized stock more than this far from its reference during + // regular hours is displayed but flagged; used only for logging. + logThresholdBps = 100.0 +) + +type Asset struct { + Symbol string // Yahoo ticker == display slug (lowercased for labels) + Token string + PoolID string + FeePPM int + USDGIsC0 bool +} + +var assets = []Asset{ + {Symbol: "NVDA", Token: "0xd0601CE157Db5bdC3162BbaC2a2C8aF5320D9EEC", PoolID: "0x3bb34a44f1b2b5f32c034c38a53065a521a47b199700fa9bd19d60985ff24bf1", FeePPM: 3000, USDGIsC0: true}, + {Symbol: "AAPL", Token: "0xaF3D76f1834A1d425780943C99Ea8A608f8a93f9", PoolID: "0xda4116b5894ee7479e64eae9276e1a2944ef0e5ce863a299d296a15618deee01", FeePPM: 10000, USDGIsC0: true}, + {Symbol: "GOOGL", Token: "0x2e0847E8910a9732eB3fb1bb4b70a580ADAD4FE3", PoolID: "0xef22239f96c6ac95dcd57b90c6b14c0cc8c3c16844def34daef68dc9dd945344", FeePPM: 10000, USDGIsC0: false}, + {Symbol: "TSLA", Token: "0x322F0929c4625eD5bAd873c95208D54E1c003b2d", PoolID: "0x8517f8071ae5b831b738052f12125e8e3d6c158b78728aa44ce3b25e5104d32e", FeePPM: 3000, USDGIsC0: false}, + {Symbol: "PLTR", Token: "0x894E1EC2D74FFE5AEF8Dc8A9e84686acCB964F2A", PoolID: "0xee430ee1003e1985e1828a01b9a20dad67ad4302994fe2abb4a173de4ac54623", FeePPM: 10000, USDGIsC0: true}, + {Symbol: "META", Token: "0xc0D6457C16Cc70d6790Dd43521C899C87ce02f35", PoolID: "0x5875d407a42965b0e768c8925cea290e06fa50603ef34fc99eb92a1050e6ae36", FeePPM: 3000, USDGIsC0: true}, + {Symbol: "AMD", Token: "0x86923f96303D656E4aa86D9d42D1e57ad2023fdC", PoolID: "0xde9f85fdd9e05a943a52f2c69ffafe3064a3287df03d02c9b431bc92d4781274", FeePPM: 10000, USDGIsC0: true}, + {Symbol: "MSFT", Token: "0xe93237C50D904957Cf27E7B1133b510C669c2e74", PoolID: "0xace02af66d24427b162f80329e039b78c226fb9a79669f5e18d5feec2aa0c056", FeePPM: 20000, USDGIsC0: true}, + {Symbol: "AMZN", Token: "0x12f190a9F9d7D37a250758b26824B97CE941bF54", PoolID: "0xa3280c768df670a535d14af8c22ad3907f2acfc0277c03309fb4d5fc8d43447e", FeePPM: 20000, USDGIsC0: false}, + {Symbol: "SPY", Token: "0x117cc2133c37B721F49dE2A7a74833232B3B4C0C", PoolID: "0x7eeda68cd84620339e6ad4bf054af9b19878ac13139991c7aaec018c40a8bb6a", FeePPM: 10000, USDGIsC0: false}, + {Symbol: "MU", Token: "0xfF080c8ce2E5feadaCa0Da81314Ae59D232d4afD", PoolID: "0x6fa3ee0048e78bf0a513eb0ab56f482944a767c21db990fcf555605e69f05659", FeePPM: 10000, USDGIsC0: true}, +} + +func rpcURL() string { + if v := strings.TrimSpace(os.Getenv("TSP_RPC_URL")); v != "" { + return v + } + return rpcDefault +} + +func listenAddr() string { + if v := strings.TrimSpace(os.Getenv("LISTEN_ADDR")); v != "" { + return v + } + return ":2112" +} diff --git a/harnesses/tokenized-stock-peg/cmd/script/loghub.go b/harnesses/tokenized-stock-peg/cmd/script/loghub.go new file mode 100644 index 00000000..9dd74488 --- /dev/null +++ b/harnesses/tokenized-stock-peg/cmd/script/loghub.go @@ -0,0 +1,114 @@ +package main + +import ( + "bufio" + "fmt" + "io" + "net/http" + "os" + "strconv" + "sync" + "time" +) + +// Auto-generated by the loghub inline pattern. Captures stdout/stderr into a +// bounded ring buffer and exposes GET /logs?tail=N protected by X-Logs-Token +// matching the LOGS_TOKEN env var. +// +// Keep in sync across miniapps (was previously the shared/loghub package; we +// inline because Railway's per-harness Docker build context can't reach a +// sibling shared module via go.mod replace). + +const logRingMax = 5000 + +type logRing struct { + mu sync.Mutex + lines []string + max int +} + +var globalLogRing = &logRing{max: logRingMax} + +func (b *logRing) push(line string) { + entry := time.Now().UTC().Format("2006-01-02T15:04:05.000Z") + " " + line + b.mu.Lock() + if len(b.lines) >= b.max { + b.lines = append(b.lines[1:], entry) + } else { + b.lines = append(b.lines, entry) + } + b.mu.Unlock() +} + +func (b *logRing) snapshot(tail int) []string { + b.mu.Lock() + defer b.mu.Unlock() + if tail <= 0 || tail >= len(b.lines) { + out := make([]string, len(b.lines)) + copy(out, b.lines) + return out + } + start := len(b.lines) - tail + out := make([]string, tail) + copy(out, b.lines[start:]) + return out +} + +var logSetupOnce sync.Once + +// installLogCapture replaces os.Stdout (and os.Stderr) with the write-end of a +// pipe, then spawns a goroutine that fan-outs every line to the original +// stdout AND the in-memory ring buffer. Call exactly once, very early in +// main(). +func installLogCapture() { logSetupOnce.Do(doInstallLogCapture) } + +func doInstallLogCapture() { + originalStdout := os.Stdout + originalStderr := os.Stderr + r, w, err := os.Pipe() + if err != nil { + fmt.Fprintf(originalStdout, "[loghub] pipe failed: %v (/logs will be empty)\n", err) + return + } + os.Stdout = w + os.Stderr = w + + go func() { + scanner := bufio.NewScanner(r) + buf := make([]byte, 0, 1024*1024) + scanner.Buffer(buf, 1024*1024) + for scanner.Scan() { + line := scanner.Text() + fmt.Fprintln(originalStdout, line) + globalLogRing.push(line) + } + _, _ = io.Copy(originalStdout, r) + _ = originalStderr + }() +} + +// logsHandler returns an http.Handler for GET /logs?tail=N. Requires header +// X-Logs-Token to match the LOGS_TOKEN env var. Returns 403 if env unset. +func logsHandler() http.Handler { + return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + expected := os.Getenv("LOGS_TOKEN") + if expected == "" { + http.Error(w, "logs disabled: LOGS_TOKEN unset", http.StatusForbidden) + return + } + if r.Header.Get("X-Logs-Token") != expected { + http.Error(w, "forbidden", http.StatusForbidden) + return + } + tail := 500 + if t := r.URL.Query().Get("tail"); t != "" { + if n, err := strconv.Atoi(t); err == nil && n > 0 { + tail = n + } + } + w.Header().Set("Content-Type", "text/plain; charset=utf-8") + for _, l := range globalLogRing.snapshot(tail) { + fmt.Fprintln(w, l) + } + }) +} diff --git a/harnesses/tokenized-stock-peg/cmd/script/main.go b/harnesses/tokenized-stock-peg/cmd/script/main.go new file mode 100644 index 00000000..bd74d17a --- /dev/null +++ b/harnesses/tokenized-stock-peg/cmd/script/main.go @@ -0,0 +1,91 @@ +package main + +import ( + "fmt" + "math" + "net/http" + "strings" + "time" +) + +func main() { + installLogCapture() + fmt.Println("=== Tokenized Stock Peg Harness ===") + fmt.Println("OpenChainBench — Robinhood Chain tokenized equities vs Nasdaq reference.") + fmt.Printf("Cohort: %d assets | poll: %s | RPC: %s\n", len(assets), pollInterval, rpcURL()) + for _, a := range assets { + fmt.Printf(" - %-6s token=%s pool=%s… fee=%.2f%%\n", a.Symbol, a.Token[:10]+"…", a.PoolID[:14], float64(a.FeePPM)/10000) + } + + go func() { + if err := startMetricsServer(listenAddr()); err != nil { + fmt.Printf("[fatal] metrics server: %v\n", err) + } + }() + + client := &http.Client{Timeout: httpTimeout} + var periods *tradingPeriods + var prevState string + tracker := newArbTracker() + + tick := func() { + now := time.Now() + if periods == nil || now.Sub(periods.FetchedAt) > 30*time.Minute { + if tp := fetchTradingPeriods(client); tp != nil { + periods = tp + } + } + state := periods.state(now) + if prevState == "regular" && state != "regular" { + tracker.closeStaleOnStateChange("robinhood") + } + prevState = state + for _, s := range []string{"pre", "regular", "post", "closed", "unknown"} { + v := 0.0 + if s == state { + v = 1.0 + } + tspMarketState.WithLabelValues(s).Set(v) + } + + refs := fetchReferencePrices(client) + onchain := fetchOnchainPrices(client) + + for _, a := range assets { + sym := strings.ToLower(a.Symbol) + ref, hasRef := refs[sym] + pool, hasPool := onchain[sym] + if hasRef { + tspPriceReference.WithLabelValues(sym).Set(ref.Price) + if ref.AsOfSec > 0 { + tspRefAge.WithLabelValues(sym).Set(float64(now.Unix() - ref.AsOfSec)) + } + } + if hasPool { + tspPriceOnchain.WithLabelValues(sym, "robinhood").Set(pool) + } + if hasRef && hasPool && ref.Price > 0 { + dev := math.Abs(pool-ref.Price) / ref.Price * 10000 + tspDeviationBps.WithLabelValues(sym, state, "robinhood").Set(dev) + if state == "regular" { + tracker.observe(sym, "robinhood", now, ref.Price, pool) + } + tspHealth.WithLabelValues(sym).Set(1) + flag := "" + if dev > logThresholdBps && state == "regular" { + flag = " <-- wide" + } + fmt.Printf("[%s][%s] pool=%.2f ref=%.2f dev=%.1fbps%s\n", sym, state, pool, ref.Price, dev, flag) + } else { + tspHealth.WithLabelValues(sym).Set(0) + } + } + } + + tick() + t := time.NewTicker(pollInterval) + defer t.Stop() + for range t.C { + tick() + } +} diff --git a/harnesses/tokenized-stock-peg/cmd/script/metrics.go b/harnesses/tokenized-stock-peg/cmd/script/metrics.go new file mode 100644 index 00000000..82072a68 --- /dev/null +++ b/harnesses/tokenized-stock-peg/cmd/script/metrics.go @@ -0,0 +1,67 @@ +package main + +import ( + "net/http" + + "github.com/prometheus/client_golang/prometheus" + "github.com/prometheus/client_golang/prometheus/promauto" + "github.com/prometheus/client_golang/prometheus/promhttp" +) + +var ( + // Headline: absolute deviation of the onchain pool price from the + // Yahoo reference, in basis points, labeled with the market session + // the sample was taken in. The bench pins its ranking to + // market_state="regular"; the closed-state series is the weekend / + // overnight drift panel. + tspDeviationBps = promauto.NewGaugeVec(prometheus.GaugeOpts{ + Name: "tsp_deviation_bps", + Help: "Absolute onchain vs reference price deviation per tokenized stock, in bps, labeled by market session state.", + }, []string{"asset", "market_state", "issuer"}) + + tspPriceOnchain = promauto.NewGaugeVec(prometheus.GaugeOpts{ + Name: "tsp_price_onchain_usdg", + Help: "Venue spot price of the tokenized stock, in the venue quote stable.", + }, []string{"asset", "issuer"}) + + tspPriceReference = promauto.NewGaugeVec(prometheus.GaugeOpts{ + Name: "tsp_price_reference_usd", + Help: "Reference equity price from Yahoo Finance (regularMarketPrice; last close when the market is closed).", + }, []string{"asset"}) + + tspRefAge = promauto.NewGaugeVec(prometheus.GaugeOpts{ + Name: "tsp_reference_age_seconds", + Help: "Age of the reference price sample (now minus regularMarketTime). Large outside regular hours by design.", + }, []string{"asset"}) + + tspMarketState = promauto.NewGaugeVec(prometheus.GaugeOpts{ + Name: "tsp_market_session", + Help: "1 for the currently active market session label, 0 otherwise.", + }, []string{"market_state"}) + + tspSourceLatency = promauto.NewGaugeVec(prometheus.GaugeOpts{ + Name: "tsp_source_latency_milliseconds", + Help: "Round-trip latency of the last fetch per source.", + }, []string{"source"}) + + tspSourceCall = promauto.NewCounterVec(prometheus.CounterOpts{ + Name: "tsp_source_call_total", + Help: "Fetch outcomes per source (onchain batch, yahoo spark, yahoo chart).", + }, []string{"source", "result"}) + + tspHealth = promauto.NewGaugeVec(prometheus.GaugeOpts{ + Name: "tsp_health", + Help: "1 when the last tick produced a deviation sample for the asset, 0 otherwise.", + }, []string{"asset"}) +) + +func startMetricsServer(addr string) error { + mux := http.NewServeMux() + mux.Handle("/metrics", promhttp.Handler()) + mux.Handle("/logs", logsHandler()) + mux.HandleFunc("/health", func(w http.ResponseWriter, _ *http.Request) { + w.WriteHeader(http.StatusOK) + _, _ = w.Write([]byte("ok")) + }) + return http.ListenAndServe(addr, mux) +} diff --git a/harnesses/tokenized-stock-peg/cmd/script/onchain.go b/harnesses/tokenized-stock-peg/cmd/script/onchain.go new file mode 100644 index 00000000..206b1a6f --- /dev/null +++ b/harnesses/tokenized-stock-peg/cmd/script/onchain.go @@ -0,0 +1,129 @@ +package main + +import ( + "bytes" + "encoding/json" + "fmt" + "io" + "math/big" + "net/http" + "strings" + "time" +) + +// Onchain leg: one JSON-RPC batch per tick, one eth_call per asset to +// StateView.getSlot0(poolId) on the Uniswap v4 singleton's view +// contract. The response's first 32-byte word is sqrtPriceX96. +// +// Price math (v4 = v3 semantics): raw = (sqrtPriceX96 / 2^96)^2 is the +// amount of currency1 base units per 1 base unit of currency0. With +// USDG at 6 decimals and stocks at 18: +// USDG is currency0 → USDG per stock = 1e12 / raw +// USDG is currency1 → USDG per stock = raw × 1e12 +// (verified against the live AAPL pool: 319.42 USDG on 2026-07-13). + +type batchReq struct { + JSONRPC string `json:"jsonrpc"` + ID int `json:"id"` + Method string `json:"method"` + Params []any `json:"params"` +} + +type batchResp struct { + ID int `json:"id"` + Result json.RawMessage `json:"result"` + Error *struct { + Code int `json:"code"` + Message string `json:"message"` + } `json:"error"` +} + +var q96 = new(big.Float).SetPrec(200).SetInt(new(big.Int).Lsh(big.NewInt(1), 96)) + +func fetchOnchainPrices(client *http.Client) map[string]float64 { + reqs := make([]batchReq, 0, len(assets)) + for i, a := range assets { + data := getSlot0Sel + strings.TrimPrefix(strings.ToLower(a.PoolID), "0x") + reqs = append(reqs, batchReq{ + JSONRPC: "2.0", ID: i, Method: "eth_call", + Params: []any{map[string]string{"to": stateView, "data": data}, "latest"}, + }) + } + body, _ := json.Marshal(reqs) + req, err := http.NewRequest("POST", rpcURL(), bytes.NewReader(body)) + if err != nil { + return nil + } + req.Header.Set("Content-Type", "application/json") + req.Header.Set("User-Agent", "OpenChainBench/1.0 (+https://openchainbench.com)") + + start := time.Now() + resp, err := client.Do(req) + if err != nil { + tspSourceCall.WithLabelValues("onchain", "network").Inc() + return nil + } + defer resp.Body.Close() + raw, err := io.ReadAll(io.LimitReader(resp.Body, 1<<22)) + if err != nil || resp.StatusCode != 200 { + tspSourceCall.WithLabelValues("onchain", fmt.Sprintf("http_%d", resp.StatusCode)).Inc() + return nil + } + tspSourceLatency.WithLabelValues("onchain").Set(float64(time.Since(start).Milliseconds())) + + var out []batchResp + if err := json.Unmarshal(raw, &out); err != nil { + tspSourceCall.WithLabelValues("onchain", "parse").Inc() + return nil + } + prices := make(map[string]float64, len(assets)) + for _, r := range out { + if r.ID < 0 || r.ID >= len(assets) { + continue + } + a := assets[r.ID] + sym := strings.ToLower(a.Symbol) + if r.Error != nil { + // Issuer pause/block or pool state error: skip the sample so + // Prom staleness ages the series out instead of freezing it. + tspSourceCall.WithLabelValues("onchain", "rpc_error").Inc() + fmt.Printf("[%s] onchain error: %s\n", sym, r.Error.Message) + continue + } + p, ok := slot0ToUSDG(string(r.Result), a.USDGIsC0) + if !ok || p <= 0 { + tspSourceCall.WithLabelValues("onchain", "decode").Inc() + continue + } + prices[sym] = p + tspSourceCall.WithLabelValues("onchain", "ok").Inc() + } + return prices +} + +// slot0ToUSDG parses the getSlot0 return blob (sqrtPriceX96 is the +// first word) and converts to USDG per stock respecting pool ordering. +func slot0ToUSDG(resultJSON string, usdgIsC0 bool) (float64, bool) { + hexStr := strings.Trim(resultJSON, `"`) + hexStr = strings.TrimPrefix(hexStr, "0x") + if len(hexStr) < 64 { + return 0, false + } + sqrtInt, ok := new(big.Int).SetString(hexStr[:64], 16) + if !ok || sqrtInt.Sign() == 0 { + return 0, false + } + sqrtF := new(big.Float).SetPrec(200).SetInt(sqrtInt) + ratio := new(big.Float).SetPrec(200).Quo(sqrtF, q96) + raw := new(big.Float).SetPrec(200).Mul(ratio, ratio) + + e12 := new(big.Float).SetPrec(200).SetFloat64(1e12) + var price *big.Float + if usdgIsC0 { + price = new(big.Float).SetPrec(200).Quo(e12, raw) + } else { + price = new(big.Float).SetPrec(200).Mul(raw, e12) + } + f, _ := price.Float64() + return f, true +} diff --git a/harnesses/tokenized-stock-peg/cmd/script/reference.go b/harnesses/tokenized-stock-peg/cmd/script/reference.go new file mode 100644 index 00000000..3ebc5c3b --- /dev/null +++ b/harnesses/tokenized-stock-peg/cmd/script/reference.go @@ -0,0 +1,207 @@ +package main + +import ( + "encoding/json" + "fmt" + "io" + "net/http" + "strings" + "time" +) + +// Reference leg: Yahoo Finance, keyless. One spark batch call fetches +// the latest price for every symbol; one chart call (AAPL as the +// bellwether) fetches the day's exact session windows, which Yahoo +// publishes holiday-aware so the harness never maintains an NYSE +// calendar. Verified 2026-07-13 from the harness host: clean 200s with +// a browser User-Agent (datacenter IP), 70-130ms. +// +// Market state is derived from currentTradingPeriod epochs: +// pre / regular / post / closed. Deviation samples carry the state as +// a label so the bench can pin its headline to regular hours and read +// the weekend drift from the closed-state series. + +const ( + sparkHost = "https://query1.finance.yahoo.com/v8/finance/spark" + chartHost = "https://query1.finance.yahoo.com/v8/finance/chart/AAPL" + yahooUA = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0 Safari/537.36" +) + +type refQuote struct { + Price float64 + AsOfSec int64 +} + +type tradingPeriods struct { + PreStart, PreEnd int64 + RegularStart, RegularEnd int64 + PostStart, PostEnd int64 + FetchedAt time.Time +} + +// fetchReferencePrices returns the freshest Yahoo price per symbol +// (lowercased) from one spark batch call. When the market is closed +// the spark meta still carries regularMarketPrice = last close, which +// is exactly the weekend reference we want. +func fetchReferencePrices(client *http.Client) map[string]refQuote { + syms := make([]string, 0, len(assets)) + for _, a := range assets { + syms = append(syms, a.Symbol) + } + url := sparkHost + "?symbols=" + strings.Join(syms, ",") + "&range=1d&interval=1m" + raw, status := yahooGet(client, url) + if raw == nil { + tspSourceCall.WithLabelValues("yahoo", status).Inc() + // Rotate to query2 once before giving up this tick. + raw, status = yahooGet(client, strings.Replace(url, "query1", "query2", 1)) + if raw == nil { + tspSourceCall.WithLabelValues("yahoo", status).Inc() + return nil + } + } + tspSourceCall.WithLabelValues("yahoo", "ok").Inc() + + // Spark response: {"spark":{"result":[{"symbol":"AAPL","response":[{"meta":{...}}]}]}} + // or the flatter {"AAPL":{...}} shape depending on edge; handle both. + prices := make(map[string]refQuote, len(syms)) + var envel struct { + Spark struct { + Result []struct { + Symbol string `json:"symbol"` + Response []struct { + Meta struct { + RegularMarketPrice float64 `json:"regularMarketPrice"` + RegularMarketTime int64 `json:"regularMarketTime"` + } `json:"meta"` + } `json:"response"` + } `json:"result"` + } `json:"spark"` + } + if err := json.Unmarshal(raw, &envel); err == nil { + for _, r := range envel.Spark.Result { + if len(r.Response) == 0 || r.Response[0].Meta.RegularMarketPrice <= 0 { + continue + } + prices[strings.ToLower(r.Symbol)] = refQuote{ + Price: r.Response[0].Meta.RegularMarketPrice, + AsOfSec: r.Response[0].Meta.RegularMarketTime, + } + } + } + if len(prices) == 0 { + // Flat spark shape (observed live 2026-07-13): + // {"MSFT":{"timestamp":[...],"close":[...],"previousClose":X},...} + // Price = last non-null close; previousClose is the fallback when + // the close array is empty (market closed all day). + var flat map[string]struct { + Timestamp []int64 `json:"timestamp"` + Close []*float64 `json:"close"` + PreviousClose *float64 `json:"previousClose"` + } + if err := json.Unmarshal(raw, &flat); err == nil { + for sym, v := range flat { + q := refQuote{} + for i := len(v.Close) - 1; i >= 0; i-- { + if v.Close[i] != nil && *v.Close[i] > 0 { + q.Price = *v.Close[i] + if i < len(v.Timestamp) { + q.AsOfSec = v.Timestamp[i] + } + break + } + } + if q.Price == 0 && v.PreviousClose != nil && *v.PreviousClose > 0 { + q.Price = *v.PreviousClose + } + if q.Price > 0 { + prices[strings.ToLower(sym)] = q + } + } + } + } + if len(prices) == 0 { + tspSourceCall.WithLabelValues("yahoo", "parse_empty").Inc() + head := string(raw) + if len(head) > 400 { + head = head[:400] + } + fmt.Printf("[yahoo] spark yielded no symbols; body head: %s\n", head) + return nil + } + return prices +} + +// fetchTradingPeriods reads currentTradingPeriod from one chart call. +// Refreshed every 30 minutes; between refreshes marketState() reuses +// the cached windows. +func fetchTradingPeriods(client *http.Client) *tradingPeriods { + raw, status := yahooGet(client, chartHost+"?range=1d&interval=5m") + if raw == nil { + tspSourceCall.WithLabelValues("yahoo_chart", status).Inc() + return nil + } + var envel struct { + Chart struct { + Result []struct { + Meta struct { + CurrentTradingPeriod struct { + Pre struct{ Start, End int64 } `json:"pre"` + Regular struct{ Start, End int64 } `json:"regular"` + Post struct{ Start, End int64 } `json:"post"` + } `json:"currentTradingPeriod"` + } `json:"meta"` + } `json:"result"` + } `json:"chart"` + } + if err := json.Unmarshal(raw, &envel); err != nil || len(envel.Chart.Result) == 0 { + tspSourceCall.WithLabelValues("yahoo_chart", "parse").Inc() + return nil + } + m := envel.Chart.Result[0].Meta.CurrentTradingPeriod + tspSourceCall.WithLabelValues("yahoo_chart", "ok").Inc() + return &tradingPeriods{ + PreStart: m.Pre.Start, PreEnd: m.Pre.End, + RegularStart: m.Regular.Start, RegularEnd: m.Regular.End, + PostStart: m.Post.Start, PostEnd: m.Post.End, + FetchedAt: time.Now(), + } +} + +func (tp *tradingPeriods) state(now time.Time) string { + if tp == nil { + return "unknown" + } + u := now.Unix() + switch { + case u >= tp.RegularStart && u < tp.RegularEnd: + return "regular" + case u >= tp.PreStart && u < tp.PreEnd: + return "pre" + case u >= tp.PostStart && u < tp.PostEnd: + return "post" + default: + return "closed" + } +} + +func yahooGet(client *http.Client, url string) ([]byte, string) { + req, err := http.NewRequest("GET", url, nil) + if err != nil { + return nil, "request_build" + } + req.Header.Set("User-Agent", yahooUA) + req.Header.Set("Accept", "application/json") + resp, err := client.Do(req) + if err != nil { + return nil, "network" + } + defer resp.Body.Close() + raw, err := io.ReadAll(io.LimitReader(resp.Body, 1<<22)) + if err != nil { + return nil, "read" + } + if resp.StatusCode != 200 { + return nil, fmt.Sprintf("http_%d", resp.StatusCode) + } + return raw, "ok" +} diff --git a/harnesses/tokenized-stock-peg/go.mod b/harnesses/tokenized-stock-peg/go.mod new file mode 100644 index 00000000..510f4f14 --- /dev/null +++ b/harnesses/tokenized-stock-peg/go.mod @@ -0,0 +1,18 @@ +module tokenized-stock-peg + +go 1.24.0 + +require github.com/prometheus/client_golang v1.23.2 + +require ( + github.com/beorn7/perks v1.0.1 // indirect + github.com/cespare/xxhash/v2 v2.3.0 // indirect + github.com/kr/text v0.2.0 // indirect + github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect + github.com/prometheus/client_model v0.6.2 // indirect + github.com/prometheus/common v0.66.1 // indirect + github.com/prometheus/procfs v0.16.1 // indirect + go.yaml.in/yaml/v2 v2.4.2 // indirect + golang.org/x/sys v0.35.0 // indirect + google.golang.org/protobuf v1.36.8 // indirect +) diff --git a/harnesses/tokenized-stock-peg/go.sum b/harnesses/tokenized-stock-peg/go.sum new file mode 100644 index 00000000..d6b8ca98 --- /dev/null +++ b/harnesses/tokenized-stock-peg/go.sum @@ -0,0 +1,46 @@ +github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= +github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= +github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs= +github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= +github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= +github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= +github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= +github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU= +github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo= +github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ= +github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= +github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= +github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= +github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= +github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc= +github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw= +github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA= +github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ= +github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= +github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/prometheus/client_golang v1.23.2 h1:Je96obch5RDVy3FDMndoUsjAhG5Edi49h0RJWRi/o0o= +github.com/prometheus/client_golang v1.23.2/go.mod h1:Tb1a6LWHB3/SPIzCoaDXI4I8UHKeFTEQ1YCr+0Gyqmg= +github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk= +github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE= +github.com/prometheus/common v0.66.1 h1:h5E0h5/Y8niHc5DlaLlWLArTQI7tMrsfQjHV+d9ZoGs= +github.com/prometheus/common v0.66.1/go.mod h1:gcaUsgf3KfRSwHY4dIMXLPV0K/Wg1oZ8+SbZk/HH/dA= +github.com/prometheus/procfs v0.16.1 h1:hZ15bTNuirocR6u0JZ6BAHHmwS1p8B4P6MRqxtzMyRg= +github.com/prometheus/procfs v0.16.1/go.mod h1:teAbpZRB1iIAJYREa1LsoWUXykVXA1KlTmWl8x/U+Is= +github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjRBZyWFQ= +github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog= +github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U= +github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U= +go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto= +go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE= +go.yaml.in/yaml/v2 v2.4.2 h1:DzmwEr2rDGHl7lsFgAHxmNz/1NlQ7xLIrlN2h5d1eGI= +go.yaml.in/yaml/v2 v2.4.2/go.mod h1:081UH+NErpNdqlCXm3TtEran0rJZGxAYx9hb/ELlsPU= +golang.org/x/sys v0.35.0 h1:vz1N37gP5bs89s7He8XuIYXpyY0+QlsKmzipCbUtyxI= +golang.org/x/sys v0.35.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k= +google.golang.org/protobuf v1.36.8 h1:xHScyCOEuuwZEc6UtSOvPbAT4zRh0xcNRYekJwfqyMc= +google.golang.org/protobuf v1.36.8/go.mod h1:fuxRtAxBytpl4zzqUh6/eyUujkJdNiuEkXntxiD/uRU= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= +gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= +gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= diff --git a/harnesses/transaction-fee/cmd/script/cardano.go b/harnesses/transaction-fee/cmd/script/cardano.go index 4cf92d28..0c7bbec6 100644 --- a/harnesses/transaction-fee/cmd/script/cardano.go +++ b/harnesses/transaction-fee/cmd/script/cardano.go @@ -51,8 +51,25 @@ type koiosEpochParams struct { MinFeeB float64 `json:"min_fee_b"` } +// Koios epoch params change once per 5-day epoch, but this fetcher runs +// on the 30s harness cycle. Caching for an hour keeps the total koios +// load around 24 calls/day (both OCB harnesses combined stay far under +// the 5,000/day public tier that we exhausted before this cache). +var cardanoParamCache struct { + minFeeA, minFeeB float64 + fetchedAt time.Time +} + func (f *cardanoFetcher) Sample(ch ChainConfig) ([]FeeSample, error) { + if time.Since(cardanoParamCache.fetchedAt) < time.Hour && cardanoParamCache.minFeeA > 0 { + feeLovelace := cardanoParamCache.minFeeB + (cardanoTypicalTxBytes * cardanoParamCache.minFeeA) + return []FeeSample{{Chain: ch.Slug, Tier: "single", NativeFee: feeLovelace}}, nil + } minFeeA, minFeeB, err := f.fetchProtocolParams(ch.RPCURL) + if err == nil { + cardanoParamCache.minFeeA, cardanoParamCache.minFeeB = minFeeA, minFeeB + cardanoParamCache.fetchedAt = time.Now() + } if err != nil { // Fall back to known-good constants rather than emit nothing. fmt.Printf("[cardano] koios fetch failed, using fallback constants: %v\n", err) diff --git a/harnesses/transaction-fee/cmd/script/config.go b/harnesses/transaction-fee/cmd/script/config.go index 24286094..15a20bb0 100644 --- a/harnesses/transaction-fee/cmd/script/config.go +++ b/harnesses/transaction-fee/cmd/script/config.go @@ -136,6 +136,15 @@ func loadConfig() *Config { HasPriorityMarket: true, Layer: "l2", }, + { + Slug: "robinhood", + Name: "Robinhood Chain", + Kind: KindEVM, + RPCURL: getenvDefault("RPC_ROBINHOOD", "https://rpc.mainnet.chain.robinhood.com"), + MobulaSlug: "ethereum", + HasPriorityMarket: true, + Layer: "l2", + }, { Slug: "base", Name: "Base", diff --git a/harnesses/usdy-nav-basis/Dockerfile b/harnesses/usdy-nav-basis/Dockerfile new file mode 100644 index 00000000..63108cfc --- /dev/null +++ b/harnesses/usdy-nav-basis/Dockerfile @@ -0,0 +1,22 @@ +FROM golang:1.24-alpine AS builder + +WORKDIR /app +RUN apk add --no-cache git + +COPY go.mod go.sum ./ +RUN go mod download + +COPY . . + +RUN CGO_ENABLED=0 GOOS=linux go build -o /app/monitor ./cmd/script + +FROM debian:bookworm-slim + +WORKDIR /app +RUN apt-get update && apt-get install -y ca-certificates && rm -rf /var/lib/apt/lists/* + +COPY --from=builder /app/monitor /app/monitor + +EXPOSE 2112 + +CMD ["/app/monitor"] diff --git a/harnesses/usdy-nav-basis/cmd/script/loghub.go b/harnesses/usdy-nav-basis/cmd/script/loghub.go new file mode 100644 index 00000000..9dd74488 --- /dev/null +++ b/harnesses/usdy-nav-basis/cmd/script/loghub.go @@ -0,0 +1,114 @@ +package main + +import ( + "bufio" + "fmt" + "io" + "net/http" + "os" + "strconv" + "sync" + "time" +) + +// Auto-generated by the loghub inline pattern. Captures stdout/stderr into a +// bounded ring buffer and exposes GET /logs?tail=N protected by X-Logs-Token +// matching the LOGS_TOKEN env var. +// +// Keep in sync across miniapps (was previously the shared/loghub package; we +// inline because Railway's per-harness Docker build context can't reach a +// sibling shared module via go.mod replace). + +const logRingMax = 5000 + +type logRing struct { + mu sync.Mutex + lines []string + max int +} + +var globalLogRing = &logRing{max: logRingMax} + +func (b *logRing) push(line string) { + entry := time.Now().UTC().Format("2006-01-02T15:04:05.000Z") + " " + line + b.mu.Lock() + if len(b.lines) >= b.max { + b.lines = append(b.lines[1:], entry) + } else { + b.lines = append(b.lines, entry) + } + b.mu.Unlock() +} + +func (b *logRing) snapshot(tail int) []string { + b.mu.Lock() + defer b.mu.Unlock() + if tail <= 0 || tail >= len(b.lines) { + out := make([]string, len(b.lines)) + copy(out, b.lines) + return out + } + start := len(b.lines) - tail + out := make([]string, tail) + copy(out, b.lines[start:]) + return out +} + +var logSetupOnce sync.Once + +// installLogCapture replaces os.Stdout (and os.Stderr) with the write-end of a +// pipe, then spawns a goroutine that fan-outs every line to the original +// stdout AND the in-memory ring buffer. Call exactly once, very early in +// main(). +func installLogCapture() { logSetupOnce.Do(doInstallLogCapture) } + +func doInstallLogCapture() { + originalStdout := os.Stdout + originalStderr := os.Stderr + r, w, err := os.Pipe() + if err != nil { + fmt.Fprintf(originalStdout, "[loghub] pipe failed: %v (/logs will be empty)\n", err) + return + } + os.Stdout = w + os.Stderr = w + + go func() { + scanner := bufio.NewScanner(r) + buf := make([]byte, 0, 1024*1024) + scanner.Buffer(buf, 1024*1024) + for scanner.Scan() { + line := scanner.Text() + fmt.Fprintln(originalStdout, line) + globalLogRing.push(line) + } + _, _ = io.Copy(originalStdout, r) + _ = originalStderr + }() +} + +// logsHandler returns an http.Handler for GET /logs?tail=N. Requires header +// X-Logs-Token to match the LOGS_TOKEN env var. Returns 403 if env unset. +func logsHandler() http.Handler { + return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + expected := os.Getenv("LOGS_TOKEN") + if expected == "" { + http.Error(w, "logs disabled: LOGS_TOKEN unset", http.StatusForbidden) + return + } + if r.Header.Get("X-Logs-Token") != expected { + http.Error(w, "forbidden", http.StatusForbidden) + return + } + tail := 500 + if t := r.URL.Query().Get("tail"); t != "" { + if n, err := strconv.Atoi(t); err == nil && n > 0 { + tail = n + } + } + w.Header().Set("Content-Type", "text/plain; charset=utf-8") + for _, l := range globalLogRing.snapshot(tail) { + fmt.Fprintln(w, l) + } + }) +} diff --git a/harnesses/usdy-nav-basis/cmd/script/main.go b/harnesses/usdy-nav-basis/cmd/script/main.go new file mode 100644 index 00000000..099fb63a --- /dev/null +++ b/harnesses/usdy-nav-basis/cmd/script/main.go @@ -0,0 +1,246 @@ +package main + +import ( + "bytes" + "encoding/base64" + "encoding/binary" + "encoding/json" + "fmt" + "io" + "math" + "math/big" + "net/http" + "os" + "strconv" + "strings" + "time" + + "github.com/prometheus/client_golang/prometheus" + "github.com/prometheus/client_golang/prometheus/promauto" + "github.com/prometheus/client_golang/prometheus/promhttp" +) + +// usdy-nav-basis: live basis between USDY's (Ondo tokenized treasury) +// onchain market price and its official redemption rate (NAV). +// +// NAV leg: Pyth Hermes "Crypto.USDY/USD.RR" redemption-rate feed. The +// same Hermes call also carries the Pyth USDY/USD market composite, +// which doubles as a cross-check venue row. Market leg: the Orca +// whirlpool USDY/USDC on Solana (deepest genuine pool, ~$2.9M), +// decoded straight from getAccountInfo (sqrtPrice u128 LE at bytes +// 65..81, both mints 6 decimals so no scale factor). +// +// Excluded, verified 2026-07-13: the Arbitrum Camelot pool holds 232 +// USDY against 7M USDC (drained), its price sits ~345 bps off NAV and +// moves nothing; kept out of the ranking, documented in the spec as +// the cautionary example of why pool depth gates peg quality. +// +// All legs keyless. One Hermes GET + one Solana RPC POST per tick. + +const ( + navFeedID = "e3d1723999820435ebab53003a542ff26847720692af92523eea613a9a28d500" + marketFeedID = "e393449f6aff8a4b6d3e1165a7c9ebec103685f3b41e60db4277b5b6d10e7326" + orcaPool = "AGXrswVDRoUf62UX9voTXv6TCGw6fBUEwDpyUd9YdZfD" + + pollInterval = 60 * time.Second + httpTimeout = 15 * time.Second +) + +var ( + basisBps = promauto.NewGaugeVec(prometheus.GaugeOpts{ + Name: "usdy_basis_bps", + Help: "Signed basis of the venue's USDY price vs the official redemption rate, in bps (positive = premium over NAV).", + }, []string{"venue"}) + + navGauge = promauto.NewGauge(prometheus.GaugeOpts{ + Name: "usdy_nav_usd", + Help: "USDY official redemption rate from the Pyth RR feed.", + }) + + marketPrice = promauto.NewGaugeVec(prometheus.GaugeOpts{ + Name: "usdy_market_price_usd", + Help: "USDY market price per venue.", + }, []string{"venue"}) + + sourceCall = promauto.NewCounterVec(prometheus.CounterOpts{ + Name: "usdy_source_call_total", + Help: "Fetch outcomes per source.", + }, []string{"source", "result"}) + + healthGauge = promauto.NewGaugeVec(prometheus.GaugeOpts{ + Name: "usdy_health", + Help: "1 when the venue produced a basis sample on the last tick.", + }, []string{"venue"}) +) + +func envDefault(key, def string) string { + if v := strings.TrimSpace(os.Getenv(key)); v != "" { + return v + } + return def +} + +func main() { + installLogCapture() + fmt.Println("=== USDY NAV Basis Harness ===") + fmt.Println("OpenChainBench — USDY market price vs official redemption rate, keyless.") + + go func() { + mux := http.NewServeMux() + mux.Handle("/metrics", promhttp.Handler()) + mux.Handle("/logs", logsHandler()) + mux.HandleFunc("/health", func(w http.ResponseWriter, _ *http.Request) { + w.WriteHeader(http.StatusOK) + _, _ = w.Write([]byte("ok")) + }) + if err := http.ListenAndServe(envDefault("LISTEN_ADDR", ":2112"), mux); err != nil { + fmt.Printf("[fatal] metrics server: %v\n", err) + } + }() + + client := &http.Client{Timeout: httpTimeout} + tick := func() { + nav, market := fetchHermes(client) + orca := fetchOrca(client) + + if nav <= 0 { + healthGauge.WithLabelValues("orca-solana").Set(0) + healthGauge.WithLabelValues("pyth-market").Set(0) + return + } + navGauge.Set(nav) + + emit := func(venue string, price float64) { + if price <= 0 { + healthGauge.WithLabelValues(venue).Set(0) + return + } + marketPrice.WithLabelValues(venue).Set(price) + bps := (price - nav) / nav * 10000 + basisBps.WithLabelValues(venue).Set(bps) + healthGauge.WithLabelValues(venue).Set(1) + fmt.Printf("[%s] price=%.6f nav=%.6f basis=%+.1fbps\n", venue, price, nav, bps) + } + emit("orca-solana", orca) + emit("pyth-market", market) + } + + tick() + t := time.NewTicker(pollInterval) + defer t.Stop() + for range t.C { + tick() + } +} + +// fetchHermes returns (nav, market) from one Hermes call. +func fetchHermes(client *http.Client) (float64, float64) { + url := "https://hermes.pyth.network/v2/updates/price/latest?ids[]=0x" + navFeedID + "&ids[]=0x" + marketFeedID + req, _ := http.NewRequest("GET", url, nil) + req.Header.Set("User-Agent", "OpenChainBench/1.0 (+https://openchainbench.com)") + resp, err := client.Do(req) + if err != nil { + sourceCall.WithLabelValues("hermes", "network").Inc() + return 0, 0 + } + defer resp.Body.Close() + raw, _ := io.ReadAll(io.LimitReader(resp.Body, 1<<20)) + if resp.StatusCode != 200 { + sourceCall.WithLabelValues("hermes", fmt.Sprintf("http_%d", resp.StatusCode)).Inc() + return 0, 0 + } + var envel struct { + Parsed []struct { + ID string `json:"id"` + Price struct { + Price string `json:"price"` + Expo int `json:"expo"` + } `json:"price"` + } `json:"parsed"` + } + if err := json.Unmarshal(raw, &envel); err != nil { + sourceCall.WithLabelValues("hermes", "parse").Inc() + return 0, 0 + } + nav, market := 0.0, 0.0 + for _, p := range envel.Parsed { + v, err := strconv.ParseFloat(p.Price.Price, 64) + if err != nil { + continue + } + val := v * math.Pow10(p.Price.Expo) + switch strings.TrimPrefix(strings.ToLower(p.ID), "0x") { + case navFeedID: + nav = val + case marketFeedID: + market = val + } + } + sourceCall.WithLabelValues("hermes", "ok").Inc() + return nav, market +} + +// fetchOrca decodes the whirlpool sqrtPrice (u128 LE at bytes 65..81); +// USDY and USDC are both 6 decimals so price = (sqrtPrice/2^64)^2. +// publicnode.com burst-rate-limits us to ~40% getAccountInfo success on +// the VPS; 2 retries with backoff take that to >95% without changing RPCs. +func fetchOrca(client *http.Client) float64 { + for attempt := 0; attempt < 3; attempt++ { + if attempt > 0 { + time.Sleep(time.Duration(attempt) * 500 * time.Millisecond) + } + if p := fetchOrcaOnce(client); p > 0 { + return p + } + } + return 0 +} + +func fetchOrcaOnce(client *http.Client) float64 { + body := []byte(fmt.Sprintf( + `{"jsonrpc":"2.0","id":%d,"method":"getAccountInfo","params":["%s",{"encoding":"base64","commitment":"confirmed"}]}`, + time.Now().UnixNano(), orcaPool, + )) + req, _ := http.NewRequest("POST", envDefault("USDY_SOLANA_RPC", "https://solana-rpc.publicnode.com"), bytes.NewReader(body)) + req.Header.Set("Content-Type", "application/json") + req.Header.Set("User-Agent", "OpenChainBench/1.0 (+https://openchainbench.com)") + resp, err := client.Do(req) + if err != nil { + sourceCall.WithLabelValues("orca", "network").Inc() + return 0 + } + defer resp.Body.Close() + raw, _ := io.ReadAll(io.LimitReader(resp.Body, 1<<20)) + if resp.StatusCode != 200 { + sourceCall.WithLabelValues("orca", fmt.Sprintf("http_%d", resp.StatusCode)).Inc() + return 0 + } + var envel struct { + Result struct { + Value struct { + Data []string `json:"data"` + } `json:"value"` + } `json:"result"` + } + if err := json.Unmarshal(raw, &envel); err != nil || len(envel.Result.Value.Data) == 0 { + sourceCall.WithLabelValues("orca", "parse").Inc() + return 0 + } + acct, err := base64.StdEncoding.DecodeString(envel.Result.Value.Data[0]) + if err != nil || len(acct) < 81 { + sourceCall.WithLabelValues("orca", "decode").Inc() + return 0 + } + lo := binary.LittleEndian.Uint64(acct[65:73]) + hi := binary.LittleEndian.Uint64(acct[73:81]) + sqrtPrice := new(big.Float).SetPrec(200).SetInt(new(big.Int).Add( + new(big.Int).Lsh(new(big.Int).SetUint64(hi), 64), + new(big.Int).SetUint64(lo), + )) + q64 := new(big.Float).SetPrec(200).SetInt(new(big.Int).Lsh(big.NewInt(1), 64)) + ratio := new(big.Float).Quo(sqrtPrice, q64) + price := new(big.Float).Mul(ratio, ratio) + f, _ := price.Float64() + sourceCall.WithLabelValues("orca", "ok").Inc() + return f +} diff --git a/harnesses/usdy-nav-basis/go.mod b/harnesses/usdy-nav-basis/go.mod new file mode 100644 index 00000000..1284070e --- /dev/null +++ b/harnesses/usdy-nav-basis/go.mod @@ -0,0 +1,18 @@ +module usdy-nav-basis + +go 1.24.0 + +require github.com/prometheus/client_golang v1.23.2 + +require ( + github.com/beorn7/perks v1.0.1 // indirect + github.com/cespare/xxhash/v2 v2.3.0 // indirect + github.com/kr/text v0.2.0 // indirect + github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect + github.com/prometheus/client_model v0.6.2 // indirect + github.com/prometheus/common v0.66.1 // indirect + github.com/prometheus/procfs v0.16.1 // indirect + go.yaml.in/yaml/v2 v2.4.2 // indirect + golang.org/x/sys v0.35.0 // indirect + google.golang.org/protobuf v1.36.8 // indirect +) diff --git a/harnesses/usdy-nav-basis/go.sum b/harnesses/usdy-nav-basis/go.sum new file mode 100644 index 00000000..d6b8ca98 --- /dev/null +++ b/harnesses/usdy-nav-basis/go.sum @@ -0,0 +1,46 @@ +github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= +github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= +github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs= +github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= +github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= +github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= +github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= +github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU= +github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo= +github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ= +github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= +github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= +github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= +github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= +github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc= +github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw= +github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA= +github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ= +github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= +github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/prometheus/client_golang v1.23.2 h1:Je96obch5RDVy3FDMndoUsjAhG5Edi49h0RJWRi/o0o= +github.com/prometheus/client_golang v1.23.2/go.mod h1:Tb1a6LWHB3/SPIzCoaDXI4I8UHKeFTEQ1YCr+0Gyqmg= +github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk= +github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE= +github.com/prometheus/common v0.66.1 h1:h5E0h5/Y8niHc5DlaLlWLArTQI7tMrsfQjHV+d9ZoGs= +github.com/prometheus/common v0.66.1/go.mod h1:gcaUsgf3KfRSwHY4dIMXLPV0K/Wg1oZ8+SbZk/HH/dA= +github.com/prometheus/procfs v0.16.1 h1:hZ15bTNuirocR6u0JZ6BAHHmwS1p8B4P6MRqxtzMyRg= +github.com/prometheus/procfs v0.16.1/go.mod h1:teAbpZRB1iIAJYREa1LsoWUXykVXA1KlTmWl8x/U+Is= +github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjRBZyWFQ= +github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog= +github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U= +github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U= +go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto= +go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE= +go.yaml.in/yaml/v2 v2.4.2 h1:DzmwEr2rDGHl7lsFgAHxmNz/1NlQ7xLIrlN2h5d1eGI= +go.yaml.in/yaml/v2 v2.4.2/go.mod h1:081UH+NErpNdqlCXm3TtEran0rJZGxAYx9hb/ELlsPU= +golang.org/x/sys v0.35.0 h1:vz1N37gP5bs89s7He8XuIYXpyY0+QlsKmzipCbUtyxI= +golang.org/x/sys v0.35.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k= +google.golang.org/protobuf v1.36.8 h1:xHScyCOEuuwZEc6UtSOvPbAT4zRh0xcNRYekJwfqyMc= +google.golang.org/protobuf v1.36.8/go.mod h1:fuxRtAxBytpl4zzqUh6/eyUujkJdNiuEkXntxiD/uRU= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= +gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= +gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= diff --git a/harnesses/ws-head-latency/Dockerfile b/harnesses/ws-head-latency/Dockerfile new file mode 100644 index 00000000..63108cfc --- /dev/null +++ b/harnesses/ws-head-latency/Dockerfile @@ -0,0 +1,22 @@ +FROM golang:1.24-alpine AS builder + +WORKDIR /app +RUN apk add --no-cache git + +COPY go.mod go.sum ./ +RUN go mod download + +COPY . . + +RUN CGO_ENABLED=0 GOOS=linux go build -o /app/monitor ./cmd/script + +FROM debian:bookworm-slim + +WORKDIR /app +RUN apt-get update && apt-get install -y ca-certificates && rm -rf /var/lib/apt/lists/* + +COPY --from=builder /app/monitor /app/monitor + +EXPOSE 2112 + +CMD ["/app/monitor"] diff --git a/harnesses/ws-head-latency/cmd/script/aggregator.go b/harnesses/ws-head-latency/cmd/script/aggregator.go new file mode 100644 index 00000000..87e384bc --- /dev/null +++ b/harnesses/ws-head-latency/cmd/script/aggregator.go @@ -0,0 +1,190 @@ +package main + +import ( + "fmt" + "sort" + "sync" + "time" +) + +// Per-chain arrival cohort. +// +// Every (provider, chain) connection reports "provider P saw block N at +// time T" into the chain's aggregator. The first report for a block +// opens a collection window (settleWindow); when it expires the cohort +// is scored: +// +// T0 = earliest arrival among providers +// lag(provider) = arrival(provider) - T0 → histogram sample +// winner = provider at T0 → wins counter +// gap = provider healthy on this chain but absent from the +// cohort → gap counter +// +// The measurement is deliberately RELATIVE: one vantage point cannot +// separate its own network path from provider pipeline latency, but the +// per-block delta between providers over the same path is a fair +// ordering. The fastest provider reads ~0 by construction; the spec's +// editorial copy discloses this. + +const ( + // settleWindow is how long a block cohort stays open after the first + // arrival. 5s covers everything we've observed (worst straggler + // ~1.5s on ethereum); providers later than this are scored as a gap, + // which is the honest reading for a push feed. + settleWindow = 5 * time.Second + // staleAfter bounds "healthy": a provider absent from a cohort only + // counts as a gap when it delivered *something* on this chain within + // staleAfter (otherwise it's just disconnected, which ws_health and + // ws_reconnects_total already report). + staleAfter = 120 * time.Second +) + +type cohortEntry struct { + arrivals map[string]time.Time +} + +type aggregator struct { + chain string + + mu sync.Mutex + blocks map[int64]*cohortEntry + maxFinalized int64 + // lastMsg tracks per-provider liveness on this chain (any accepted + // head/slot). Used for gap attribution and the ws_health sweeper. + lastMsg map[string]time.Time +} + +var ( + aggregatorsMu sync.Mutex + aggregators = map[string]*aggregator{} +) + +func aggregatorFor(chain string) *aggregator { + aggregatorsMu.Lock() + defer aggregatorsMu.Unlock() + if a, ok := aggregators[chain]; ok { + return a + } + a := &aggregator{ + chain: chain, + blocks: map[int64]*cohortEntry{}, + lastMsg: map[string]time.Time{}, + } + aggregators[chain] = a + return a +} + +// record ingests one arrival. Returns false when the arrival was +// dropped (duplicate from the same provider, or a straggler for an +// already-scored block). +func (a *aggregator) record(provider string, height int64, now time.Time) bool { + a.mu.Lock() + defer a.mu.Unlock() + + a.lastMsg[provider] = now + + // Straggler for a cohort that already settled. From a push-feed + // consumer's perspective this block was missed (the gap counter was + // bumped at settle time); opening a fresh single-provider cohort + // here would fabricate a lag=0 "win", so drop it. + if height <= a.maxFinalized { + return false + } + + e, ok := a.blocks[height] + if !ok { + e = &cohortEntry{arrivals: map[string]time.Time{}} + a.blocks[height] = e + time.AfterFunc(settleWindow, func() { a.settle(height) }) + } + if _, dup := e.arrivals[provider]; dup { + return false + } + e.arrivals[provider] = now + headBlocksSeen.WithLabelValues(provider, a.chain).Inc() + return true +} + +// settle scores one block cohort and releases it. +func (a *aggregator) settle(height int64) { + a.mu.Lock() + e, ok := a.blocks[height] + if !ok { + a.mu.Unlock() + return + } + delete(a.blocks, height) + if height > a.maxFinalized { + a.maxFinalized = height + } + // Snapshot healthy providers for gap attribution while still locked. + settleTime := time.Now() + healthy := make([]string, 0, len(a.lastMsg)) + for p, t := range a.lastMsg { + if settleTime.Sub(t) <= staleAfter { + healthy = append(healthy, p) + } + } + a.mu.Unlock() + + // A single-provider cohort carries no relative information: lag + // would be 0 by definition and the "win" unearned. Count gaps for + // the healthy absentees (they truly missed a block others pushed) + // but emit no lag samples. + if len(e.arrivals) >= 2 { + providers := make([]string, 0, len(e.arrivals)) + for p := range e.arrivals { + providers = append(providers, p) + } + // Deterministic tie-break: earliest arrival wins; equal + // timestamps resolve alphabetically. + sort.Strings(providers) + winner := providers[0] + t0 := e.arrivals[winner] + for _, p := range providers[1:] { + if e.arrivals[p].Before(t0) { + winner, t0 = p, e.arrivals[p] + } + } + for _, p := range providers { + lagMs := float64(e.arrivals[p].Sub(t0).Milliseconds()) + headLagHist.WithLabelValues(p, a.chain).Observe(lagMs) + } + headWins.WithLabelValues(winner, a.chain).Inc() + fmt.Printf("[%s] block=%d cohort=%d winner=%s\n", a.chain, height, len(e.arrivals), winner) + } + + for _, p := range healthy { + if _, saw := e.arrivals[p]; !saw { + headBlockGap.WithLabelValues(p, a.chain).Inc() + } + } +} + +// startHealthSweeper flips ws_health to 0 for any (provider, chain) +// that stopped delivering messages, per the gauge's contract (1 = +// connected AND received a message in the last 120s). The connection +// goroutines set it back to 1 on every accepted message. +func startHealthSweeper() { + go func() { + t := time.NewTicker(15 * time.Second) + defer t.Stop() + for now := range t.C { + aggregatorsMu.Lock() + aggs := make([]*aggregator, 0, len(aggregators)) + for _, a := range aggregators { + aggs = append(aggs, a) + } + aggregatorsMu.Unlock() + for _, a := range aggs { + a.mu.Lock() + for p, last := range a.lastMsg { + if now.Sub(last) > staleAfter { + wsHealth.WithLabelValues(p, a.chain).Set(0) + } + } + a.mu.Unlock() + } + } + }() +} diff --git a/harnesses/ws-head-latency/cmd/script/config.go b/harnesses/ws-head-latency/cmd/script/config.go new file mode 100644 index 00000000..1c30cfaa --- /dev/null +++ b/harnesses/ws-head-latency/cmd/script/config.go @@ -0,0 +1,70 @@ +package main + +import ( + "os" + "strings" +) + +// Endpoint is one (provider, chain) WebSocket subscription. Each entry +// gets its own goroutine holding a persistent connection; the arrival +// timestamps from every endpoint on the same chain are pooled by the +// per-chain cohort (see cohort.go logic in evm_ws.go / solana_ws.go and +// the shared aggregator below). +type Endpoint struct { + Provider string + Chain string // ethereum | base | solana + Kind string // "evm" (eth_subscribe newHeads) | "solana" (slotSubscribe) + URL string +} + +// endpoints is the source of truth for the measured cohort. All default +// URLs are public and keyless, verified live at harness inception: +// +// publicnode: ethereum / base / solana all push events +// drpc: ethereum verified; base + solana declared and allowed to +// fail at runtime (per-connection reconnect loop degrades +// gracefully, never crashes the process) +// tenderly: ethereum only (no keyless base/solana WS) +// +// Optional keyed providers are appended from env vars so contributor +// keys can widen the cohort without a rebuild. Every default URL is +// also env-overridable via WS_URL__. +func endpoints() []Endpoint { + eps := []Endpoint{ + {Provider: "publicnode", Chain: "ethereum", Kind: "evm", URL: envDefault("WS_URL_PUBLICNODE_ETHEREUM", "wss://ethereum-rpc.publicnode.com")}, + {Provider: "publicnode", Chain: "base", Kind: "evm", URL: envDefault("WS_URL_PUBLICNODE_BASE", "wss://base-rpc.publicnode.com")}, + {Provider: "publicnode", Chain: "solana", Kind: "solana", URL: envDefault("WS_URL_PUBLICNODE_SOLANA", "wss://solana-rpc.publicnode.com")}, + {Provider: "drpc", Chain: "ethereum", Kind: "evm", URL: envDefault("WS_URL_DRPC_ETHEREUM", "wss://eth.drpc.org")}, + {Provider: "drpc", Chain: "base", Kind: "evm", URL: envDefault("WS_URL_DRPC_BASE", "wss://base.drpc.org")}, + {Provider: "drpc", Chain: "solana", Kind: "solana", URL: envDefault("WS_URL_DRPC_SOLANA", "wss://solana.drpc.org")}, + {Provider: "tenderly", Chain: "ethereum", Kind: "evm", URL: envDefault("WS_URL_TENDERLY_ETHEREUM", "wss://mainnet.gateway.tenderly.co")}, + {Provider: "solana-labs", Chain: "solana", Kind: "solana", URL: envDefault("WS_URL_SOLANALABS_SOLANA", "wss://api.mainnet-beta.solana.com")}, + } + // Keyed providers, skipped when the env var is unset. URLs carry the + // key inline (e.g. wss://eth-mainnet.g.alchemy.com/v2/) so they + // live in the deployment env, never in the repo. + for _, opt := range []struct{ env, provider, chain string }{ + {"WS_URL_ALCHEMY_ETHEREUM", "alchemy", "ethereum"}, + {"WS_URL_INFURA_ETHEREUM", "infura", "ethereum"}, + {"WS_URL_CHAINSTACK_ETHEREUM", "chainstack", "ethereum"}, + } { + if url := strings.TrimSpace(os.Getenv(opt.env)); url != "" { + eps = append(eps, Endpoint{Provider: opt.provider, Chain: opt.chain, Kind: "evm", URL: url}) + } + } + return eps +} + +func envDefault(key, def string) string { + if v := strings.TrimSpace(os.Getenv(key)); v != "" { + return v + } + return def +} + +// listenAddr defaults to :2112, the OCB harness convention (scraped by +// the shared Prometheus). LISTEN_ADDR overrides for local runs so a dev +// copy doesn't collide with a deployed harness. +func listenAddr() string { + return envDefault("LISTEN_ADDR", ":2112") +} diff --git a/harnesses/ws-head-latency/cmd/script/evm_ws.go b/harnesses/ws-head-latency/cmd/script/evm_ws.go new file mode 100644 index 00000000..b69fa750 --- /dev/null +++ b/harnesses/ws-head-latency/cmd/script/evm_ws.go @@ -0,0 +1,212 @@ +package main + +import ( + "encoding/json" + "fmt" + "strconv" + "strings" + "sync/atomic" + "time" + + "github.com/gorilla/websocket" +) + +// Persistent WebSocket machinery, ported from the l2-block-time +// harness. One goroutine per (provider, chain) endpoint: +// +// - dial with timeout, subscribe, read loop with rolling deadline +// - client ping every 30s; pong handler extends the read deadline +// - head watchdog: the pong handler keeps the read deadline alive, so +// a provider that heartbeats while silently dropping the +// subscription (observed on publicnode) would wedge the goroutine +// forever. If no head/slot arrives for headWatchdog, slam the +// connection closed and let the outer loop redial. +// - reconnect with exponential backoff 2s → 60s; per-connection, so a +// dead endpoint (e.g. drpc/base if delisted) degrades to health=0 + +// periodic redials without touching the rest of the cohort. + +const ( + dialTimeout = 15 * time.Second + readDeadline = 60 * time.Second + pingEvery = 30 * time.Second + minBackoff = 2 * time.Second + maxBackoff = 60 * time.Second + // headWatchdog must sit well above the slowest cadence in the panel + // (ethereum, 12s slots; solana pushes multiple times per second). + // 90s of silence is unambiguously a broken subscription. + headWatchdog = 90 * time.Second +) + +type rpcReq struct { + JSONRPC string `json:"jsonrpc"` + ID int `json:"id"` + Method string `json:"method"` + Params []any `json:"params"` +} + +// evmEnvelope matches both the eth_subscription push frame and the +// subscribe ack. +type evmEnvelope struct { + Method string `json:"method"` + Params *struct { + Result json.RawMessage `json:"result"` + } `json:"params,omitempty"` +} + +type evmHead struct { + Number string `json:"number"` + Hash string `json:"hash"` +} + +// StartEndpoints spawns the reconnect loop for every configured +// (provider, chain) connection. +func StartEndpoints() { + for _, ep := range endpoints() { + ep := ep + go func() { + backoff := minBackoff + for { + start := time.Now() + var err error + switch ep.Kind { + case "solana": + err = runSolanaConn(ep) + default: + err = runEVMConn(ep) + } + wsHealth.WithLabelValues(ep.Provider, ep.Chain).Set(0) + wsReconnects.WithLabelValues(ep.Provider, ep.Chain).Inc() + if err != nil { + fmt.Printf("[%s/%s] WS error: %v (reconnecting in %v)\n", ep.Provider, ep.Chain, err, backoff) + } + // A connection that held for a while earns a fresh + // backoff; only consecutive fast failures escalate. + if time.Since(start) > 5*time.Minute { + backoff = minBackoff + } + time.Sleep(backoff) + if backoff < maxBackoff { + backoff *= 2 + if backoff > maxBackoff { + backoff = maxBackoff + } + } + } + }() + } +} + +// dialAndGuard opens the connection and installs ping + watchdog +// goroutines shared by the EVM and Solana read loops. The returned +// cleanup must be deferred; bumpHead must be called on every accepted +// head/slot. +func dialAndGuard(ep Endpoint) (conn *websocket.Conn, bumpHead func(), cleanup func(), err error) { + dialer := *websocket.DefaultDialer + dialer.HandshakeTimeout = dialTimeout + conn, _, err = dialer.Dial(ep.URL, nil) + if err != nil { + return nil, nil, nil, fmt.Errorf("dial: %w", err) + } + + done := make(chan struct{}) + var lastHeadUnixNano atomic.Int64 + lastHeadUnixNano.Store(time.Now().UnixNano()) + + go func() { + t := time.NewTicker(pingEvery) + defer t.Stop() + for { + select { + case <-done: + return + case <-t.C: + _ = conn.WriteControl(websocket.PingMessage, nil, time.Now().Add(5*time.Second)) + } + } + }() + go func() { + t := time.NewTicker(15 * time.Second) + defer t.Stop() + for { + select { + case <-done: + return + case now := <-t.C: + silent := now.Sub(time.Unix(0, lastHeadUnixNano.Load())) + if silent > headWatchdog { + fmt.Printf("[%s/%s] watchdog: no event for %s, forcing reconnect\n", ep.Provider, ep.Chain, silent.Round(time.Second)) + _ = conn.Close() + return + } + } + } + }() + + conn.SetPongHandler(func(string) error { + _ = conn.SetReadDeadline(time.Now().Add(readDeadline)) + return nil + }) + + bumpHead = func() { lastHeadUnixNano.Store(time.Now().UnixNano()) } + cleanup = func() { + close(done) + _ = conn.Close() + } + return conn, bumpHead, cleanup, nil +} + +func runEVMConn(ep Endpoint) error { + conn, bumpHead, cleanup, err := dialAndGuard(ep) + if err != nil { + return err + } + defer cleanup() + + if err := conn.WriteJSON(rpcReq{ + JSONRPC: "2.0", ID: 1, Method: "eth_subscribe", + Params: []any{"newHeads"}, + }); err != nil { + return fmt.Errorf("subscribe: %w", err) + } + + wsHealth.WithLabelValues(ep.Provider, ep.Chain).Set(1) + fmt.Printf("[%s/%s] connected, subscribed to newHeads\n", ep.Provider, ep.Chain) + + agg := aggregatorFor(ep.Chain) + for { + _ = conn.SetReadDeadline(time.Now().Add(readDeadline)) + _, msg, err := conn.ReadMessage() + if err != nil { + return fmt.Errorf("read: %w", err) + } + + var env evmEnvelope + if err := json.Unmarshal(msg, &env); err != nil { + continue + } + if env.Method != "eth_subscription" || env.Params == nil { + continue // subscribe ack or unrelated frame + } + var head evmHead + if err := json.Unmarshal(env.Params.Result, &head); err != nil { + continue + } + blockNum := parseHexInt64(head.Number) + if blockNum == 0 { + continue + } + now := time.Now() + bumpHead() + wsHealth.WithLabelValues(ep.Provider, ep.Chain).Set(1) + agg.record(ep.Provider, blockNum, now) + } +} + +func parseHexInt64(s string) int64 { + s = strings.TrimPrefix(s, "0x") + if s == "" { + return 0 + } + n, _ := strconv.ParseInt(s, 16, 64) + return n +} diff --git a/harnesses/ws-head-latency/cmd/script/loghub.go b/harnesses/ws-head-latency/cmd/script/loghub.go new file mode 100644 index 00000000..9dd74488 --- /dev/null +++ b/harnesses/ws-head-latency/cmd/script/loghub.go @@ -0,0 +1,114 @@ +package main + +import ( + "bufio" + "fmt" + "io" + "net/http" + "os" + "strconv" + "sync" + "time" +) + +// Auto-generated by the loghub inline pattern. Captures stdout/stderr into a +// bounded ring buffer and exposes GET /logs?tail=N protected by X-Logs-Token +// matching the LOGS_TOKEN env var. +// +// Keep in sync across miniapps (was previously the shared/loghub package; we +// inline because Railway's per-harness Docker build context can't reach a +// sibling shared module via go.mod replace). + +const logRingMax = 5000 + +type logRing struct { + mu sync.Mutex + lines []string + max int +} + +var globalLogRing = &logRing{max: logRingMax} + +func (b *logRing) push(line string) { + entry := time.Now().UTC().Format("2006-01-02T15:04:05.000Z") + " " + line + b.mu.Lock() + if len(b.lines) >= b.max { + b.lines = append(b.lines[1:], entry) + } else { + b.lines = append(b.lines, entry) + } + b.mu.Unlock() +} + +func (b *logRing) snapshot(tail int) []string { + b.mu.Lock() + defer b.mu.Unlock() + if tail <= 0 || tail >= len(b.lines) { + out := make([]string, len(b.lines)) + copy(out, b.lines) + return out + } + start := len(b.lines) - tail + out := make([]string, tail) + copy(out, b.lines[start:]) + return out +} + +var logSetupOnce sync.Once + +// installLogCapture replaces os.Stdout (and os.Stderr) with the write-end of a +// pipe, then spawns a goroutine that fan-outs every line to the original +// stdout AND the in-memory ring buffer. Call exactly once, very early in +// main(). +func installLogCapture() { logSetupOnce.Do(doInstallLogCapture) } + +func doInstallLogCapture() { + originalStdout := os.Stdout + originalStderr := os.Stderr + r, w, err := os.Pipe() + if err != nil { + fmt.Fprintf(originalStdout, "[loghub] pipe failed: %v (/logs will be empty)\n", err) + return + } + os.Stdout = w + os.Stderr = w + + go func() { + scanner := bufio.NewScanner(r) + buf := make([]byte, 0, 1024*1024) + scanner.Buffer(buf, 1024*1024) + for scanner.Scan() { + line := scanner.Text() + fmt.Fprintln(originalStdout, line) + globalLogRing.push(line) + } + _, _ = io.Copy(originalStdout, r) + _ = originalStderr + }() +} + +// logsHandler returns an http.Handler for GET /logs?tail=N. Requires header +// X-Logs-Token to match the LOGS_TOKEN env var. Returns 403 if env unset. +func logsHandler() http.Handler { + return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + expected := os.Getenv("LOGS_TOKEN") + if expected == "" { + http.Error(w, "logs disabled: LOGS_TOKEN unset", http.StatusForbidden) + return + } + if r.Header.Get("X-Logs-Token") != expected { + http.Error(w, "forbidden", http.StatusForbidden) + return + } + tail := 500 + if t := r.URL.Query().Get("tail"); t != "" { + if n, err := strconv.Atoi(t); err == nil && n > 0 { + tail = n + } + } + w.Header().Set("Content-Type", "text/plain; charset=utf-8") + for _, l := range globalLogRing.snapshot(tail) { + fmt.Fprintln(w, l) + } + }) +} diff --git a/harnesses/ws-head-latency/cmd/script/main.go b/harnesses/ws-head-latency/cmd/script/main.go new file mode 100644 index 00000000..2079aeaa --- /dev/null +++ b/harnesses/ws-head-latency/cmd/script/main.go @@ -0,0 +1,38 @@ +package main + +import ( + "fmt" + "os" + "os/signal" + "syscall" +) + +func main() { + installLogCapture() // capture stdout into /logs ring buffer + fmt.Println("=== WS Head Latency Monitor ===") + fmt.Println("Bench № 081 — per-block WebSocket push lag across RPC providers, relative to the earliest arrival in the cohort.") + fmt.Println() + + eps := endpoints() + addr := listenAddr() + fmt.Printf("Connections tracked: %d\n", len(eps)) + for _, ep := range eps { + fmt.Printf(" %-12s %-9s %s\n", ep.Provider, ep.Chain, ep.URL) + } + fmt.Printf("Metrics server: %s/metrics\n\n", addr) + + go func() { + if err := StartMetricsServer(addr); err != nil { + fmt.Printf("[fatal] metrics server: %v\n", err) + os.Exit(1) + } + }() + + StartEndpoints() + startHealthSweeper() + + sig := make(chan os.Signal, 1) + signal.Notify(sig, os.Interrupt, syscall.SIGTERM) + s := <-sig + fmt.Printf("\n[shutdown] received %v\n", s) +} diff --git a/harnesses/ws-head-latency/cmd/script/metrics.go b/harnesses/ws-head-latency/cmd/script/metrics.go new file mode 100644 index 00000000..e2940ee6 --- /dev/null +++ b/harnesses/ws-head-latency/cmd/script/metrics.go @@ -0,0 +1,81 @@ +package main + +import ( + "net/http" + + "github.com/prometheus/client_golang/prometheus" + "github.com/prometheus/client_golang/prometheus/promauto" + "github.com/prometheus/client_golang/prometheus/promhttp" +) + +// Prom metrics for the ws-head-latency harness. Names follow the +// `__` OCB convention. All series carry +// {provider, chain} so the spec can pin headline queries to one chain +// while keeping base/solana series available for future tabs. +var ( + headLagHist = promauto.NewHistogramVec( + prometheus.HistogramOpts{ + Name: "ws_head_lag_milliseconds", + Help: "Per-block push lag vs the earliest provider in the cohort, in milliseconds. Relative measurement: the fastest provider on a block reads 0 by construction.", + // 5ms → 10.24s exponential; covers same-frame ties through + // multi-second stragglers. + Buckets: prometheus.ExponentialBuckets(5, 2, 12), + }, + []string{"provider", "chain"}, + ) + + headWins = promauto.NewCounterVec( + prometheus.CounterOpts{ + Name: "ws_head_wins_total", + Help: "Blocks where this provider was the first in the cohort to push the head/slot. Only counted when at least 2 providers saw the block.", + }, + []string{"provider", "chain"}, + ) + + headBlocksSeen = promauto.NewCounterVec( + prometheus.CounterOpts{ + Name: "ws_head_blocks_seen_total", + Help: "Distinct blocks/slots this provider pushed within the cohort settle window. Ratio against the cohort max is the coverage score.", + }, + []string{"provider", "chain"}, + ) + + wsReconnects = promauto.NewCounterVec( + prometheus.CounterOpts{ + Name: "ws_reconnects_total", + Help: "WebSocket reconnections per (provider, chain) since process start. Sustained non-zero rate flags an unstable endpoint.", + }, + []string{"provider", "chain"}, + ) + + wsHealth = promauto.NewGaugeVec( + prometheus.GaugeOpts{ + Name: "ws_health", + Help: "1 when the subscription is connected and delivered a message in the last 120s, 0 otherwise.", + }, + []string{"provider", "chain"}, + ) + + headBlockGap = promauto.NewCounterVec( + prometheus.CounterOpts{ + Name: "ws_block_gap_total", + Help: "Blocks the cohort saw but this provider missed (absent from the 5s settle window while its connection was otherwise live).", + }, + []string{"provider", "chain"}, + ) +) + +// StartMetricsServer binds /metrics + /health + /logs on addr. Blocking +// call, run in its own goroutine. +func StartMetricsServer(addr string) error { + mux := http.NewServeMux() + mux.Handle("/metrics", promhttp.Handler()) + mux.Handle("/logs", logsHandler()) + mux.HandleFunc("/health", func(w http.ResponseWriter, r *http.Request) { + _, _ = w.Write([]byte("ok")) + }) + mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) { + _, _ = w.Write([]byte("ws-head-latency harness · OpenChainBench № 081")) + }) + return http.ListenAndServe(addr, mux) +} diff --git a/harnesses/ws-head-latency/cmd/script/solana_ws.go b/harnesses/ws-head-latency/cmd/script/solana_ws.go new file mode 100644 index 00000000..6ebf8cc3 --- /dev/null +++ b/harnesses/ws-head-latency/cmd/script/solana_ws.go @@ -0,0 +1,80 @@ +package main + +import ( + "encoding/json" + "fmt" + "time" +) + +// Solana slot push via `slotSubscribe`. The notification frame looks +// like: +// +// {"jsonrpc":"2.0","method":"slotNotification", +// "params":{"result":{"parent":364,"root":362,"slot":365},"subscription":0}} +// +// The slot number is the cohort key, playing the role the block number +// plays on EVM chains. Solana slots tick every ~400ms, so the cohort +// churn is higher but the settle/score path is identical. + +type solanaEnvelope struct { + Method string `json:"method"` + Params *struct { + Result struct { + Slot int64 `json:"slot"` + } `json:"result"` + } `json:"params,omitempty"` + // Subscribe ack: {"jsonrpc":"2.0","result":23784,"id":1} + Error *struct { + Code int `json:"code"` + Message string `json:"message"` + } `json:"error,omitempty"` +} + +func runSolanaConn(ep Endpoint) error { + conn, bumpHead, cleanup, err := dialAndGuard(ep) + if err != nil { + return err + } + defer cleanup() + + if err := conn.WriteJSON(rpcReq{ + JSONRPC: "2.0", ID: 1, Method: "slotSubscribe", + Params: []any{}, + }); err != nil { + return fmt.Errorf("subscribe: %w", err) + } + + wsHealth.WithLabelValues(ep.Provider, ep.Chain).Set(1) + fmt.Printf("[%s/%s] connected, subscribed to slotSubscribe\n", ep.Provider, ep.Chain) + + agg := aggregatorFor(ep.Chain) + for { + _ = conn.SetReadDeadline(time.Now().Add(readDeadline)) + _, msg, err := conn.ReadMessage() + if err != nil { + return fmt.Errorf("read: %w", err) + } + + var env solanaEnvelope + if err := json.Unmarshal(msg, &env); err != nil { + continue + } + if env.Error != nil { + // Endpoint answered the subscribe with a JSON-RPC error + // (method disabled, quota, ...). Surface it and let the + // outer loop back off instead of holding a dead socket. + return fmt.Errorf("subscribe error %d: %s", env.Error.Code, env.Error.Message) + } + if env.Method != "slotNotification" || env.Params == nil { + continue // subscribe ack or unrelated frame + } + slot := env.Params.Result.Slot + if slot == 0 { + continue + } + now := time.Now() + bumpHead() + wsHealth.WithLabelValues(ep.Provider, ep.Chain).Set(1) + agg.record(ep.Provider, slot, now) + } +} diff --git a/harnesses/ws-head-latency/go.mod b/harnesses/ws-head-latency/go.mod new file mode 100644 index 00000000..3bca706e --- /dev/null +++ b/harnesses/ws-head-latency/go.mod @@ -0,0 +1,21 @@ +module ws-head-latency + +go 1.24.0 + +require ( + github.com/gorilla/websocket v1.5.3 + github.com/prometheus/client_golang v1.23.2 +) + +require ( + github.com/beorn7/perks v1.0.1 // indirect + github.com/cespare/xxhash/v2 v2.3.0 // indirect + github.com/kr/text v0.2.0 // indirect + github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect + github.com/prometheus/client_model v0.6.2 // indirect + github.com/prometheus/common v0.66.1 // indirect + github.com/prometheus/procfs v0.16.1 // indirect + go.yaml.in/yaml/v2 v2.4.2 // indirect + golang.org/x/sys v0.35.0 // indirect + google.golang.org/protobuf v1.36.8 // indirect +) diff --git a/harnesses/ws-head-latency/go.sum b/harnesses/ws-head-latency/go.sum new file mode 100644 index 00000000..02807ba4 --- /dev/null +++ b/harnesses/ws-head-latency/go.sum @@ -0,0 +1,48 @@ +github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= +github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= +github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs= +github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= +github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= +github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= +github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= +github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU= +github.com/gorilla/websocket v1.5.3 h1:saDtZ6Pbx/0u+bgYQ3q96pZgCzfhKXGPqt7kZ72aNNg= +github.com/gorilla/websocket v1.5.3/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE= +github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo= +github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ= +github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= +github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= +github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= +github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= +github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc= +github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw= +github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA= +github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ= +github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= +github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/prometheus/client_golang v1.23.2 h1:Je96obch5RDVy3FDMndoUsjAhG5Edi49h0RJWRi/o0o= +github.com/prometheus/client_golang v1.23.2/go.mod h1:Tb1a6LWHB3/SPIzCoaDXI4I8UHKeFTEQ1YCr+0Gyqmg= +github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk= +github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE= +github.com/prometheus/common v0.66.1 h1:h5E0h5/Y8niHc5DlaLlWLArTQI7tMrsfQjHV+d9ZoGs= +github.com/prometheus/common v0.66.1/go.mod h1:gcaUsgf3KfRSwHY4dIMXLPV0K/Wg1oZ8+SbZk/HH/dA= +github.com/prometheus/procfs v0.16.1 h1:hZ15bTNuirocR6u0JZ6BAHHmwS1p8B4P6MRqxtzMyRg= +github.com/prometheus/procfs v0.16.1/go.mod h1:teAbpZRB1iIAJYREa1LsoWUXykVXA1KlTmWl8x/U+Is= +github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjRBZyWFQ= +github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog= +github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U= +github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U= +go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto= +go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE= +go.yaml.in/yaml/v2 v2.4.2 h1:DzmwEr2rDGHl7lsFgAHxmNz/1NlQ7xLIrlN2h5d1eGI= +go.yaml.in/yaml/v2 v2.4.2/go.mod h1:081UH+NErpNdqlCXm3TtEran0rJZGxAYx9hb/ELlsPU= +golang.org/x/sys v0.35.0 h1:vz1N37gP5bs89s7He8XuIYXpyY0+QlsKmzipCbUtyxI= +golang.org/x/sys v0.35.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k= +google.golang.org/protobuf v1.36.8 h1:xHScyCOEuuwZEc6UtSOvPbAT4zRh0xcNRYekJwfqyMc= +google.golang.org/protobuf v1.36.8/go.mod h1:fuxRtAxBytpl4zzqUh6/eyUujkJdNiuEkXntxiD/uRU= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= +gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= +gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= diff --git a/harnesses/xstocks-peg/Dockerfile b/harnesses/xstocks-peg/Dockerfile new file mode 100644 index 00000000..63108cfc --- /dev/null +++ b/harnesses/xstocks-peg/Dockerfile @@ -0,0 +1,22 @@ +FROM golang:1.24-alpine AS builder + +WORKDIR /app +RUN apk add --no-cache git + +COPY go.mod go.sum ./ +RUN go mod download + +COPY . . + +RUN CGO_ENABLED=0 GOOS=linux go build -o /app/monitor ./cmd/script + +FROM debian:bookworm-slim + +WORKDIR /app +RUN apt-get update && apt-get install -y ca-certificates && rm -rf /var/lib/apt/lists/* + +COPY --from=builder /app/monitor /app/monitor + +EXPOSE 2112 + +CMD ["/app/monitor"] diff --git a/harnesses/xstocks-peg/cmd/script/config.go b/harnesses/xstocks-peg/cmd/script/config.go new file mode 100644 index 00000000..b151be10 --- /dev/null +++ b/harnesses/xstocks-peg/cmd/script/config.go @@ -0,0 +1,60 @@ +package main + +import ( + "os" + "strings" + "time" +) + +// xstocks-peg: Backed's xStocks tokenized equities on Solana vs their +// Yahoo reference, deviation in bps, session-labeled. Same metric +// contract as the Robinhood harness (tsp_* family) with +// issuer="xstocks", so cross-issuer comparisons are pure PromQL. +// +// Price read: Jupiter lite-api swap quotes in BOTH directions per +// symbol (sell 1 share to USDC, buy with the equivalent USDC); the mid +// is the executable peg price, immune to the price/v3 drift observed +// on thin routes (PLTRx v3 was $1.93 off its executable quote). +// A single batched price/v3 call per tick supplies the Token-2022 +// ScaledUiAmount multiplier (7 of 12 mints carry one, ~1.0009), which +// converts raw 1e8 quote units to exactly one UI share. +// +// Cohort: 12 xStocks with verified Jupiter routes at 1-share impact +// under 2bp (2026-07-13). All mints 8 decimals. + +const ( + usdcMint = "EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v" + oneShareRaw = 100000000 // 1e8 = 1 share at 8 decimals, pre-multiplier + pollInterval = 60 * time.Second + httpTimeout = 15 * time.Second + quoteGap = 1100 * time.Millisecond // lite tier: stay well under 60 req/min + issuerLabel = "xstocks" + logThresholdBps = 100.0 +) + +type Asset struct { + Symbol string // Yahoo ticker (HOODx maps to HOOD, etc.) + Mint string +} + +var assets = []Asset{ + {Symbol: "TSLA", Mint: "XsDoVfqeBukxuZHWhdvWHBhgEHjGNst4MLodqsJHzoB"}, + {Symbol: "NVDA", Mint: "Xsc9qvGR1efVDFGLrVsmkzv3qi45LTBjeUKSPmx9qEh"}, + {Symbol: "AAPL", Mint: "XsbEhLAtcf6HdfpFZ5xEMdqW8nfAvcsP5bdudRLJzJp"}, + {Symbol: "MSFT", Mint: "XspzcW1PRtgf6Wj92HCiZdjzKCyFekVD8P5Ueh3dRMX"}, + {Symbol: "AMZN", Mint: "Xs3eBt7uRfJX8QUs4suhyU8p2M6DoUDrJyWBa8LLZsg"}, + {Symbol: "GOOGL", Mint: "XsCPL9dNWBMvFtTmwcCA5v3xWPSMEBCszbQdiLLq6aN"}, + {Symbol: "META", Mint: "Xsa62P5mvPszXL1krVUnU5ar38bBSVcWAB6fmPCo5Zu"}, + {Symbol: "HOOD", Mint: "XsvNBAYkrDRNhA7wPHQfX3ZUXZyZLdnCQDfHZ56bzpg"}, + {Symbol: "SPY", Mint: "XsoCS1TfEyfFhfvj8EtZ528L3CaKBDBRqRapnBbDF2W"}, + {Symbol: "QQQ", Mint: "Xs8S1uUs1zvS2p7iwtsG3b6fkhpvmwz4GYU3gWAmWHZ"}, + {Symbol: "COIN", Mint: "Xs7ZdzSHLU9ftNJsii5fCeJhoRWSC32SQGzGQtePxNu"}, + {Symbol: "PLTR", Mint: "XsoBhf2ufR8fTyNSjqfU71DYGaE6Z3SUGAidpzriAA4"}, +} + +func listenAddr() string { + if v := strings.TrimSpace(os.Getenv("LISTEN_ADDR")); v != "" { + return v + } + return ":2112" +} diff --git a/harnesses/xstocks-peg/cmd/script/jupiter.go b/harnesses/xstocks-peg/cmd/script/jupiter.go new file mode 100644 index 00000000..a34a4708 --- /dev/null +++ b/harnesses/xstocks-peg/cmd/script/jupiter.go @@ -0,0 +1,143 @@ +package main + +import ( + "encoding/json" + "fmt" + "io" + "net/http" + "strconv" + "strings" + "time" +) + +// Jupiter lite-api legs. Two swap quotes per symbol per tick (sell one +// share, buy one share worth of USDC back), spaced quoteGap apart to +// stay far under the lite tier's 60 req/min. The mid of the two +// implied prices is the executable peg price. One batched price/v3 +// call per tick provides the ScaledUiAmount multiplier per mint. + +const jupUA = "OpenChainBench/1.0 (+https://openchainbench.com)" + +func jupGet(client *http.Client, url string) ([]byte, string) { + req, err := http.NewRequest("GET", url, nil) + if err != nil { + return nil, "request_build" + } + req.Header.Set("User-Agent", jupUA) + req.Header.Set("Accept", "application/json") + resp, err := client.Do(req) + if err != nil { + return nil, "network" + } + defer resp.Body.Close() + raw, err := io.ReadAll(io.LimitReader(resp.Body, 1<<20)) + if err != nil { + return nil, "read" + } + if resp.StatusCode != 200 { + return nil, fmt.Sprintf("http_%d", resp.StatusCode) + } + return raw, "ok" +} + +// fetchMultipliers reads price/v3 for every mint in one call and +// derives the ScaledUiAmount multiplier as usdPrice / usdPricePrescaled +// (1.0 when the mint carries no scaling config). +func fetchMultipliers(client *http.Client) map[string]float64 { + ids := make([]string, 0, len(assets)) + for _, a := range assets { + ids = append(ids, a.Mint) + } + raw, status := jupGet(client, "https://lite-api.jup.ag/price/v3?ids="+strings.Join(ids, ",")) + if raw == nil { + tspSourceCall.WithLabelValues("jup_price", status).Inc() + return nil + } + var flat map[string]struct { + USDPrice float64 `json:"usdPrice"` + USDPricePrescaled float64 `json:"usdPricePrescaled"` + } + if err := json.Unmarshal(raw, &flat); err != nil { + tspSourceCall.WithLabelValues("jup_price", "parse").Inc() + return nil + } + out := make(map[string]float64, len(flat)) + for mint, v := range flat { + m := 1.0 + if v.USDPricePrescaled > 0 && v.USDPrice > 0 { + m = v.USDPrice / v.USDPricePrescaled + } + out[mint] = m + } + tspSourceCall.WithLabelValues("jup_price", "ok").Inc() + return out +} + +type quoteResp struct { + OutAmount string `json:"outAmount"` +} + +func quoteOut(client *http.Client, inMint, outMint string, amount int64) (float64, bool) { + url := fmt.Sprintf( + "https://lite-api.jup.ag/swap/v1/quote?inputMint=%s&outputMint=%s&amount=%d&slippageBps=100", + inMint, outMint, amount, + ) + raw, status := jupGet(client, url) + if raw == nil { + tspSourceCall.WithLabelValues("jup_quote", status).Inc() + return 0, false + } + var q quoteResp + if err := json.Unmarshal(raw, &q); err != nil || q.OutAmount == "" { + tspSourceCall.WithLabelValues("jup_quote", "parse").Inc() + return 0, false + } + n, err := strconv.ParseFloat(q.OutAmount, 64) + if err != nil || n <= 0 { + tspSourceCall.WithLabelValues("jup_quote", "decode").Inc() + return 0, false + } + tspSourceCall.WithLabelValues("jup_quote", "ok").Inc() + return n, true +} + +// fetchOnchainPrices returns the executable mid price in USDC per UI +// share for every symbol. Sequential with quoteGap spacing: ~26s for +// the 12-symbol cohort, comfortably inside the 60s tick. +func fetchOnchainPrices(client *http.Client, multipliers map[string]float64) map[string]float64 { + prices := make(map[string]float64, len(assets)) + start := time.Now() + for _, a := range assets { + mult := 1.0 + if m, ok := multipliers[a.Mint]; ok && m > 0 { + mult = m + } + + // Sell leg: 1 raw share -> USDC. + sellOut, okSell := quoteOut(client, a.Mint, usdcMint, oneShareRaw) + time.Sleep(quoteGap) + sellPx := 0.0 + if okSell { + sellPx = sellOut / 1e6 * mult + } + + // Buy leg: spend the sell proceeds, see how many shares return. + buyPx := 0.0 + if okSell { + gotRaw, okBuy := quoteOut(client, usdcMint, a.Mint, int64(sellOut)) + if okBuy && gotRaw > 0 { + buyPx = (sellOut / 1e6) / (gotRaw / 1e8 / mult) + } + time.Sleep(quoteGap) + } + + switch { + case sellPx > 0 && buyPx > 0: + prices[strings.ToLower(a.Symbol)] = (sellPx + buyPx) / 2 + case sellPx > 0: + prices[strings.ToLower(a.Symbol)] = sellPx + } + } + tspSourceLatency.WithLabelValues("onchain").Set(float64(time.Since(start).Milliseconds())) + return prices +} diff --git a/harnesses/xstocks-peg/cmd/script/loghub.go b/harnesses/xstocks-peg/cmd/script/loghub.go new file mode 100644 index 00000000..9dd74488 --- /dev/null +++ b/harnesses/xstocks-peg/cmd/script/loghub.go @@ -0,0 +1,114 @@ +package main + +import ( + "bufio" + "fmt" + "io" + "net/http" + "os" + "strconv" + "sync" + "time" +) + +// Auto-generated by the loghub inline pattern. Captures stdout/stderr into a +// bounded ring buffer and exposes GET /logs?tail=N protected by X-Logs-Token +// matching the LOGS_TOKEN env var. +// +// Keep in sync across miniapps (was previously the shared/loghub package; we +// inline because Railway's per-harness Docker build context can't reach a +// sibling shared module via go.mod replace). + +const logRingMax = 5000 + +type logRing struct { + mu sync.Mutex + lines []string + max int +} + +var globalLogRing = &logRing{max: logRingMax} + +func (b *logRing) push(line string) { + entry := time.Now().UTC().Format("2006-01-02T15:04:05.000Z") + " " + line + b.mu.Lock() + if len(b.lines) >= b.max { + b.lines = append(b.lines[1:], entry) + } else { + b.lines = append(b.lines, entry) + } + b.mu.Unlock() +} + +func (b *logRing) snapshot(tail int) []string { + b.mu.Lock() + defer b.mu.Unlock() + if tail <= 0 || tail >= len(b.lines) { + out := make([]string, len(b.lines)) + copy(out, b.lines) + return out + } + start := len(b.lines) - tail + out := make([]string, tail) + copy(out, b.lines[start:]) + return out +} + +var logSetupOnce sync.Once + +// installLogCapture replaces os.Stdout (and os.Stderr) with the write-end of a +// pipe, then spawns a goroutine that fan-outs every line to the original +// stdout AND the in-memory ring buffer. Call exactly once, very early in +// main(). +func installLogCapture() { logSetupOnce.Do(doInstallLogCapture) } + +func doInstallLogCapture() { + originalStdout := os.Stdout + originalStderr := os.Stderr + r, w, err := os.Pipe() + if err != nil { + fmt.Fprintf(originalStdout, "[loghub] pipe failed: %v (/logs will be empty)\n", err) + return + } + os.Stdout = w + os.Stderr = w + + go func() { + scanner := bufio.NewScanner(r) + buf := make([]byte, 0, 1024*1024) + scanner.Buffer(buf, 1024*1024) + for scanner.Scan() { + line := scanner.Text() + fmt.Fprintln(originalStdout, line) + globalLogRing.push(line) + } + _, _ = io.Copy(originalStdout, r) + _ = originalStderr + }() +} + +// logsHandler returns an http.Handler for GET /logs?tail=N. Requires header +// X-Logs-Token to match the LOGS_TOKEN env var. Returns 403 if env unset. +func logsHandler() http.Handler { + return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + expected := os.Getenv("LOGS_TOKEN") + if expected == "" { + http.Error(w, "logs disabled: LOGS_TOKEN unset", http.StatusForbidden) + return + } + if r.Header.Get("X-Logs-Token") != expected { + http.Error(w, "forbidden", http.StatusForbidden) + return + } + tail := 500 + if t := r.URL.Query().Get("tail"); t != "" { + if n, err := strconv.Atoi(t); err == nil && n > 0 { + tail = n + } + } + w.Header().Set("Content-Type", "text/plain; charset=utf-8") + for _, l := range globalLogRing.snapshot(tail) { + fmt.Fprintln(w, l) + } + }) +} diff --git a/harnesses/xstocks-peg/cmd/script/main.go b/harnesses/xstocks-peg/cmd/script/main.go new file mode 100644 index 00000000..5cc18415 --- /dev/null +++ b/harnesses/xstocks-peg/cmd/script/main.go @@ -0,0 +1,83 @@ +package main + +import ( + "fmt" + "math" + "net/http" + "strings" + "time" +) + +func main() { + installLogCapture() + fmt.Println("=== xStocks Peg Harness ===") + fmt.Println("OpenChainBench — Backed xStocks on Solana vs Nasdaq reference.") + fmt.Printf("Cohort: %d assets | poll: %s | Jupiter lite-api\n", len(assets), pollInterval) + for _, a := range assets { + fmt.Printf(" - %-6s mint=%s\n", a.Symbol, a.Mint) + } + + go func() { + if err := startMetricsServer(listenAddr()); err != nil { + fmt.Printf("[fatal] metrics server: %v\n", err) + } + }() + + client := &http.Client{Timeout: httpTimeout} + var periods *tradingPeriods + + tick := func() { + now := time.Now() + if periods == nil || now.Sub(periods.FetchedAt) > 30*time.Minute { + if tp := fetchTradingPeriods(client); tp != nil { + periods = tp + } + } + state := periods.state(now) + for _, s := range []string{"pre", "regular", "post", "closed", "unknown"} { + v := 0.0 + if s == state { + v = 1.0 + } + tspMarketState.WithLabelValues(s).Set(v) + } + + refs := fetchReferencePrices(client) + multipliers := fetchMultipliers(client) + onchain := fetchOnchainPrices(client, multipliers) + + for _, a := range assets { + sym := strings.ToLower(a.Symbol) + ref, hasRef := refs[sym] + pool, hasPool := onchain[sym] + if hasRef { + tspPriceReference.WithLabelValues(sym).Set(ref.Price) + if ref.AsOfSec > 0 { + tspRefAge.WithLabelValues(sym).Set(float64(now.Unix() - ref.AsOfSec)) + } + } + if hasPool { + tspPriceOnchain.WithLabelValues(sym, issuerLabel).Set(pool) + } + if hasRef && hasPool && ref.Price > 0 { + dev := math.Abs(pool-ref.Price) / ref.Price * 10000 + tspDeviationBps.WithLabelValues(sym, state, issuerLabel).Set(dev) + tspHealth.WithLabelValues(sym).Set(1) + flag := "" + if dev > logThresholdBps && state == "regular" { + flag = " <-- wide" + } + fmt.Printf("[%s][%s] pool=%.2f ref=%.2f dev=%.1fbps%s\n", sym, state, pool, ref.Price, dev, flag) + } else { + tspHealth.WithLabelValues(sym).Set(0) + } + } + } + + tick() + t := time.NewTicker(pollInterval) + defer t.Stop() + for range t.C { + tick() + } +} diff --git a/harnesses/xstocks-peg/cmd/script/metrics.go b/harnesses/xstocks-peg/cmd/script/metrics.go new file mode 100644 index 00000000..82072a68 --- /dev/null +++ b/harnesses/xstocks-peg/cmd/script/metrics.go @@ -0,0 +1,67 @@ +package main + +import ( + "net/http" + + "github.com/prometheus/client_golang/prometheus" + "github.com/prometheus/client_golang/prometheus/promauto" + "github.com/prometheus/client_golang/prometheus/promhttp" +) + +var ( + // Headline: absolute deviation of the onchain pool price from the + // Yahoo reference, in basis points, labeled with the market session + // the sample was taken in. The bench pins its ranking to + // market_state="regular"; the closed-state series is the weekend / + // overnight drift panel. + tspDeviationBps = promauto.NewGaugeVec(prometheus.GaugeOpts{ + Name: "tsp_deviation_bps", + Help: "Absolute onchain vs reference price deviation per tokenized stock, in bps, labeled by market session state.", + }, []string{"asset", "market_state", "issuer"}) + + tspPriceOnchain = promauto.NewGaugeVec(prometheus.GaugeOpts{ + Name: "tsp_price_onchain_usdg", + Help: "Venue spot price of the tokenized stock, in the venue quote stable.", + }, []string{"asset", "issuer"}) + + tspPriceReference = promauto.NewGaugeVec(prometheus.GaugeOpts{ + Name: "tsp_price_reference_usd", + Help: "Reference equity price from Yahoo Finance (regularMarketPrice; last close when the market is closed).", + }, []string{"asset"}) + + tspRefAge = promauto.NewGaugeVec(prometheus.GaugeOpts{ + Name: "tsp_reference_age_seconds", + Help: "Age of the reference price sample (now minus regularMarketTime). Large outside regular hours by design.", + }, []string{"asset"}) + + tspMarketState = promauto.NewGaugeVec(prometheus.GaugeOpts{ + Name: "tsp_market_session", + Help: "1 for the currently active market session label, 0 otherwise.", + }, []string{"market_state"}) + + tspSourceLatency = promauto.NewGaugeVec(prometheus.GaugeOpts{ + Name: "tsp_source_latency_milliseconds", + Help: "Round-trip latency of the last fetch per source.", + }, []string{"source"}) + + tspSourceCall = promauto.NewCounterVec(prometheus.CounterOpts{ + Name: "tsp_source_call_total", + Help: "Fetch outcomes per source (onchain batch, yahoo spark, yahoo chart).", + }, []string{"source", "result"}) + + tspHealth = promauto.NewGaugeVec(prometheus.GaugeOpts{ + Name: "tsp_health", + Help: "1 when the last tick produced a deviation sample for the asset, 0 otherwise.", + }, []string{"asset"}) +) + +func startMetricsServer(addr string) error { + mux := http.NewServeMux() + mux.Handle("/metrics", promhttp.Handler()) + mux.Handle("/logs", logsHandler()) + mux.HandleFunc("/health", func(w http.ResponseWriter, _ *http.Request) { + w.WriteHeader(http.StatusOK) + _, _ = w.Write([]byte("ok")) + }) + return http.ListenAndServe(addr, mux) +} diff --git a/harnesses/xstocks-peg/cmd/script/reference.go b/harnesses/xstocks-peg/cmd/script/reference.go new file mode 100644 index 00000000..3ebc5c3b --- /dev/null +++ b/harnesses/xstocks-peg/cmd/script/reference.go @@ -0,0 +1,207 @@ +package main + +import ( + "encoding/json" + "fmt" + "io" + "net/http" + "strings" + "time" +) + +// Reference leg: Yahoo Finance, keyless. One spark batch call fetches +// the latest price for every symbol; one chart call (AAPL as the +// bellwether) fetches the day's exact session windows, which Yahoo +// publishes holiday-aware so the harness never maintains an NYSE +// calendar. Verified 2026-07-13 from the harness host: clean 200s with +// a browser User-Agent (datacenter IP), 70-130ms. +// +// Market state is derived from currentTradingPeriod epochs: +// pre / regular / post / closed. Deviation samples carry the state as +// a label so the bench can pin its headline to regular hours and read +// the weekend drift from the closed-state series. + +const ( + sparkHost = "https://query1.finance.yahoo.com/v8/finance/spark" + chartHost = "https://query1.finance.yahoo.com/v8/finance/chart/AAPL" + yahooUA = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0 Safari/537.36" +) + +type refQuote struct { + Price float64 + AsOfSec int64 +} + +type tradingPeriods struct { + PreStart, PreEnd int64 + RegularStart, RegularEnd int64 + PostStart, PostEnd int64 + FetchedAt time.Time +} + +// fetchReferencePrices returns the freshest Yahoo price per symbol +// (lowercased) from one spark batch call. When the market is closed +// the spark meta still carries regularMarketPrice = last close, which +// is exactly the weekend reference we want. +func fetchReferencePrices(client *http.Client) map[string]refQuote { + syms := make([]string, 0, len(assets)) + for _, a := range assets { + syms = append(syms, a.Symbol) + } + url := sparkHost + "?symbols=" + strings.Join(syms, ",") + "&range=1d&interval=1m" + raw, status := yahooGet(client, url) + if raw == nil { + tspSourceCall.WithLabelValues("yahoo", status).Inc() + // Rotate to query2 once before giving up this tick. + raw, status = yahooGet(client, strings.Replace(url, "query1", "query2", 1)) + if raw == nil { + tspSourceCall.WithLabelValues("yahoo", status).Inc() + return nil + } + } + tspSourceCall.WithLabelValues("yahoo", "ok").Inc() + + // Spark response: {"spark":{"result":[{"symbol":"AAPL","response":[{"meta":{...}}]}]}} + // or the flatter {"AAPL":{...}} shape depending on edge; handle both. + prices := make(map[string]refQuote, len(syms)) + var envel struct { + Spark struct { + Result []struct { + Symbol string `json:"symbol"` + Response []struct { + Meta struct { + RegularMarketPrice float64 `json:"regularMarketPrice"` + RegularMarketTime int64 `json:"regularMarketTime"` + } `json:"meta"` + } `json:"response"` + } `json:"result"` + } `json:"spark"` + } + if err := json.Unmarshal(raw, &envel); err == nil { + for _, r := range envel.Spark.Result { + if len(r.Response) == 0 || r.Response[0].Meta.RegularMarketPrice <= 0 { + continue + } + prices[strings.ToLower(r.Symbol)] = refQuote{ + Price: r.Response[0].Meta.RegularMarketPrice, + AsOfSec: r.Response[0].Meta.RegularMarketTime, + } + } + } + if len(prices) == 0 { + // Flat spark shape (observed live 2026-07-13): + // {"MSFT":{"timestamp":[...],"close":[...],"previousClose":X},...} + // Price = last non-null close; previousClose is the fallback when + // the close array is empty (market closed all day). + var flat map[string]struct { + Timestamp []int64 `json:"timestamp"` + Close []*float64 `json:"close"` + PreviousClose *float64 `json:"previousClose"` + } + if err := json.Unmarshal(raw, &flat); err == nil { + for sym, v := range flat { + q := refQuote{} + for i := len(v.Close) - 1; i >= 0; i-- { + if v.Close[i] != nil && *v.Close[i] > 0 { + q.Price = *v.Close[i] + if i < len(v.Timestamp) { + q.AsOfSec = v.Timestamp[i] + } + break + } + } + if q.Price == 0 && v.PreviousClose != nil && *v.PreviousClose > 0 { + q.Price = *v.PreviousClose + } + if q.Price > 0 { + prices[strings.ToLower(sym)] = q + } + } + } + } + if len(prices) == 0 { + tspSourceCall.WithLabelValues("yahoo", "parse_empty").Inc() + head := string(raw) + if len(head) > 400 { + head = head[:400] + } + fmt.Printf("[yahoo] spark yielded no symbols; body head: %s\n", head) + return nil + } + return prices +} + +// fetchTradingPeriods reads currentTradingPeriod from one chart call. +// Refreshed every 30 minutes; between refreshes marketState() reuses +// the cached windows. +func fetchTradingPeriods(client *http.Client) *tradingPeriods { + raw, status := yahooGet(client, chartHost+"?range=1d&interval=5m") + if raw == nil { + tspSourceCall.WithLabelValues("yahoo_chart", status).Inc() + return nil + } + var envel struct { + Chart struct { + Result []struct { + Meta struct { + CurrentTradingPeriod struct { + Pre struct{ Start, End int64 } `json:"pre"` + Regular struct{ Start, End int64 } `json:"regular"` + Post struct{ Start, End int64 } `json:"post"` + } `json:"currentTradingPeriod"` + } `json:"meta"` + } `json:"result"` + } `json:"chart"` + } + if err := json.Unmarshal(raw, &envel); err != nil || len(envel.Chart.Result) == 0 { + tspSourceCall.WithLabelValues("yahoo_chart", "parse").Inc() + return nil + } + m := envel.Chart.Result[0].Meta.CurrentTradingPeriod + tspSourceCall.WithLabelValues("yahoo_chart", "ok").Inc() + return &tradingPeriods{ + PreStart: m.Pre.Start, PreEnd: m.Pre.End, + RegularStart: m.Regular.Start, RegularEnd: m.Regular.End, + PostStart: m.Post.Start, PostEnd: m.Post.End, + FetchedAt: time.Now(), + } +} + +func (tp *tradingPeriods) state(now time.Time) string { + if tp == nil { + return "unknown" + } + u := now.Unix() + switch { + case u >= tp.RegularStart && u < tp.RegularEnd: + return "regular" + case u >= tp.PreStart && u < tp.PreEnd: + return "pre" + case u >= tp.PostStart && u < tp.PostEnd: + return "post" + default: + return "closed" + } +} + +func yahooGet(client *http.Client, url string) ([]byte, string) { + req, err := http.NewRequest("GET", url, nil) + if err != nil { + return nil, "request_build" + } + req.Header.Set("User-Agent", yahooUA) + req.Header.Set("Accept", "application/json") + resp, err := client.Do(req) + if err != nil { + return nil, "network" + } + defer resp.Body.Close() + raw, err := io.ReadAll(io.LimitReader(resp.Body, 1<<22)) + if err != nil { + return nil, "read" + } + if resp.StatusCode != 200 { + return nil, fmt.Sprintf("http_%d", resp.StatusCode) + } + return raw, "ok" +} diff --git a/harnesses/xstocks-peg/go.mod b/harnesses/xstocks-peg/go.mod new file mode 100644 index 00000000..815ee407 --- /dev/null +++ b/harnesses/xstocks-peg/go.mod @@ -0,0 +1,18 @@ +module xstocks-peg + +go 1.24.0 + +require github.com/prometheus/client_golang v1.23.2 + +require ( + github.com/beorn7/perks v1.0.1 // indirect + github.com/cespare/xxhash/v2 v2.3.0 // indirect + github.com/kr/text v0.2.0 // indirect + github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect + github.com/prometheus/client_model v0.6.2 // indirect + github.com/prometheus/common v0.66.1 // indirect + github.com/prometheus/procfs v0.16.1 // indirect + go.yaml.in/yaml/v2 v2.4.2 // indirect + golang.org/x/sys v0.35.0 // indirect + google.golang.org/protobuf v1.36.8 // indirect +) diff --git a/harnesses/xstocks-peg/go.sum b/harnesses/xstocks-peg/go.sum new file mode 100644 index 00000000..d6b8ca98 --- /dev/null +++ b/harnesses/xstocks-peg/go.sum @@ -0,0 +1,46 @@ +github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= +github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= +github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs= +github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= +github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= +github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= +github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= +github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU= +github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo= +github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ= +github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= +github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= +github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= +github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= +github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc= +github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw= +github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA= +github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ= +github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= +github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/prometheus/client_golang v1.23.2 h1:Je96obch5RDVy3FDMndoUsjAhG5Edi49h0RJWRi/o0o= +github.com/prometheus/client_golang v1.23.2/go.mod h1:Tb1a6LWHB3/SPIzCoaDXI4I8UHKeFTEQ1YCr+0Gyqmg= +github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk= +github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE= +github.com/prometheus/common v0.66.1 h1:h5E0h5/Y8niHc5DlaLlWLArTQI7tMrsfQjHV+d9ZoGs= +github.com/prometheus/common v0.66.1/go.mod h1:gcaUsgf3KfRSwHY4dIMXLPV0K/Wg1oZ8+SbZk/HH/dA= +github.com/prometheus/procfs v0.16.1 h1:hZ15bTNuirocR6u0JZ6BAHHmwS1p8B4P6MRqxtzMyRg= +github.com/prometheus/procfs v0.16.1/go.mod h1:teAbpZRB1iIAJYREa1LsoWUXykVXA1KlTmWl8x/U+Is= +github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjRBZyWFQ= +github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog= +github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U= +github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U= +go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto= +go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE= +go.yaml.in/yaml/v2 v2.4.2 h1:DzmwEr2rDGHl7lsFgAHxmNz/1NlQ7xLIrlN2h5d1eGI= +go.yaml.in/yaml/v2 v2.4.2/go.mod h1:081UH+NErpNdqlCXm3TtEran0rJZGxAYx9hb/ELlsPU= +golang.org/x/sys v0.35.0 h1:vz1N37gP5bs89s7He8XuIYXpyY0+QlsKmzipCbUtyxI= +golang.org/x/sys v0.35.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k= +google.golang.org/protobuf v1.36.8 h1:xHScyCOEuuwZEc6UtSOvPbAT4zRh0xcNRYekJwfqyMc= +google.golang.org/protobuf v1.36.8/go.mod h1:fuxRtAxBytpl4zzqUh6/eyUujkJdNiuEkXntxiD/uRU= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= +gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= +gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= diff --git a/public/logos/aapl.svg b/public/logos/aapl.svg new file mode 100644 index 00000000..915e8909 --- /dev/null +++ b/public/logos/aapl.svg @@ -0,0 +1 @@ +Apple \ No newline at end of file diff --git a/public/logos/amd.svg b/public/logos/amd.svg new file mode 100644 index 00000000..ae3de073 --- /dev/null +++ b/public/logos/amd.svg @@ -0,0 +1 @@ +AMD \ No newline at end of file diff --git a/public/logos/amzn.png b/public/logos/amzn.png new file mode 100644 index 00000000..d466c792 Binary files /dev/null and b/public/logos/amzn.png differ diff --git a/public/logos/beaverbuild.jpg b/public/logos/beaverbuild.jpg new file mode 100644 index 00000000..2f6794a6 Binary files /dev/null and b/public/logos/beaverbuild.jpg differ diff --git a/public/logos/bobthebuilder.jpg b/public/logos/bobthebuilder.jpg new file mode 100644 index 00000000..b47283ba Binary files /dev/null and b/public/logos/bobthebuilder.jpg differ diff --git a/public/logos/btcs.png b/public/logos/btcs.png new file mode 100644 index 00000000..c947ae54 Binary files /dev/null and b/public/logos/btcs.png differ diff --git a/public/logos/buildernet.png b/public/logos/buildernet.png new file mode 100644 index 00000000..4ca16514 Binary files /dev/null and b/public/logos/buildernet.png differ diff --git a/public/logos/eureka.png b/public/logos/eureka.png new file mode 100644 index 00000000..833b3f5e Binary files /dev/null and b/public/logos/eureka.png differ diff --git a/public/logos/googl.svg b/public/logos/googl.svg new file mode 100644 index 00000000..088288fa --- /dev/null +++ b/public/logos/googl.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/public/logos/leorpc.png b/public/logos/leorpc.png new file mode 100644 index 00000000..27485d76 Binary files /dev/null and b/public/logos/leorpc.png differ diff --git a/public/logos/markets-mobile.svg b/public/logos/markets-mobile.svg new file mode 100644 index 00000000..05a28b14 --- /dev/null +++ b/public/logos/markets-mobile.svg @@ -0,0 +1 @@ + diff --git a/public/logos/meta.svg b/public/logos/meta.svg new file mode 100644 index 00000000..385fb0a1 --- /dev/null +++ b/public/logos/meta.svg @@ -0,0 +1 @@ +Meta \ No newline at end of file diff --git a/public/logos/msft.svg b/public/logos/msft.svg new file mode 100644 index 00000000..5334aa7c --- /dev/null +++ b/public/logos/msft.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/public/logos/mu.png b/public/logos/mu.png new file mode 100644 index 00000000..f8f9a294 Binary files /dev/null and b/public/logos/mu.png differ diff --git a/public/logos/nvda.svg b/public/logos/nvda.svg new file mode 100644 index 00000000..e427e2ce --- /dev/null +++ b/public/logos/nvda.svg @@ -0,0 +1 @@ +NVIDIA \ No newline at end of file diff --git a/public/logos/orca.png b/public/logos/orca.png new file mode 100644 index 00000000..35bf2a7d Binary files /dev/null and b/public/logos/orca.png differ diff --git a/public/logos/parity.png b/public/logos/parity.png new file mode 100644 index 00000000..3d06da06 Binary files /dev/null and b/public/logos/parity.png differ diff --git a/public/logos/pltr.svg b/public/logos/pltr.svg new file mode 100644 index 00000000..6bb694d7 --- /dev/null +++ b/public/logos/pltr.svg @@ -0,0 +1 @@ +Palantir \ No newline at end of file diff --git a/public/logos/polkadot.png b/public/logos/polkadot.png new file mode 100644 index 00000000..23b3782e Binary files /dev/null and b/public/logos/polkadot.png differ diff --git a/public/logos/pyth.jpg b/public/logos/pyth.jpg new file mode 100644 index 00000000..4bbb1cac Binary files /dev/null and b/public/logos/pyth.jpg differ diff --git a/public/logos/qqq.png b/public/logos/qqq.png new file mode 100644 index 00000000..dab7e300 Binary files /dev/null and b/public/logos/qqq.png differ diff --git a/public/logos/quasar.png b/public/logos/quasar.png new file mode 100644 index 00000000..364ef935 Binary files /dev/null and b/public/logos/quasar.png differ diff --git a/public/logos/redstone.png b/public/logos/redstone.png new file mode 100644 index 00000000..a333e41c Binary files /dev/null and b/public/logos/redstone.png differ diff --git a/public/logos/slash.png b/public/logos/slash.png new file mode 100644 index 00000000..d002b793 Binary files /dev/null and b/public/logos/slash.png differ diff --git a/public/logos/spy.svg b/public/logos/spy.svg new file mode 100644 index 00000000..d3d540a1 --- /dev/null +++ b/public/logos/spy.svg @@ -0,0 +1,35 @@ + + + diff --git a/public/logos/titan.png b/public/logos/titan.png new file mode 100644 index 00000000..cbbbcd1e Binary files /dev/null and b/public/logos/titan.png differ diff --git a/public/logos/topdog.jpg b/public/logos/topdog.jpg new file mode 100644 index 00000000..ea41a198 Binary files /dev/null and b/public/logos/topdog.jpg differ diff --git a/public/logos/tsla.svg b/public/logos/tsla.svg new file mode 100644 index 00000000..e990fab5 --- /dev/null +++ b/public/logos/tsla.svg @@ -0,0 +1 @@ +Tesla \ No newline at end of file diff --git a/public/logos/vanilla.svg b/public/logos/vanilla.svg new file mode 100644 index 00000000..91efc9cd --- /dev/null +++ b/public/logos/vanilla.svg @@ -0,0 +1,4 @@ + + + + diff --git a/src/app/alternatives/[slug]/page.tsx b/src/app/alternatives/[slug]/page.tsx index 0d427d19..f3afe8ec 100644 --- a/src/app/alternatives/[slug]/page.tsx +++ b/src/app/alternatives/[slug]/page.tsx @@ -15,6 +15,7 @@ import { SITE } from "@/data/site"; import { loadAlternative } from "@/lib/alternatives"; import { Breadcrumb } from "@/components/breadcrumb"; import { buildBreadcrumbJsonLd, safeJsonLd } from "@/lib/jsonld"; +import { citableAsOf } from "@/lib/citation"; import { ProviderLogo } from "@/components/provider-logo"; import { ProviderTypeBadge } from "@/components/provider-type-badge"; import { isRegion } from "@/lib/brand"; @@ -124,7 +125,7 @@ export default async function AlternativePage({ isAccessibleForFree: true, license: "https://creativecommons.org/licenses/by/4.0/", datePublished: getBenchCreatedAt(bench.slug).toISOString(), - dateModified: bench.lastRunAt, + ...(citableAsOf(bench) ? { dateModified: bench.lastRunAt } : {}), variableMeasured: bench.metric, isBasedOn: benchUrl, }, @@ -137,7 +138,7 @@ export default async function AlternativePage({ mainEntityOfPage: url, articleBody: alt.intro, datePublished: getBenchCreatedAt(bench.slug).toISOString(), - dateModified: bench.lastRunAt, + ...(citableAsOf(bench) ? { dateModified: bench.lastRunAt } : {}), author: { "@id": `${SITE.url}/#org` }, publisher: { "@id": `${SITE.url}/#org` }, about: { "@id": `${url}#dataset` }, diff --git a/src/app/answers/[slug]/page.tsx b/src/app/answers/[slug]/page.tsx index 97db48d8..ad3bee58 100644 --- a/src/app/answers/[slug]/page.tsx +++ b/src/app/answers/[slug]/page.tsx @@ -9,7 +9,7 @@ import { hasLiveDataTokens, benchDataPendingFallback, } from "@/lib/answers-template"; -import { leader } from "@/lib/citation"; +import { citableAsOf, leader } from "@/lib/citation"; import { Breadcrumb } from "@/components/breadcrumb"; import { Pill } from "@/components/pill"; import { ProviderLogo } from "@/components/provider-logo"; @@ -105,7 +105,13 @@ export default async function AnswerPage({ // YAML's slug reference) fall through to a neutral fallback via // cleanLeftoverTokens so a placeholder string never reaches the SERP. const render = (s: string) => cleanLeftoverTokens(renderTemplate(s, bench)); - const asOfUtc = fmtAsOfUtc(bench.lastRunAt); + // Real measurement timestamp on live benches, null on drafts (whose + // lastRunAt is a wall-clock placeholder from the loader). Prevents the + // visible "Data as of ..." line and the associated

- Data as of , + Data as of , refreshed continuously.

)} diff --git a/src/app/api/citable/[date]/route.ts b/src/app/api/citable/[date]/route.ts index debd0be0..bbbbdd4b 100644 --- a/src/app/api/citable/[date]/route.ts +++ b/src/app/api/citable/[date]/route.ts @@ -2,7 +2,8 @@ import { NextResponse } from "next/server"; import { getBenchmarks } from "@/data/benchmarks"; import { SITE } from "@/data/site"; import { AllBenchmarksDraftError } from "@/lib/spec"; -import { citeBundle, fieldValue, leader, headlineSentence } from "@/lib/citation"; +import { citableAsOf, citeBundle, fieldValue, leader, headlineSentence } from "@/lib/citation"; +import { valueInDeclaredUnit } from "@/lib/format"; import { clientKey, rateLimit, tooManyRequests } from "@/lib/rate-limit"; export const runtime = "nodejs"; @@ -111,6 +112,13 @@ export async function GET( const data = benches.map((b) => { const top = leader(b); const insufficient = b.dataConfidence === "insufficient"; + // `value` and `leader.value` are published in the declared `unit`. + // Latency benches with unit "s" store ms internally (fmtUnit + // convention); valueInDeclaredUnit converts so a snapshot never + // claims 627 seconds for a 627 ms head lag. Same conversion as + // the live /api/citable route; missing here caused sub-1 unit + // values to leak through as their internal ms representation. + const raw = insufficient ? null : fieldValue(b); return { slug: b.slug, title: b.title, @@ -118,17 +126,21 @@ export async function GET( metric: b.metric, unit: b.unit, status: b.status, - value: insufficient ? null : fieldValue(b), + value: raw == null ? null : valueInDeclaredUnit(raw, b.unit), leader: insufficient ? null : top - ? { name: top.name, slug: top.slug, value: top.value } + ? { + name: top.name, + slug: top.slug, + value: valueInDeclaredUnit(top.value, b.unit), + } : null, sampleSize: b.sampleSize, expectedN: b.expectedN, dataConfidence: b.dataConfidence, - asOf: b.lastRunAt, + asOf: citableAsOf(b), headline: headlineSentence(b), url: `${SITE.url}/benchmarks/${b.slug}`, api: `${SITE.url}/api/stat/${b.slug}`, diff --git a/src/app/api/citable/route.ts b/src/app/api/citable/route.ts index 1440ab17..0e84b380 100644 --- a/src/app/api/citable/route.ts +++ b/src/app/api/citable/route.ts @@ -2,7 +2,7 @@ import { NextResponse } from "next/server"; import { getBenchmarks } from "@/data/benchmarks"; import { SITE } from "@/data/site"; import { AllBenchmarksDraftError } from "@/lib/spec"; -import { citeBundle, fieldValue, leader, headlineSentence } from "@/lib/citation"; +import { citableAsOf, citeBundle, fieldValue, leader, headlineSentence } from "@/lib/citation"; import { valueInDeclaredUnit } from "@/lib/format"; import { clientKey, rateLimit, tooManyRequests } from "@/lib/rate-limit"; @@ -80,7 +80,7 @@ export async function GET(req: Request) { sampleSize: b.sampleSize, expectedN: b.expectedN, dataConfidence: b.dataConfidence, - asOf: b.lastRunAt, + asOf: citableAsOf(b), headline: headlineSentence(b), url: `${SITE.url}/benchmarks/${b.slug}`, api: `${SITE.url}/api/stat/${b.slug}`, diff --git a/src/app/api/cite/[slug]/[format]/route.ts b/src/app/api/cite/[slug]/[format]/route.ts new file mode 100644 index 00000000..f09a44a9 --- /dev/null +++ b/src/app/api/cite/[slug]/[format]/route.ts @@ -0,0 +1,106 @@ +import { NextResponse } from "next/server"; +import { getBenchmark } from "@/data/benchmarks"; +import { SITE } from "@/data/site"; +import { citeBundle } from "@/lib/citation"; +import { clientKey, rateLimit, tooManyRequests } from "@/lib/rate-limit"; +import { SLUG_RE } from "@/lib/slug"; + +export const runtime = "nodejs"; +export const revalidate = 3600; + +/** + * Per-bench citation file with the correct MIME type so Zotero, + * EndNote, Mendeley, Google Scholar's "Cite" button and Perplexity's + * cite-picker recognize it as an importable citation record rather than + * a plain text blob. + * + * Four formats: + * - `bib` (application/x-bibtex) BibTeX record for LaTeX toolchains. + * - `ris` (application/x-research-info-systems) RIS record for + * Zotero, EndNote, Mendeley, Zotero-compatible tools. + * - `apa` (text/plain) Ready-to-paste APA string. + * - `txt` (text/plain) Ready-to-paste plain-language attribution. + * + * The content is the same as `/api/stat/{slug}.cite.*` inside JSON, but + * exposed here as a separate URL with the right Content-Type so a + * browser or reference manager triggers "import" rather than "display". + */ + +const FORMATS = { + bib: { + contentType: "application/x-bibtex; charset=utf-8", + field: "bibtex", + fileExt: "bib", + }, + ris: { + contentType: "application/x-research-info-systems; charset=utf-8", + field: "ris", + fileExt: "ris", + }, + apa: { + contentType: "text/plain; charset=utf-8", + field: "apa", + fileExt: "apa.txt", + }, + txt: { + contentType: "text/plain; charset=utf-8", + field: "plain", + fileExt: "txt", + }, +} as const; + +type Format = keyof typeof FORMATS; + +function isFormat(x: string): x is Format { + return x in FORMATS; +} + +export async function GET( + req: Request, + { params }: { params: Promise<{ slug: string; format: string }> }, +) { + const r = rateLimit(clientKey(req, "cite"), 60, 60, req); + if (!r.ok) return tooManyRequests(r.retryAfterSec); + + const { slug, format } = await params; + if (!SLUG_RE.test(slug)) { + return NextResponse.json({ error: "bad_slug" }, { status: 400 }); + } + if (!isFormat(format)) { + return NextResponse.json( + { + error: "bad_format", + allowed: Object.keys(FORMATS), + }, + { status: 400 }, + ); + } + + const bench = await getBenchmark(slug); + if (!bench || bench.editorialStatus !== "live") { + return NextResponse.json( + { error: "unknown_slug", slug }, + { status: 404, headers: { "cache-control": "public, s-maxage=60" } }, + ); + } + + const cfg = FORMATS[format]; + const bundle = citeBundle(bench, SITE.url); + const body = bundle[cfg.field as keyof typeof bundle]; + const filename = `openchainbench-${slug}.${cfg.fileExt}`; + + return new Response(body, { + status: 200, + headers: { + "content-type": cfg.contentType, + // `attachment` triggers the reference-manager import flow in + // browsers; `inline` would fight with Firefox+Zotero. Filename + // exposed as ASCII fallback + UTF-8 (RFC 5987) to survive + // non-ASCII bench titles. Slug is ASCII-only by SLUG_RE, so + // filename never needs escaping in practice. + "content-disposition": `attachment; filename="${filename}"`, + "cache-control": "public, s-maxage=3600, stale-while-revalidate=86400", + "access-control-allow-origin": "*", + }, + }); +} diff --git a/src/app/api/compare/[a]/[b]/route.ts b/src/app/api/compare/[a]/[b]/route.ts new file mode 100644 index 00000000..54e70cef --- /dev/null +++ b/src/app/api/compare/[a]/[b]/route.ts @@ -0,0 +1,156 @@ +import { NextResponse } from "next/server"; +import { canonicalize, getProvider } from "@/lib/providers"; +import { valueInDeclaredUnit } from "@/lib/format"; +import { canonicalPairSlug } from "@/lib/compare-pairing-shared"; +import { citableAsOf } from "@/lib/citation"; +import { clientKey, rateLimit, tooManyRequests } from "@/lib/rate-limit"; +import { SLUG_RE } from "@/lib/slug"; +import { SITE } from "@/data/site"; + +export const runtime = "nodejs"; +export const revalidate = 60; + +/** + * Head-to-head comparison of two providers as a single citable payload. + * + * LLMs asked "Mobula vs Codex head lag" would otherwise need two + * /api/stat calls plus their own delta reasoning; this endpoint returns + * one flat structure with per-benchmark p50 for each side, the winner + * on the metric's higher-is-better convention, and the absolute delta. + * + * The intersection logic mirrors src/lib/related-providers.ts and the + * HTML /compare/{slug} page: shared benchmark = both providers appear + * in it. This route intentionally returns a flat leader-p50 payload + * only (no chain/region breakdowns, no cache-heavy fan out) so the + * agent tool call stays fast and cheap. + */ +export async function GET( + req: Request, + { params }: { params: Promise<{ a: string; b: string }> }, +) { + const r = rateLimit(clientKey(req, "compare"), 60, 60, req); + if (!r.ok) return tooManyRequests(r.retryAfterSec); + + const { a: aRaw, b: bRaw } = await params; + if (!SLUG_RE.test(aRaw) || !SLUG_RE.test(bRaw)) { + return NextResponse.json({ error: "bad_slug" }, { status: 400 }); + } + const aCanon = canonicalize(aRaw).slug; + const bCanon = canonicalize(bRaw).slug; + if (aCanon === bCanon) { + return NextResponse.json( + { error: "same_provider", slug: aCanon }, + { status: 400 }, + ); + } + + const [aProfile, bProfile] = await Promise.all([ + getProvider(aCanon), + getProvider(bCanon), + ]); + if (!aProfile || !bProfile) { + return NextResponse.json( + { + error: "unknown_provider", + missing: !aProfile ? aCanon : bCanon, + }, + { status: 404, headers: { "cache-control": "public, s-maxage=60" } }, + ); + } + + // Intersection of the two providers' bench appearances. Same shape as + // src/lib/related-providers.ts:78 so this endpoint and the "compare + // with" cards on /products/[slug] never disagree on which benches show + // up in the head-to-head. + const aByBench = new Map( + aProfile.appearances.map((x) => [x.benchmark.slug, x] as const), + ); + const sharedSlugs = bProfile.appearances + .filter((x) => aByBench.has(x.benchmark.slug)) + .map((x) => x.benchmark.slug); + + if (sharedSlugs.length === 0) { + return NextResponse.json( + { + error: "no_shared_benchmark", + a: { slug: aProfile.slug, name: aProfile.name }, + b: { slug: bProfile.slug, name: bProfile.name }, + }, + { status: 404, headers: { "cache-control": "public, s-maxage=60" } }, + ); + } + + // Publish p50 in the declared unit (unit "s" benches store ms + // internally — same rule as /api/citable and /api/stat). + const shared = sharedSlugs + .map((slug) => { + const aEntry = aByBench.get(slug); + const bEntry = bProfile.appearances.find( + (x) => x.benchmark.slug === slug, + ); + if (!aEntry || !bEntry) return null; + const bench = aEntry.benchmark; + const higherIsBetter = bench.higherIsBetter === true; + const aP50Raw = aEntry.result.ms.p50; + const bP50Raw = bEntry.result.ms.p50; + const aValue = + aP50Raw > 0 ? valueInDeclaredUnit(aP50Raw, bench.unit) : null; + const bValue = + bP50Raw > 0 ? valueInDeclaredUnit(bP50Raw, bench.unit) : null; + let winner: "a" | "b" | "tie"; + if (aValue == null && bValue == null) winner = "tie"; + else if (aValue == null) winner = "b"; + else if (bValue == null) winner = "a"; + else if (aValue === bValue) winner = "tie"; + else if (higherIsBetter) winner = aValue > bValue ? "a" : "b"; + else winner = aValue < bValue ? "a" : "b"; + const delta = + aValue != null && bValue != null ? aValue - bValue : null; + return { + slug: bench.slug, + title: bench.title, + category: bench.category, + metric: bench.metric, + unit: bench.unit, + higherIsBetter, + aValue, + bValue, + winner, + delta, + pageUrl: `${SITE.url}/benchmarks/${bench.slug}`, + asOf: citableAsOf(bench), + }; + }) + .filter( + (row): row is Exclude => row !== null, + ); + + const pairSlug = canonicalPairSlug(aProfile.slug, bProfile.slug); + const totalWins = shared.reduce( + (acc, row) => { + if (row.winner === "a") acc.a += 1; + else if (row.winner === "b") acc.b += 1; + else acc.tie += 1; + return acc; + }, + { a: 0, b: 0, tie: 0 }, + ); + + return NextResponse.json( + { + a: { slug: aProfile.slug, name: aProfile.name }, + b: { slug: bProfile.slug, name: bProfile.name }, + totalWins, + shared, + comparePageUrl: `${SITE.url}/compare/${pairSlug}`, + license: "CC-BY-4.0", + }, + { + headers: { + "cache-control": + "public, s-maxage=60, stale-while-revalidate=300", + "access-control-allow-origin": "*", + }, + }, + ); +} diff --git a/src/app/api/llm-context/route.ts b/src/app/api/llm-context/route.ts index c6617f49..27aada6f 100644 --- a/src/app/api/llm-context/route.ts +++ b/src/app/api/llm-context/route.ts @@ -7,6 +7,7 @@ import { headlineSentence, isInsufficient, leader, + rankedCandidates, } from "@/lib/citation"; import { clientKey, rateLimit, tooManyRequests } from "@/lib/rate-limit"; @@ -84,11 +85,11 @@ export async function GET(req: Request) { lines.push(`- Headline: ${headlineSentence(b)}`); lines.push(""); lines.push(`**Rankings (p50, 24h):**`); - const ranked = [...b.results] - .filter((r) => r.ms.p50 > 0) - .sort((a, c) => - b.higherIsBetter ? c.ms.p50 - a.ms.p50 : a.ms.p50 - c.ms.p50, - ); + // Shares `rankedCandidates` with `leader()` so the numbered list + // below matches the Headline sentence above. Without the shared + // filter, an LLM pasting this block would see e.g. "Etherscan + // leads" then a rankings list with Owlracle at #1. + const ranked = rankedCandidates(b); for (let i = 0; i < ranked.length; i++) { const r = ranked[i]; lines.push( diff --git a/src/app/api/mcp/[transport]/route.ts b/src/app/api/mcp/[transport]/route.ts index 809e4978..e9461cb6 100644 --- a/src/app/api/mcp/[transport]/route.ts +++ b/src/app/api/mcp/[transport]/route.ts @@ -4,11 +4,13 @@ import { z } from "zod"; import { getBenchmark, getBenchmarks } from "@/data/benchmarks"; import { SITE } from "@/data/site"; import { + citableAsOf, citationQuote, fieldValue, headlineSentence, isInsufficient, leader, + rankedCandidates, sparklineFor, } from "@/lib/citation"; import { fmtUnit } from "@/lib/format"; @@ -225,7 +227,7 @@ const mcpHandler = createMcpHandler( leader: top, headline: headlineSentence(b), url: `${SITE.url}/benchmarks/${b.slug}`, - asOf: b.lastRunAt, + asOf: citableAsOf(b), }; }); return { @@ -296,17 +298,12 @@ const mcpHandler = createMcpHandler( ms: { p50: null, p90: null, p99: null, mean: null }, successRate: r.successRate, })) - : b.results - .filter((r) => r.ms.p50 > 0) - .sort((a, c) => - b.higherIsBetter ? c.ms.p50 - a.ms.p50 : a.ms.p50 - c.ms.p50, - ) - .map((r) => ({ - name: r.name, - slug: r.slug, - ms: r.ms, - successRate: r.successRate, - })); + : rankedCandidates(b).map((r) => ({ + name: r.name, + slug: r.slug, + ms: r.ms, + successRate: r.successRate, + })); const payload = { slug: b.slug, title: b.title, @@ -321,7 +318,7 @@ const mcpHandler = createMcpHandler( quote: citationQuote(b, SITE.url), pageUrl: `${SITE.url}/benchmarks/${b.slug}`, ogImage: `${SITE.url}/api/og/${b.slug}`, - asOf: b.lastRunAt, + asOf: citableAsOf(b), methodology: b.methodology, source: b.source, }; @@ -479,13 +476,12 @@ const mcpHandler = createMcpHandler( } const insufficient = isInsufficient(b); const top = insufficient ? null : leader(b); - const ranked = insufficient - ? [] - : b.results - .filter((r) => r.ms.p50 > 0) - .sort((a, c) => - b.higherIsBetter ? c.ms.p50 - a.ms.p50 : a.ms.p50 - c.ms.p50, - ); + // Shares `rankedCandidates` with `leader()` so the Markdown + // Rankings list matches the Headline sentence above and the + // `rankings` field on the JSON tool response below. Without + // this, an agent reading this resource would see e.g. + // "Etherscan leads" then a numbered list with Owlracle at #1. + const ranked = insufficient ? [] : rankedCandidates(b); const md: string[] = []; md.push(`# ${b.title}`); @@ -497,7 +493,10 @@ const mcpHandler = createMcpHandler( md.push(`- Page: ${SITE.url}/benchmarks/${b.slug}`); md.push(`- Source: ${b.source}`); md.push(`- License: CC-BY-4.0`); - md.push(`- Last sample: ${b.lastRunAt}`); + { + const asOf = citableAsOf(b); + md.push(`- Last sample: ${asOf ?? "(no measurement yet, draft)"}`); + } md.push(""); md.push(`**Headline.** ${headlineSentence(b)}`); md.push(""); @@ -555,7 +554,7 @@ const mcpHandler = createMcpHandler( headline: headlineSentence(b), quote: citationQuote(b, SITE.url), pageUrl: `${SITE.url}/benchmarks/${b.slug}`, - asOf: b.lastRunAt, + asOf: citableAsOf(b), methodology: b.methodology, source: b.source, }; diff --git a/src/app/api/openapi.json/route.ts b/src/app/api/openapi.json/route.ts index d7abc397..dfe847d2 100644 --- a/src/app/api/openapi.json/route.ts +++ b/src/app/api/openapi.json/route.ts @@ -32,6 +32,30 @@ export async function GET() { }, }, }, + "/api/citable/{date}": { + get: { + summary: + "Immutable per-day snapshot of the citable index. Same shape as /api/citable; values freeze at end-of-day for past dates. Lets citers pin a citation to a stable URL that keeps returning the number they quoted.", + operationId: "list_benchmarks_snapshot", + parameters: [ + { + name: "date", + in: "path", + required: true, + schema: { type: "string", pattern: "^\\d{4}-\\d{2}-\\d{2}$" }, + description: "ISO 8601 calendar date (YYYY-MM-DD).", + }, + ], + responses: { + "200": { + description: "OK", + content: { "application/json": { schema: { $ref: "#/components/schemas/CitableIndex" } } }, + }, + "400": { description: "Malformed or out-of-range date" }, + "503": { description: "Benchmarks temporarily unavailable" }, + }, + }, + }, "/api/stat/{slug}": { get: { summary: "Single benchmark with rankings, sparkline, and a ready-to-paste citation.", @@ -44,13 +68,80 @@ export async function GET() { schema: { type: "string" }, description: "Benchmark slug (e.g. 'aggregator-head-lag').", }, + { + name: "chain", + in: "query", + required: false, + schema: { type: "string" }, + description: + "Restrict the response to a specific chain dimension (e.g. 'ethereum'). Unknown values 404.", + }, + { + name: "region", + in: "query", + required: false, + schema: { type: "string" }, + description: + "Restrict the response to a specific region dimension (e.g. 'us-east'). Unknown values 404.", + }, + { + name: "kind", + in: "query", + required: false, + schema: { type: "string" }, + description: + "Restrict the response to a specific bench-defined `kind` dimension when the spec declares one.", + }, + { + name: "venue", + in: "query", + required: false, + schema: { type: "string" }, + description: + "Restrict the response to a specific bench-defined `venue` dimension when the spec declares one.", + }, ], responses: { "200": { description: "OK", content: { "application/json": { schema: { $ref: "#/components/schemas/Stat" } } }, }, - "404": { description: "Unknown slug" }, + "404": { description: "Unknown slug or unknown dimension value" }, + }, + }, + }, + "/api/compare/{a}/{b}": { + get: { + summary: + "Head-to-head comparison of two providers on every benchmark where they both appear. One tool call resolves an X-vs-Y query end to end instead of two /api/stat lookups plus reasoning across them.", + operationId: "compare_providers", + parameters: [ + { + name: "a", + in: "path", + required: true, + schema: { type: "string" }, + description: "Provider slug for the left side of the comparison.", + }, + { + name: "b", + in: "path", + required: true, + schema: { type: "string" }, + description: "Provider slug for the right side of the comparison.", + }, + ], + responses: { + "200": { + description: "OK", + content: { + "application/json": { + schema: { $ref: "#/components/schemas/Compare" }, + }, + }, + }, + "400": { description: "Malformed or identical provider slugs" }, + "404": { description: "One of the providers or no shared benchmark" }, }, }, }, @@ -67,6 +158,83 @@ export async function GET() { }, }, }, + "/api/sparkline/{slug}": { + get: { + summary: + "Standalone SVG sparkline of the current leader's 24 h series. Small enough to inline in Perplexity Pages, Notion, dev.to, and any embed that speaks img/svg. Query params: w (default 240), h (default 60), theme (light|dark).", + operationId: "get_sparkline_svg", + parameters: [ + { name: "slug", in: "path", required: true, schema: { type: "string" } }, + { + name: "w", + in: "query", + required: false, + schema: { type: "integer", minimum: 40, maximum: 1200 }, + description: "Width in pixels. Defaults to 240.", + }, + { + name: "h", + in: "query", + required: false, + schema: { type: "integer", minimum: 20, maximum: 400 }, + description: "Height in pixels. Defaults to 60.", + }, + { + name: "theme", + in: "query", + required: false, + schema: { type: "string", enum: ["light", "dark"] }, + description: "Stroke and background. Defaults to `light`.", + }, + ], + responses: { + "200": { + description: "SVG", + content: { "image/svg+xml": {} }, + }, + "404": { description: "Unknown slug" }, + }, + }, + }, + "/api/cite/{slug}/{format}": { + get: { + summary: + "Per-bench citation record with the correct MIME so Zotero, EndNote, Mendeley, Google Scholar and Perplexity's cite-picker recognize the response as an importable citation rather than plain text.", + operationId: "get_citation_file", + parameters: [ + { + name: "slug", + in: "path", + required: true, + schema: { type: "string" }, + description: "Benchmark slug.", + }, + { + name: "format", + in: "path", + required: true, + schema: { + type: "string", + enum: ["bib", "ris", "apa", "txt"], + }, + description: + "bib -> application/x-bibtex, ris -> application/x-research-info-systems, apa/txt -> text/plain.", + }, + ], + responses: { + "200": { + description: "Citation record", + content: { + "application/x-bibtex": {}, + "application/x-research-info-systems": {}, + "text/plain": {}, + }, + }, + "400": { description: "Malformed slug or unknown format" }, + "404": { description: "Unknown slug" }, + }, + }, + }, "/api/llm-context": { get: { summary: @@ -163,12 +331,101 @@ export async function GET() { description: "Current leading value, expressed in `unit`.", }, unit: { type: "string" }, + filters: { + type: "object", + nullable: true, + description: + "Echo of the applied dimension filter (chain, region, kind, venue) when the request scoped the response to a sub-cell; null when the response is the cross-dimension aggregate.", + properties: { + chain: { type: "string" }, + region: { type: "string" }, + kind: { type: "string" }, + venue: { type: "string" }, + }, + }, rankings: { type: "array" }, sparkline: { type: "array", items: { type: "number" } }, headline: { type: "string" }, quote: { type: "string" }, pageUrl: { type: "string", format: "uri" }, - asOf: { type: "string", format: "date-time" }, + asOf: { + type: "string", + format: "date-time", + nullable: true, + description: + "Last measurement timestamp. Null when the bench has no live samples yet (draft state).", + }, + }, + }, + Compare: { + type: "object", + properties: { + a: { + type: "object", + description: "Left side of the head-to-head.", + properties: { + slug: { type: "string" }, + name: { type: "string" }, + }, + }, + b: { + type: "object", + description: "Right side of the head-to-head.", + properties: { + slug: { type: "string" }, + name: { type: "string" }, + }, + }, + totalWins: { + type: "object", + description: + "Head-to-head win tally across the shared benchmark set. Sum of a + b + tie equals shared.length.", + properties: { + a: { type: "integer" }, + b: { type: "integer" }, + tie: { type: "integer" }, + }, + }, + shared: { + type: "array", + description: + "Per benchmark head-to-head rows for every bench where both providers appear. Each row publishes p50 for each side in the declared unit, which side is faster on the metric's higher-is-better convention, and the numeric delta.", + items: { + type: "object", + properties: { + slug: { type: "string" }, + title: { type: "string" }, + metric: { type: "string" }, + unit: { type: "string" }, + higherIsBetter: { type: "boolean" }, + aValue: { type: "number", nullable: true }, + bValue: { type: "number", nullable: true }, + winner: { + type: "string", + enum: ["a", "b", "tie"], + description: "Which side wins on this benchmark.", + }, + delta: { + type: "number", + nullable: true, + description: + "Absolute difference between aValue and bValue in the declared unit. Positive means a > b regardless of the higher-is-better convention.", + }, + pageUrl: { type: "string", format: "uri" }, + asOf: { + type: "string", + format: "date-time", + nullable: true, + }, + }, + }, + }, + comparePageUrl: { + type: "string", + format: "uri", + description: + "Canonical HTML comparison page (openchainbench.com/compare/{a}-vs-{b}) with the full leaderboard tables.", + }, }, }, }, diff --git a/src/app/api/sparkline/[slug]/route.ts b/src/app/api/sparkline/[slug]/route.ts new file mode 100644 index 00000000..49fbbf0a --- /dev/null +++ b/src/app/api/sparkline/[slug]/route.ts @@ -0,0 +1,160 @@ +import { getBenchmark } from "@/data/benchmarks"; +import { NextResponse } from "next/server"; +import { leader, sparklineFor } from "@/lib/citation"; +import { valueInDeclaredUnit } from "@/lib/format"; +import { clientKey, rateLimit, tooManyRequests } from "@/lib/rate-limit"; +import { SLUG_RE } from "@/lib/slug"; + +export const runtime = "nodejs"; +export const revalidate = 300; + +/** + * Standalone SVG sparkline for one bench, keyed to the current leader's + * 24 h series. Lets Perplexity Pages, ChatGPT, Notion, dev.to and any + * third-party embed inline a live micro-chart with `` instead + * of hotlinking the heavier OG PNG or hand-rendering the JSON array. + * + * /api/sparkline/aggregator-head-lag -> defaults (240x60) + * /api/sparkline/aggregator-head-lag?w=320 -> width override + * /api/sparkline/aggregator-head-lag?theme=dark -> dark stroke + * + * Returns a minimal, hand-authored polyline SVG. Deliberately no gzip + * dependency, no charting library, no runtime dep on satori: the payload + * is under 1 KB and every LLM embed target already handles it. + */ + +const DEFAULT_W = 240; +const DEFAULT_H = 60; +const PAD_X = 4; +const PAD_Y = 4; +const MIN_W = 40; +const MAX_W = 1200; +const MIN_H = 20; +const MAX_H = 400; + +/** Minimal XML escape for user-controlled strings interpolated into the + * SVG's element and aria-label attribute. Bench titles and + * provider names come from YAML / a registry and can contain `&`, `<`, + * `>`, `"`; librsvg and Chrome inline-SVG both refuse to render + * malformed XML, so an embedder would silently see a broken image. */ +function xmlEscape(s: string): string { + return s + .replace(/&/g, "&") + .replace(/</g, "<") + .replace(/>/g, ">") + .replace(/"/g, """); +} + +function clampDim(raw: string | null, def: number, min: number, max: number) { + const n = raw == null ? NaN : Number(raw); + if (!Number.isFinite(n) || n < min) return def; + return Math.min(Math.floor(n), max); +} + +function buildPolyline( + values: number[], + width: number, + height: number, + higherIsBetter: boolean, +): { points: string; min: number; max: number } { + const min = Math.min(...values); + const max = Math.max(...values); + const span = max - min || 1; + const drawW = width - 2 * PAD_X; + const drawH = height - 2 * PAD_Y; + const step = values.length > 1 ? drawW / (values.length - 1) : 0; + const points = values + .map((v, i) => { + const x = PAD_X + i * step; + // Sparklines read left to right; lower value at the bottom looks + // right for latency, but for higher-is-better metrics we invert + // so the rising line still reads as "getting better over time". + const norm = (v - min) / span; + const y = higherIsBetter + ? PAD_Y + drawH - norm * drawH + : PAD_Y + norm * drawH; + return `${x.toFixed(2)},${y.toFixed(2)}`; + }) + .join(" "); + return { points, min, max }; +} + +export async function GET( + req: Request, + { params }: { params: Promise<{ slug: string }> }, +) { + const r = rateLimit(clientKey(req, "sparkline"), 120, 60, req); + if (!r.ok) return tooManyRequests(r.retryAfterSec); + + const { slug } = await params; + if (!SLUG_RE.test(slug)) { + return NextResponse.json({ error: "bad_slug" }, { status: 400 }); + } + const bench = await getBenchmark(slug); + if (!bench || bench.editorialStatus !== "live") { + return NextResponse.json( + { error: "unknown_slug", slug }, + { status: 404, headers: { "cache-control": "public, s-maxage=60" } }, + ); + } + + const url = new URL(req.url); + const width = clampDim(url.searchParams.get("w"), DEFAULT_W, MIN_W, MAX_W); + const height = clampDim(url.searchParams.get("h"), DEFAULT_H, MIN_H, MAX_H); + const theme = url.searchParams.get("theme") === "dark" ? "dark" : "light"; + const stroke = theme === "dark" ? "#e5e5e5" : "#111111"; + const bg = theme === "dark" ? "#111111" : "#ffffff"; + + // Prefer the current leader's series; fall back to whichever series is + // first (mirrors sparklineFor default). Empty series → tiny SVG with a + // placeholder line so an embedder never gets a broken image. + const top = leader(bench); + const rawValues = sparklineFor(bench, top?.slug); + // Convert to declared unit (unit "s" benches store ms internally) so + // the min/max labels and future annotations match the page value. + const values = rawValues.map((v) => valueInDeclaredUnit(v, bench.unit)); + + const strokeWidth = Math.max(1, Math.round(Math.min(width, height) / 40)); + let svgBody: string; + if (values.length < 2) { + // Under-populated series: single flat placeholder. Better than a + // blank rectangle because the alt text still reads as "sparkline". + const midY = height / 2; + svgBody = + `<line x1="${PAD_X}" y1="${midY}" x2="${width - PAD_X}" y2="${midY}" ` + + `stroke="${stroke}" stroke-width="${strokeWidth}" ` + + `stroke-linecap="round" stroke-dasharray="2 3" opacity="0.5"/>`; + } else { + const { points } = buildPolyline( + values, + width, + height, + bench.higherIsBetter === true, + ); + svgBody = + `<polyline points="${points}" fill="none" stroke="${stroke}" ` + + `stroke-width="${strokeWidth}" stroke-linecap="round" ` + + `stroke-linejoin="round"/>`; + } + const title = xmlEscape( + `${bench.title} — 24h sparkline${top ? ` (${top.name})` : ""}`, + ); + const svg = + `<?xml version="1.0" encoding="UTF-8"?>\n` + + `<svg xmlns="http://www.w3.org/2000/svg" ` + + `viewBox="0 0 ${width} ${height}" width="${width}" height="${height}" ` + + `role="img" aria-label="${title}">` + + `<title>${title}` + + `` + + svgBody + + `\n`; + + return new Response(svg, { + status: 200, + headers: { + "content-type": "image/svg+xml; charset=utf-8", + "cache-control": "public, s-maxage=300, stale-while-revalidate=900", + "access-control-allow-origin": "*", + }, + }); +} diff --git a/src/app/api/stat/[slug]/route.ts b/src/app/api/stat/[slug]/route.ts index 973f1e16..49efc2da 100644 --- a/src/app/api/stat/[slug]/route.ts +++ b/src/app/api/stat/[slug]/route.ts @@ -2,11 +2,13 @@ import { NextResponse } from "next/server"; import { getBenchmark } from "@/data/benchmarks"; import { SITE } from "@/data/site"; import { + citableAsOf, citationQuote, citeBundle, fieldValue, headlineSentence, leader, + rankedCandidates, sparklineFor, } from "@/lib/citation"; import { valueInDeclaredUnit } from "@/lib/format"; @@ -33,7 +35,31 @@ export async function GET( if (!SLUG_RE.test(slug)) { return NextResponse.json({ error: "bad_slug" }, { status: 400 }); } - const b = await getBenchmark(slug); + // Dimension query params (?chain=, ?region=, ?kind=, ?venue=) mirror + // the same client-side selector on the bench page, so a citer asking + // "fastest ethereum us-east RPC" gets the per cell leader instead of + // the cross chain aggregate. Values pass through to the loader + // unchanged. An unknown value (`?chain=nonexistent`) does NOT fall + // back to the unfiltered aggregate — the loader returns undefined and + // the route below 404s. That is intentional: silently substituting + // the aggregate for a mistyped filter would make citers cite the + // wrong number without knowing. + const url = new URL(req.url); + const filters: { + chain?: string; + region?: string; + kind?: string; + venue?: string; + } = {}; + const chainParam = url.searchParams.get("chain"); + const regionParam = url.searchParams.get("region"); + const kindParam = url.searchParams.get("kind"); + const venueParam = url.searchParams.get("venue"); + if (chainParam && chainParam !== "all") filters.chain = chainParam; + if (regionParam && regionParam !== "all") filters.region = regionParam; + if (kindParam && kindParam !== "all") filters.kind = kindParam; + if (venueParam && venueParam !== "all") filters.venue = venueParam; + const b = await getBenchmark(slug, filters); if (!b || b.editorialStatus !== "live") { return NextResponse.json( { error: "unknown_slug", slug }, @@ -58,6 +84,11 @@ export async function GET( unit: b.unit, status: b.status, higherIsBetter: b.higherIsBetter, + // Echo the applied dimension filter so a citer can verify which + // cell (chain / region / kind / venue) their answer covers. + // Missing key = "all" for that dimension. + filters: + Object.keys(filters).length > 0 ? filters : null, // Aggregate is "insufficient" (median per-provider sample health // below 10 percent of expected_n): refuse to publish a value or // leader; the headline is rewritten by headlineSentence so the @@ -68,27 +99,24 @@ export async function GET( insufficient || !top ? null : { ...top, value: valueInDeclaredUnit(top.value, b.unit) }, - rankings: b.results - .filter((r) => r.ms.p50 > 0) - // Drop "insufficient" rows from the machine-readable ranking too: - // a row that the page hides from the leaderboard must not surface - // here either. - .filter((r) => r.dataConfidence !== "insufficient") - .sort((a, c) => (b.higherIsBetter ? c.ms.p50 - a.ms.p50 : a.ms.p50 - c.ms.p50)) - .map((r) => ({ - name: r.name, - slug: r.slug, - ms: r.ms, - successRate: r.successRate, - sampleSize: r.sampleSize, - sampleHealth: r.sampleHealth, - dataConfidence: r.dataConfidence, - })), + // Shares `rankedCandidates` with `leader()` so `rankings[0]` + // stays consistent with the `leader` field on the same JSON blob: + // a document that names Etherscan as leader must not also list + // Owlracle first here. + rankings: rankedCandidates(b).map((r) => ({ + name: r.name, + slug: r.slug, + ms: r.ms, + successRate: r.successRate, + sampleSize: r.sampleSize, + sampleHealth: r.sampleHealth, + dataConfidence: r.dataConfidence, + })), sparkline: sparklineFor(b, top?.slug), sampleSize: b.sampleSize, expectedN: b.expectedN, dataConfidence: b.dataConfidence, - asOf: b.lastRunAt, + asOf: citableAsOf(b), headline: headlineSentence(b), quote: citationQuote(b, SITE.url), cite: citeBundle(b, SITE.url), diff --git a/src/app/benchmarks/[slug]/[chain]/page.tsx b/src/app/benchmarks/[slug]/[chain]/page.tsx index e44f23fd..82429fed 100644 --- a/src/app/benchmarks/[slug]/[chain]/page.tsx +++ b/src/app/benchmarks/[slug]/[chain]/page.tsx @@ -4,6 +4,7 @@ import Link from "next/link"; import { ArrowLeft, ArrowUpRight } from "lucide-react"; import { getBenchmark } from "@/data/benchmarks"; import { Breadcrumb } from "@/components/breadcrumb"; +import { citableAsOf } from "@/lib/citation"; import { liveResults } from "@/lib/provider-filters"; import { fmtUnit } from "@/lib/format"; import { capDescription } from "@/lib/seo-text"; @@ -363,7 +364,9 @@ export default async function BenchmarkChainPage({ articleBody: `${keyFacts} ${stripInlineMarkdown(explainer.body)}`, image: `${SITE.url}/api/og/${benchmark.slug}`, datePublished: getBenchCreatedAt(benchmark.slug).toISOString(), - dateModified: benchmark.lastRunAt, + ...(citableAsOf(benchmark) + ? { dateModified: benchmark.lastRunAt } + : {}), author: { "@id": `${SITE.url}/#org` }, publisher: { "@id": `${SITE.url}/#org` }, about: { "@id": `${benchmarkUrl}#dataset` }, diff --git a/src/app/benchmarks/[slug]/page.tsx b/src/app/benchmarks/[slug]/page.tsx index 39bc536b..14215e18 100644 --- a/src/app/benchmarks/[slug]/page.tsx +++ b/src/app/benchmarks/[slug]/page.tsx @@ -20,6 +20,7 @@ import { ExportVideoSection } from "@/components/export-video-section"; import { ReportSection } from "@/components/report-section"; import { CATEGORY_COLOR } from "@/lib/category-colors"; import { + citableAsOf, groundingTraceLine, groundingTraceParts, headlineSentence, @@ -34,6 +35,7 @@ import { buildBreadcrumbJsonLd, buildFaqPageJsonLd, safeJsonLd } from "@/lib/jso import { buildBenchDatasetJsonLd, buildBenchStatReportJsonLd, + buildBenchVariableMeasured, } from "@/lib/dataset-jsonld"; import { renderTemplate } from "@/lib/bench-template"; import { canonicalChainSlug } from "@/lib/chain-aliases"; @@ -287,16 +289,29 @@ export default async function BenchmarkPage({ // never emit a structured "Observation" with a fabricated value. const currentLeader = leader(benchmark); // variableMeasured ships as an array so Google's Dataset validator - // reports each statistical aggregate individually rather than as one - // opaque string. Order matches what /api/stat/ returns per - // provider, so a crawler can map field names 1:1. - const variableMeasured = [ - benchmark.metric, - `${benchmark.metric}_p50`, - `${benchmark.metric}_p90`, - `${benchmark.metric}_p99`, - "sample_size", - ]; + // reports each statistical aggregate individually. When a defensible + // leader exists the p50/p90/p99 aggregates ship as PropertyValue + // objects with the numeric value in the declared unit, so Google + // Dataset Search and academic LLM tools extract the measurement as a + // structured fact rather than opaque axis labels. Drafts and + // insufficient benches fall back to bare strings so we never publish + // a fabricated numeric aggregate. + const leaderResult = currentLeader + ? benchmark.results.find((r) => r.slug === currentLeader.slug) + : undefined; + const variableMeasured = buildBenchVariableMeasured({ + metric: benchmark.metric, + unit: benchmark.unit, + leader: + currentLeader && leaderResult + ? { + name: currentLeader.name, + p50: valueInDeclaredUnit(leaderResult.ms.p50, benchmark.unit), + p90: valueInDeclaredUnit(leaderResult.ms.p90, benchmark.unit), + p99: valueInDeclaredUnit(leaderResult.ms.p99, benchmark.unit), + } + : null, + }); const datasetNode = { ...buildBenchDatasetJsonLd({ slug: benchmark.slug, @@ -310,7 +325,12 @@ export default async function BenchmarkPage({ variableMeasured, category: benchmark.category, datePublished: getBenchCreatedAt(benchmark.slug).toISOString(), - dateModified: benchmark.lastRunAt, + // For drafts (no measurement history) skip dateModified so the + // structured-data channel does not spoof freshness. Same rule + // applied to /api/citable, /api/stat and the MCP surface. + ...(citableAsOf(benchmark) + ? { dateModified: benchmark.lastRunAt } + : {}), measurementTechnique: benchmark.methodology.join(" "), }), // Re-bind creator + publisher to the global @id reference so the bench @@ -365,7 +385,9 @@ export default async function BenchmarkPage({ // Clears the "Missing field image" warning in Rich Results Test. image: `${SITE.url}/api/og/${benchmark.slug}`, datePublished: getBenchCreatedAt(benchmark.slug).toISOString(), - dateModified: benchmark.lastRunAt, + ...(citableAsOf(benchmark) + ? { dateModified: benchmark.lastRunAt } + : {}), author: { "@id": `${SITE.url}/#org` }, publisher: { "@id": `${SITE.url}/#org` }, about: { "@id": `${benchmarkUrl}#dataset` }, diff --git a/src/app/hyperliquid/[slug]/page.tsx b/src/app/hyperliquid/[slug]/page.tsx index 7f0b419c..2c937d45 100644 --- a/src/app/hyperliquid/[slug]/page.tsx +++ b/src/app/hyperliquid/[slug]/page.tsx @@ -10,6 +10,7 @@ import { } from "@/lib/hl-builder-stats"; import { Breadcrumb } from "@/components/breadcrumb"; import { HlBuilderDashboard } from "@/components/hl-builder-dashboard"; +import { ProviderLogo } from "@/components/provider-logo"; import { pageMetadata } from "@/lib/page-metadata"; import { safeJsonLd } from "@/lib/jsonld"; @@ -145,9 +146,12 @@ export default async function HlFrontendPage({

Hyperliquid frontend

-

- {frontend.name} -

+
+ +

+ {frontend.name} +

+

x.ms.p50 - a.ms.p50 : (a, x) => a.ms.p50 - x.ms.p50, ); - const leader = sorted[0]; + const leader = b.leaderSlug + ? (sorted.find((r) => r.slug === b.leaderSlug) ?? sorted[0]) + : sorted[0]; const headlineValue = !isDraft && !insufficient && leader ? fmtValue(leader.ms.p50, b.unit) : "n/a"; // Pass the leader's p50 so unitSuffix mirrors fmtUnit's auto-conversion diff --git a/src/components/chain-kpi-strip.tsx b/src/components/chain-kpi-strip.tsx index 24c8da33..b8da98d5 100644 --- a/src/components/chain-kpi-strip.tsx +++ b/src/components/chain-kpi-strip.tsx @@ -20,6 +20,8 @@ type CardDef = { label: string; value: number | null; tip: string; + /** Optional formatter override; defaults to fmtUSD. */ + fmt?: (v: number) => string; }; export function ChainKpiStrip({ @@ -35,7 +37,32 @@ export function ChainKpiStrip({ tvlHistory?: ChainTvlHistory | null; chainLabel: string; }) { + const subsidyCards: CardDef[] = slug === "robinhood" + ? [ + { + label: "Gas subsidy days left", + value: kpis.subsidyDaysRemaining, + tip: + "Days remaining until Robinhood Chain's 90-day launch gas subsidy ends on 2026-09-29. Users pay $0 in gas during this window; Robinhood covers the sequencer cost.", + fmt: fmtDays, + }, + { + label: "Subsidy paid so far", + value: kpis.subsidyCostToDate, + tip: + "Estimated cumulative USD Robinhood has paid the sequencer since launch to cover user gas. This is NOT growthepie's fees_paid_by_users (~$0 during the subsidy by design). Refreshed hourly.", + }, + { + label: "Projected subsidy total", + value: kpis.subsidyProjectedTotal, + tip: + "Where the subsidy lands if the current daily burn rate holds through Sep 29. Linear projection: cumulative_cost x (90 / days_elapsed).", + }, + ] + : []; + const staticCards: CardDef[] = [ + ...subsidyCards, { label: "TVL", value: kpis.tvl, @@ -133,7 +160,7 @@ function KpiCard({ card }: { card: CardDef }) { {card.label}

- {fmtUSD(card.value!)} + {(card.fmt ?? fmtUSD)(card.value!)}

{/* Empty slot mirrors the live-card sparkline space so static and live KpiCards share the exact same baseline geometry. */} @@ -142,6 +169,12 @@ function KpiCard({ card }: { card: CardDef }) { ); } +function fmtDays(v: number): string { + if (!Number.isFinite(v)) return "-"; + if (v < 0) return "ended"; + return `${Math.round(v)}d`; +} + function fmtUSD(v: number): string { if (!Number.isFinite(v) || v === 0) return "$0"; const abs = Math.abs(v); diff --git a/src/components/provider-logo.tsx b/src/components/provider-logo.tsx index 62d4b0f4..4125ae79 100644 --- a/src/components/provider-logo.tsx +++ b/src/components/provider-logo.tsx @@ -80,6 +80,11 @@ export function ProviderLogo({ // hairline shadow so they pop on both light and dark page backgrounds. const NEEDS_LIGHT_CHIP = new Set([ "aptos", + "slash", + "markets-mobile", + "aapl", + "pltr", + "leorpc", "blinklabs", "mobula", "lighter", diff --git a/src/data/benchmarks.ts b/src/data/benchmarks.ts index 0808aaf5..7aa14f07 100644 --- a/src/data/benchmarks.ts +++ b/src/data/benchmarks.ts @@ -9,6 +9,7 @@ import { promises as fs } from "node:fs"; import path from "node:path"; import yaml from "js-yaml"; import { cache } from "react"; +import { leader } from "@/lib/citation"; import { downsample, MINI_CHART_POINTS } from "@/lib/downsample"; import type { Benchmark } from "@/types/benchmark"; import { @@ -51,6 +52,14 @@ export type BenchmarkCardData = { sampleSize: number; lastRunAt: string; metric: string; + /** Slug of the citation-eligible leader for this bench (result of + * `leader(b)` in citation.ts). Passed at the projection boundary so + * the hub card can render the same headline the bench page names + * without duplicating the reliability + insufficient-sample filter + * logic. Null when the bench has no defensible leader (draft, + * insufficient, or every provider below the reliability floor with + * an empty live pool). */ + leaderSlug: string | null; results: { slug: string; name: string; @@ -76,6 +85,7 @@ export function toBenchmarkCardData(b: Benchmark): BenchmarkCardData { sampleSize: b.sampleSize, lastRunAt: b.lastRunAt, metric: b.metric, + leaderSlug: leader(b)?.slug ?? null, results: b.results.map((r) => ({ slug: r.slug, name: r.name, @@ -115,7 +125,7 @@ export const getBenchmarksSafe = cache(loadAllBenchmarksSafe); export async function getBenchmark( slug: string, - options: { chain?: string; region?: string } = {} + options: { chain?: string; region?: string; kind?: string; venue?: string } = {} ): Promise { return loadBenchmark(slug, options); } diff --git a/src/data/provider-registry.ts b/src/data/provider-registry.ts index 8b89ddbc..a49929fe 100644 --- a/src/data/provider-registry.ts +++ b/src/data/provider-registry.ts @@ -214,6 +214,112 @@ export const PROVIDER_REGISTRY: Record = { "Solana transaction landing service by Temporal Labs. Direct-to-leader submission from 9 colocated regions, tip paid only on successful landing.", twitter: "@temporal_xyz", }, + "solana-official": { + url: "https://solana.com", + description: + "api.mainnet-beta.solana.com is the chain-official public RPC operated by Solana Labs, the endpoint every tutorial and SDK defaults to, documented at roughly 100 requests per 10 seconds per IP.", + twitter: "@solana", + }, + leorpc: { + url: "https://leorpc.com", + description: + "LeoRPC is a community Solana RPC operation exposing a publicly documented FREE tier through a query key, one of the few remaining keyless ways to read Solana mainnet.", + }, + aapl: { + url: "https://www.apple.com", + description: + "Apple tokenized equity measured across issuers on the RWA benchmarks: Robinhood's token on Robinhood Chain (Uniswap v4 vs USDG) and Backed's xStock on Solana (Jupiter executable mid vs USDC), each tracked live against the real market price.", + }, + nvda: { + url: "https://www.nvidia.com", + description: + "Nvidia tokenized equity measured across issuers on the RWA benchmarks: Robinhood's token on Robinhood Chain (Uniswap v4 vs USDG) and Backed's xStock on Solana (Jupiter executable mid vs USDC), each tracked live against the real market price.", + }, + googl: { + url: "https://www.google.com", + description: + "Alphabet tokenized equity measured across issuers on the RWA benchmarks: Robinhood's token on Robinhood Chain (Uniswap v4 vs USDG) and Backed's xStock on Solana (Jupiter executable mid vs USDC), each tracked live against the real market price.", + }, + tsla: { + url: "https://www.tesla.com", + description: + "Tesla tokenized equity measured across issuers on the RWA benchmarks: Robinhood's token on Robinhood Chain (Uniswap v4 vs USDG) and Backed's xStock on Solana (Jupiter executable mid vs USDC), each tracked live against the real market price.", + }, + pltr: { + url: "https://www.palantir.com", + description: + "Palantir tokenized equity measured across issuers on the RWA benchmarks: Robinhood's token on Robinhood Chain (Uniswap v4 vs USDG) and Backed's xStock on Solana (Jupiter executable mid vs USDC), each tracked live against the real market price.", + }, + meta: { + url: "https://www.meta.com", + description: + "Meta Platforms tokenized equity measured across issuers on the RWA benchmarks: Robinhood's token on Robinhood Chain (Uniswap v4 vs USDG) and Backed's xStock on Solana (Jupiter executable mid vs USDC), each tracked live against the real market price.", + }, + amd: { + url: "https://www.amd.com", + description: + "AMD tokenized equity issued by Robinhood on Robinhood Chain (contract 0x86923f96303D656E4aa86D9d42D1e57ad2023fdC), trading against USDG on Uniswap v4 around the clock. OpenChainBench measures its live price deviation from the real market on the tokenized stock peg benchmark.", + }, + msft: { + url: "https://www.microsoft.com", + description: + "Microsoft tokenized equity measured across issuers on the RWA benchmarks: Robinhood's token on Robinhood Chain (Uniswap v4 vs USDG) and Backed's xStock on Solana (Jupiter executable mid vs USDC), each tracked live against the real market price.", + }, + amzn: { + url: "https://www.amazon.com", + description: + "Amazon tokenized equity measured across issuers on the RWA benchmarks: Robinhood's token on Robinhood Chain (Uniswap v4 vs USDG) and Backed's xStock on Solana (Jupiter executable mid vs USDC), each tracked live against the real market price.", + }, + spy: { + url: "https://www.ssga.com", + description: + "SPDR S&P 500 ETF (State Street) tokenized equity measured across issuers on the RWA benchmarks: Robinhood's token on Robinhood Chain (Uniswap v4 vs USDG) and Backed's xStock on Solana (Jupiter executable mid vs USDC), each tracked live against the real market price.", + }, + mu: { + url: "https://www.micron.com", + description: + "Micron Technology tokenized equity issued by Robinhood on Robinhood Chain (contract 0xfF080c8ce2E5feadaCa0Da81314Ae59D232d4afD), trading against USDG on Uniswap v4 around the clock. OpenChainBench measures its live price deviation from the real market on the tokenized stock peg benchmark.", + }, + hood: { + url: "https://robinhood.com", + description: + "Robinhood Markets tokenized equity, issued by Backed as HOODx on Solana. Notably absent from Robinhood's own chain: the only onchain HOOD is third party. Tracked live against the Nasdaq price on the xStocks peg benchmark.", + }, + qqq: { + url: "https://www.invesco.com", + description: + "Invesco QQQ Nasdaq 100 ETF, tokenized by Backed as QQQx on Solana. Tracked live against the real ETF price on the xStocks peg benchmark.", + }, + coin: { + url: "https://www.coinbase.com", + description: + "Coinbase tokenized equity, issued by Backed as COINx on Solana, a crypto exchange's stock trading on a blockchain. Tracked live against the Nasdaq price on the xStocks peg benchmark.", + }, + "orca-solana": { + url: "https://www.orca.so", + description: + "Orca is Solana's concentrated liquidity DEX. Its USDY/USDC whirlpool is the deepest genuine venue for Ondo's tokenized treasury and the market leg of the USDY NAV basis benchmark.", + }, + "pyth-market": { + url: "https://pyth.network", + description: + "Pyth Network's USDY/USD market composite aggregates USDY trading into one feed. Measured against Pyth's own USDY redemption rate feed on the NAV basis benchmark.", + }, + slash: { + url: "https://slash.trade", + description: + "Slash is a Telegram trading bot for Hyperliquid perps, routing user orders via its builder code and collecting builder fees. Tracked live on the Hyperliquid frontends leaderboard.", + }, + topdog: { + url: "https://t.me/topdog_trade_bot", + description: + "TopDog is a Telegram-native social trading bot on Hyperliquid, tagline 'Never Trade Alone', built around copying and coordinating around active traders. Routes orders via its builder code.", + }, + "markets-mobile": { + url: "https://markets.xyz", + description: + "Markets by Kinetiq is a mobile-first Hyperliquid frontend focused on clean UX for perp trading on iOS and Android. Routes flow through its builder code and appears on the Hyperliquid frontends leaderboard.", + }, bloxroute: { url: "https://bloxroute.com", description: @@ -488,6 +594,24 @@ export const PROVIDER_REGISTRY: Record = { }, // ─── Public RPC providers ───────────────────────────────────── + parity: { + url: "https://www.parity.io", + description: + "Parity Technologies is the primary implementation shop behind Polkadot and Substrate, and the operator of the reference public relay RPC at rpc.polkadot.io. Also builds the Polkadot-JS Apps UI, Kusama tooling, and enterprise Substrate deployments.", + twitter: "@ParityTech", + }, + "polkadot-official": { + url: "https://rpc.polkadot.io", + description: + "Parity-operated Polkadot relay-chain public RPC endpoint (rpc.polkadot.io). Substrate JSON-RPC (chain_getHeader, state_getRuntimeVersion, system_health), keyless, rate-limited per IP. The reference endpoint any Polkadot integrator points at before wiring a dedicated node.", + twitter: "@ParityTech", + }, + dwellir: { + url: "https://www.dwellir.com", + description: + "Web3 Foundation grantee running production-grade Substrate infrastructure across globally distributed bare-metal servers. Keyless public gateways on Polkadot, Kusama and every major parachain, plus dedicated node service.", + twitter: "@dwellir", + }, "monad-official": { url: "https://docs.monad.xyz", description: @@ -952,11 +1076,6 @@ export const PROVIDER_REGISTRY: Record = { "Owly.fi is a non custodial AI powered copy trading and asset management terminal on Hyperliquid, executing perp orders via a registered builder code.", twitter: "@owlyfi", }, - topdog: { - url: "https://topdog.gg", - description: - "TopDog is a Discord signal bot delivering real time trade alerts from top performing Hyperliquid wallets, with execution routed via a registered builder code.", - }, goodcryptox: { url: "https://goodcrypto.app", description: @@ -1457,6 +1576,76 @@ export const PROVIDER_REGISTRY: Record = { "zkSync hybrid CEX/DEX perp venue with privacy-preserving order matching.", twitter: "@grvt_io", }, + + // ─── Oracles (bench № 082 oracle-freshness) ─────────────────── + pyth: { + url: "https://www.pyth.network", + description: + "Pull-based oracle network publishing sub-second price updates through the Hermes service; consumers post the signed update onchain only when they need it. Prices are contributed by exchanges and market makers.", + twitter: "@PythNetwork", + }, + redstone: { + url: "https://www.redstone.finance", + description: + "Modular oracle delivering signed price packages every few seconds through its data gateway, consumed push or pull. The freshness bench reads the primary-prod feed keyless.", + twitter: "@redstone_defi", + }, + + // ─── Ethereum block builders (bench № 085) ──────────────────── + // Attribution rows come from the extraData self-label each builder + // stamps into its blocks. Where no legal entity is public, the + // description says so instead of guessing. + titan: { + url: "https://www.titanbuilder.xyz", + description: + "Dominant private Ethereum block builder, self-labeled `Titan (titanbuilder.xyz)` in block extraData. Regularly builds around half of all MEV-Boost blocks and also operates the Titan relay.", + twitter: "@titanbuilderxyz", + }, + quasar: { + url: "https://quasar.win", + description: + "Ethereum block builder self-described as neutral and non-censoring, second-largest by extraData share. The operating entity is not publicly named; identity rests on the `quasar` extraData self-label and the quasarbuilder.eth fee recipient.", + twitter: "@QuasarBuilder", + }, + eureka: { + url: "https://eurekalabs.xyz", + description: + "Ethereum block builder by Eureka Labs, founded late 2024 and backed by a $6.7M seed round (Spark Capital, Collider Ventures) to develop programmable blocks. Attributed via the `eureka` extraData self-label.", + twitter: "@EurekaBuilder", + }, + buildernet: { + url: "https://buildernet.org", + description: + "Decentralized block-building network initiated by Flashbots, with Beaverbuild and Nethermind among the early node operators. Blocks are built inside TEEs by multiple operators and carry the `BuilderNet` extraData tag; conceived as the mutualized counterweight to private-builder concentration.", + }, + beaverbuild: { + url: "https://beaverbuild.org", + description: + "Pseudonymous Ethereum block builder that dominated block building through 2023-2024 alongside rsync. Labels its blocks `beaverbuild.org` in extraData; no legal entity is publicly identified.", + twitter: "@beaverbuild", + }, + btcs: { + url: "https://www.btcs.com/builder/", + description: + "Builder+ is the Ethereum block-building operation of BTCS Inc., a Nasdaq-listed blockchain infrastructure company (NodeOps staking, Builder+, Imperium DeFi). Attributed via the `Builder+` extraData self-label.", + twitter: "@NasdaqBTCS", + }, + bobthebuilder: { + url: "https://bobthebuilder.xyz", + description: + "Pseudonymous Ethereum block builder, self-labeled `bob-the-builder` in block extraData with the bob-the-builder.eth fee recipient. No operating entity is publicly identified; identity rests on the extraData self-label.", + twitter: "@0xb0bthebuilder", + }, + vanilla: { + url: "https://ethereum.org/en/developers/docs/mev/", + description: + "Not a company: blocks whose extraData is empty or carries an execution client's default tag (geth, reth, nethermind, besu, erigon), meaning the proposer built the block locally instead of outsourcing to a MEV-Boost builder. The live measure of how much of Ethereum still self-builds.", + }, + other: { + url: "https://github.com/ChainBench/OpenChainBench/tree/main/harnesses/evm-block-builders", + description: + "Aggregate long-tail bucket, not a single provider. On the block-builder bench it collects blocks whose extraData tag is not in the curated attribution table; every raw string is logged so named rows can be added as the table grows.", + }, }; /** diff --git a/src/lib/bench-template.ts b/src/lib/bench-template.ts index f36e4300..f1addc5f 100644 --- a/src/lib/bench-template.ts +++ b/src/lib/bench-template.ts @@ -38,6 +38,7 @@ import type { Benchmark, ProviderResult } from "@/types/benchmark"; import { liveResults } from "@/lib/provider-filters"; +import { citationCandidates } from "@/lib/citation"; import { rankResults } from "@/lib/ranking"; import { fmtUnit } from "@/lib/format"; @@ -87,8 +88,18 @@ function worstForChain(b: Benchmark, chain: string): ProviderResult | undefined export function renderTemplate(text: string, benchmark: Benchmark): string { if (!text || text.indexOf("{{") === -1) return text; + // `live` still drives per-slug lookups so callers of {{p50:some-slug}} + // can still address unreliable providers by name (the token is + // explicit). `bestPool` applies the same reliability floor as + // `leader()` so {{best_name}} and {{best_p50}} tokens in bench copy + // never elevate a provider that would be filtered out of the + // citation headline. const live = liveResults(benchmark.results); - const sorted = rankResults(live, benchmark.higherIsBetter); + const bestPool = citationCandidates(benchmark); + const sorted = rankResults( + bestPool.length > 0 ? bestPool : live, + benchmark.higherIsBetter, + ); const best = sorted[0]; const worst = sorted[sorted.length - 1]; diff --git a/src/lib/categories.ts b/src/lib/categories.ts index d432767f..091bde4e 100644 --- a/src/lib/categories.ts +++ b/src/lib/categories.ts @@ -30,7 +30,8 @@ export type Category = | "Wallets" | "RPCs" | "NFT APIs" - | "Explorers"; + | "Explorers" + | "RWA"; export type CategoryEntry = { /** URL slug. Lowercase, kebab-case, ASCII. */ @@ -93,6 +94,13 @@ export const CATEGORIES: readonly CategoryEntry[] = [ description: "Live measurements for block explorer APIs. Compare how many chains each family actually serves with a working indexer, probe-verified daily against the vendors' own registries.", }, + { + slug: "rwa", + label: "RWA", + heading: "RWA benchmarks", + description: + "Live measurements for tokenized real world assets. How closely tokenized stocks and treasuries track their underlying market price, measured onchain around the clock, including nights and weekends when the real markets are closed.", + }, { slug: "nft-apis", label: "NFT APIs", diff --git a/src/lib/category-colors.ts b/src/lib/category-colors.ts index e06035d5..6cf03306 100644 --- a/src/lib/category-colors.ts +++ b/src/lib/category-colors.ts @@ -18,4 +18,6 @@ export const CATEGORY_COLOR: Record = { // in use. Picks up OpenSea/Alchemy/Moralis brand palettes which all // cluster around blues/purples. "NFT APIs": "#6366f1", + // RWA: teal, the tradfi-meets-onchain lane gets its own hue. + RWA: "#0d9488", }; diff --git a/src/lib/chain-kpis.ts b/src/lib/chain-kpis.ts index aa5960f3..eaa28e35 100644 --- a/src/lib/chain-kpis.ts +++ b/src/lib/chain-kpis.ts @@ -34,6 +34,12 @@ export type ChainKpis = { nativePrice: number | null; /** Mobula native token circulating market cap in USD. */ nativeMcap: number | null; + /** Robinhood Chain gas subsidy: days remaining until 2026-09-29 (null on every other chain). */ + subsidyDaysRemaining: number | null; + /** Robinhood Chain gas subsidy: cumulative USD Robinhood has paid the sequencer since launch. */ + subsidyCostToDate: number | null; + /** Robinhood Chain gas subsidy: projected total cost at the current daily rate through Sep 29. */ + subsidyProjectedTotal: number | null; }; /** TVL daily samples for the sparkline chart. Last 30 days, oldest first. */ @@ -78,14 +84,28 @@ export async function fetchChainKpisFresh( } const sel = `{chain="${slug}"}`; - const [tvl, dexVolume24h, stablesMcap, nativePrice, nativeMcap] = - await Promise.all([ - prom.scalar(`chain_tvl_usd${sel}`), - prom.scalar(`chain_dex_volume_24h_usd${sel}`), - prom.scalar(`chain_stables_mcap_usd${sel}`), - prom.scalar(`chain_native_price_usd${sel}`), - prom.scalar(`chain_native_mcap_usd${sel}`), - ]); + const isRobinhood = slug === "robinhood"; + const [ + tvl, + dexVolume24h, + stablesMcap, + nativePrice, + nativeMcap, + subsidyDaysRemaining, + subsidyCostToDate, + subsidyProjectedTotal, + ] = await Promise.all([ + prom.scalar(`chain_tvl_usd${sel}`), + prom.scalar(`chain_dex_volume_24h_usd${sel}`), + prom.scalar(`chain_stables_mcap_usd${sel}`), + prom.scalar(`chain_native_price_usd${sel}`), + prom.scalar(`chain_native_mcap_usd${sel}`), + // Robinhood-only. On every other chain these queries return null, + // and the strip skips the cards silently. + isRobinhood ? prom.scalar(`robinhood_subsidy_days_remaining`) : Promise.resolve(null), + isRobinhood ? prom.scalar(`robinhood_subsidy_chainside_cost_total_usd`) : Promise.resolve(null), + isRobinhood ? prom.scalar(`robinhood_subsidy_projected_total_usd`) : Promise.resolve(null), + ]); return { slug, @@ -94,6 +114,9 @@ export async function fetchChainKpisFresh( stablesMcap, nativePrice, nativeMcap, + subsidyDaysRemaining, + subsidyCostToDate, + subsidyProjectedTotal, }; } @@ -146,7 +169,10 @@ export function hasAnyKpi(k: ChainKpis | null): boolean { k.dexVolume24h != null || k.stablesMcap != null || k.nativePrice != null || - k.nativeMcap != null + k.nativeMcap != null || + k.subsidyDaysRemaining != null || + k.subsidyCostToDate != null || + k.subsidyProjectedTotal != null ); } diff --git a/src/lib/chains.ts b/src/lib/chains.ts index 29fd4bff..2e1fed7b 100644 --- a/src/lib/chains.ts +++ b/src/lib/chains.ts @@ -155,6 +155,14 @@ export const CHAINS: ChainEntry[] = [ description: "OP Stack reference rollup, 2 s sequencer cadence, EIP-4844 blob calldata settlement on Ethereum.", }, + { + slug: "robinhood", + label: "Robinhood Chain", + category: "L2", + nativeSymbol: "ETH", + description: + "Robinhood's Arbitrum Orbit rollup for tokenized stocks, ~100 ms blocks, ETH gas, Ethereum blob settlement, sequencer operated by Robinhood with 10% of chain fees shared to ArbitrumDAO.", + }, { slug: "base", label: "Base", @@ -301,6 +309,14 @@ export const CHAINS: ChainEntry[] = [ description: "Sony Block Solutions OP Stack rollup in the Optimism Superchain, 2 s blocks, consumer and entertainment focus.", }, + { + slug: "polkadot", + label: "Polkadot", + category: "L1", + nativeSymbol: "DOT", + description: + "Substrate-based relay chain coordinating a parachain ecosystem via shared security. GRANDPA finality with BABE block production, ~6 s block time.", + }, ]; export const CHAIN_BY_SLUG = new Map(CHAINS.map((c) => [c.slug, c])); diff --git a/src/lib/citation.test.ts b/src/lib/citation.test.ts new file mode 100644 index 00000000..376995f8 --- /dev/null +++ b/src/lib/citation.test.ts @@ -0,0 +1,113 @@ +import { describe, expect, test } from "bun:test"; +import { leader, fieldValue, rankedCandidates } from "./citation"; +import type { Benchmark, ProviderResult } from "@/types/benchmark"; + +function r( + slug: string, + name: string, + p50: number, + successRate = 100, +): ProviderResult { + return { + slug, + name, + ms: { p50, p90: p50, p99: p50, mean: p50 }, + successRate, + availability: "live", + }; +} + +function bench(results: ProviderResult[]): Benchmark { + return { + slug: "test", + number: "001", + title: "Test bench", + subtitle: "", + lastRunAt: "2026-07-14T00:00:00.000Z", + status: "live", + editorialStatus: "live", + sampleSize: 100, + abstract: "", + metric: "Latency", + unit: "ms", + higherIsBetter: false, + category: "RPCs", + results, + findings: [], + methodology: [], + source: "", + extras: { series24h: {}, regions: {} }, + }; +} + +describe("citation reliability threshold", () => { + test("leader excludes providers with success rate below 50 percent", () => { + // Owlracle scenario: technically most accurate (lowest p50 gap) but + // fails 93 percent of calls. Etherscan should win despite a higher p50. + const b = bench([ + r("owlracle", "Owlracle", 0.001, 6.48), + r("etherscan", "Etherscan", 1.0, 95.7), + r("publicnode", "PublicNode", 1.5, 99.99), + ]); + const top = leader(b); + expect(top?.slug).toBe("etherscan"); + expect(top?.value).toBe(1.0); + }); + + test("fieldValue reflects the reliability-filtered leader", () => { + const b = bench([ + r("owlracle", "Owlracle", 0.001, 6.48), + r("etherscan", "Etherscan", 1.0, 95.7), + ]); + expect(fieldValue(b)).toBe(1.0); + }); + + test("falls back to the full live pool when every provider is unreliable", () => { + // Bench in a totally degraded state: rather than vanishing from + // downstream surfaces, surface the least-bad provider so readers + // still see a live number with the caveats their spec already + // documents. + const b = bench([ + r("a", "A", 5, 20), + r("b", "B", 10, 30), + ]); + const top = leader(b); + expect(top?.slug).toBe("a"); + }); + + test("providers with the default 100 percent success rate pass through", () => { + // Freshness / gauge-only benches never emit a `success` query; + // the loader defaults to 100 percent, so the guard is inert. + const b = bench([r("a", "A", 100), r("b", "B", 200)]); + expect(leader(b)?.slug).toBe("a"); + }); + + test("higher-is-better ranks the correct reliable leader", () => { + const b = bench([ + r("high-but-flaky", "Flaky", 999, 10), + r("modest-reliable", "Reliable", 50, 100), + r("mid-reliable", "Mid", 80, 100), + ]); + b.higherIsBetter = true; + // higherIsBetter=true reverses sort so the biggest p50 wins. + // Flaky wins on raw value but is filtered out; Mid (80) wins the + // reliable pool. + expect(leader(b)?.slug).toBe("mid-reliable"); + }); + + test("rankedCandidates[0] matches leader for the same bench", () => { + // Locks the invariant that downstream surfaces (/api/stat rankings, + // llm-context, MCP resource) can share `rankedCandidates` with + // `leader()` and never emit a document where the "leader" field + // contradicts the first entry of the "rankings" list. + const b = bench([ + r("owlracle", "Owlracle", 0.001, 6.48), + r("etherscan", "Etherscan", 1.0, 95.7), + r("publicnode", "PublicNode", 1.5, 99.99), + ]); + const top = leader(b); + const ranks = rankedCandidates(b); + expect(top?.slug).toBe(ranks[0].slug); + expect(top?.value).toBe(ranks[0].ms.p50); + }); +}); diff --git a/src/lib/citation.ts b/src/lib/citation.ts index ad3eb923..09062a88 100644 --- a/src/lib/citation.ts +++ b/src/lib/citation.ts @@ -8,16 +8,71 @@ import type { Benchmark, ProviderResult } from "@/types/benchmark"; import { liveResults } from "@/lib/provider-filters"; import { fmtUnit } from "@/lib/format"; +/** Minimum measured success rate (in percent, 0-100) for a provider to + * contribute to the headline leader claim. Providers with a real + * success measurement below this floor are excluded from citation + * candidates because an "unreliable but accurate when it works" + * outlier should not top the leaderboard: it misleads AI agents citing + * the bench and any human reader glancing at the headline. Benches + * whose harness does not emit a `success` query default to 100 in the + * load path (see materialize/load.ts), so this guard is a no-op for + * freshness / gauge-only benches and only bites where the harness + * actually measures polling reliability (gas-estimation, RPC + * benches). */ +const LEADER_MIN_SUCCESS_PCT = 50; + /** Provider set used to derive the headline figures. Drops rows whose * per-provider sample-health is "insufficient" (set on the load path * when the bench declares expected_n and the row falls below the 10 - * percent of expected floor). Those rows can still render in some - * surfaces with a soft tag, but they must not contribute to the leader - * claim shipped to AI agents and journalists via the citable APIs. */ -function citationCandidates(b: Benchmark): ProviderResult[] { + * percent of expected floor) and rows whose measured success rate + * sits below the reliability floor. Those rows can still render in + * some surfaces with a soft tag, but they must not contribute to the + * leader claim shipped to AI agents and journalists via the citable + * APIs. Falls back to the full live pool when every provider is + * below the reliability floor so a totally-degraded bench still + * reports a best-of-bad-options leader instead of vanishing. + * + * Exported so downstream machine-readable surfaces (`/api/stat` + * rankings, `/api/llm-context`, MCP `get_benchmark`, `/api/compare`) + * can share the same eligibility rule as `leader()` and stay + * internally consistent: a JSON that names Etherscan as leader + * should not simultaneously rank Owlracle first in its `rankings` + * array. */ +export function citationCandidates(b: Benchmark): ProviderResult[] { const live = liveResults(b.results); - if (!b.expectedN) return live; - return live.filter((r) => r.dataConfidence !== "insufficient"); + const reliable = live.filter( + (r) => (r.successRate ?? 100) >= LEADER_MIN_SUCCESS_PCT, + ); + const pool = reliable.length > 0 ? reliable : live; + if (!b.expectedN) return pool; + return pool.filter((r) => r.dataConfidence !== "insufficient"); +} + +/** Sorted candidate pool for the machine-readable `rankings` array on + * `/api/stat`, MCP, llm-context and any downstream that ranks the + * full field. Applies the same reliability + insufficient-sample + * filters as `leader()` so a document that names X as leader ranks X + * first in its own list. Sort direction honors the bench's + * `higherIsBetter` flag. */ +export function rankedCandidates(b: Benchmark): ProviderResult[] { + return [...citationCandidates(b)].sort((a, c) => + b.higherIsBetter ? c.ms.p50 - a.ms.p50 : a.ms.p50 - c.ms.p50, + ); +} + +/** Timestamp of the last real measurement, or null when the bench has + * no measurement history yet (draft placeholder). Draft benches carry + * a wall-clock `lastRunAt` for type safety (Benchmark.lastRunAt is a + * non-nullable string), which downstream JSON, JSON-LD and MCP surfaces + * would otherwise expose as a real freshness signal to LLM crawlers. + * Use this helper on every machine-readable surface. + * + * Accepts any object that carries `status` + `lastRunAt` so the slim + * `ProviderAppearance.benchmark` (a Pick of Benchmark) can use it too. */ +export function citableAsOf( + b: Pick, +): string | null { + return b.status === "draft" ? null : b.lastRunAt; } /** Median value of the benchmark (the field shown in the headline). */ @@ -27,11 +82,8 @@ export function fieldValue(b: Benchmark): number | null { // (downstream LLM tools and SERP snippets would otherwise quote a // number drawn from a wildly undersized field). if (b.dataConfidence === "insufficient") return null; - const candidates = citationCandidates(b); - if (candidates.length === 0) return null; - const sorted = [...candidates].sort((a, c) => - b.higherIsBetter ? c.ms.p50 - a.ms.p50 : a.ms.p50 - c.ms.p50 - ); + const sorted = rankedCandidates(b); + if (sorted.length === 0) return null; return sorted[0].ms.p50; } @@ -39,11 +91,8 @@ export function fieldValue(b: Benchmark): number | null { export function leader(b: Benchmark): { name: string; slug: string; value: number } | null { if (b.status !== "live") return null; if (b.dataConfidence === "insufficient") return null; - const candidates = citationCandidates(b); - if (candidates.length === 0) return null; - const sorted = [...candidates].sort((a, c) => - b.higherIsBetter ? c.ms.p50 - a.ms.p50 : a.ms.p50 - c.ms.p50 - ); + const sorted = rankedCandidates(b); + if (sorted.length === 0) return null; return { name: sorted[0].name, slug: sorted[0].slug, value: sorted[0].ms.p50 }; } @@ -154,6 +203,13 @@ export type CiteBundle = { plain: string; bibtex: string; apa: string; + /** RIS record (Research Information Systems) for Zotero, EndNote, + * Mendeley and every academic reference manager that speaks the + * format. Same fields as bibtex, RIS field codes: TY (type), + * AU (author), TI (title), PY (year), UR (url), Y2 (retrieved + * date), ER (end record). Type GEN is the neutral fallback for a + * dataset citation — RIS has no "benchmark" type. */ + ris: string; }; const MONTHS = [ @@ -187,6 +243,17 @@ export function citeBundle( plain: `OpenChainBench. "${b.title}". Retrieved ${isoDate}. ${url}`, bibtex: `@misc{${bibKey},\n author = {OpenChainBench},\n title = {${b.title}},\n year = {${yyyy}},\n url = {${url}},\n note = {Retrieved ${isoDate}}\n}`, apa: `OpenChainBench. (${yyyy}). ${b.title}. Retrieved ${longDate}, from ${url}`, + // RIS records use CRLF line endings by convention (RFC-style), + // which every consumer we've tested (Zotero 6+, EndNote 20+, + // Mendeley) accepts either way — using \r\n stays safe. + ris: + `TY - GEN\r\n` + + `AU - OpenChainBench\r\n` + + `TI - ${b.title}\r\n` + + `PY - ${yyyy}\r\n` + + `UR - ${url}\r\n` + + `Y2 - ${isoDate}\r\n` + + `ER - \r\n`, }; } diff --git a/src/lib/dataset-jsonld.ts b/src/lib/dataset-jsonld.ts index 567080b1..951cc20c 100644 --- a/src/lib/dataset-jsonld.ts +++ b/src/lib/dataset-jsonld.ts @@ -117,22 +117,85 @@ export const GLOBAL_DATASET_JSONLD = { /** Inputs for a per-bench Dataset entry. Keeps the call site at * /benchmarks/[slug] decoupled from the Benchmark type so the helper is * trivially reusable from the per-chain page and future variants. */ +/** Schema.org PropertyValue node used inside variableMeasured to publish + * a measured aggregate with its numeric value. This is the shape Google + * Dataset Search + academic LLM tools extract as a structured fact, + * unlike a bare label string which is opaque to a crawler. + * + * `value` is the current leader's p50 (or leader's p90/p99) expressed in + * the same unit as `unitText`; downstream consumers can key on + * `propertyID` for machine matching and on `name` for display. */ +export type VariableMeasuredValue = { + "@type": "PropertyValue"; + propertyID: string; + name: string; + value: number; + unitText: string; +}; + export type BenchDatasetInput = { slug: string; name: string; alternateName?: string; description: string; url: string; - /** Schema.org Dataset accepts variableMeasured as a string or array. - * We pass an array of metric labels (p50, p90, p99, sample_size, ...) - * so Google's validator reports each metric individually. */ - variableMeasured: string[]; + /** Schema.org Dataset accepts variableMeasured as a string, a + * PropertyValue, or an array mixing both. Pass PropertyValue objects + * when the current leader's numeric aggregates are available (Google + * Dataset Search / academic LLM tools index the numeric `value` + * directly); fall back to bare label strings for drafts or when the + * aggregate is `sample_size`-shaped rather than a per-percentile + * measurement. */ + variableMeasured: Array; category: string; datePublished: string; dateModified?: string; measurementTechnique?: string; }; +/** Build the `variableMeasured` array for a bench Dataset node. + * - When the leader carries a real numeric p50 (and unit), the p50 / + * p90 / p99 percentiles ship as PropertyValue objects so Google + * Dataset Search and academic LLM tools extract the numeric fact + * without parsing prose. + * - `sample_size` and the bare metric label stay as strings — they + * describe axes, not measured values. + * - When no defensible leader exists (draft, insufficient) the array + * degrades to the legacy string-only shape so we never publish a + * fabricated PropertyValue with a zero-fallback value. */ +export function buildBenchVariableMeasured(input: { + metric: string; + unit: string; + leader: { name: string; p50: number; p90: number; p99: number } | null; +}): Array { + if (!input.leader || input.leader.p50 <= 0) { + return [ + input.metric, + `${input.metric}_p50`, + `${input.metric}_p90`, + `${input.metric}_p99`, + "sample_size", + ]; + } + const mk = ( + suffix: "p50" | "p90" | "p99", + value: number, + ): VariableMeasuredValue => ({ + "@type": "PropertyValue", + propertyID: `${input.metric.toLowerCase().replace(/\s+/g, "_")}_${suffix}`, + name: `${input.metric} ${suffix}`, + value, + unitText: input.unit, + }); + return [ + input.metric, + mk("p50", input.leader.p50), + mk("p90", input.leader.p90), + mk("p99", input.leader.p99), + "sample_size", + ]; +} + /** Inputs for the per-bench StatisticalReport companion node. Wrapping * the single-leader claim as `StatisticalReport` + inline `Observation` * is the shape Google Dataset Search and Perplexity's grounding pipe @@ -220,7 +283,12 @@ export function buildBenchDatasetJsonLd( keywords: [ input.category, ...KEYWORDS, - ...input.variableMeasured, + // Keywords should stay flat strings for indexer compat, so pull + // the display name off any PropertyValue entries rather than + // leaking `[object Object]` into the graph. + ...input.variableMeasured.map((v) => + typeof v === "string" ? v : v.name, + ), ], creator: CREATOR_PUBLISHER, publisher: CREATOR_PUBLISHER, diff --git a/src/lib/logo-manifest.ts b/src/lib/logo-manifest.ts index 3df2764e..a4f4382f 100644 --- a/src/lib/logo-manifest.ts +++ b/src/lib/logo-manifest.ts @@ -108,6 +108,8 @@ const RAW: Record = { drpc: "/logos/drpc.webp", "1rpc": "/logos/1rpc.svg", cloudflare: "/logos/cloudflare.svg", + parity: "/logos/parity.png", + polkadot: "/logos/polkadot.png", "base-official": "/logos/base.jpeg", binance: "/logos/binance.png", lava: "/logos/lava.webp", @@ -164,6 +166,21 @@ const RAW: Record = { nextblock: "/logos/nextblock.png", astralane: "/logos/astralane.svg", solanavibestation: "/logos/solanavibestation.png", + leorpc: "/logos/leorpc.png", + aapl: "/logos/aapl.svg", + nvda: "/logos/nvda.svg", + googl: "/logos/googl.svg", + tsla: "/logos/tsla.svg", + pltr: "/logos/pltr.svg", + meta: "/logos/meta.svg", + amd: "/logos/amd.svg", + msft: "/logos/msft.svg", + amzn: "/logos/amzn.png", + spy: "/logos/spy.svg", + mu: "/logos/mu.png", + qqq: "/logos/qqq.png", + "orca-solana": "/logos/orca.png", + "pyth-market": "/logos/pyth.jpg", // ─── Buyback audit (bench 018) ─── sky: "/logos/sky.svg", @@ -247,6 +264,9 @@ const RAW: Record = { coinpilot: "/logos/coinpilot.png", echosync: "/logos/echosync.png", fomo: "/logos/fomo.png", + slash: "/logos/slash.png", + topdog: "/logos/topdog.jpg", + "markets-mobile": "/logos/markets-mobile.svg", dextrabot: "/logos/dextrabot.png", kinto: "/logos/kinto.jpg", hypersignals: "/logos/hypersignals.png", @@ -314,6 +334,21 @@ const RAW: Record = { pancakeswap: "/logos/pancakeswap.png", mevblocker: "/logos/mevblocker.svg", + // ─── Ethereum block builders (bench № 085) ─── + // `other` (aggregate long-tail row) reuses /logos/other.svg above. + titan: "/logos/titan.png", + quasar: "/logos/quasar.png", + eureka: "/logos/eureka.png", + buildernet: "/logos/buildernet.png", + beaverbuild: "/logos/beaverbuild.jpg", + btcs: "/logos/btcs.png", + bobthebuilder: "/logos/bobthebuilder.jpg", + vanilla: "/logos/vanilla.svg", + + // ─── Oracles (bench № 082 oracle-freshness) ─── + pyth: "/logos/pyth.jpg", + redstone: "/logos/redstone.png", + // ─── Hyperliquid frontends registry expansion (60 → 66, 2026-06-28) ─── // Identified via on-chain HL referral codes + brand cross-reference // (DFS, UNITYWALLET, INVO, MARSGO, BITGETWALLET). See builders.json @@ -365,6 +400,10 @@ const ALIASES: Record = { // Long-tail RPC cluster (055-066). "sonic-official": "sonic", "monad-official": "monad", + "solana-official": "solana", + "polkadot-official": "parity", + hood: "robinhood", + coin: "coinbase", "megaeth-official": "megaeth", "celo-official": "celo", "blast-official": "blast", diff --git a/src/lib/providers.ts b/src/lib/providers.ts index cd65780e..01043029 100644 --- a/src/lib/providers.ts +++ b/src/lib/providers.ts @@ -12,6 +12,7 @@ import { cache } from "react"; import { unstable_cache } from "next/cache"; import { getBenchmarksSafe } from "@/data/benchmarks"; import { liveResults } from "@/lib/provider-filters"; +import { citationCandidates } from "@/lib/citation"; import { readBestPerChain } from "@/lib/per-chain-contract"; import type { Benchmark, ProviderResult } from "@/types/benchmark"; @@ -40,6 +41,11 @@ const PRODUCT_ALIASES: Record = { "avalanche-official": "avalanche", "base-official": "base", "optimism-official": "optimism", + // Polkadot's "official" endpoint is Parity-operated, so the product + // page for the endpoint slug collapses to the Parity brand rather + // than a synthetic "Polkadot Foundation RPC" entry (there is no such + // separate operator). + "polkadot-official": "parity", // Oracle pair → underlying chain / asset "eth-usd": "ethereum", "sol-usd": "solana", @@ -74,6 +80,10 @@ const CANONICAL_NAMES: Record = { arbitrum: "Arbitrum", base: "Base", optimism: "Optimism", + polkadot: "Polkadot", + parity: "Parity", + onfinality: "OnFinality", + dwellir: "Dwellir", // NFT bench 040 providers — explicit brand casing. opensea: "OpenSea", // Other brand casings frequently referenced. @@ -212,7 +222,15 @@ export type ProviderProfile = { }; function rankProviders(b: Benchmark): ProviderResult[] { - const live = liveResults(b.results); + // Drives the /products hub `wins` count and per-chain leadership + // chips. Shares the reliability + insufficient-sample filters with + // `leader()` (citation.ts) so a provider that is filtered from the + // bench headline never gets credited as a bench winner on its + // product page. Falls back to the raw live pool when every + // candidate is filtered so a totally degraded bench still surfaces + // a best-of-bad-options ranking. + const pool = citationCandidates(b); + const live = pool.length > 0 ? pool : liveResults(b.results); return [...live].sort((a, c) => b.higherIsBetter ? c.ms.p50 - a.ms.p50 : a.ms.p50 - c.ms.p50, ); diff --git a/src/lib/removed-benches.ts b/src/lib/removed-benches.ts index d7823ee9..e5a4c60c 100644 --- a/src/lib/removed-benches.ts +++ b/src/lib/removed-benches.ts @@ -36,6 +36,13 @@ export const REMOVED_BENCH_SLUGS = new Set([ "indexing-freshness", "rpc-keyed-latency", "explorer-chain-coverage", - "solana-rpc", + "tokenized-stock-arb-latency", "portfolio-chain-coverage", + // bench vague 2 (081-085): validating on staging until harnesses have + // 48h of clean data on the VPS, then ship dev -> main + "ws-head-latency", + "oracle-freshness", + "rpc-reliability", + "indexer-latency", + "evm-block-builders", ]); diff --git a/src/lib/spec-schema.ts b/src/lib/spec-schema.ts index 47fc2eeb..75535ac0 100644 --- a/src/lib/spec-schema.ts +++ b/src/lib/spec-schema.ts @@ -158,6 +158,7 @@ export const Category = z.enum([ "RPCs", "NFT APIs", "Explorers", + "RWA", ]); // Em-dash (—) and en-dash (–) are the classic "AI tells" that hurt our brand diff --git a/src/lib/spec.ts b/src/lib/spec.ts index 7f9ea805..93926846 100644 --- a/src/lib/spec.ts +++ b/src/lib/spec.ts @@ -201,6 +201,16 @@ const loadBenchmarkUnfilteredCached = unstable_cache( // ISR re-render bursts. const stored = await benchFromStore(slug, ""); if (stored) return slimBenchmarkForCache(overlayEditorial(stored, spec)); + if (spec.status === "live") { + // A live spec with no readable blob is a transient store failure + // (proxy timeout, worker mid-write), not a real state. Returning + // undefined would cache the miss for the whole revalidate window: + // home row + /api/stat then serve "Awaiting samples" while the + // bench page reads fine (bnb-rpc incident, 2026-07-13). Throwing + // makes unstable_cache keep the last good entry; cold-cache + // callers already catch and fall back to the draft placeholder. + throw new Error(`store blob missing for live bench ${slug}`); + } return undefined; }, // Version key bumped when Benchmark shape changes so stale cache entries @@ -273,7 +283,9 @@ const loadBenchmarkUnfilteredCached = unstable_cache( // buckets (gap rendering fix). Cached v26 entries hold the old // hole-compressed arrays whose indices no longer map onto the nominal // step grid the chart back-computes timestamps from. - ["bench-unfiltered-v27", process.env.VERCEL_ENV === "production" ? "prod" : "all"], +// v28: bench vague 2 ships 081-085 (ws-head-latency, oracle-freshness, + // rpc-reliability, indexer-latency, evm-block-builders). Bench SET changed. + ["bench-unfiltered-v36", process.env.VERCEL_ENV === "production" ? "prod" : "all"], { revalidate: 300, tags: ["benchmarks"] }, ); @@ -428,7 +440,8 @@ const loadAllBenchmarksCached = unstable_cache( // gated catalog to /products for 30+ min after the deploy). // v29: bumped with bench-unfiltered-v26 (monad-rpc + megaeth-rpc ship). // v30: bumped with bench-unfiltered-v27 (dense series with nulls). - ["all-benchmarks-v30", process.env.VERCEL_ENV === "production" ? "prod" : "all"], +// v31: bumped with bench-unfiltered-v28 (bench vague 2, 081-085). + ["all-benchmarks-v39", process.env.VERCEL_ENV === "production" ? "prod" : "all"], { revalidate: 300, tags: ["benchmarks"] }, ); export const loadAllBenchmarks = cache(loadAllBenchmarksCached); @@ -507,7 +520,8 @@ const loadBenchmarkFiltered = unstable_cache( // v16: bumped with the bench 074 ship (lockstep rule, see all-benchmarks-v28). // v17: bumped with the monad-rpc + megaeth-rpc ship (lockstep rule). // v18: bumped with bench-unfiltered-v27 (dense series with nulls). - ["bench-filters-v18", process.env.VERCEL_ENV === "production" ? "prod" : "all"], + // v19: bumped with bench-unfiltered-v28 (bench vague 2, 081-085). + ["bench-filters-v27", process.env.VERCEL_ENV === "production" ? "prod" : "all"], { revalidate: 300, tags: ["benchmarks"] } ); diff --git a/src/types/benchmark.ts b/src/types/benchmark.ts index c213c3ef..47d29a00 100644 --- a/src/types/benchmark.ts +++ b/src/types/benchmark.ts @@ -215,7 +215,7 @@ export type Benchmark = { kind?: { value: string; label: string }[]; venue?: { value: string; label: string }[]; }; - category: "Aggregators" | "Bridges" | "Blockchains" | "Trading" | "Wallets" | "RPCs" | "NFT APIs" | "Explorers"; + category: "Aggregators" | "Bridges" | "Blockchains" | "Trading" | "Wallets" | "RPCs" | "NFT APIs" | "Explorers" | "RWA"; results: ProviderResult[]; /** Per-chain leader, computed only on the unfiltered ("All chains") view * when the spec declares `dimensions.chain`. Key = chain slug from the