feat(3jane): monitor Accountable Proof of Solvency - #329
Draft
spalen0 wants to merge 2 commits into
Draft
Conversation
Adds a client for Accountable's public Proof of Solvency feed and wires 3Jane's dashboard (DFID 100000026) into the existing hourly monitor. The collateral ratio is recomputed from total_reserves / total_supply at full precision rather than read from the API's `collateralization` field, which is rounded to six decimals -- near the alert boundary a true ratio of 0.9999996 would present as 1.0 and pass a `< 1.00` test. The reported field is kept as a consistency cross-check. `net` and `collateralization` are defined against liabilities, which equal total_supply only when total_supply.fx == 1, so the client asserts that rather than assuming it. Staleness budgets are keyed by source type: the reserves side leans on manually uploaded document reports that routinely run past their declared cadence, so those get a 7-day grace while on-chain sources get 2 hours. A single global threshold would fire on day one. Alerts fire on band transitions (OK/HIGH/CRITICAL) rather than on every worsening tick, since the live margin sits a few basis points above 100%. CRITICAL additionally requires two consecutive sub-100% runs; a single reading is reported as HIGH so it stays visible without escalating on what is more likely a stale document-report refresh. Alerts use protocol key `3jane-accountable` with channel `3jane`: they reach the normal Telegram channel at full severity, but the key is absent from DISPATCHABLE_PROTOCOLS so a CRITICAL cannot trigger the emergency cap-zeroing webhook while the feed's noise profile is unproven. Refs #327 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #327.
Adds monitoring for Accountable's Proof of Solvency feed and wires 3Jane's dashboard (
DFID=100000026) into the existing hourly3janemonitor.Data access
No scraping and no Weavechain client needed.
GET https://accountable.3jane.xyz/dashboardserves the full report as public JSON — no auth, no headers. The SPA only falls back to the encrypted Weavechain plugin path if that endpoint fails, so the plain JSON path is the integration.Three things that shaped the implementation
The reported ratio is rounded, so it isn't used for alerting.
collateralizationcomes back rounded to six decimals. Near the boundary that's a missed-insolvency path: a true0.9999996presents as1.0and passes a< 1.00test. The monitor recomputes fromtotal_reserves / total_supplyat full precision and keeps the reported field as a consistency cross-check (tolerance ≥1e-6, since the server's own rounding sets the floor).netandcollateralizationare defined against liabilities, which equaltotal_supplyonly whentotal_supply.fx == 1. The client asserts that rather than assuming it, so a non-USD-pegged feed fails loudly instead of silently comparing against the wrong denominator.Freshness is per source type. A fresh aggregate timestamp doesn't prove the inputs are fresh, and that matters here —
reserves_splitis essentially all "Morpho Credit", the bulk of which is off-chain loan receivables priced by manually uploaded document reports. Live, two of the four sources are already past their declared cadence:So budgets are keyed by source
type—Document Reportgets a 7-day grace on top of its cadence, everything else gets 2 hours. A single global threshold would have fired on day one. Sources with an unrecognised cadence are skipped rather than flagged, so a schema addition on Accountable's side can't spuriously page us.Band transitions, not drop-based dedupe. The live margin sits a few basis points above 100%, so the existing
should_alert_value_drop()helper would alert on essentially every run. Alerts fire onOK → HIGH → CRITICALtransitions; recovering re-arms the bands above without alerting. CRITICAL additionally requires two consecutive sub-100% runs — a single reading is reported as HIGH so it stays visible without escalating on what is more likely a stale document-report refresh.No emergency dispatch in v1
Alerts use protocol key
3jane-accountablewith channel3jane. They reach the normal 3Jane Telegram channel at full severity, but the key is deliberately absent fromutils.dispatch.DISPATCHABLE_PROTOCOLS, so a CRITICAL here cannot fire the emergency cap-zeroing webhook.Alertalready separates dispatch keying from Telegram routing, so this needed no changes to shared alert infrastructure. Intentional for v1 — the margin is basis points and the feed's noise profile is unproven. There's a test asserting this stays true.Heads-up: this alerts on first run
Live collateralization is ~100.03%, which is below the 105% HIGH threshold. Deploying this will fire one HIGH alert on the first run, then go quiet (band dedupe). That's a real signal about a genuinely thin margin, not a false positive — but worth knowing before merge. The 105% threshold is the value flagged as TBD in #327 and is easy to adjust.
Changes
utils/accountable.py— DFID-keyed client: fetch, strict validation, consistency cross-checks, per-source-type freshness, typedOK/STALE/UNAVAILABLEoutcomes. Never returns a silentNone.protocols/3jane/main.py— Accountable check in its own failure boundary, run before the onchain reads so it still executes when RPC is degraded.protocols/3jane/README.md,monitoring.yaml— docs and metadata.tests/test_accountable.py,tests/test_3jane.py— 48 new tests against a recorded live fixture.Verification
740 passed, 6 skipped— full suiteruff check .andruff format --check .cleanmypy utils/accountable.pyclean (the one reported error is pre-existing inutils/http_client.py)OK, classifies the live 100.029% as HIGH, routes to3jane-accountable/3jane, and the second run is silentFollow-ups (not in scope)
Questions for Accountable, none blocking: whether
/dashboardis a supported endpoint with versioning guarantees, whether the response can carry a feed identifier so a misrouted response can be rejected, intended grace periods for the document-report inputs, and the supported verification procedure for the HTTP signatures / TEE attestations. Signature verification is deferred — v1 relies on TLS plus the consistency checks.🤖 Generated with Claude Code