Skip to content

feat(morpho-v2): group per-vault alerts into one Telegram message; sync KATANA V2 vaults - #328

Open
spalen0 wants to merge 2 commits into
feat/peg-monitoring-l3-eventsfrom
feat/morpho-v2-katana-vaults-and-grouped-alerts
Open

feat(morpho-v2): group per-vault alerts into one Telegram message; sync KATANA V2 vaults#328
spalen0 wants to merge 2 commits into
feat/peg-monitoring-l3-eventsfrom
feat/morpho-v2-katana-vaults-and-grouped-alerts

Conversation

@spalen0

@spalen0 spalen0 commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator

Summary

Two changes bundled because they unblock the same workflow.

A KATANA Gauntlet USDT increaseTimelock(setSendAssetsGate) landed without a follow-up "executed" alert because the vault wasn't in VAULTS_V2_BY_CHAIN for KATANA. And even for vaults we do track, a 22-call batch operation (e.g. the same InfiniFi LongTimelock upgrade that surfaced the Wavey Gist bug) would have spammed 22 separate "pending config" Telegram messages.

  1. Add KATANA V2 vaults the team already tracks in prod
    protocols/morpho/_shared.py: Gauntlet USDT, Steakhouse High Yield USDC, Steakhouse Prime USDC on KATANA. Addresses and risk levels mirror /srv/monitoring production config — the only place these were already tracked. Result: they get the same pending/executed/role/set-diff treatment as mainnet+BASE.

    Note these three are curated by Gauntlet and Steakhouse, not Yearn, so the "Yearn-curated" wording on VAULTS_V2_BY_CHAIN (and the matching docstrings in markets_v2.py) is corrected. The list is shared with markets_v2.py, so adding a row here also enrols the vault in market/allocation checks — that's intended, and now stated in the comment.

  2. Group per-vault alerts into one Telegram message
    protocols/morpho/governance_v2.py: every diff category (pending new, pending resolved, owner, curator, sentinels, allocators, adapters) now buffers into a per-vault _VaultDiff, then diff_and_alert flushes ONE Telegram message per vault with:

    • one header (V2 [name](url) on chain)
    • one severity (highest of the group: CRITICAL > HIGH > MEDIUM > LOW)
    • sections separated by ---

    Worst case (InfiniFi 22-call batch) goes from a 22-message Telegram burst to 2 numbered parts (see below).

Message length

Telegram caps a message at 4096 chars, and utils.telegram truncates past that (and drops Markdown with it), so naively concatenating would have lost the tail of exactly the batch this PR exists to fix — the 22-call case renders ~6.3k chars, i.e. 7 of 22 sections silently dropped.

_send_vault_alerts therefore packs sections into as many messages as needed, each under budget, with the header repeated and an (i/N) suffix:

messages: 2   lengths: [3783, 2649]   sections delivered: 22 / 22

Delivery vs. cache

Cache cursors are buffered in _VaultDiff and committed only after the send succeeds. Writing them during the diff pass would mark a change as alerted even when Telegram failed — main() logs the exception and moves on, so the alert would be lost permanently. A partial send (part 1 lands, part 2 fails) now repeats the whole group next run; duplicates beat a governance change nobody ever sees.

Before / after

Before (each diff category fires its own Telegram message):

⚠️ ⏳ V2 Gauntlet USDT on katana
📥 Submitted: increaseTimelock(setSendAssetsGate → 604800s)
⏰ Executable at: 2026-08-06 19:11:34 (6 days)
🔗 Tx: 0x603be77d…

ℹ️ ✅ V2 Steakhouse Prime USDC on katana
Pending operation IncreaseAbsoluteCap() was executed (was due 2026-07-30 20:32:41).

ℹ️ ✅ V2 Steakhouse Prime USDC on katana
Pending operation IncreaseAbsoluteCap() was executed (was due 2026-07-30 20:32:41).

After (one grouped message per vault):

🚨 V2 [Gauntlet USDT](https://app.morpho.org/katana/vault/0xaC59…) on katana

---

📥 Submitted: increaseTimelock(setSendAssetsGate → 604800s)
⏰ Executable at: 2026-08-06 19:11:34 (6 days)
🔗 Tx: [0x603be77d…](https://katanascan.com/tx/0x603be77d…)

---

🚨 👑 Owner changed: `0xold` → `0xnew`

---

🧩 adapters changed
  + `0xdd…`
  − `0xee…`

Tests

  • Existing test_morpho_v2_governance.py updated for the buffer-based API (_diff_* / _alert_* now take a _VaultDiff); the pending-label test drives the public diff_and_alert instead of the private helpers.
  • New test_multiple_alerts_for_one_vault_are_grouped_into_single_message: 3 pending + 1 owner + 3 set changes produce exactly 1 alert at HIGH severity, with the vault header once and all 3 pending Txs present.
  • New test_no_alerts_emits_no_message: vaults with no diffs stay silent (no spam).
  • New test_only_low_severity_changes_emit_low_severity_alert: pure allocator/pending-resolved diffs are LOW, not the HIGH of the previous "first-message-wins" style.
  • New test_oversized_group_is_split_into_numbered_parts_without_dropping_sections: the 22-call batch splits into >1 message, each ≤4096 chars, with all 22 sections and tx hashes present.
  • New test_cache_writes_are_deferred_until_the_send_succeeds: a raising send_alert leaves the cache untouched; a working one commits the cursors.

All 657 tests pass; ruff format and ruff check clean.

Note

Based on feat/peg-monitoring-l3-events (#307) — retarget to main once that lands.

spalen0 and others added 2 commits July 31, 2026 08:51
…nc KATANA V2 vaults

Two changes bundled because they unblock the same workflow: a KATANA
Gauntlet USDT increaseTimelock landed without a follow-up 'executed'
alert because the vault was never in our V2 list, and even for vaults
we DO track, a 22-call batch operation (like the same InfiniFi upgrade
that surfaced the Wavey Gist bug) would have spammed 22 separate
'pending config' Telegram messages.

1) protocols/morpho/_shared.py: add Gauntlet USDT, Steakhouse High
   Yield USDC, and Steakhouse Prime USDC to KATANA in VAULTS_V2_BY_CHAIN
   so they get the same pending/executed/role/set diff treatment as
   mainnet+BASE. Addresses and risk levels mirror /srv/monitoring prod
   config (the only place these were already tracked).

2) protocols/morpho/governance_v2.py: per-vault diff categories
   (pending new, pending resolved, owner, curator, sentinels,
   allocators, adapters) now buffer into a single list and flush as
   ONE grouped Telegram message per vault. One header ('V2 [name](url)
   on chain'), one severity (highest of the group), sections separated
   by '---'. With the InfiniFi batch as the worst case, this takes a
   22-message burst down to 1.

Tests:
- test_morpho_v2_governance updated for the new buffer-based API
  (_diff_pending / _alert_* now take an alerts list).
- New test_multiple_alerts_for_one_vault_are_grouped_into_single_message
  asserts 3 pending + 1 owner + 3 set changes produce exactly 1 alert
  at HIGH severity.
- New test_no_alerts_emits_no_message — vaults with no diffs stay
  silent.
- New test_only_low_severity_changes_emit_low_severity_alert — pure
  allocator diffs are LOW, not the HIGH of the previous
  'first-message-wins' style (which had no winner because each message
  was independent).

All 655 tests pass; ruff format/check clean.
Review follow-ups on the per-vault alert grouping:

* Split a grouped alert into "(i/N)" parts instead of letting
  utils.telegram truncate it. The 22-call batch that motivated the
  grouping renders ~6.3k chars, so the previous single message lost 7 of
  22 sections and dropped Markdown; it now sends as 2 parts with every
  section intact.
* Buffer cache writes in _VaultDiff and commit them only after the send
  succeeds. Writing during the diff pass marked changes as alerted even
  when delivery failed — main() logs and moves on, so the alert was lost
  for good.
* Replace the _SEVERITY_RANK dict with an ascending _SEVERITY_ORDER
  tuple, and drop the snapshot argument from the _alert_* helpers that
  no longer use it (the header moved out of the section bodies).
* Correct the "Yearn-curated" docstrings: VAULTS_V2_BY_CHAIN also holds
  Gauntlet/Steakhouse vaults, and a row there enrols the vault in
  markets_v2 checks as well as governance diffs.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@spalen0
spalen0 marked this pull request as ready for review July 31, 2026 14:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant