fix(observability): first cache write after below-threshold calls is first_write, not miss_avoidable#701
Merged
Conversation
…d calls as miss_avoidable When every prior call in a session was uncached (prompt below the model's minimum cacheable prefix, e.g. ~4096 tokens on Claude Haiku 4.5), the first call that crosses the threshold does cacheWrite>0/cacheRead=0 and was classified miss_avoidable — inflating the AvoidableMiss and WastedUsd EMF metrics and the admin Session Cost Anatomy page. Verified live in dev-ai session 9a1f25b2 (calls 1-5 uncached at 3.5-4k tokens, call 6 falsely flagged with write=4122/read=0). classify_cache_status now takes the previous call's cached-prefix token total: when the immediately preceding call had zero cache activity there was no entry to read from, so the write is classified first_write (the expected initial population) and excluded from waste pricing. Unknown (None) keeps the previous behavior. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The prompt-cache classifier (PR #697) produces a false positive: in a session where every prior call was
uncached(prompt below the model's minimum cacheable prefix — e.g. ~4096 tokens for Claude Haiku 4.5), the first call that crosses the threshold doescacheWrite>0 / cacheRead=0within the TTL window and gets classifiedmiss_avoidable. That inflates the AvoidableMiss and WastedUsd EMF metrics and the admin Session Cost Anatomy page.Verified live in dev-ai session
9a1f25b2-8ea4-487f-b0e7-80a60deff945: calls 1–5 uncached at 3.5–4k tokens, call 6 falsely flaggedmiss_avoidablewith write=4122/read=0, calls 7–8 clean hits.Fix
classify_cache_statusgains an optionalprevious_cached_prefix_tokensparameter (the previous call's cacheRead + cacheWrite total, which_derive_cache_observabilityalready computed for waste pricing). When the immediately preceding call had zero cache activity, there was no cache entry to read from — so the threshold-crossing write is classifiedfirst_write(the expected initial population) and excluded from waste pricing.Reusing
first_write(rather than adding a new status) keeps the EMF schema, dashboard, and admin anatomy page enums unchanged.None(unknown split) preserves the previous behavior.Tests
first_write; unknown previous prefix staysmiss_avoidable.first_write,wastedUsd == 0.0.🤖 Generated with Claude Code