feat(kvstore): decay eviction freshness and split hit half-life by role#550
Open
unsaltedbutter-ai wants to merge 1 commit into
Open
feat(kvstore): decay eviction freshness and split hit half-life by role#550unsaltedbutter-ai wants to merge 1 commit into
unsaltedbutter-ai wants to merge 1 commit into
Conversation
Replace the constant +1 score floor with a freshness term that decays with idle time, so a never-reused checkpoint loses its density-driven score within about a day instead of persisting on raw size forever. Decay the hit count on a per-role clock: cold session prefixes, reused across independent sessions, use a 21-day half-life while conversation tips keep the 6h half-life. Classify only cold as an entry anchor, so evict and shutdown tips no longer receive the durable-anchor bonus.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
KV disk cache evicts small, frequently-reused prefixes before large single-use checkpoints
Under budget-pressure eviction,
ds4_kvstore_entry_eviction_scoredrops small cold prefixes reused across many sessions and keeps large multi-turn checkpoints that were hit 0-1 times and will not be resumed.Seen after lowering
DS4_KV_DISK_SPACE_MBfrom 256 GiB to 32 GiB. The boot eviction removed every reused prefix and kept ~25 one-shot conversation snapshots:evicted disk-cache-full tokens=533 hits=16 size=29.86 MiB (reused prefix)
evicted disk-cache-full tokens=564 hits=16 size=30.27 MiB
kept ... tokens=91920 hits=1 size=1229.8 MiB (one-shot chat)
Two effects combine:
Net: at a tight budget the cache keeps low-value large one-shot conversations even if quite old and evicts often-but-infrequently reused small prefixes.
Two changes to
ds4_kvstore_entry_eviction_score:Effect: a large one-shot conversation stays protected inside its resume window, then falls below small, frequently-reused prefixes once idle. The manual pin (future last_used) and the superseded-prefix penalty are unchanged.
Replaying an actual purge with the new scoring keeps the reused prefixes that were evicted at every budget and idle-age tested. Half-lives are #defines; the values are logged in the KV disk cache banner.
Related to #444
Orthogonal to #448