Skip to content

Sync L2 hit never backfills L1; sync L2-hit stats stringify the whole value #164

Description

@27Bslash6

Summary

Two sync-vs-async asymmetries on the L2 hit path:

  1. No L1 backfill on sync L2 hit. The async wrapper re-warms L1 on every L2 hit; the sync wrapper does not. After L1 eviction/restart, sync decorators re-pay the full L2 read + deserialize on every call until the next miss-store.
  2. Stats stringify the whole object. size_bytes = len(str(cached_result[1]).encode('utf-8')) renders the entire deserialized value (e.g. str(df) on a whole DataFrame) just for a stats number — twice, once unconditionally.

Evidence

  • Backfill: only _l1_cache.put in the sync wrapper is the miss-store at decorators/wrapper.py:797; async backfills at :1015,:1096,:1122. Sync L2-hit block :700-757 has none.
  • Stringify: decorators/wrapper.py:726 (unconditional) and :739; sync L1 already uses len(l1_bytes) correctly at :669.

Impact

Throughput cliff for sync large-object reads + a payload-proportional throwaway allocation/CPU on every sync L2 hit.

Fix

Backfill L1 on sync L2 hits (mirror async); reuse the known serialized length for stats instead of str(value).

Metadata

Metadata

Assignees

No one assigned

    Labels

    performancePerformance improvementspythonPython library

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions