feat: add memoize middleware, drop parking_lot dep

[bobrykov]

No description provided.

[coderabbitai]

Warning

Review limit reached

@bobrykov, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 15 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 38c5881f-b9ce-49e4-aa26-08f7786a7169

📥 Commits

Reviewing files that changed from the base of the PR and between 1844b2a and 832a827.

📒 Files selected for processing (4)

• CHANGELOG.md
• Cargo.toml
• src/engine/bare.rs
• src/middleware/memoize.rs

📝 Walkthrough

Walkthrough

Adds MemoizingMiddleware with canonical-input caching, TTL expiry, path-aware invalidation, and cached-result markers. It also removes parking_lot, introduces centralized poisoned-lock recovery, and migrates runtime and test mutex usage to std::sync.

Changes

Tool-result memoization

Layer / File(s)	Summary
Memoization contract and exports src/middleware/memoize.rs, src/middleware.rs, CHANGELOG.md	Adds PathExtractor, MemoizingMiddleware, cache state, public exports, and unreleased feature documentation.
Cache lookup, storage, and invalidation src/middleware/memoize.rs, src/tool.rs	Caches successful memoized results by canonical tool input, applies TTL expiry, invalidates overlapping write paths, and marks cached output with zero duration.
Memoization behavior tests src/middleware/memoize.rs	Covers hits, misses, canonical JSON keys, TTL, invalidation, pass-through, error handling, and text or multipart markers.

Standard mutex migration

Layer / File(s)	Summary
Centralized poison recovery Cargo.toml, src/error.rs, CHANGELOG.md	Removes parking_lot and adds tested recover_guard handling for poisoned locks.
Runtime lock recovery migration src/memory/builtin.rs, src/provider/*.rs, src/tool/health.rs, src/tool/shield.rs	Migrates storage, provider model state, health registries, circuit breakers, and shield history to standard mutex recovery.
Test and example lock migration src/engine/bare.rs, src/engine/bare/dispatch.rs, src/observer.rs, src/testing.rs	Updates examples, mocks, observers, streaming utilities, and test state to use std::sync locks with recover_guard.

Sequence Diagram(s)

sequenceDiagram
  participant Caller
  participant MemoizingMiddleware
  participant ToolPipeline
  Caller->>MemoizingMiddleware: Dispatch tool call
  MemoizingMiddleware->>MemoizingMiddleware: Build canonical cache key
  MemoizingMiddleware->>ToolPipeline: Forward cache miss
  ToolPipeline-->>MemoizingMiddleware: Successful ToolDispatchResult
  MemoizingMiddleware-->>Caller: Cache result or return cached marker

Loading

Possibly related PRs

• dch-labs/loopctl#31: Introduced the tool health and circuit-breaker code whose mutex handling is migrated here.
• dch-labs/loopctl#33: Introduced the shield implementation whose history locking is updated here.
• dch-labs/loopctl#34: Defined ToolDispatchResult, whose duration and output semantics are used by memoization.

🚥 Pre-merge checks | ✅ 4

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately summarizes the main changes: adding memoize middleware and removing the parking_lot dependency.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 50.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/memoize-middleware

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 4

🧹 Nitpick comments (1)

src/middleware/memoize.rs (1)

280-290: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick win

Failed writes still trigger path invalidation.

is_write invalidation (line 282-284) runs regardless of result.is_error — a permission-denied or failed write still evicts cache entries for paths it never actually touched. The module docs justify over-invalidation as "safe" (never causes staleness), so this isn't incorrect, just an avoidable cache-thrashing case (e.g. repeated denied writes to the same file will keep evicting an otherwise-valid Read cache entry for no reason).

♻️ Optional tightening

             if is_write {
-                let write_paths = self.path_extractor.paths(&ctx.tool_name, &ctx.input);
-                invalidate_paths(&self.cache, &write_paths);
+                if !result.is_error {
+                    let write_paths = self.path_extractor.paths(&ctx.tool_name, &ctx.input);
+                    invalidate_paths(&self.cache, &write_paths);
+                }
             } else if is_memoized && !result.is_error {

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/middleware/memoize.rs` around lines 280 - 290, Update the write-handling
branch after next.dispatch in the memoization flow to invalidate paths only when
is_write is true and result.is_error is false. Preserve the existing path
extraction and invalidation behavior for successful writes, and leave
memoized-read insertion unchanged.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/engine/bare.rs`:
- Line 691: Update the set_text_streamer example’s documented buffer-locking
expression to remove the crate-private crate::error::recover_guard helper and
use a public Mutex lock pattern such as unwrap or explicit poisoned-lock
recovery, keeping the example copy-pasteable for external users.
- Line 691: Update the `set_text_streamer` documentation example to replace the
crate-private `crate::error::recover_guard` call with the public mutex guard
recovery pattern using `buf.lock().unwrap_or_else(|e| e.into_inner())`, while
preserving the existing `push_str(delta)` behavior.

In `@src/middleware/memoize.rs`:
- Around line 63-71: Correct the “Generosity” documentation for PathExtractor to
remove the claim that returning an empty string forces intersection with any
write. State that paths are matched by exact verbatim membership, so
implementations must return every concrete or otherwise shared path that can be
affected, consistent with the existing no-prefix/no-wildcard matching guidance.
- Around line 269-278: Remove the let-chain syntax from the memoized lookup
conditions in the async closure and the corresponding block around the later
cache lookup, rewriting both to equivalent nested conditionals compatible with
Rust 1.85. Preserve the existing is_memoized, key, and lookup_fresh checks and
cached-result handling unchanged.

---

Nitpick comments:
In `@src/middleware/memoize.rs`:
- Around line 280-290: Update the write-handling branch after next.dispatch in
the memoization flow to invalidate paths only when is_write is true and
result.is_error is false. Preserve the existing path extraction and invalidation
behavior for successful writes, and leave memoized-read insertion unchanged.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 2ce25019-fe2e-4749-b54b-f7c5e4132d98

📥 Commits

Reviewing files that changed from the base of the PR and between 20a83a4 and 1844b2a.

📒 Files selected for processing (16)

• CHANGELOG.md
• Cargo.toml
• src/engine/bare.rs
• src/engine/bare/dispatch.rs
• src/error.rs
• src/memory/builtin.rs
• src/middleware.rs
• src/middleware/memoize.rs
• src/observer.rs
• src/provider/anthropic.rs
• src/provider/gemini.rs
• src/provider/openai.rs
• src/testing.rs
• src/tool.rs
• src/tool/health.rs
• src/tool/shield.rs

💤 Files with no reviewable changes (1)

• Cargo.toml