diff --git a/CHANGELOG.md b/CHANGELOG.md index b2dd4383..c6b30c72 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,37 @@ All notable changes to this project are documented in this file. Format follows For narrative release notes written for operators and product owners, see [RELEASE_NOTES.md](RELEASE_NOTES.md). +## [1.9.0] - 2026-07-20 + +Feature release making **Bedrock prompt-cache economics stable and measurable**. Three cache-busting defects in the model-call path are fixed (per-turn history mutation, nondeterministic skill ordering, single-cachePoint fragility), and a new observability layer makes every model call's cache behavior diagnosable: prefix fingerprints and a `cacheStatus` classification on each cost row, an admin Session Cost Anatomy drill-down page, CloudWatch EMF metrics, and a dashboard with alarms. Also fixes chat-input textarea sizing and points the deployed runtime's Word-document tools at the real user-files bucket. Requires a CDK deploy (new dashboard construct + one runtime env var); no data migration. + +### 🚀 Added + +- Prompt-cache observability per model call — `PrefixFingerprintHook` hashes the three cacheable prefix components (toolConfig, effective system prompt, message history) onto each cost row; write-time `cacheStatus` classification (`first_write | hit | miss_ttl_expired | miss_avoidable | uncached`) with `wastedUsd` for avoidable misses priced from the row's own pricing snapshot; session-row cache rollups (`totalCacheReadTokens`, `totalCacheWriteTokens`, `avoidableMissCount`, `wastedUsd`); `GET /admin/costs/sessions/{sessionId}/calls` admin endpoint; and per-call CloudWatch EMF metrics. Kill switch `PROMPT_CACHE_OBSERVABILITY_ENABLED=false` (default ON) (#697) +- Admin **Session Cost Anatomy** page at `/admin/costs/sessions/:id` — chronological per-call table with color-coded `cacheStatus` badges and prefix-fingerprint diffing that names which hash (tools/system/history) flipped versus the previous call, plus session-level cache summary and a session-id lookup form on the Cost Analytics dashboard (#700) + +### ⚡ Performance + +- Restored conversation history is now byte-stable between turns — tool-content truncation is driven by a persisted `truncation_anchor` in compaction state instead of a sliding window that re-mutated history every turn, which forced a full prompt-cache prefix re-write (~$2.50/MTok on a 35k–150k-token prefix) nearly every turn (#697) +- Skill records reach the `` system-prompt block in deterministic order (sorted at the repository, RBAC-union, and injection layers), removing a per-turn prompt-cache invalidator (#697) +- Model calls now carry 3 of Bedrock's max-4 cachePoints (toolConfig tail, system tail, last-user-message) so a message-level cache miss still reads the stable tools+system prefix from cache instead of re-writing everything; the added points are gated to models that support them (#697) + +### 🐛 Fixed + +- Chat input textarea is scrollable once content exceeds its max height, clamps its growth, and resets to its base height after sending (#696) +- The first cache write after a run of below-threshold (uncached) calls is classified `first_write`, not `miss_avoidable` — it no longer inflates the AvoidableMiss/WastedUsd metrics or the Session Cost Anatomy page (#701) +- Word-document tools on the deployed runtime save to the real user-files bucket — `S3_USER_FILES_BUCKET_NAME` is now set on the inference-api Runtime, where its absence made the tools fall back to a literal `user-files` bucket and fail with `AccessDenied` (#702) +- Word-document tools fail fast with a clear "storage is not configured" message when the bucket env var is unset, instead of surfacing a confusing S3 `PermanentRedirect`/`AccessDenied` mid-run (#706) + +### 🏗️ Infrastructure + +- `PromptCacheObservabilityConstruct` (`lib/constructs/observability/`) — CloudWatch dashboard over the `AgentCoreStack/PromptCache` EMF metrics (cache read/write tokens, cache-efficiency expression, AvoidableMiss, WastedUsd) with a Logs Insights widget grouped by `cacheStatus`, plus console-only alarms on AvoidableMiss and WastedUsd (stricter in prod; `NOT_BREACHING` on missing data so the kill switch stays quiet) (#699) +- `S3_USER_FILES_BUCKET_NAME` added to the inference-api AgentCore Runtime environment (#702) + +### 📚 Docs + +- CLAUDE.md — prompt-cache determinism/byte-stability contract, the fingerprint-based cache-miss debugging recipe, and a token-cost-effectiveness design tenet (#697, #698) + ## [1.8.0] - 2026-07-19 Feature release delivering **Skills v2** — skills are redesigned from tool-binding containers into pure, portable knowledge bundles that Agents load on demand, and for the first time **any signed-in user can author their own**. Skills ship enabled by default. Also completes the session-metadata static-sort-key migration (issue #175 Phases 2–3) with an operator backfill and a GSI-only read contraction, and consolidates the two artifact tool-catalog rows into a single "Artifacts" toggle. **Three backfill scripts must be run per environment** — see [RELEASE_NOTES.md](RELEASE_NOTES.md) deployment notes. No new AWS resources; a CDK deploy is needed only to set the `SKILLS_ENABLED` env var explicitly. diff --git a/README.md b/README.md index 5b9906bb..c014c9a7 100644 --- a/README.md +++ b/README.md @@ -8,7 +8,7 @@ **An open-source, production-ready Generative AI platform for institutions** *Built by Boise State University, designed for everyone.* -[![Release](https://img.shields.io/badge/Release-v1.8.0-6366f1?style=flat&logo=github&logoColor=white)](RELEASE_NOTES.md) +[![Release](https://img.shields.io/badge/Release-v1.9.0-6366f1?style=flat&logo=github&logoColor=white)](RELEASE_NOTES.md) [![Nightly](https://github.com/Boise-State-Development/agentcore-public-stack/actions/workflows/nightly.yml/badge.svg)](https://github.com/Boise-State-Development/agentcore-public-stack/actions/workflows/nightly.yml) ![Python](https://img.shields.io/badge/Python-3.13+-3776AB?style=flat&logo=python&logoColor=white) @@ -296,7 +296,7 @@ agentcore-public-stack/ See [RELEASE_NOTES.md](RELEASE_NOTES.md) for the full changelog, including new features, bug fixes, platform upgrades, and deployment notes for each release. -**Current release:** v1.8.0 +**Current release:** v1.9.0 --- diff --git a/RELEASE_NOTES.md b/RELEASE_NOTES.md index 5c9db597..dd55d368 100644 --- a/RELEASE_NOTES.md +++ b/RELEASE_NOTES.md @@ -1,3 +1,69 @@ +# Release Notes — v1.9.0 + +**Release Date:** July 20, 2026 +**Previous Release:** v1.8.0 (July 19, 2026) + +--- + +> 🏗️ **CDK deploy required this release** — a new CloudWatch dashboard construct and one new env var on the inference-api Runtime. No data migration, no backfills, no dependency changes. Standard order: `platform.yml` → `backend.yml` → `frontend-deploy.yml`. + +--- + +## Highlights + +v1.9.0 makes **Bedrock prompt-cache spend stable and measurable**. A production conversation audit found 75% of one session's cost was avoidable cache re-writes; this release fixes the three defects behind that waste — restored history that mutated every turn, skills injected in nondeterministic order, and a single fragile cachePoint — and adds the observability to prove it and catch regressions: every model call now records prefix fingerprints and a `cacheStatus` verdict, admins get a per-session **Cost Anatomy** drill-down page, and operators get a CloudWatch dashboard with alarms on avoidable waste. Smaller fixes: the chat input textarea scrolls and resets correctly, and Word-document saves work on the deployed runtime (the container wasn't told which S3 bucket to use). + +## Prompt-cache stability — stop paying for avoidable re-writes + +Bedrock prompt caching is exact-prefix-match: if any byte of the cached prefix changes between turns, the whole prefix re-writes at the $2.50/MTok cache-write premium instead of reading at the ~$0.30/MTok cache-read rate. On a typical 35k–150k-token session prefix, one silent cache-buster costs more per turn than most turns' actual work. Three were found and fixed (#697): + +- **Restored history is now byte-stable.** Tool-content truncation ran on every session restore behind a sliding protected-turns window, so each new turn re-mutated the turn that had just aged past the window — invalidating the cache nearly every turn. Truncation is now driven by a persisted `truncation_anchor` in the session's compaction state: it advances only when the compaction checkpoint advances (where the slice already pays a single re-write) or opportunistically when the cache TTL (default 300s, `AGENTCORE_MEMORY_COMPACTION_CACHE_TTL_SECONDS`) has lapsed since the previous turn and the cache entry is dead anyway — so pending truncations apply for free. +- **Skills inject in deterministic order.** Skill records reached the `` system-prompt block in whatever order DynamoDB `batch_get_item` and Python set iteration produced, changing the system prompt between turns of the same session. Ordering is now sorted at three layers: the skills repository, the RBAC grant-union resolver (which returned `list(set)` — order varies per process via hash randomization), and the injection point itself. +- **Three cachePoints instead of one.** The auto strategy places a single message-level cachePoint; when its lookup misses (one proven mode: a wide parallel tool fan-out pushes the previous checkpoint past Anthropic's ~20-block cache lookback), *nothing* was read and the entire prefix re-wrote. Requests now carry 3 of Bedrock's max-4 cachePoints — toolConfig tail, system-prompt tail, and the existing last-user-message point — so a message-level miss still reads the stable tools+system prefix from cache. The added points are gated on `ModelConfig.bedrock_cache_points_supported()` since non-Anthropic models reject them. + +## Prompt-cache observability — every model call explains its cache behavior + +Diagnosing the waste above originally took hours of manual forensics against raw DynamoDB cost rows. That whole class of investigation is now a column diff (#697, #700, #699, #701). + +### Backend + +- `PrefixFingerprintHook` (a Strands `BeforeModelCallEvent` hook) hashes the three cacheable prefix components per model call — toolConfig (order-sensitive canonical JSON), the effective system prompt captured *after* AgentSkills injection, and message history excluding the newest message — and the stream coordinator persists them on the turn's cost rows. When a cache miss happens, the hash that changed between consecutive calls names the cache-buster. +- Each cost row gets a write-time `cacheStatus` — `first_write`, `hit`, `miss_ttl_expired`, `miss_avoidable`, or `uncached` — derived against the session's previous cost row, plus `wastedUsd` for avoidable misses priced at the cache-write premium over cache-read from the row's own pricing snapshot. Turn rows now write sequentially so each call classifies against its true predecessor. A follow-up fix (#701) classifies the first write after a run of below-threshold calls as `first_write` rather than `miss_avoidable`, so short-prompt sessions don't inflate the waste metrics. +- Session rows carry rollups next to `totalCost` — `totalCacheReadTokens`, `totalCacheWriteTokens`, `avoidableMissCount`, `wastedUsd` — so lists and admin views get a cache-efficiency ratio without scanning cost rows. +- `GET /admin/costs/sessions/{sessionId}/calls` (admin-only) returns the chronological per-call rows with token splits, cost, `cacheStatus`, and fingerprints, plus a session-level cache summary. +- Everything derived is behind `PROMPT_CACHE_OBSERVABILITY_ENABLED` (default ON, `=false` to disable the hook, the classification's extra GSI read, and EMF emission). Raw cache read/write token rollups are unaffected — they're usage passthrough, not derived. + +### Frontend + +- New **Session Cost Anatomy** page at `/admin/costs/sessions/:id`: summary tiles (total cost, cache efficiency, avoidable misses, wasted USD, cache read/write tokens) over a chronological calls table with color-coded `cacheStatus` badges. Fingerprint diffing flags which hash — tools, system, or history — flipped versus the previous fingerprinted call, which is the diagnosis on any `miss_avoidable` row. Expandable rows show full hashes and message counts; a session-id lookup form on the Cost Analytics dashboard is the entry point. + +### Infrastructure + +- `PromptCacheObservabilityConstruct` (new `lib/constructs/observability/` area, composed into `PlatformStack`) builds a CloudWatch dashboard over the `AgentCoreStack/PromptCache` EMF namespace both APIs emit into: cache read/write token trends, a cache-efficiency MathExpression, AvoidableMiss and WastedUsd, and a Logs Insights widget grouped by `cacheStatus`. Console-only alarms on AvoidableMiss and WastedUsd Sums (stricter thresholds in prod, `NOT_BREACHING` on missing data so the kill switch keeps them quiet). Deliberately no SNS — alerting infra remains out of scope, matching kb-sync and scheduled runs. + +### Test Coverage + +~1,800 lines of new tests: fingerprint/classification unit tests (including the below-threshold regression), cachePoint position and budget assertions, forced-order skill-sorting regressions, CDK construct assertions, and Vitest specs for the anatomy page, diff util, and HTTP service. + +## 🐛 Bug fixes + +- **The chat input became unusable on long prompts.** The textarea carried `overflow-hidden` with unclamped height growth, so past its 200px cap the content could neither be seen nor scrolled — and after sending, the box stayed expanded. Growth is now clamped with scrolling enabled past the cap, and the input resets to its base height on submit (#696) +- **Word-document saves failed on the deployed runtime with `AccessDenied`.** The AgentCore Runtime env set the user-files *table* name but not `S3_USER_FILES_BUCKET_NAME`, so the Word tools fell back to a literal `user-files` bucket the role has no access to. The env var is now set on the Runtime (#702), and the tools fail fast with a clear "storage is not configured" message — before spending a Code Interpreter run — if the variable is ever missing again (#706) + +## 🏗️ Infrastructure + +- New CloudWatch dashboard + alarms construct (see spotlight above) — CloudWatch-console resources only, no SNS, no new IAM of note. +- `S3_USER_FILES_BUCKET_NAME` on the inference-api Runtime environment; the role's existing `UserFilesBucketAccess` grant already covers the bucket (#702) +- New env var `PROMPT_CACHE_OBSERVABILITY_ENABLED` on app-api and inference-api (default ON; set `=false` per environment to disable the observability layer — caching itself stays on). + +## 🚀 Deployment notes + +Standard order, and this release uses all three: **`platform.yml` first** (the dashboard construct and the Runtime env var are CDK changes; the runtime picks up its current image via SSM, so the infra deploy is safe on its own) → `backend.yml` → `frontend-deploy.yml`. No backfills, no data migration, no dependency changes. + +After deploy, the **PromptCache dashboard** appears in the CloudWatch console. Expect `first_write` rows at the start of sessions and after idle gaps — only `miss_avoidable` indicates regression. The observability layer is per-call metadata; if it ever needs to be silenced in an environment, set `PROMPT_CACHE_OBSERVABILITY_ENABLED=false` and redeploy that service — the alarms go quiet on missing data by design. + +--- + # Release Notes — v1.8.0 **Release Date:** July 19, 2026 diff --git a/VERSION b/VERSION index 27f9cd32..f8e233b2 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -1.8.0 +1.9.0 diff --git a/backend/pyproject.toml b/backend/pyproject.toml index 57f66059..5ccd017e 100644 --- a/backend/pyproject.toml +++ b/backend/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta" [project] name = "agentcore-stack" -version = "1.8.0" +version = "1.9.0" requires-python = ">=3.10" description = "Multi-agent conversational AI system with AWS Bedrock AgentCore" readme = "README.md" diff --git a/backend/uv.lock b/backend/uv.lock index 1c575a2f..48a5cada 100644 --- a/backend/uv.lock +++ b/backend/uv.lock @@ -12,7 +12,7 @@ resolution-markers = [ [[package]] name = "agentcore-stack" -version = "1.8.0" +version = "1.9.0" source = { editable = "." } dependencies = [ { name = "aiofiles" }, diff --git a/frontend/ai.client/package-lock.json b/frontend/ai.client/package-lock.json index 92fcfcfe..0e6b860e 100644 --- a/frontend/ai.client/package-lock.json +++ b/frontend/ai.client/package-lock.json @@ -1,12 +1,12 @@ { "name": "ai.client", - "version": "1.8.0", + "version": "1.9.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "ai.client", - "version": "1.8.0", + "version": "1.9.0", "dependencies": { "@angular/cdk": "21.2.14", "@angular/common": "21.2.17", diff --git a/frontend/ai.client/package.json b/frontend/ai.client/package.json index 359bc4d2..05141990 100644 --- a/frontend/ai.client/package.json +++ b/frontend/ai.client/package.json @@ -1,6 +1,6 @@ { "name": "ai.client", - "version": "1.8.0", + "version": "1.9.0", "scripts": { "ng": "ng", "start": "ng serve", diff --git a/infrastructure/package-lock.json b/infrastructure/package-lock.json index a5ac74c8..d9666af9 100644 --- a/infrastructure/package-lock.json +++ b/infrastructure/package-lock.json @@ -1,12 +1,12 @@ { "name": "infrastructure", - "version": "1.8.0", + "version": "1.9.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "infrastructure", - "version": "1.8.0", + "version": "1.9.0", "dependencies": { "aws-cdk-lib": "2.260.0", "constructs": "10.6.0" diff --git a/infrastructure/package.json b/infrastructure/package.json index 75781bdc..d1245228 100644 --- a/infrastructure/package.json +++ b/infrastructure/package.json @@ -1,6 +1,6 @@ { "name": "infrastructure", - "version": "1.8.0", + "version": "1.9.0", "bin": { "infrastructure": "bin/infrastructure.js" },