From eb9d614d007827abd342cc95bdb769ad3966dc49 Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Thu, 2 Jul 2026 15:32:17 -0700 Subject: [PATCH 01/30] Add EdgeZero full-migration umbrella design spec Defines the end-state, current-state gap analysis, and an ordered set of phases (stores, config injection, secret externalization, extractors, legacy-path removal) for moving Trusted Server completely onto EdgeZero. Phase 0 (State extractor + nested #[secret]) is an upstream edgezero prerequisite tracked separately. --- ...26-07-02-edgezero-full-migration-design.md | 212 ++++++++++++++++++ 1 file changed, 212 insertions(+) create mode 100644 docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md diff --git a/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md b/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md new file mode 100644 index 00000000..a5bc3137 --- /dev/null +++ b/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md @@ -0,0 +1,212 @@ +# Trusted Server → EdgeZero — Full Migration (umbrella design) + +- **Status:** Draft for review +- **Date:** 2026-07-02 +- **Scope:** Move trusted-server **completely** onto EdgeZero primitives: config push, KV, secret store, config injection without an embedded `trusted-server.toml`, extractor-based handlers, and deletion of every pre-EdgeZero workaround. +- **Shape:** Umbrella roadmap. Defines the end-state, the current-state gap, and an ordered set of phases with dependencies. **Each phase gets its own implementation plan** (`writing-plans`) before code is written. +- **Companion spec:** Phase 0 (`State` extractor + nested `#[secret]`) is an **edgezero-repo** change, specified separately (`…-state-and-nested-secrets-design.md`) and tracked via its own edgezero PR. This umbrella depends on it but does not re-specify it. + +--- + +## 1. End-state + +trusted-server is a fully EdgeZero-native app: adapter binaries call `run_app::`; core is platform-neutral; config, KV, and secrets flow exclusively through EdgeZero's `StoreRegistry`; app config is a signed blob published by `ts config push` and read back typed at request time with secrets resolved from the secret store; handlers are `#[action]` functions taking `FromRequest` extractors; and no Fastly-specific or pre-EdgeZero shim remains in core or the adapters. + +Concretely, at the end of this migration: + +- **No `include_str!` of any `*.toml` config** in any adapter. All four adapters load app config from the EdgeZero config store. +- **No app-level secrets embedded in the pushed config blob.** Secrets live in the EdgeZero secret store; the blob carries only key names, resolved at request time. +- **No bespoke `PlatformConfigStore` / `PlatformSecretStore` / `RuntimeServices`.** Core and adapters use EdgeZero `ConfigStore` / `SecretStore` / `KvStore` via `StoreRegistry`. +- **No `FastlyManagementApiClient`, no `settings_data.rs` chunk resolver, no `config`-crate env overlay, no `Redacted`.** +- **Core handlers are extractor-based**; the per-adapter handler shims are gone. +- **The legacy Fastly `route_request` path, `compat.rs`, and the `edgezero_enabled` / `edgezero_rollout_pct` flags are deleted** (final phase, gated on 100% rollout). + +--- + +## 2. Current-state gap analysis + +Verified across `trusted-server-core`, the four adapters, `trusted-server-cli`, and the pinned `edgezero` dependency. + +| Concern | Today | Gap to close | +|---|---|---| +| **KV** | ✅ 100% on EdgeZero (`KvStore`/`KvHandle`, re-exported as `PlatformKvStore`) | None (baseline for the pattern) | +| **Routing** | ✅ All 4 adapters route through EdgeZero `RouterService` + `Hooks` | None structurally; handler authoring changes in Phase 4 | +| **Core off `fastly::` types** | ✅ Enforced by `migration_guards.rs` | Keep the guard; extend coverage as adapters shrink | +| **Config load** | ⚠️ Fastly + Axum load the blob from the config store; **Cloudflare + Spin `include_str!` `trusted-server.example.toml`** | Phase 2 | +| **Config injection** | ⚠️ `TrustedServerAppConfig` wraps `Settings`, `SECRET_FIELDS = &[]` → secrets inline in blob; `#[derive(AppConfig)]`/`#[secret]` unused | Phases 2–3 | +| **Config / Secret stores** | ❌ Core uses bespoke `PlatformConfigStore`/`PlatformSecretStore` + `RuntimeServices`; 4× per-adapter `platform.rs` impls; `FastlyManagementApiClient` for writes | Phase 1 | +| **Fastly config chunking** | ❌ `settings_data.rs` re-implements EdgeZero's `chunked_config.rs` verbatim in core | Phase 1 | +| **Env overlay** | ❌ `from_toml_and_env` + `TRUSTED_SERVER__*` via the `config` crate (test-only, but keeps the dep) | Phase 2 | +| **Handlers → extractors** | ❌ Hand-written `Fn(RequestContext)` shims calling `(&Settings, &RuntimeServices, Request)`; `#[action]`/`FromRequest` unused; **no `State` extractor exists upstream** | Phase 0 (upstream) → Phase 4 | +| **Legacy Fastly path** | ❌ `legacy_main`/`route_request` + `compat.rs` + rollout flags live (marked "TODO delete after Phase 5 cutover — #495") | Phase 5 | + +**Key architectural constraints discovered:** + +1. **No `State`/`Extension` extractor in EdgeZero.** trusted-server threads `Arc` (`Settings`, `AuctionOrchestrator`, `IntegrationRegistry`) via closures. Extractor migration needs an upstream `State` → **Phase 0**. +2. **`AppConfig` re-parses + verifies + secret-walks the whole blob every request** — too costly for `Settings`. Decision: keep loading `Settings` once at startup into `Arc`, exposed via `State`, rather than the per-request `AppConfig` extractor (see §4, Decision D1). +3. **Full secret externalization needs nested/array `#[secret]`** because `Settings` is deeply nested → **Phase 0** (edgezero derive change). +4. **Integration proxies are a second, nested `matchit` router** with their own `IntegrationProxy::handle(&Settings, &RuntimeServices, req)` convention — orthogonal to the core route handlers (see §4, Decision D2). + +--- + +## 3. Phase map (ordered, foundation-first) + +Each phase leaves **all four adapters building and green**. Dependencies are explicit; within a phase, work can parallelize. + +``` +Phase 0 (edgezero, external) ── State extractor ─────────────┐ + └─ nested/array #[secret] ───┐ │ + v v +Phase 1 (stores) ──> Phase 2 (config) ──> Phase 3 (secrets) Phase 4 (extractors) + │ + Phase 5 (legacy removal, gated 100% rollout) +``` + +- **Phase 1** depends on nothing upstream (EdgeZero store APIs already exist). +- **Phase 2** depends on Phase 1 (config store must be EdgeZero-native first). +- **Phase 3** depends on Phase 2 **and** Phase 0's nested `#[secret]`. +- **Phase 4** depends on Phase 0's `State`; it can run in parallel with 1–3 once Phase 0 lands, but is cleaner after Phase 1 (so handlers pull EdgeZero stores, not `RuntimeServices`). +- **Phase 5** is last and **gated on the edgezero rollout reaching 100%** (issue #495). + +--- + +## 4. Cross-cutting design decisions + +**D1 — Config caching, not per-request extraction.** Keep the load-once model: at startup each adapter reads the blob from the config store, verifies the envelope, resolves secrets, validates, and stores `Arc` in `AppState`. Handlers read it via `State` (Phase 4). Rationale: `AppConfig`'s per-request re-parse/verify/secret-walk is prohibitive for a struct this large. Trade-off: config changes require a new deploy/boot to take effect (already true today). *This diverges deliberately from the stock `AppConfig` extractor; documented as such.* + +**D2 — Integration proxy router stays put (for now).** Phase 4 migrates the **named/core routes** to extractors. The integration registry's nested `matchit` dispatch and `IntegrationProxy::handle` signature are internal, working, and orthogonal; migrating them is a **follow-up** (Phase 4b, optional), not silently dropped. Called out so the extractor migration isn't mistaken for "all handlers." + +**D3 — Secret resolution happens at startup, not per request.** With D1, the startup config load resolves `#[secret]` fields against the secret store once (Phase 3). Adapters must therefore have a secret-store handle available at boot, not only per request. Fastly/Axum already open stores eagerly; Cloudflare/Spin resolve from bindings — confirm boot-time access in Phase 3 scoping. + +**D4 — One typed `Settings` as the AppConfig root.** Replace `TrustedServerAppConfig` (wrapper with empty `SECRET_FIELDS`) by deriving `AppConfig` directly on `Settings`, with `#[secret]` on the real secret fields (Phase 3). Removes a transitional indirection. + +--- + +## 5. Phases + +### Phase 0 — EdgeZero prerequisites (external, edgezero repo) + +**Owner:** edgezero. **Tracked by:** its own spec + PR (link to be added). +**Delivers:** (A) `State` extractor + `RouterBuilder::with_state`; (B) nested/array `#[secret]` in `#[derive(AppConfig)]` + path-aware `secret_walk`. +**Blocks:** Phase 3 (B), Phase 4 (A). **This umbrella consumes it as a versioned dependency** — bump the pinned `edgezero` rev once merged. + +--- + +### Phase 1 — Stores onto EdgeZero `StoreRegistry` + +**Goal:** delete trusted-server's bespoke config/secret store layer; route all store access through EdgeZero `ConfigStore` / `SecretStore` / `StoreRegistry` (KV is already there). + +**Changes:** +- Replace `PlatformConfigStore` / `PlatformSecretStore` (`platform/traits.rs`, `types.rs`) and the `RuntimeServices` config/secret fields with EdgeZero `ConfigStoreHandle` / `BoundSecretStore` resolved from the per-request registries (`ConfigRegistry` / `SecretRegistry`), matching how KV already works. +- Migrate core secret consumers to `secrets.named(id)?.require_str(key)` / config consumers to the config binding: `proxy.rs` (S3), `request_signing/{signing,rotation}.rs`, `integrations/datadome/{protection,protection_scope}.rs`. +- Delete the 4× per-adapter `platform.rs` config/secret store impls (`FastlyPlatformConfigStore`, `AxumPlatformConfigStore`, `NoopConfigStore`, `Cloudflare…`, and secret equivalents); adapters instead build `ConfigRegistry`/`SecretRegistry` via `dispatch_with_registries` from `[stores.*]` metadata. +- Delete `FastlyManagementApiClient` (`management_api.rs`) — store writes/provisioning move to the EdgeZero CLI provision path. +- Delete `settings_data.rs`'s `FastlyChunkPointer` resolver — EdgeZero's `FastlyConfigStore` resolves chunks transparently. `get_settings_from_config_store` collapses to `ConfigStore::get` + `settings_from_config_blob`. + +**Deletions:** `management_api.rs`, `settings_data.rs` chunk resolver, `platform/traits.rs` config/secret traits, 4× `platform.rs` config/secret impls. +**Keeps:** `RuntimeServices` as a shrinking bundle for the still-explicit-arg handlers (removed in Phase 4); `StoreName`/`StoreId` only where the CLI provisioning still needs the management-id split (revisit). +**Acceptance:** all adapters build; `cargo test-fastly/-axum/-cloudflare/-spin` green; secret/config reads exercised in tests go through EdgeZero registries; parity test passes. + +--- + +### Phase 2 — Finish config injection (no embedded `trusted-server.toml`) + +**Goal:** every adapter loads app config from the EdgeZero config store; kill compile-time config baking and the legacy env overlay. + +**Changes:** +- Derive `AppConfig` on the config root (interim: still `TrustedServerAppConfig` until Phase 3 collapses it onto `Settings`) so all adapters use the same store-load path. +- **Cloudflare** (`adapter-cloudflare/src/app.rs`) and **Spin** (`adapter-spin/src/app.rs`): replace `Settings::from_toml(include_str!(".../trusted-server.example.toml"))` with `get_settings_from_config_store(...)` (now the EdgeZero `ConfigStore` path from Phase 1). Seed each platform's config store (`wrangler.toml` / `runtime-config.toml` / `fastly.toml` local blocks) with the pushed blob. +- Delete `Settings::from_toml_and_env`, `ENVIRONMENT_VARIABLE_PREFIX/SEPARATOR`, and the `config` **dev-dependency**. Any remaining env overlay uses EdgeZero's `EDGEZERO__*` / AppConfig `__…` layers. + +**Deletions:** both `include_str!` config paths, `from_toml_and_env`, `config` crate dep. +**Acceptance:** Cloudflare + Spin serve with store-loaded config (no baked TOML); `ts config push` blob is the single source on all four adapters; tests green. + +--- + +### Phase 3 — Secret externalization (full) + +**Goal:** no app-level secret is stored inside the config blob; secrets live in the EdgeZero secret store and resolve at startup (D3). + +**Depends on:** Phase 0 (B) nested `#[secret]`, Phase 2. + +**Changes:** +- Collapse `TrustedServerAppConfig` onto `Settings` (D4): `#[derive(AppConfig)]` on `Settings`, `#[secret]` / `#[secret(store_ref)]` on the real secret fields (S3 keys, request-signing key refs, DataDome server-side key, integration API keys, etc.), including the **nested** ones enabled by Phase 0. +- Audit `Settings` for the secret inventory **before implementation** — this settles Phase 0's open question B-1 (are any secrets inside arrays?). Feed the answer back to the edgezero PR. +- Delete `Redacted` and its manual redaction handling; `#[secret]` + the secret store replace it. +- Operator migration: `ts` provisions secrets into the secret store (via EdgeZero provision), and a migration guide moves existing inline secrets out of `trusted-server.toml`. `reject_placeholder_secrets` becomes a check on the resolved values at boot. +- Startup load resolves `#[secret]` fields against the secret store (D1/D3), then validates. + +**Deletions:** inline secrets in the blob, `Redacted`, `SECRET_FIELDS = &[]` wrapper. +**Acceptance:** pushed blob contains only secret **key names**; boot resolves them; a config with a nested secret validates and serves; operator migration guide published; tests green. + +--- + +### Phase 4 — Handlers → extractors + +**Goal:** core route handlers become `#[action]` functions taking `FromRequest` extractors; per-adapter handler shims deleted. + +**Depends on:** Phase 0 (A) `State`; cleaner after Phase 1. + +**Changes:** +- Introduce `State>` (or narrower `State>` / `State>` / `State>`) wired via `RouterBuilder::with_state` in each adapter's `Hooks::routes()`. +- Rewrite core `handle_*` (`proxy.rs`, `publisher.rs`, `auction/endpoints.rs`, `request_signing/endpoints.rs`, `ec/*.rs`) from `(&Settings, &RuntimeServices, Request)` to `#[action]` signatures using `State<…>`, `Json`/`Query`/`Path`/`Headers`/`Host`, and the store extractors (`Kv`, `Secrets`, `Config`). +- Delete the per-adapter shims (`execute_handler`/`execute_named`/`named_route_handler` + `NamedRouteHandler` enums) and shrink/retire `RuntimeServices` (its store fields already gone in Phase 1; remaining bundle folds into `State` + extractors). +- **EC lifecycle & pre-route filters** (`build_ec_request_state`, `run_pre_route_filters`, `attach_dispatch_extensions`, `FinalizeResponseMiddleware`) are cross-cutting — keep them as **middleware**, not per-arg extractors. +- **Phase 4b (optional follow-up, D2):** migrate the integration proxy nested router / `IntegrationProxy::handle` onto `RouterService` + extractors. Deferred by default. + +**Deletions:** per-adapter handler shims, `NamedRouteHandler` enums, `RuntimeServices` (final form). +**Acceptance:** all named routes served via `#[action]` handlers on all adapters; middleware carries EC lifecycle; parity test green. + +--- + +### Phase 5 — Delete the legacy Fastly path (gated on 100% rollout) + +**Goal:** remove the pre-EdgeZero Fastly entry path once the EdgeZero rollout is complete. + +**Gate:** edgezero rollout at 100% (issue #495). Do not start until confirmed. + +**Changes:** +- Delete `legacy_main` / `route_request` (`adapter-fastly/src/main.rs`), `compat.rs` (fastly↔http shim), and the flag machinery (`edgezero_enabled`, `edgezero_rollout_pct`, `select_edgezero_entrypoint`, `should_route_to_edgezero`, IP-bucket hashing). +- `main()` calls the EdgeZero path unconditionally (`run_app::` shape). +- Retire the `trusted_server_config` rollout-flag reads. + +**Deletions:** `legacy_main`, `route_request`, `compat.rs`, rollout flags. +**Acceptance:** Fastly adapter has a single EdgeZero entry path; no rollout flags; full CI gate green; production traffic unaffected (already 100% on EdgeZero by gate definition). + +--- + +## 6. Cruft deletion ledger (rolled into phases) + +| Item | File(s) | Phase | Replaced by | +|---|---|---|---| +| Fastly chunk-pointer resolver | `core/src/settings_data.rs` | 1 | EdgeZero `FastlyConfigStore` + `chunked_config.rs` | +| Bespoke config/secret store traits | `core/src/platform/{traits,types,mod}.rs` | 1 | EdgeZero `ConfigStore`/`SecretStore`/`StoreRegistry` | +| 4× per-adapter store impls | `adapter-*/src/platform.rs` | 1 | per-adapter EdgeZero store impls | +| Fastly management REST client | `adapter-fastly/src/management_api.rs` | 1 | EdgeZero CLI provision | +| `include_str!` config baking | `adapter-{cloudflare,spin}/src/app.rs` | 2 | store-loaded config | +| Legacy env overlay + `config` dep | `core/src/settings.rs` (`from_toml_and_env`, `ENVIRONMENT_VARIABLE_*`) | 2 | `EDGEZERO__*` / AppConfig env layers | +| AppConfig wrapper w/ empty `SECRET_FIELDS` | `core/src/config.rs` | 3 | `#[derive(AppConfig)]` on `Settings` | +| `Redacted` | `core/src/redacted.rs` | 3 | `#[secret]` + secret store | +| Per-adapter handler shims | `adapter-*/src/app.rs` | 4 | `#[action]` + extractors | +| Legacy Fastly path + flags + compat | `adapter-fastly/src/{main.rs,compat.rs}` | 5 | single EdgeZero entry path | + +**Explicitly NOT cruft (do not remove):** `migration_guards.rs` (intentional `fastly::` ban test), `s3_sigv4.rs` (AWS-domain canonical/hashing), `platform/image_optimizer.rs` (no EdgeZero equivalent yet), EC KV CAS wrapper (`ec/kv*.rs` — needs EdgeZero generation-CAS parity first; revisit, don't delete). + +--- + +## 7. Risks & open questions + +| ID | Question | Owner / resolution | +|----|----------|--------------------| +| R1 | Do any `Settings` secrets live inside **arrays**? | Phase 3 audit; feeds edgezero Phase 0 B-1. | +| R2 | `StoreName` vs `StoreId` split — still needed after `management_api.rs` deletion? | Phase 1; drop if only the CLI provision path used it. | +| R3 | EC identity API + Fastly rate limiter are Fastly-only today | Out of scope here; note as a portability follow-up (not blocking). | +| R4 | Cloudflare/Spin boot-time secret-store access for D3 | Confirm in Phase 3 scoping. | +| R5 | Config-change-requires-redeploy (D1) acceptable to operators? | Already true today; confirm no regression expectation. | +| R6 | Phase 4 handler rewrite is large — split by route group? | Yes; per-implementation-plan, group by file (`proxy`, `auction`, `ec`, `request_signing`, `publisher`). | + +--- + +## 8. Next step + +Per phase, run `writing-plans` to produce an implementation plan **at phase start** (not upfront for all five) — the plan for Phase N should reflect the state left by Phase N-1. Begin with **Phase 1** once this umbrella is approved and the Phase 0 edgezero PR is merged (or Phase 1 can start immediately since it has no upstream dependency). From 1744f162b74e85c486c1363698493b140be1cdfc Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Thu, 2 Jul 2026 15:45:23 -0700 Subject: [PATCH 02/30] Tighten cruft-ledger precision after self-review Clarify that platform/mod.rs and types.rs are edited (KV re-export and a shrinking RuntimeServices remain), not deleted, in Phase 1. --- .../specs/2026-07-02-edgezero-full-migration-design.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md b/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md index a5bc3137..bf7fdb9f 100644 --- a/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md +++ b/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md @@ -180,7 +180,7 @@ Phase 1 (stores) ──> Phase 2 (config) ──> Phase 3 (secrets) Phase 4 (e | Item | File(s) | Phase | Replaced by | |---|---|---|---| | Fastly chunk-pointer resolver | `core/src/settings_data.rs` | 1 | EdgeZero `FastlyConfigStore` + `chunked_config.rs` | -| Bespoke config/secret store traits | `core/src/platform/{traits,types,mod}.rs` | 1 | EdgeZero `ConfigStore`/`SecretStore`/`StoreRegistry` | +| Bespoke config/secret store traits | `core/src/platform/traits.rs` (config+secret trait defs); `mod.rs`/`types.rs` edited, not deleted (KV re-export + shrinking `RuntimeServices` stay) | 1 | EdgeZero `ConfigStore`/`SecretStore`/`StoreRegistry` | | 4× per-adapter store impls | `adapter-*/src/platform.rs` | 1 | per-adapter EdgeZero store impls | | Fastly management REST client | `adapter-fastly/src/management_api.rs` | 1 | EdgeZero CLI provision | | `include_str!` config baking | `adapter-{cloudflare,spin}/src/app.rs` | 2 | store-loaded config | From 7266ef7716b8351a10a236a230c848ade9deb3b7 Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Thu, 2 Jul 2026 15:53:56 -0700 Subject: [PATCH 03/30] Link edgezero Phase 0 PR #305 in migration spec --- .../specs/2026-07-02-edgezero-full-migration-design.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md b/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md index bf7fdb9f..8cd20683 100644 --- a/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md +++ b/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md @@ -4,7 +4,7 @@ - **Date:** 2026-07-02 - **Scope:** Move trusted-server **completely** onto EdgeZero primitives: config push, KV, secret store, config injection without an embedded `trusted-server.toml`, extractor-based handlers, and deletion of every pre-EdgeZero workaround. - **Shape:** Umbrella roadmap. Defines the end-state, the current-state gap, and an ordered set of phases with dependencies. **Each phase gets its own implementation plan** (`writing-plans`) before code is written. -- **Companion spec:** Phase 0 (`State` extractor + nested `#[secret]`) is an **edgezero-repo** change, specified separately (`…-state-and-nested-secrets-design.md`) and tracked via its own edgezero PR. This umbrella depends on it but does not re-specify it. +- **Companion spec:** Phase 0 (`State` extractor + nested `#[secret]`) is an **edgezero-repo** change, specified separately (`…-state-and-nested-secrets-design.md`) and tracked via edgezero PR [stackpop/edgezero#305](https://github.com/stackpop/edgezero/pull/305). This umbrella depends on it but does not re-specify it. --- @@ -86,7 +86,7 @@ Phase 1 (stores) ──> Phase 2 (config) ──> Phase 3 (secrets) Phase 4 (e ### Phase 0 — EdgeZero prerequisites (external, edgezero repo) -**Owner:** edgezero. **Tracked by:** its own spec + PR (link to be added). +**Owner:** edgezero. **Tracked by:** its own spec + PR [stackpop/edgezero#305](https://github.com/stackpop/edgezero/pull/305) — "add State + nested #[secret] design spec". **Delivers:** (A) `State` extractor + `RouterBuilder::with_state`; (B) nested/array `#[secret]` in `#[derive(AppConfig)]` + path-aware `secret_walk`. **Blocks:** Phase 3 (B), Phase 4 (A). **This umbrella consumes it as a versioned dependency** — bump the pinned `edgezero` rev once merged. From 2971467aa33ad91c4c35d8d66141ad0326f57621 Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Thu, 2 Jul 2026 16:43:16 -0700 Subject: [PATCH 04/30] Amend migration spec per review: Fastly dispatch, boot-time config, store id - P0-C: Fastly bypasses run_app (multi-value Set-Cookie, logger reinit, JA4/H2 capture) - add EdgeZero dispatch prereq or documented exception - P-BOOT: specify boot-time config/secret store access for Cloudflare/Spin (build_state runs before request context; registry is per-request) - D5: unify the split app-config store id (app_config vs trusted_server_config) - Promote the secret inventory to a spec artifact; array + optional secrets confirmed present, so edgezero #305 must ship ArrayEach + Option - Phase 4 acceptance: per-adapter route parity (EC routes are Fastly-only) - Phase 5: expand deletion ledger (route_tests, viceroy config, fastly.toml, runbook) - Scope the include_str! ban to adapter/runtime app-config only --- ...26-07-02-edgezero-full-migration-design.md | 77 +++++++++++++++---- 1 file changed, 60 insertions(+), 17 deletions(-) diff --git a/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md b/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md index 8cd20683..3d2a56a2 100644 --- a/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md +++ b/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md @@ -10,11 +10,13 @@ ## 1. End-state -trusted-server is a fully EdgeZero-native app: adapter binaries call `run_app::`; core is platform-neutral; config, KV, and secrets flow exclusively through EdgeZero's `StoreRegistry`; app config is a signed blob published by `ts config push` and read back typed at request time with secrets resolved from the secret store; handlers are `#[action]` functions taking `FromRequest` extractors; and no Fastly-specific or pre-EdgeZero shim remains in core or the adapters. +trusted-server is a fully EdgeZero-native app: adapter binaries are thin entry points (`run_app::` where the platform allows it, or a **documented adapter-level dispatch shim** where it does not — see Fastly below); core is platform-neutral; config, KV, and secrets flow exclusively through EdgeZero's `StoreRegistry`; app config is a signed blob published by `ts config push` and read back typed with secrets resolved from the secret store; handlers are `#[action]` functions taking `FromRequest` extractors; and no *pre-EdgeZero* shim remains in core or the adapters. + +**Fastly is not `run_app::` today and may not be at the end** (Blocker, verified `adapter-fastly/src/main.rs`): Fastly deliberately calls `app.router().oneshot()` directly instead of the standard dispatch helpers, because (a) the helpers convert through `fastly::Response` via `set_header`, which **drops duplicate `Set-Cookie` values** from publisher/origin responses, and (b) `run_app_*` triggers a **logger reinit** Fastly must avoid. Fastly also injects `client_info` + `device_signals` (TLS JA4 / H2 fingerprint) into request extensions from the *original* `FastlyRequest` before conversion — signals a reconstructed EdgeZero request cannot expose. This is an **EdgeZero-adapter capability gap**, not trusted-server cruft. Resolution is a **prerequisite (P0-C)**, see §4a. Concretely, at the end of this migration: -- **No `include_str!` of any `*.toml` config** in any adapter. All four adapters load app config from the EdgeZero config store. +- **No adapter/runtime app-config baking** — no `include_str!` of `*.toml` **app config** in any adapter runtime path; all four adapters load app config from the EdgeZero config store. (The `ts config init` CLI command still embeds `trusted-server.example.toml` as a scaffolding template — that is not runtime config baking and is out of scope.) - **No app-level secrets embedded in the pushed config blob.** Secrets live in the EdgeZero secret store; the blob carries only key names, resolved at request time. - **No bespoke `PlatformConfigStore` / `PlatformSecretStore` / `RuntimeServices`.** Core and adapters use EdgeZero `ConfigStore` / `SecretStore` / `KvStore` via `StoreRegistry`. - **No `FastlyManagementApiClient`, no `settings_data.rs` chunk resolver, no `config`-crate env overlay, no `Redacted`.** @@ -32,7 +34,7 @@ Verified across `trusted-server-core`, the four adapters, `trusted-server-cli`, | **KV** | ✅ 100% on EdgeZero (`KvStore`/`KvHandle`, re-exported as `PlatformKvStore`) | None (baseline for the pattern) | | **Routing** | ✅ All 4 adapters route through EdgeZero `RouterService` + `Hooks` | None structurally; handler authoring changes in Phase 4 | | **Core off `fastly::` types** | ✅ Enforced by `migration_guards.rs` | Keep the guard; extend coverage as adapters shrink | -| **Config load** | ⚠️ Fastly + Axum load the blob from the config store; **Cloudflare + Spin `include_str!` `trusted-server.example.toml`** | Phase 2 | +| **Config load** | ⚠️ Fastly + Axum load the blob from the config store; **Cloudflare** reads a `TRUSTED_SERVER_CONFIG` env side-channel (native fallback `include_str!`); **Spin `include_str!` `trusted-server.example.toml`** — none of these is a boot-time config-store read | Phase 2 (P-BOOT) | | **Config injection** | ⚠️ `TrustedServerAppConfig` wraps `Settings`, `SECRET_FIELDS = &[]` → secrets inline in blob; `#[derive(AppConfig)]`/`#[secret]` unused | Phases 2–3 | | **Config / Secret stores** | ❌ Core uses bespoke `PlatformConfigStore`/`PlatformSecretStore` + `RuntimeServices`; 4× per-adapter `platform.rs` impls; `FastlyManagementApiClient` for writes | Phase 1 | | **Fastly config chunking** | ❌ `settings_data.rs` re-implements EdgeZero's `chunked_config.rs` verbatim in core | Phase 1 | @@ -46,6 +48,9 @@ Verified across `trusted-server-core`, the four adapters, `trusted-server-cli`, 2. **`AppConfig` re-parses + verifies + secret-walks the whole blob every request** — too costly for `Settings`. Decision: keep loading `Settings` once at startup into `Arc`, exposed via `State`, rather than the per-request `AppConfig` extractor (see §4, Decision D1). 3. **Full secret externalization needs nested/array `#[secret]`** because `Settings` is deeply nested → **Phase 0** (edgezero derive change). 4. **Integration proxies are a second, nested `matchit` router** with their own `IntegrationProxy::handle(&Settings, &RuntimeServices, req)` convention — orthogonal to the core route handlers (see §4, Decision D2). +5. **Fastly cannot use `run_app::` today** — it bypasses standard dispatch for multi-value `Set-Cookie` preservation, to skip a logger reinit, and to capture TLS JA4 / H2 fingerprints from the raw `FastlyRequest`. Needs an EdgeZero-adapter capability (**P0-C**, §4a) or a permanent documented exception → **Phase 0 / §4a**. +6. **Config is loaded at boot, before any request context exists** (`build_state()` → `load_startup_settings()`), but EdgeZero's config-store handle is only wired *per request* (`ConfigRegistry` in request extensions). On Cloudflare, config arrives via a `TRUSTED_SERVER_CONFIG` env side-channel injected at the worker entry; on Spin it's baked example TOML. So "load config from the store at startup" needs a **boot-time store-access mechanism**, not the per-request registry → **§4a + Phase 2**. +7. **The logical app-config store id is inconsistent** — `settings_data.rs` defaults to `app_config`, `edgezero.toml` declares `trusted_server_config`, and Fastly splits rollout flags (`trusted_server_config`) from the app-config blob (`app_config`). Must be unified → **Decision D5**, before Phase 1/2 planning. --- @@ -76,10 +81,28 @@ Phase 1 (stores) ──> Phase 2 (config) ──> Phase 3 (secrets) Phase 4 (e **D2 — Integration proxy router stays put (for now).** Phase 4 migrates the **named/core routes** to extractors. The integration registry's nested `matchit` dispatch and `IntegrationProxy::handle` signature are internal, working, and orthogonal; migrating them is a **follow-up** (Phase 4b, optional), not silently dropped. Called out so the extractor migration isn't mistaken for "all handlers." -**D3 — Secret resolution happens at startup, not per request.** With D1, the startup config load resolves `#[secret]` fields against the secret store once (Phase 3). Adapters must therefore have a secret-store handle available at boot, not only per request. Fastly/Axum already open stores eagerly; Cloudflare/Spin resolve from bindings — confirm boot-time access in Phase 3 scoping. +**D3 — Secret resolution happens at startup, not per request.** With D1, the startup config load resolves `#[secret]` fields against the secret store once (Phase 3). Adapters must therefore have a secret-store handle available **at boot**, not only per request — the same boot-time-store-access problem as config (constraint 6). Resolution is shared with §4a. On Cloudflare `env` and on Spin the host component are both available at the `run_app` entry, so a boot-time handle is constructible; the gap is that EdgeZero currently only exposes stores via the per-request registry. **D4 — One typed `Settings` as the AppConfig root.** Replace `TrustedServerAppConfig` (wrapper with empty `SECRET_FIELDS`) by deriving `AppConfig` directly on `Settings`, with `#[secret]` on the real secret fields (Phase 3). Removes a transitional indirection. +**D5 — Single logical app-config store id.** Unify on **one** logical config store id and blob key before Phase 1/2 planning. Recommendation: the app-config blob lives in the `edgezero.toml`-declared config store id **`trusted_server_config`** under key **`app_config`** (the current `CONFIG_BLOB_KEY`); reconcile `settings_data.rs`'s `DEFAULT_CONFIG_STORE_ID = "app_config"` to that store id. The competing `app_config` **store** id exists only because rollout flags were parked in `trusted_server_config`; those flags are deleted in Phase 5, removing the reason for two stores. *Open sub-question: keep flags and app-config in the same store until Phase 5, or move flags out first — decide in the Phase 1 plan.* + +--- + +## 4a. Prerequisites (must resolve before or during Phase 1/2) + +These are not trusted-server refactors; they are EdgeZero-adapter capability gaps or up-front decisions that gate the phases. + +**P0-C — EdgeZero adapter dispatch that preserves multi-value headers and skips logger reinit (Fastly).** For Fastly to reach a thin entry point, EdgeZero's Fastly adapter dispatch must: (1) preserve duplicate response headers (esp. `Set-Cookie`) instead of collapsing via `set_header`; (2) allow the app to opt out of the per-call logger reinit; and (3) provide a hook to inject request-scoped extensions (`client_info`, `device_signals`) derived from the raw `FastlyRequest` before conversion. **Two resolutions:** +- **(Recommended) Upstream to EdgeZero** as a header-preserving `run_app`/dispatch variant + a pre-dispatch extension hook. Add to the edgezero prerequisite set alongside Phase 0 (A/B). Then Fastly's `main.rs` collapses to that variant. +- **(Fallback) Permanent documented exception** — Fastly keeps a small adapter-level dispatch shim calling `app.router().oneshot()`. The end-state (§1) already allows this. This is *not* pre-EdgeZero cruft and would survive Phase 5. +Decision needed with the edgezero maintainer; feeds the same PR track as #305. + +**P-BOOT — Boot-time store access for startup config + secret load.** Define, per adapter, how `build_state()` obtains a config-store (and secret-store) handle at boot, before request context. Options: +- **(a) Boot-time handle from the adapter environment** — Cloudflare builds a config-store handle from the `env` binding passed to `run_app`; Spin from the host component config; Fastly/Axum open the store eagerly (already do). Requires EdgeZero to expose a boot-time store constructor (or trusted-server constructs it from the adapter's env directly, mirroring today's `TRUSTED_SERVER_CONFIG` side-channel but reading the store instead). +- **(b) Lazy first-request load + cache** — defer the config load to the first request (where the registry exists), cache `Arc` in a `OnceCell`. Keeps D1's load-once semantics but moves the load off the boot path. Trade-off: first request pays the cost and must handle a config-load error as a request error. +Recommendation: **(a)** where the adapter env is available at boot (Cloudflare/Spin both pass it to `run_app`), falling back to **(b)** only if an adapter genuinely cannot construct a boot-time handle. Settle in the Phase 2 plan; this is the load-bearing detail that makes "no baked TOML on Cloudflare/Spin" actually implementable. + --- ## 5. Phases @@ -87,8 +110,9 @@ Phase 1 (stores) ──> Phase 2 (config) ──> Phase 3 (secrets) Phase 4 (e ### Phase 0 — EdgeZero prerequisites (external, edgezero repo) **Owner:** edgezero. **Tracked by:** its own spec + PR [stackpop/edgezero#305](https://github.com/stackpop/edgezero/pull/305) — "add State + nested #[secret] design spec". -**Delivers:** (A) `State` extractor + `RouterBuilder::with_state`; (B) nested/array `#[secret]` in `#[derive(AppConfig)]` + path-aware `secret_walk`. -**Blocks:** Phase 3 (B), Phase 4 (A). **This umbrella consumes it as a versioned dependency** — bump the pinned `edgezero` rev once merged. +**Delivers:** (A) `State` extractor + `RouterBuilder::with_state`; (B) nested/array `#[secret]` in `#[derive(AppConfig)]` + path-aware `secret_walk`; **(C, if resolved upstream) P0-C** header-preserving Fastly dispatch + pre-dispatch extension hook (§4a). +**Blocks:** Phase 3 (B), Phase 4 (A), Phase 5/Fastly end-state (C). **This umbrella consumes it as a versioned dependency** — bump the pinned `edgezero` rev once merged. +**Note for #305:** the trusted-server secret audit (Phase 3 / §5) confirms **array secrets exist** (`ec.partners[].api_token`, `handlers[].password`) and **optional-string secrets exist** (`ts_pull_token`). So edgezero #305's `ArrayEach` and `Option` support are **required**, not deferrable — this settles that PR's open question B-1. --- @@ -115,7 +139,7 @@ Phase 1 (stores) ──> Phase 2 (config) ──> Phase 3 (secrets) Phase 4 (e **Changes:** - Derive `AppConfig` on the config root (interim: still `TrustedServerAppConfig` until Phase 3 collapses it onto `Settings`) so all adapters use the same store-load path. -- **Cloudflare** (`adapter-cloudflare/src/app.rs`) and **Spin** (`adapter-spin/src/app.rs`): replace `Settings::from_toml(include_str!(".../trusted-server.example.toml"))` with `get_settings_from_config_store(...)` (now the EdgeZero `ConfigStore` path from Phase 1). Seed each platform's config store (`wrangler.toml` / `runtime-config.toml` / `fastly.toml` local blocks) with the pushed blob. +- **Cloudflare** (`adapter-cloudflare/src/app.rs`) and **Spin** (`adapter-spin/src/app.rs`): replace startup config sourcing (Cloudflare's `TRUSTED_SERVER_CONFIG` env side-channel + the native `include_str!` fallback; Spin's baked example TOML) with a **boot-time config-store read** per **P-BOOT (§4a)**. `build_state()` obtains a config-store handle from the adapter env passed to `run_app` (option a) or defers to a lazy first-request cached load (option b). This is the load-bearing detail — settle the mechanism in the Phase 2 plan. Seed each platform's config store (`wrangler.toml` / `runtime-config.toml` / `fastly.toml` local blocks) with the pushed blob under the D5 store id/key. - Delete `Settings::from_toml_and_env`, `ENVIRONMENT_VARIABLE_PREFIX/SEPARATOR`, and the `config` **dev-dependency**. Any remaining env overlay uses EdgeZero's `EDGEZERO__*` / AppConfig `__…` layers. **Deletions:** both `include_str!` config paths, `from_toml_and_env`, `config` crate dep. @@ -131,13 +155,27 @@ Phase 1 (stores) ──> Phase 2 (config) ──> Phase 3 (secrets) Phase 4 (e **Changes:** - Collapse `TrustedServerAppConfig` onto `Settings` (D4): `#[derive(AppConfig)]` on `Settings`, `#[secret]` / `#[secret(store_ref)]` on the real secret fields (S3 keys, request-signing key refs, DataDome server-side key, integration API keys, etc.), including the **nested** ones enabled by Phase 0. -- Audit `Settings` for the secret inventory **before implementation** — this settles Phase 0's open question B-1 (are any secrets inside arrays?). Feed the answer back to the edgezero PR. - Delete `Redacted` and its manual redaction handling; `#[secret]` + the secret store replace it. - Operator migration: `ts` provisions secrets into the secret store (via EdgeZero provision), and a migration guide moves existing inline secrets out of `trusted-server.toml`. `reject_placeholder_secrets` becomes a check on the resolved values at boot. - Startup load resolves `#[secret]` fields against the secret store (D1/D3), then validates. +**Secret inventory (spec artifact — verify + extend during the Phase 3 plan).** Preliminary audit of `Settings`; shapes drive the edgezero #305 requirements: + +| Secret | Path | Shape | Notes | +|---|---|---|---| +| Partner API tokens | `ec.partners[].api_token` | **array element** | needs `ArrayEach` (edgezero #305) | +| Handler passwords | `handlers[].password` | **array element** | needs `ArrayEach` | +| EC passphrase | `ec.passphrase` | scalar `String` | nested | +| Pull token | `ts_pull_token` | **`Option`** | needs optional-secret support (edgezero #305) | +| Publisher proxy secret | `publisher.proxy_secret` | scalar `String` | nested | +| DataDome server-side key | `integrations.datadome.*` (store-ref name+key) | store-ref | already resolves via secret-store name+key | +| S3 / proxy secret access key | `proxy.secret_access_key` (+ `proxy.secret_store`) | store-ref | already store-backed | +| Request-signing keys | `request_signing.*` (`secret_store_id`) | store-ref | already store-backed | + +Two consequences: (1) edgezero #305 **must** ship `ArrayEach` + `Option` (see Phase 0 note); (2) the already-store-backed secrets (DataDome, S3, request-signing) need only re-expression as `#[secret(store_ref)]`, not relocation. + **Deletions:** inline secrets in the blob, `Redacted`, `SECRET_FIELDS = &[]` wrapper. -**Acceptance:** pushed blob contains only secret **key names**; boot resolves them; a config with a nested secret validates and serves; operator migration guide published; tests green. +**Acceptance:** pushed blob contains only secret **key names**; boot resolves them; a config with nested **and array** secrets validates and serves; operator migration guide published; tests green. --- @@ -148,14 +186,14 @@ Phase 1 (stores) ──> Phase 2 (config) ──> Phase 3 (secrets) Phase 4 (e **Depends on:** Phase 0 (A) `State`; cleaner after Phase 1. **Changes:** -- Introduce `State>` (or narrower `State>` / `State>` / `State>`) wired via `RouterBuilder::with_state` in each adapter's `Hooks::routes()`. +- Introduce `State>` (or narrower `State>` / `State>` / `State>`) wired via `RouterBuilder::with_state` in each adapter's `Hooks::routes()`. *Granularity (one `Arc` vs per-component states) is a Phase 4 plan decision.* - Rewrite core `handle_*` (`proxy.rs`, `publisher.rs`, `auction/endpoints.rs`, `request_signing/endpoints.rs`, `ec/*.rs`) from `(&Settings, &RuntimeServices, Request)` to `#[action]` signatures using `State<…>`, `Json`/`Query`/`Path`/`Headers`/`Host`, and the store extractors (`Kv`, `Secrets`, `Config`). - Delete the per-adapter shims (`execute_handler`/`execute_named`/`named_route_handler` + `NamedRouteHandler` enums) and shrink/retire `RuntimeServices` (its store fields already gone in Phase 1; remaining bundle folds into `State` + extractors). - **EC lifecycle & pre-route filters** (`build_ec_request_state`, `run_pre_route_filters`, `attach_dispatch_extensions`, `FinalizeResponseMiddleware`) are cross-cutting — keep them as **middleware**, not per-arg extractors. - **Phase 4b (optional follow-up, D2):** migrate the integration proxy nested router / `IntegrationProxy::handle` onto `RouterService` + extractors. Deferred by default. **Deletions:** per-adapter handler shims, `NamedRouteHandler` enums, `RuntimeServices` (final form). -**Acceptance:** all named routes served via `#[action]` handlers on all adapters; middleware carries EC lifecycle; parity test green. +**Acceptance:** every named route **that a given adapter supports** is served via an `#[action]` handler on that adapter (route sets are *not* uniform — Fastly exposes EC identity routes `/_ts/api/v1/{identify,batch-sync}`; Spin and Axum deliberately omit them to match non-Fastly adapters); middleware carries EC lifecycle, and **Fastly-only EC after-send / finalize ordering** is preserved; parity test green. --- @@ -167,11 +205,12 @@ Phase 1 (stores) ──> Phase 2 (config) ──> Phase 3 (secrets) Phase 4 (e **Changes:** - Delete `legacy_main` / `route_request` (`adapter-fastly/src/main.rs`), `compat.rs` (fastly↔http shim), and the flag machinery (`edgezero_enabled`, `edgezero_rollout_pct`, `select_edgezero_entrypoint`, `should_route_to_edgezero`, IP-bucket hashing). -- `main()` calls the EdgeZero path unconditionally (`run_app::` shape). -- Retire the `trusted_server_config` rollout-flag reads. +- `main()` calls the EdgeZero path unconditionally — the P0-C dispatch variant, or the documented Fastly dispatch shim (§4a), depending on how P0-C resolves. +- Retire the `trusted_server_config` rollout-flag reads (the flags, not the config store — after D5 the store may still hold app config). +- **Ancillary cleanup (easy to miss):** Fastly route tests importing legacy stores + `route_request` (`adapter-fastly/src/route_tests.rs`); generated Viceroy config rollout flags (`integration-tests/src/bin/generate-viceroy-config.rs`); `fastly.toml` local `edgezero_enabled`/`edgezero_rollout_pct` config; and the rollout runbook `docs/internal/EDGEZERO_MIGRATION.md`. -**Deletions:** `legacy_main`, `route_request`, `compat.rs`, rollout flags. -**Acceptance:** Fastly adapter has a single EdgeZero entry path; no rollout flags; full CI gate green; production traffic unaffected (already 100% on EdgeZero by gate definition). +**Deletions:** `legacy_main`, `route_request`, `compat.rs`, rollout flags, `route_tests.rs` legacy imports, viceroy-config flags, `fastly.toml` flag config, `EDGEZERO_MIGRATION.md` runbook. +**Acceptance:** Fastly adapter has a single EdgeZero entry path; no rollout flags anywhere (adapter, tests, generated config, `fastly.toml`, docs); full CI gate green; production traffic unaffected (already 100% on EdgeZero by gate definition). --- @@ -183,12 +222,13 @@ Phase 1 (stores) ──> Phase 2 (config) ──> Phase 3 (secrets) Phase 4 (e | Bespoke config/secret store traits | `core/src/platform/traits.rs` (config+secret trait defs); `mod.rs`/`types.rs` edited, not deleted (KV re-export + shrinking `RuntimeServices` stay) | 1 | EdgeZero `ConfigStore`/`SecretStore`/`StoreRegistry` | | 4× per-adapter store impls | `adapter-*/src/platform.rs` | 1 | per-adapter EdgeZero store impls | | Fastly management REST client | `adapter-fastly/src/management_api.rs` | 1 | EdgeZero CLI provision | -| `include_str!` config baking | `adapter-{cloudflare,spin}/src/app.rs` | 2 | store-loaded config | +| Adapter/runtime app-config baking | `adapter-{cloudflare,spin}/src/app.rs` (`include_str!` + Cloudflare `TRUSTED_SERVER_CONFIG` side-channel) | 2 | boot-time store-loaded config (P-BOOT). *`ts config init` template embed is out of scope.* | | Legacy env overlay + `config` dep | `core/src/settings.rs` (`from_toml_and_env`, `ENVIRONMENT_VARIABLE_*`) | 2 | `EDGEZERO__*` / AppConfig env layers | | AppConfig wrapper w/ empty `SECRET_FIELDS` | `core/src/config.rs` | 3 | `#[derive(AppConfig)]` on `Settings` | | `Redacted` | `core/src/redacted.rs` | 3 | `#[secret]` + secret store | | Per-adapter handler shims | `adapter-*/src/app.rs` | 4 | `#[action]` + extractors | | Legacy Fastly path + flags + compat | `adapter-fastly/src/{main.rs,compat.rs}` | 5 | single EdgeZero entry path | +| Rollout-flag ancillaries | `adapter-fastly/src/route_tests.rs` (legacy imports), `integration-tests/src/bin/generate-viceroy-config.rs` (flags), `fastly.toml` (local flag config), `docs/internal/EDGEZERO_MIGRATION.md` (runbook) | 5 | — (deleted with the rollout mechanism) | **Explicitly NOT cruft (do not remove):** `migration_guards.rs` (intentional `fastly::` ban test), `s3_sigv4.rs` (AWS-domain canonical/hashing), `platform/image_optimizer.rs` (no EdgeZero equivalent yet), EC KV CAS wrapper (`ec/kv*.rs` — needs EdgeZero generation-CAS parity first; revisit, don't delete). @@ -198,7 +238,10 @@ Phase 1 (stores) ──> Phase 2 (config) ──> Phase 3 (secrets) Phase 4 (e | ID | Question | Owner / resolution | |----|----------|--------------------| -| R1 | Do any `Settings` secrets live inside **arrays**? | Phase 3 audit; feeds edgezero Phase 0 B-1. | +| R1 | Do any `Settings` secrets live inside **arrays**? | **Resolved: yes** (`ec.partners[].api_token`, `handlers[].password`) + optional (`ts_pull_token`). edgezero #305 must ship `ArrayEach` + `Option` (see §5 Phase 3 inventory + Phase 0 note). | +| R7 | P0-C: upstream a header-preserving Fastly dispatch, or keep a permanent Fastly dispatch shim? | Decide with edgezero maintainer (§4a); gates the Fastly end-state and Phase 5. | +| R8 | P-BOOT: boot-time store handle (a) vs lazy cached first-request load (b), per adapter? | Phase 2 plan (§4a). | +| R9 | D5: single config-store id `trusted_server_config` (key `app_config`) — confirm and reconcile `settings_data.rs`. | Phase 1 plan. | | R2 | `StoreName` vs `StoreId` split — still needed after `management_api.rs` deletion? | Phase 1; drop if only the CLI provision path used it. | | R3 | EC identity API + Fastly rate limiter are Fastly-only today | Out of scope here; note as a portability follow-up (not blocking). | | R4 | Cloudflare/Spin boot-time secret-store access for D3 | Confirm in Phase 3 scoping. | From eceaf8160b925c34e0f9cf99a349620537796545 Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Thu, 2 Jul 2026 20:59:02 -0700 Subject: [PATCH 05/30] Amend Phase 1 per review: runtime write path (D6) + store-id reconciliation (D5) - D6: EdgeZero stores are read-only, but KeyRotationManager writes/deletes config+secrets at runtime for /_ts/admin/keys/*. management_api.rs cannot be deleted in Phase 1 unconditionally; gate on a keep/move-to-ops/upstream decision (R10) - D5 expanded: reconcile ALL runtime store ids (app_config, secrets, JWKS, DataDome ts_secrets, S3, fixtures) with edgezero.toml or strict lookup fails - Fastly needs explicit registry injection into its custom oneshot path - Phase 1 plan starts with a store-capability inventory, not deletions --- ...26-07-02-edgezero-full-migration-design.md | 32 ++++++++++++------- 1 file changed, 21 insertions(+), 11 deletions(-) diff --git a/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md b/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md index 3d2a56a2..a9a494a9 100644 --- a/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md +++ b/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md @@ -85,7 +85,13 @@ Phase 1 (stores) ──> Phase 2 (config) ──> Phase 3 (secrets) Phase 4 (e **D4 — One typed `Settings` as the AppConfig root.** Replace `TrustedServerAppConfig` (wrapper with empty `SECRET_FIELDS`) by deriving `AppConfig` directly on `Settings`, with `#[secret]` on the real secret fields (Phase 3). Removes a transitional indirection. -**D5 — Single logical app-config store id.** Unify on **one** logical config store id and blob key before Phase 1/2 planning. Recommendation: the app-config blob lives in the `edgezero.toml`-declared config store id **`trusted_server_config`** under key **`app_config`** (the current `CONFIG_BLOB_KEY`); reconcile `settings_data.rs`'s `DEFAULT_CONFIG_STORE_ID = "app_config"` to that store id. The competing `app_config` **store** id exists only because rollout flags were parked in `trusted_server_config`; those flags are deleted in Phase 5, removing the reason for two stores. *Open sub-question: keep flags and app-config in the same store until Phase 5, or move flags out first — decide in the Phase 1 plan.* +**D5 — Reconcile ALL runtime store ids with `edgezero.toml`.** Not just the app-config blob: EdgeZero's registry lookup is **strict** (unknown id → `None`), so every logical store id any config field or call site names at runtime must appear in `edgezero.toml` `[stores.config]`/`[stores.secrets]` `ids`. Today `edgezero.toml` declares only `trusted_server_config` / `trusted_server_secrets`, while config references `app_config`, `secrets`, JWKS/config-list stores, DataDome `ts_secrets`, S3 secret store, etc. Phase 1 must either (a) declare all these ids in `edgezero.toml` and map each to a platform store via `EDGEZERO__STORES__*__NAME`, or (b) collapse them onto the declared defaults and update every config field + fixture. Recommendation: the app-config blob lives in `trusted_server_config` under key `app_config` (`CONFIG_BLOB_KEY`); collapse incidental ids onto the declared defaults where semantically identical, and declare the genuinely-separate ones (e.g. JWKS store). The full id inventory is the Phase 1 plan's task 1. + +**D6 — Runtime write path for request-signing key rotation.** EdgeZero `ConfigStore`/`SecretStore` are **read-only** at runtime; writes go through provisioning (author/ops time). But `KeyRotationManager` writes+deletes config (JWKS) and secrets (private keys) **at request time** via `/_ts/admin/keys/{rotate,deactivate,delete}`, backed by `management_api.rs`. Three resolutions: +- **(a) Keep a write-capable admin abstraction** — retain the trusted-server `put`/`create`/`delete` traits + `management_api.rs` for the admin write path only; EdgeZero read-only stores serve the read path. Least disruptive; leaves a non-EdgeZero write path (so "completely on EdgeZero" is not literally met for admin writes). +- **(b) Move key rotation out of runtime** — make rotate/deactivate/delete an **ops/CLI** operation (`ts keys …`) that writes via EdgeZero provisioning; the runtime only reads current keys. Most EdgeZero-native; changes the admin surface from an HTTP endpoint to an operator command — a **product/ops decision**. +- **(c) Add an EdgeZero runtime store-write/provision API** — upstream change; broadens the platform. +**Decision needed before Phase 1 deletions.** Until then `management_api.rs` stays. This is R10; recommend (a) as the interim (unblocks Phase 1 read migration) with (b) as the target end-state if ops agrees. --- @@ -118,18 +124,21 @@ Recommendation: **(a)** where the adapter env is available at boot (Cloudflare/S ### Phase 1 — Stores onto EdgeZero `StoreRegistry` -**Goal:** delete trusted-server's bespoke config/secret store layer; route all store access through EdgeZero `ConfigStore` / `SecretStore` / `StoreRegistry` (KV is already there). +**Goal:** route all **read** store access through EdgeZero `ConfigStore` / `SecretStore` / `StoreRegistry` (KV is already there), and delete the bespoke read layer + duplicated chunk resolver. **Runtime writes and store-id reconciliation must be resolved first** (see D5, D6 below and the plan's task 1). + +> **Plan ordering (per review):** the Phase 1 plan does NOT start with deletions. Task 1 is a **store-capability inventory** — enumerate every runtime store id and every read vs write call site — and a **decision on D6** (runtime writes). Deletions come only after the write path is settled. **Changes:** -- Replace `PlatformConfigStore` / `PlatformSecretStore` (`platform/traits.rs`, `types.rs`) and the `RuntimeServices` config/secret fields with EdgeZero `ConfigStoreHandle` / `BoundSecretStore` resolved from the per-request registries (`ConfigRegistry` / `SecretRegistry`), matching how KV already works. -- Migrate core secret consumers to `secrets.named(id)?.require_str(key)` / config consumers to the config binding: `proxy.rs` (S3), `request_signing/{signing,rotation}.rs`, `integrations/datadome/{protection,protection_scope}.rs`. -- Delete the 4× per-adapter `platform.rs` config/secret store impls (`FastlyPlatformConfigStore`, `AxumPlatformConfigStore`, `NoopConfigStore`, `Cloudflare…`, and secret equivalents); adapters instead build `ConfigRegistry`/`SecretRegistry` via `dispatch_with_registries` from `[stores.*]` metadata. -- Delete `FastlyManagementApiClient` (`management_api.rs`) — store writes/provisioning move to the EdgeZero CLI provision path. +- **Reads:** replace the `PlatformConfigStore`/`PlatformSecretStore` **read** methods and the `RuntimeServices` config/secret fields with EdgeZero `ConfigStoreHandle` / `BoundSecretStore` resolved from the per-request registries (`ConfigRegistry`/`SecretRegistry`), matching KV. Migrate read consumers: `proxy.rs` (S3), `request_signing/{signing,rotation}.rs` (reads), `integrations/datadome/{protection,protection_scope}.rs`. +- **Writes (D6):** `KeyRotationManager` writes+deletes **config and secrets at request time** (`store_private_key`/`store_public_jwk`/`delete_key` for `/_ts/admin/keys/rotate` + deactivate/delete). EdgeZero `ConfigStore`/`SecretStore` are **read-only by design**. So `management_api.rs` **cannot be deleted in Phase 1 as originally written**. Resolve per D6 before touching it. +- **Store-id reconciliation (D5, expanded):** every runtime store id referenced by config must be declared in `edgezero.toml` `[stores.config]`/`[stores.secrets]` `ids` or strict registry lookup returns `None`. Reconcile at least: the app-config blob store, `request_signing.config_store_id` (`app_config` today) + `secret_store_id` (`secrets` today), the JWKS/config-list store, DataDome config-list + secret stores (`ts_secrets`), the S3 secret store, and all `trusted-server.example.toml` + integration/test fixtures. +- **Fastly registry injection (ties to P0-C):** Fastly's custom `oneshot` path (§1) currently inserts only a `ConfigStoreHandle`, not registries via `dispatch_with_registries`. Phase 1 must add explicit `Kv`/`Config`/`Secret` registry construction + insertion into extensions compatible with that custom path (not just the standard dispatch helper the other adapters use). +- Delete the 4× per-adapter `platform.rs` config/secret **read** impls; adapters build registries from `[stores.*]` metadata (via `dispatch_with_registries` on Axum/Cloudflare/Spin, via the Fastly-specific injection above). - Delete `settings_data.rs`'s `FastlyChunkPointer` resolver — EdgeZero's `FastlyConfigStore` resolves chunks transparently. `get_settings_from_config_store` collapses to `ConfigStore::get` + `settings_from_config_blob`. -**Deletions:** `management_api.rs`, `settings_data.rs` chunk resolver, `platform/traits.rs` config/secret traits, 4× `platform.rs` config/secret impls. -**Keeps:** `RuntimeServices` as a shrinking bundle for the still-explicit-arg handlers (removed in Phase 4); `StoreName`/`StoreId` only where the CLI provisioning still needs the management-id split (revisit). -**Acceptance:** all adapters build; `cargo test-fastly/-axum/-cloudflare/-spin` green; secret/config reads exercised in tests go through EdgeZero registries; parity test passes. +**Deletions (after D6/D5 resolved):** `settings_data.rs` chunk resolver, `platform/traits.rs` config/secret **read** traits, 4× `platform.rs` config/secret read impls. **`management_api.rs` deletion is conditional on D6** (may move to CLI/ops instead, or stay as a runtime write path). +**Keeps:** `RuntimeServices` as a shrinking bundle (removed in Phase 4); the runtime write path until D6 resolves it; `StoreName`/`StoreId` where writes/provisioning need the management-id split. +**Acceptance:** all adapters build; `cargo test-fastly/-axum/-cloudflare/-spin` + parity green; secret/config **reads** go through EdgeZero registries; **key rotation/delete still works** (per the D6 resolution); every declared store id resolves (no strict-lookup `None`). --- @@ -221,7 +230,7 @@ Two consequences: (1) edgezero #305 **must** ship `ArrayEach` + `Option` | Fastly chunk-pointer resolver | `core/src/settings_data.rs` | 1 | EdgeZero `FastlyConfigStore` + `chunked_config.rs` | | Bespoke config/secret store traits | `core/src/platform/traits.rs` (config+secret trait defs); `mod.rs`/`types.rs` edited, not deleted (KV re-export + shrinking `RuntimeServices` stay) | 1 | EdgeZero `ConfigStore`/`SecretStore`/`StoreRegistry` | | 4× per-adapter store impls | `adapter-*/src/platform.rs` | 1 | per-adapter EdgeZero store impls | -| Fastly management REST client | `adapter-fastly/src/management_api.rs` | 1 | EdgeZero CLI provision | +| Fastly management REST client (**runtime writes**) | `adapter-fastly/src/management_api.rs` | **conditional (D6)** — 1 only if key rotation moves to ops/CLI; otherwise retained as the admin write path | EdgeZero provisioning (if writes leave runtime) — else no replacement | | Adapter/runtime app-config baking | `adapter-{cloudflare,spin}/src/app.rs` (`include_str!` + Cloudflare `TRUSTED_SERVER_CONFIG` side-channel) | 2 | boot-time store-loaded config (P-BOOT). *`ts config init` template embed is out of scope.* | | Legacy env overlay + `config` dep | `core/src/settings.rs` (`from_toml_and_env`, `ENVIRONMENT_VARIABLE_*`) | 2 | `EDGEZERO__*` / AppConfig env layers | | AppConfig wrapper w/ empty `SECRET_FIELDS` | `core/src/config.rs` | 3 | `#[derive(AppConfig)]` on `Settings` | @@ -241,7 +250,8 @@ Two consequences: (1) edgezero #305 **must** ship `ArrayEach` + `Option` | R1 | Do any `Settings` secrets live inside **arrays**? | **Resolved: yes** (`ec.partners[].api_token`, `handlers[].password`) + optional (`ts_pull_token`). edgezero #305 must ship `ArrayEach` + `Option` (see §5 Phase 3 inventory + Phase 0 note). | | R7 | P0-C: upstream a header-preserving Fastly dispatch, or keep a permanent Fastly dispatch shim? | Decide with edgezero maintainer (§4a); gates the Fastly end-state and Phase 5. | | R8 | P-BOOT: boot-time store handle (a) vs lazy cached first-request load (b), per adapter? | Phase 2 plan (§4a). | -| R9 | D5: single config-store id `trusted_server_config` (key `app_config`) — confirm and reconcile `settings_data.rs`. | Phase 1 plan. | +| R9 | D5: reconcile **all** runtime store ids (`app_config`, `secrets`, JWKS, DataDome `ts_secrets`, S3, fixtures) with `edgezero.toml` — strict lookup fails otherwise. | Phase 1 plan task 1. | +| R10 | D6: runtime write path for key rotation — keep write-capable admin abstraction (a), move to ops/CLI (b), or upstream an EdgeZero write API (c)? | **Blocks Phase 1 deletions.** Decide before deleting `management_api.rs`. | | R2 | `StoreName` vs `StoreId` split — still needed after `management_api.rs` deletion? | Phase 1; drop if only the CLI provision path used it. | | R3 | EC identity API + Fastly rate limiter are Fastly-only today | Out of scope here; note as a portability follow-up (not blocking). | | R4 | Cloudflare/Spin boot-time secret-store access for D3 | Confirm in Phase 3 scoping. | From 62d54372b4b5ffd47cf6c3df2bd17c0b1786d96d Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Thu, 2 Jul 2026 21:02:32 -0700 Subject: [PATCH 06/30] Add Phase 1 store-registry migration implementation plan Task 1 is a decision gate (store-id inventory + D5/D6) per review, not deletions. Read-path migration and Fastly custom-dispatch registry injection are fully specified; management_api.rs deletion is gated on the D6 decision. --- ...07-02-edgezero-store-registry-migration.md | 338 ++++++++++++++++++ 1 file changed, 338 insertions(+) create mode 100644 docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md diff --git a/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md b/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md new file mode 100644 index 00000000..20db0054 --- /dev/null +++ b/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md @@ -0,0 +1,338 @@ +# EdgeZero Store-Registry Migration (Phase 1) Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Route trusted-server's runtime **config and secret reads** through EdgeZero's `ConfigRegistry`/`SecretRegistry` (as KV already is), reconcile all logical store ids with `edgezero.toml`, and delete the duplicated Fastly chunk resolver — without breaking the runtime **write** path (key rotation) or Fastly's custom dispatch. + +**Architecture:** trusted-server core reads stores through the bespoke `PlatformConfigStore`/`PlatformSecretStore` traits (read `get`/`get_string` + write `put`/`create`/`delete`), surfaced via `RuntimeServices`. EdgeZero's `ConfigStore`/`SecretStore` are **read-only**; the per-request `ConfigRegistry`/`SecretRegistry` live in request extensions. This phase makes core reads resolve from those registries while **keeping** the write-capable path until D6 decides its fate. Every adapter must wire the registries, including Fastly's custom `oneshot` path. + +**Tech Stack:** Rust 2024, `error-stack` (`Report`), EdgeZero (`edgezero-core` git dep), Viceroy (Fastly test sim), `cargo test-{fastly,axum,cloudflare,spin}`. + +**Spec:** `docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md` §5 Phase 1, decisions D5 + D6, §4a. + +## Global Constraints + +- Rust **2024 edition**, toolchain **1.95.0** (`rust-toolchain.toml`); WASM target `wasm32-wasip1`. +- Errors: `error-stack` `Report` only (no `anyhow` outside the Spin entry point); `derive_more::Display` for error types; import `Error` from `core::error::`. +- No `unwrap()` in production; `expect("should …")`. No `println!`/`eprintln!`; use `log` macros. +- No wildcard imports (except `use super::*` in `#[cfg(test)]`). No local imports inside functions. +- Commit style: sentence case, imperative, no semantic prefixes, no `Co-Authored-By`/AI footers. +- CI gate (must pass before PR): `cargo fmt --all -- --check`; `cargo clippy-{fastly,axum,cloudflare,spin-native,spin-wasm}`; `cargo test-{fastly,axum,cloudflare,spin}`; `cargo test --manifest-path crates/trusted-server-integration-tests/Cargo.toml --test parity`. +- **Every phase step leaves all four adapters building and green.** +- **EdgeZero `ConfigStore`/`SecretStore` are read-only.** Never assume a runtime write API on them. +- **Registry lookup is strict:** an unknown logical id yields `None`. Every id any config field names at runtime must be declared in `edgezero.toml` `[stores.config]`/`[stores.secrets]` `ids`. + +--- + +## Task 1: Store-capability inventory + D5/D6 decision gate (no code deletion) + +This task produces decisions, not deletions. Its deliverable is a written **decision record** appended to this plan (section "Task 1 Output") that Tasks 2+ depend on. Per spec review, Phase 1 must not begin with deletions. + +**Files:** +- Modify (append decision record): `docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md` +- Read-only inventory across: `crates/trusted-server-core/src/**`, `crates/trusted-server-adapter-*/src/**`, `edgezero.toml`, `trusted-server.example.toml`, `crates/trusted-server-integration-tests/fixtures/**` + +**Interfaces:** +- Produces: the **store-id map** (logical id → platform name → declared-in-edgezero.toml?) and the **write-site list** (every runtime `put`/`create`/`delete` call), consumed by Tasks 2, 3, 6. + +- [ ] **Step 1: Enumerate every logical store id referenced at runtime** + +Run: +```bash +cd /Users/ag/projects/iab/trusted-server/.claude/worktrees/edgezero-migration-spec +rg -n 'config_store_id|secret_store_id|secret_store\s*=|config_store\s*=|StoreName::from|StoreId::from|"app_config"|"secrets"|"jwks|ts_secrets|signing_keys|"api-keys"' \ + crates/trusted-server-core crates/trusted-server-adapter-* trusted-server.example.toml \ + crates/trusted-server-integration-tests/fixtures +rg -n '\[stores\.' edgezero.toml +``` +Expected: a list of ids including at least `app_config`, `secrets`, `signing_keys`, JWKS config-list store, DataDome `ts_secrets`, S3 secret store — versus `edgezero.toml` declaring only `trusted_server_config`/`trusted_server_kv`/`trusted_server_secrets`. + +- [ ] **Step 2: Enumerate every runtime store WRITE call site** + +Run: +```bash +rg -n '\.config_store\(\)\.(put|delete)|\.secret_store\(\)\.(create|delete)' crates/trusted-server-core +``` +Expected: the `KeyRotationManager` write sites in `crates/trusted-server-core/src/request_signing/rotation.rs` (`store_private_key`, `store_public_jwk`, `deactivate_key`, `delete_key`) — the only runtime writers. Confirm no other runtime writers exist. + +- [ ] **Step 3: Record the D5 decision (store-id reconciliation map)** + +Append to "Task 1 Output" a table: each runtime logical id → chosen resolution (declare a new id in `edgezero.toml`, or collapse onto `trusted_server_config`/`trusted_server_secrets`) → the `EDGEZERO__STORES______NAME` mapping. Default recommendation from the spec (D5): app-config blob in `trusted_server_config` key `app_config`; declare JWKS as its own config id; collapse `secrets`→`trusted_server_secrets`; keep DataDome/S3 as declared secret ids. Confirm or adjust against Step 1's actual list. + +- [ ] **Step 4: Record the D6 decision (runtime write path)** + +Append to "Task 1 Output" the chosen option: **(a)** keep a write-capable admin abstraction (`management_api.rs` + the `put`/`create`/`delete` trait methods stay for the admin path); **(b)** move rotate/deactivate/delete to an ops/CLI command; or **(c)** upstream an EdgeZero write API. Spec recommendation: **(a) as the Phase 1 interim** (unblocks the read migration without changing the admin surface), with (b) as the target end-state pending an ops decision. Record which is chosen; Tasks 3–6 assume **(a)** unless this step records otherwise. + +- [ ] **Step 5: Commit the decision record** + +```bash +git add docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md +git commit -m "Record Phase 1 store-id map and runtime-write decision (D5, D6)" +``` + +--- + +## Task 2: Declare all runtime store ids in `edgezero.toml` + reconcile config fields/fixtures + +Implements the D5 map from Task 1. Makes every referenced id resolvable so strict registry lookup never returns `None`. + +**Files:** +- Modify: `edgezero.toml` (`[stores.config]`/`[stores.secrets]` `ids`) +- Modify: `trusted-server.example.toml` (`request_signing.config_store_id`, `secret_store_id`, and any other store-id fields to match the D5 map) +- Modify: `crates/trusted-server-integration-tests/fixtures/configs/trusted-server.integration.toml` +- Test: `crates/trusted-server-core/src/settings.rs` (`#[cfg(test)]`) — assert declared ids cover the config's referenced ids + +**Interfaces:** +- Consumes: Task 1 store-id map. +- Produces: an `edgezero.toml` whose `[stores.*].ids` is a superset of every id named by `Settings`. + +- [ ] **Step 1: Write the failing test** + +Add to `crates/trusted-server-core/src/settings.rs` under `#[cfg(test)]`: +```rust +#[test] +fn every_referenced_store_id_is_declared() { + // Arrange: parse the example config and the manifest's declared ids. + let settings = Settings::from_toml(include_str!("../../../trusted-server.example.toml")) + .expect("should parse example config"); + let declared = declared_store_ids_from_manifest(); // helper reads edgezero.toml + // Act: collect the store ids the settings reference. + let referenced = settings.referenced_store_ids(); + // Assert: manifest declares every referenced id. + for id in &referenced { + assert!( + declared.contains(id), + "store id `{id}` referenced by Settings is not declared in edgezero.toml", + ); + } +} +``` + +- [ ] **Step 2: Run the test to verify it fails** + +Run: `cargo test-fastly every_referenced_store_id_is_declared` +Expected: FAIL — `app_config`/`secrets`/JWKS ids referenced but not declared. + +- [ ] **Step 3: Add `Settings::referenced_store_ids()` + the manifest helper** + +In `settings.rs`, implement `referenced_store_ids(&self) -> std::collections::BTreeSet` returning every `*_store_id` / `*_store` value (request-signing config+secret ids, DataDome, S3, EC, consent). Add a test-only `declared_store_ids_from_manifest()` that parses `edgezero.toml`'s `[stores.config]`/`[stores.secrets]` `ids`. + +- [ ] **Step 4: Update `edgezero.toml` + config fields per the D5 map** + +Edit `edgezero.toml` `[stores.config].ids` / `[stores.secrets].ids` to declare every id from the map; update `trusted-server.example.toml` and the integration fixture so their store-id fields use declared ids. + +- [ ] **Step 5: Run the test to verify it passes** + +Run: `cargo test-fastly every_referenced_store_id_is_declared` +Expected: PASS. + +- [ ] **Step 6: Run all adapter tests + commit** + +Run: `cargo test-fastly && cargo test-axum && cargo test-cloudflare && cargo test-spin` +Expected: PASS. +```bash +git add edgezero.toml trusted-server.example.toml crates/trusted-server-integration-tests/fixtures crates/trusted-server-core/src/settings.rs +git commit -m "Declare all runtime store ids in edgezero.toml and reconcile config fields" +``` + +--- + +## Task 3: Bridge `RuntimeServices` config/secret READS to EdgeZero registries + +Make `RuntimeServices::config_store()`/`secret_store()` **reads** resolve from the request's `ConfigRegistry`/`SecretRegistry` instead of the per-adapter `Platform*Store` read impls. The write methods (`put`/`create`/`delete`) stay routed to the existing path (D6 option a). + +**Files:** +- Modify: `crates/trusted-server-core/src/platform/types.rs` (`RuntimeServices` build/accessors) +- Modify: `crates/trusted-server-core/src/platform/traits.rs` (split read vs write if needed) +- Test: `crates/trusted-server-core/src/platform/types.rs` (`#[cfg(test)]`) + +**Interfaces:** +- Consumes: EdgeZero `ConfigRegistry`/`SecretRegistry` (from request extensions), `edgezero_core::config_store::ConfigStoreHandle`, `edgezero_core::store_registry::BoundSecretStore`. +- Produces: a `RuntimeServices` whose `config_store().get(name,key)` / `secret_store().get_string(name,key)` read through EdgeZero; write methods unchanged. + +- [ ] **Step 1: Write the failing test** (config read resolves via an EdgeZero-backed registry) + +```rust +#[test] +fn runtime_services_config_read_resolves_via_edgezero_registry() { + // Arrange: a RuntimeServices built from an EdgeZero ConfigRegistry with a fixed value. + let services = runtime_services_with_config_registry("trusted_server_config", "greeting", "hi"); + // Act + let value = services + .config_store() + .get(&StoreName::from("trusted_server_config"), "greeting") + .expect("should read via edgezero registry"); + // Assert + assert_eq!(value, "hi", "should return the EdgeZero-backed value"); +} +``` + +- [ ] **Step 2: Run the test to verify it fails** + +Run: `cargo test-fastly runtime_services_config_read_resolves_via_edgezero_registry` +Expected: FAIL (no such constructor / still uses the old read path). + +- [ ] **Step 3: Implement the read bridge** + +Add an EdgeZero-backed `PlatformConfigStore`/`PlatformSecretStore` read adapter in `platform/` that wraps a `ConfigStoreHandle`/`BoundSecretStore` resolved from the registry, mapping `edgezero_core` errors → `PlatformError`. Route `RuntimeServices::config_store()/secret_store()` reads through it; keep `put`/`create`/`delete` delegating to the existing write impl (per D6-a). Reads use `block_on` on the async EdgeZero handle (mirrors `storage/kv_store.rs`). + +- [ ] **Step 4: Run the test to verify it passes** + +Run: `cargo test-fastly runtime_services_config_read_resolves_via_edgezero_registry` +Expected: PASS. + +- [ ] **Step 5: Commit** + +```bash +git add crates/trusted-server-core/src/platform/ +git commit -m "Resolve RuntimeServices config/secret reads via EdgeZero registries" +``` + +--- + +## Task 4: Wire registries in the standard adapters (Axum, Cloudflare, Spin) + +These adapters use `dispatch_with_registries`; ensure `Config`/`Secret` registries are built from `[stores.*]` metadata and reach `build_runtime_services`. + +**Files:** +- Modify: `crates/trusted-server-adapter-axum/src/platform.rs`, `.../adapter-cloudflare/src/platform.rs`, `.../adapter-spin/src/platform.rs` +- Test: each adapter's route tests + +**Interfaces:** +- Consumes: Task 3's read bridge; `StoresMetadata` from `Hooks::stores()`. +- Produces: `RuntimeServices` on these adapters whose reads flow through EdgeZero registries. + +- [ ] **Step 1: Write a failing route test (per adapter) that reads a config/secret value through a handler** + +For Axum, add a test hitting a route whose handler reads a known config value; assert 200 + expected body. (Mirror existing `adapter-axum` route tests.) + +- [ ] **Step 2: Run to verify it fails** + +Run: `cargo test-axum ` +Expected: FAIL. + +- [ ] **Step 3: Build `Config`/`Secret` registries in each adapter's `build_runtime_services`** + +Use the EdgeZero registries the adapter's `dispatch_with_registries` already inserts into request extensions; construct `RuntimeServices` via Task 3's bridge instead of the old `Platform*Store` read impls. + +- [ ] **Step 4: Run per-adapter tests to verify pass** + +Run: `cargo test-axum && cargo test-cloudflare && cargo test-spin` +Expected: PASS. + +- [ ] **Step 5: Commit** + +```bash +git add crates/trusted-server-adapter-axum crates/trusted-server-adapter-cloudflare crates/trusted-server-adapter-spin +git commit -m "Wire EdgeZero config/secret registries in Axum, Cloudflare, and Spin adapters" +``` + +--- + +## Task 5: Fastly-specific registry injection into the custom `oneshot` path + +Fastly bypasses `dispatch_with_registries` (inserts only a `ConfigStoreHandle` before `app.router().oneshot()`). Add explicit `Config`/`Secret`/`Kv` registry construction + insertion compatible with that path. + +**Files:** +- Modify: `crates/trusted-server-adapter-fastly/src/main.rs` (the `oneshot` dispatch block, ~L470–490) +- Modify: `crates/trusted-server-adapter-fastly/src/platform.rs` (registry builders) +- Test: `crates/trusted-server-adapter-fastly/src/route_tests.rs` + +**Interfaces:** +- Consumes: Task 3 bridge; the same registry builders the standard path uses. +- Produces: Fastly requests whose extensions carry `ConfigRegistry`/`SecretRegistry`/`KvRegistry`, resolvable by `RuntimeServices`. + +- [ ] **Step 1: Write a failing Fastly route test** that exercises a handler reading a config value and asserts the EdgeZero-backed value is returned. + +Run: `cargo test-fastly ` → Expected: FAIL. + +- [ ] **Step 2: Build + insert the registries before `oneshot`** + +In `main.rs`, replace the lone `core_req.extensions_mut().insert(config_store)` with construction of `ConfigRegistry`/`SecretRegistry`/`KvRegistry` (from `[stores.*]` metadata + `EnvConfig`) and insert each into `core_req.extensions_mut()`, preserving the existing `client_info`/`device_signals` inserts. + +- [ ] **Step 3: Run to verify pass** + +Run: `cargo test-fastly ` → Expected: PASS. + +- [ ] **Step 4: Full Fastly suite + parity + commit** + +Run: `cargo test-fastly && cargo test --manifest-path crates/trusted-server-integration-tests/Cargo.toml --test parity` +Expected: PASS. +```bash +git add crates/trusted-server-adapter-fastly +git commit -m "Inject EdgeZero registries into the Fastly custom oneshot dispatch path" +``` + +--- + +## Task 6: Delete the duplicated Fastly chunk resolver + +`settings_data.rs`'s `FastlyChunkPointer` resolver duplicates EdgeZero's `FastlyConfigStore` chunk handling. With reads flowing through EdgeZero (Task 3–5), collapse `get_settings_from_config_store` to a plain `ConfigStore::get` + `settings_from_config_blob`. + +**Files:** +- Modify: `crates/trusted-server-core/src/settings_data.rs` +- Test: `crates/trusted-server-core/src/settings_data.rs` (`#[cfg(test)]`) + +**Interfaces:** +- Consumes: EdgeZero-backed config read (Task 3). +- Produces: a chunk-free `get_settings_from_config_store`. + +- [ ] **Step 1: Confirm the existing multi-chunk test now passes against the EdgeZero-resolved value** (EdgeZero's `FastlyConfigStore` reassembles chunks). If a `settings_data` test asserts the local resolver's behavior, rewrite it to assert the blob is read + parsed, not chunk-reassembled. + +- [ ] **Step 2: Delete `FastlyChunkPointer`, `FastlyChunkRef`, `resolve_fastly_chunk_pointer`, `sha256_hex`, and the chunk constants**; collapse `get_settings_from_config_store` to `ConfigStore::get` + `settings_from_config_blob`. + +- [ ] **Step 3: Run tests to verify pass** + +Run: `cargo test-fastly && cargo test-axum && cargo test-cloudflare && cargo test-spin` +Expected: PASS. + +- [ ] **Step 4: Commit** + +```bash +git add crates/trusted-server-core/src/settings_data.rs +git commit -m "Delete duplicated Fastly config-chunk resolver; read via EdgeZero FastlyConfigStore" +``` + +--- + +## Task 7: Retire the per-adapter config/secret READ impls; keep the write path (D6-a) + +Delete the four `platform.rs` config/secret **read** implementations now that reads flow through EdgeZero. Keep the write-capable path (`management_api.rs` + `put`/`create`/`delete`) per D6-a, or execute D6-b/c if Task 1 chose it. + +**Files:** +- Modify: `crates/trusted-server-adapter-{fastly,axum,cloudflare,spin}/src/platform.rs` +- Modify (only if D6-b chosen): `crates/trusted-server-adapter-fastly/src/management_api.rs`, request-signing endpoints + +**Interfaces:** +- Consumes: Tasks 3–5. +- Produces: adapters with no bespoke config/secret **read** impls; write path intact per D6. + +- [ ] **Step 1: Delete the config/secret read impls** (`FastlyPlatformConfigStore::get`, `AxumPlatformConfigStore`, `NoopConfigStore`, Cloudflare/Spin equivalents, and secret read impls) that are now unused after Tasks 4–5. Keep the write impls (D6-a). + +- [ ] **Step 2: If Task 1 chose D6-b** (move rotation to ops/CLI): delete `management_api.rs` and the `put`/`create`/`delete` trait methods; move `KeyRotationManager` writes behind a `ts keys` CLI command using EdgeZero provisioning; make the runtime endpoints return `501`/redirect per the ops decision. **Otherwise skip this step.** + +- [ ] **Step 3: Run the full CI gate** + +Run: `cargo fmt --all -- --check && cargo clippy-fastly && cargo clippy-axum && cargo clippy-cloudflare && cargo clippy-spin-native && cargo clippy-spin-wasm && cargo test-fastly && cargo test-axum && cargo test-cloudflare && cargo test-spin && cargo test --manifest-path crates/trusted-server-integration-tests/Cargo.toml --test parity` +Expected: PASS. Key rotation/delete still works (per D6 resolution). + +- [ ] **Step 4: Commit** + +```bash +git add crates/trusted-server-adapter-* +git commit -m "Retire per-adapter config/secret read impls; reads flow through EdgeZero registries" +``` + +--- + +## Task 1 Output (filled in during execution) + +_D5 store-id map and D6 decision are recorded here by Task 1 before Tasks 2+ run._ + +--- + +## Notes on scope and gating + +- **Blocked-until-decided:** Tasks 3–7 assume D6-a (keep the write path). If Task 1 selects D6-b, Task 7 Step 2 activates and the `management_api.rs` deletion (spec ledger, conditional) proceeds; if D6-c, add an upstream-EdgeZero prerequisite task before Task 7. +- **Not in this phase:** `RuntimeServices` full removal (Phase 4), `include_str!` config removal on Cloudflare/Spin (Phase 2), `from_toml_and_env`/`config`-dep removal (Phase 2), `Redacted` / secret externalization (Phase 3). +- **No dependency on edgezero #305** — Phase 1 uses only the already-shipped EdgeZero store APIs. From dc5b605dbc269528df4ba80cdba92da99c91f4f9 Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Thu, 2 Jul 2026 21:54:47 -0700 Subject: [PATCH 07/30] Revise Phase 1 plan to be execution-ready (D6-a locked) - Lock plan to D6-a; D6-b/c stop after Task 1 (separate plan) - Kind-aware store-id reconciliation (kv/config/secrets), incl. ec_store as KV - Composite read/write store bridge with a write-delegation test - New task: migrate Fastly/Axum BOOT config read to EdgeZero before deleting impls - Local Fastly registry builders (EdgeZero builders are pub(crate)); R11 - Concrete named tests, files, routes, fixtures (no placeholders) - Sync spec D5 (kind-aware), Phase 1 boot-read, R11 --- ...07-02-edgezero-store-registry-migration.md | 333 ++++++++++-------- ...26-07-02-edgezero-full-migration-design.md | 9 +- 2 files changed, 186 insertions(+), 156 deletions(-) diff --git a/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md b/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md index 20db0054..e4b0384c 100644 --- a/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md +++ b/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md @@ -1,338 +1,367 @@ -# EdgeZero Store-Registry Migration (Phase 1) Implementation Plan +# EdgeZero Store-Registry Migration (Phase 1, D6-a) Implementation Plan > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. -**Goal:** Route trusted-server's runtime **config and secret reads** through EdgeZero's `ConfigRegistry`/`SecretRegistry` (as KV already is), reconcile all logical store ids with `edgezero.toml`, and delete the duplicated Fastly chunk resolver — without breaking the runtime **write** path (key rotation) or Fastly's custom dispatch. +**Goal:** Route trusted-server's runtime **and boot-time** config/secret **reads** through EdgeZero stores/registries (as KV already is), reconcile every logical store id (kv/config/secrets) with `edgezero.toml`, and delete the duplicated Fastly chunk resolver — while **keeping** the runtime **write** path (key rotation) intact via a composite store (decision **D6-a**). -**Architecture:** trusted-server core reads stores through the bespoke `PlatformConfigStore`/`PlatformSecretStore` traits (read `get`/`get_string` + write `put`/`create`/`delete`), surfaced via `RuntimeServices`. EdgeZero's `ConfigStore`/`SecretStore` are **read-only**; the per-request `ConfigRegistry`/`SecretRegistry` live in request extensions. This phase makes core reads resolve from those registries while **keeping** the write-capable path until D6 decides its fate. Every adapter must wire the registries, including Fastly's custom `oneshot` path. +**Architecture:** trusted-server core reads/writes stores through the bespoke `PlatformConfigStore`/`PlatformSecretStore` traits (each mixes read `get`/`get_string` + write `put`/`create`/`delete`), surfaced via `RuntimeServices` (one trait object per kind). EdgeZero's `ConfigStore`/`SecretStore` are **read-only**; per-request `ConfigRegistry`/`SecretRegistry` live in request extensions. This phase introduces a **composite store** whose *reads* resolve from EdgeZero and whose *writes* delegate to the existing management-API-backed impl, migrates the Fastly/Axum **boot** config read to EdgeZero, and adds **local** registry builders for Fastly's custom `oneshot` dispatch (EdgeZero's builders are `pub(crate)`). -**Tech Stack:** Rust 2024, `error-stack` (`Report`), EdgeZero (`edgezero-core` git dep), Viceroy (Fastly test sim), `cargo test-{fastly,axum,cloudflare,spin}`. +**Tech Stack:** Rust 2024, toolchain 1.95.0, `error-stack` `Report`, EdgeZero (`edgezero-core`/`edgezero-adapter-fastly` git dep), Viceroy, `cargo test-{fastly,axum,cloudflare,spin}`. -**Spec:** `docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md` §5 Phase 1, decisions D5 + D6, §4a. +**Spec:** `docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md` §5 Phase 1, D5, D6, §4a. ## Global Constraints -- Rust **2024 edition**, toolchain **1.95.0** (`rust-toolchain.toml`); WASM target `wasm32-wasip1`. -- Errors: `error-stack` `Report` only (no `anyhow` outside the Spin entry point); `derive_more::Display` for error types; import `Error` from `core::error::`. -- No `unwrap()` in production; `expect("should …")`. No `println!`/`eprintln!`; use `log` macros. -- No wildcard imports (except `use super::*` in `#[cfg(test)]`). No local imports inside functions. -- Commit style: sentence case, imperative, no semantic prefixes, no `Co-Authored-By`/AI footers. -- CI gate (must pass before PR): `cargo fmt --all -- --check`; `cargo clippy-{fastly,axum,cloudflare,spin-native,spin-wasm}`; `cargo test-{fastly,axum,cloudflare,spin}`; `cargo test --manifest-path crates/trusted-server-integration-tests/Cargo.toml --test parity`. -- **Every phase step leaves all four adapters building and green.** -- **EdgeZero `ConfigStore`/`SecretStore` are read-only.** Never assume a runtime write API on them. -- **Registry lookup is strict:** an unknown logical id yields `None`. Every id any config field names at runtime must be declared in `edgezero.toml` `[stores.config]`/`[stores.secrets]` `ids`. +- Rust **2024 edition**, toolchain **1.95.0**; WASM target `wasm32-wasip1`. +- Errors: `error-stack` `Report` only (no `anyhow` outside the Spin entry point); `derive_more::Display`; import `Error` from `core::error::`. +- No `unwrap()` in production (`expect("should …")`); no `println!`/`eprintln!` (use `log`). +- No wildcard imports (except `use super::*` in `#[cfg(test)]`); no imports inside functions. +- Commits: sentence case, imperative, no semantic prefixes, no `Co-Authored-By`/AI footers. +- CI gate before PR: `cargo fmt --all -- --check`; `cargo clippy-{fastly,axum,cloudflare,spin-native,spin-wasm}`; `cargo test-{fastly,axum,cloudflare,spin}`; `cargo test --manifest-path crates/trusted-server-integration-tests/Cargo.toml --test parity`. +- **Every task leaves all four adapters building and green.** +- **EdgeZero `ConfigStore`/`SecretStore` are read-only.** Runtime writes stay on the management path (D6-a). +- **Registry lookup is strict:** an unknown logical id yields `None`. Every id any config field names — in **any** kind (kv/config/secrets) — must be declared in `edgezero.toml`. +- **This plan is D6-a-locked.** If Task 1 selects D6-b (move key rotation to ops/CLI) or D6-c (upstream write API), **stop after Task 1** and write a separate plan — those change the admin API surface and are out of Phase 1 scope. --- -## Task 1: Store-capability inventory + D5/D6 decision gate (no code deletion) +## Task 1: Kind-aware store inventory + confirm D6-a (decision gate, no deletions) -This task produces decisions, not deletions. Its deliverable is a written **decision record** appended to this plan (section "Task 1 Output") that Tasks 2+ depend on. Per spec review, Phase 1 must not begin with deletions. +Deliverable: a **decision record** appended to "Task 1 Output" that Tasks 2+ consume. No code is deleted here. **Files:** -- Modify (append decision record): `docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md` -- Read-only inventory across: `crates/trusted-server-core/src/**`, `crates/trusted-server-adapter-*/src/**`, `edgezero.toml`, `trusted-server.example.toml`, `crates/trusted-server-integration-tests/fixtures/**` +- Modify (append record): this plan file. +- Read-only inventory: `crates/trusted-server-core/src/**`, `crates/trusted-server-adapter-*/src/**`, `edgezero.toml`, `trusted-server.example.toml`, `crates/trusted-server-integration-tests/fixtures/**`. **Interfaces:** -- Produces: the **store-id map** (logical id → platform name → declared-in-edgezero.toml?) and the **write-site list** (every runtime `put`/`create`/`delete` call), consumed by Tasks 2, 3, 6. +- Produces: the **kind-partitioned store-id map** (`{kv, config, secrets}` → each logical id → platform name → declared?) and the **write-site list**, consumed by Tasks 2, 3, 8. -- [ ] **Step 1: Enumerate every logical store id referenced at runtime** +- [ ] **Step 1: Enumerate store ids by kind** Run: ```bash cd /Users/ag/projects/iab/trusted-server/.claude/worktrees/edgezero-migration-spec -rg -n 'config_store_id|secret_store_id|secret_store\s*=|config_store\s*=|StoreName::from|StoreId::from|"app_config"|"secrets"|"jwks|ts_secrets|signing_keys|"api-keys"' \ - crates/trusted-server-core crates/trusted-server-adapter-* trusted-server.example.toml \ - crates/trusted-server-integration-tests/fixtures +# KV ids +rg -n 'ec_store|consent_store|creative_store|ec_identity_store|counter_store|opid_store' crates/trusted-server-core/src/settings.rs trusted-server.example.toml +# config ids +rg -n 'config_store_id|jwks|JWKS_CONFIG_STORE_NAME|"app_config"|config_store\s*=' crates/trusted-server-core trusted-server.example.toml +# secret ids +rg -n 'secret_store_id|secret_store\s*=|"secrets"|ts_secrets|signing_keys|SIGNING_SECRET_STORE_NAME' crates/trusted-server-core trusted-server.example.toml rg -n '\[stores\.' edgezero.toml ``` -Expected: a list of ids including at least `app_config`, `secrets`, `signing_keys`, JWKS config-list store, DataDome `ts_secrets`, S3 secret store — versus `edgezero.toml` declaring only `trusted_server_config`/`trusted_server_kv`/`trusted_server_secrets`. +Expected: KV ids include `ec_identity_store` (from `ec.ec_store`), consent/creative/counter/opid stores; config ids include `app_config` + the JWKS store (`JWKS_CONFIG_STORE_NAME`); secret ids include `secrets`, `signing_keys`, DataDome `ts_secrets`, the S3 secret store — versus `edgezero.toml` declaring only `trusted_server_kv`/`trusted_server_config`/`trusted_server_secrets`. -- [ ] **Step 2: Enumerate every runtime store WRITE call site** +- [ ] **Step 2: Enumerate runtime WRITE sites** Run: ```bash rg -n '\.config_store\(\)\.(put|delete)|\.secret_store\(\)\.(create|delete)' crates/trusted-server-core ``` -Expected: the `KeyRotationManager` write sites in `crates/trusted-server-core/src/request_signing/rotation.rs` (`store_private_key`, `store_public_jwk`, `deactivate_key`, `delete_key`) — the only runtime writers. Confirm no other runtime writers exist. +Expected: only `KeyRotationManager` in `crates/trusted-server-core/src/request_signing/rotation.rs` (`store_private_key`, `store_public_jwk`, `deactivate_key`, `delete_key`). Confirm no other runtime writers. -- [ ] **Step 3: Record the D5 decision (store-id reconciliation map)** +- [ ] **Step 3: Record the kind-partitioned D5 map** -Append to "Task 1 Output" a table: each runtime logical id → chosen resolution (declare a new id in `edgezero.toml`, or collapse onto `trusted_server_config`/`trusted_server_secrets`) → the `EDGEZERO__STORES______NAME` mapping. Default recommendation from the spec (D5): app-config blob in `trusted_server_config` key `app_config`; declare JWKS as its own config id; collapse `secrets`→`trusted_server_secrets`; keep DataDome/S3 as declared secret ids. Confirm or adjust against Step 1's actual list. +Append a table to "Task 1 Output": for each `{kv|config|secrets}` id → resolution (declare in `edgezero.toml`, or collapse onto the kind's default) → `EDGEZERO__STORES______NAME`. Spec default: app-config blob → config id `trusted_server_config` key `app_config`; JWKS → its own config id; `ec_identity_store` → kv id; collapse `secrets`→`trusted_server_secrets` where identical; declare DataDome/S3/signing as distinct secret ids. -- [ ] **Step 4: Record the D6 decision (runtime write path)** +- [ ] **Step 4: Confirm D6-a (or STOP)** -Append to "Task 1 Output" the chosen option: **(a)** keep a write-capable admin abstraction (`management_api.rs` + the `put`/`create`/`delete` trait methods stay for the admin path); **(b)** move rotate/deactivate/delete to an ops/CLI command; or **(c)** upstream an EdgeZero write API. Spec recommendation: **(a) as the Phase 1 interim** (unblocks the read migration without changing the admin surface), with (b) as the target end-state pending an ops decision. Record which is chosen; Tasks 3–6 assume **(a)** unless this step records otherwise. +Confirm this phase keeps the write-capable composite (D6-a). Record it. **If the team instead chooses D6-b/c, stop here** and open a separate plan (`…-key-rotation-ops-migration.md`); do not proceed to Task 2. -- [ ] **Step 5: Commit the decision record** +- [ ] **Step 5: Commit the record** ```bash git add docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md -git commit -m "Record Phase 1 store-id map and runtime-write decision (D5, D6)" +git commit -m "Record Phase 1 kind-aware store-id map and confirm D6-a" ``` --- -## Task 2: Declare all runtime store ids in `edgezero.toml` + reconcile config fields/fixtures - -Implements the D5 map from Task 1. Makes every referenced id resolvable so strict registry lookup never returns `None`. +## Task 2: Declare all store ids (kv/config/secrets) in `edgezero.toml` + reconcile fields/fixtures **Files:** -- Modify: `edgezero.toml` (`[stores.config]`/`[stores.secrets]` `ids`) -- Modify: `trusted-server.example.toml` (`request_signing.config_store_id`, `secret_store_id`, and any other store-id fields to match the D5 map) -- Modify: `crates/trusted-server-integration-tests/fixtures/configs/trusted-server.integration.toml` -- Test: `crates/trusted-server-core/src/settings.rs` (`#[cfg(test)]`) — assert declared ids cover the config's referenced ids +- Modify: `edgezero.toml` (`[stores.kv]`, `[stores.config]`, `[stores.secrets]` `ids`) +- Modify: `trusted-server.example.toml`, `crates/trusted-server-integration-tests/fixtures/configs/trusted-server.integration.toml` +- Test: `crates/trusted-server-core/src/settings.rs` (`#[cfg(test)]`) **Interfaces:** -- Consumes: Task 1 store-id map. -- Produces: an `edgezero.toml` whose `[stores.*].ids` is a superset of every id named by `Settings`. +- Consumes: Task 1 map. +- Produces: `Settings::referenced_store_ids_by_kind() -> ReferencedStoreIds { kv: BTreeSet, config: BTreeSet, secrets: BTreeSet }`; an `edgezero.toml` whose per-kind `ids` are supersets. - [ ] **Step 1: Write the failing test** -Add to `crates/trusted-server-core/src/settings.rs` under `#[cfg(test)]`: +Add to `settings.rs` under `#[cfg(test)]`: ```rust #[test] -fn every_referenced_store_id_is_declared() { - // Arrange: parse the example config and the manifest's declared ids. +fn every_referenced_store_id_is_declared_by_kind() { let settings = Settings::from_toml(include_str!("../../../trusted-server.example.toml")) .expect("should parse example config"); - let declared = declared_store_ids_from_manifest(); // helper reads edgezero.toml - // Act: collect the store ids the settings reference. - let referenced = settings.referenced_store_ids(); - // Assert: manifest declares every referenced id. - for id in &referenced { - assert!( - declared.contains(id), - "store id `{id}` referenced by Settings is not declared in edgezero.toml", - ); + let referenced = settings.referenced_store_ids_by_kind(); + let declared = declared_store_ids_by_kind_from_manifest(); // reads edgezero.toml + for (kind, ids) in [ + ("kv", &referenced.kv), + ("config", &referenced.config), + ("secrets", &referenced.secrets), + ] { + let declared_for_kind = declared.for_kind(kind); + for id in ids { + assert!( + declared_for_kind.contains(id), + "{kind} store id `{id}` referenced by Settings is not declared in edgezero.toml", + ); + } } } ``` -- [ ] **Step 2: Run the test to verify it fails** - -Run: `cargo test-fastly every_referenced_store_id_is_declared` -Expected: FAIL — `app_config`/`secrets`/JWKS ids referenced but not declared. +- [ ] **Step 2: Run to verify it fails** -- [ ] **Step 3: Add `Settings::referenced_store_ids()` + the manifest helper** +Run: `cargo test-fastly every_referenced_store_id_is_declared_by_kind` +Expected: FAIL — `ec_identity_store` (kv), `app_config`/JWKS (config), `secrets`/`ts_secrets` (secrets) referenced but not declared. -In `settings.rs`, implement `referenced_store_ids(&self) -> std::collections::BTreeSet` returning every `*_store_id` / `*_store` value (request-signing config+secret ids, DataDome, S3, EC, consent). Add a test-only `declared_store_ids_from_manifest()` that parses `edgezero.toml`'s `[stores.config]`/`[stores.secrets]` `ids`. +- [ ] **Step 3: Implement `referenced_store_ids_by_kind()` + manifest helper** -- [ ] **Step 4: Update `edgezero.toml` + config fields per the D5 map** +Add the `ReferencedStoreIds` struct + method returning KV ids (`ec.ec_store`, consent/creative/counter/opid), config ids (`request_signing.config_store_id`, `JWKS_CONFIG_STORE_NAME`, app-config), secret ids (`request_signing.secret_store_id`, DataDome, S3, `SIGNING_SECRET_STORE_NAME`). Add test-only `declared_store_ids_by_kind_from_manifest()` parsing `edgezero.toml`. -Edit `edgezero.toml` `[stores.config].ids` / `[stores.secrets].ids` to declare every id from the map; update `trusted-server.example.toml` and the integration fixture so their store-id fields use declared ids. +- [ ] **Step 4: Update `edgezero.toml` + config fields/fixtures per the Task 1 map** -- [ ] **Step 5: Run the test to verify it passes** +- [ ] **Step 5: Run to verify pass** -Run: `cargo test-fastly every_referenced_store_id_is_declared` +Run: `cargo test-fastly every_referenced_store_id_is_declared_by_kind` Expected: PASS. -- [ ] **Step 6: Run all adapter tests + commit** +- [ ] **Step 6: Full adapter tests + commit** Run: `cargo test-fastly && cargo test-axum && cargo test-cloudflare && cargo test-spin` -Expected: PASS. ```bash git add edgezero.toml trusted-server.example.toml crates/trusted-server-integration-tests/fixtures crates/trusted-server-core/src/settings.rs -git commit -m "Declare all runtime store ids in edgezero.toml and reconcile config fields" +git commit -m "Declare kv/config/secret store ids in edgezero.toml and reconcile config fields" ``` --- -## Task 3: Bridge `RuntimeServices` config/secret READS to EdgeZero registries +## Task 3: Composite read/write store bridge (reads → EdgeZero, writes → management path) -Make `RuntimeServices::config_store()`/`secret_store()` **reads** resolve from the request's `ConfigRegistry`/`SecretRegistry` instead of the per-adapter `Platform*Store` read impls. The write methods (`put`/`create`/`delete`) stay routed to the existing path (D6 option a). +Concrete D6-a mechanism. Introduce a composite that implements `PlatformConfigStore`/`PlatformSecretStore` by routing **reads** to an EdgeZero-backed handle and **writes** (`put`/`create`/`delete`) to the existing management-API-backed impl (`inner_writer`). This preserves `KeyRotationManager` writes with zero call-site changes. **Files:** -- Modify: `crates/trusted-server-core/src/platform/types.rs` (`RuntimeServices` build/accessors) -- Modify: `crates/trusted-server-core/src/platform/traits.rs` (split read vs write if needed) -- Test: `crates/trusted-server-core/src/platform/types.rs` (`#[cfg(test)]`) +- Create: `crates/trusted-server-core/src/platform/composite.rs` (`CompositeConfigStore`, `CompositeSecretStore`) +- Modify: `crates/trusted-server-core/src/platform/mod.rs` (export composite) +- Test: `crates/trusted-server-core/src/platform/composite.rs` (`#[cfg(test)]`) **Interfaces:** -- Consumes: EdgeZero `ConfigRegistry`/`SecretRegistry` (from request extensions), `edgezero_core::config_store::ConfigStoreHandle`, `edgezero_core::store_registry::BoundSecretStore`. -- Produces: a `RuntimeServices` whose `config_store().get(name,key)` / `secret_store().get_string(name,key)` read through EdgeZero; write methods unchanged. +- Consumes: `edgezero_core::config_store::ConfigStoreHandle`, `edgezero_core::store_registry::BoundSecretStore`, an `Arc`/`Arc` writer. +- Produces: + - `CompositeConfigStore::new(reader: ConfigStoreHandle, writer: Arc) -> Self` implementing `PlatformConfigStore` (get→reader, put/delete→writer). + - `CompositeSecretStore::new(reader: BoundSecretStore, writer: Arc) -> Self` implementing `PlatformSecretStore` (get_bytes→reader, create/delete→writer). -- [ ] **Step 1: Write the failing test** (config read resolves via an EdgeZero-backed registry) +- [ ] **Step 1: Write the failing test — read via EdgeZero, write delegates to writer** ```rust #[test] -fn runtime_services_config_read_resolves_via_edgezero_registry() { - // Arrange: a RuntimeServices built from an EdgeZero ConfigRegistry with a fixed value. - let services = runtime_services_with_config_registry("trusted_server_config", "greeting", "hi"); - // Act - let value = services - .config_store() +fn composite_config_reads_edgezero_and_writes_delegate() { + // Arrange: an EdgeZero reader returning "hi"; a recording writer. + let reader = fixed_config_handle("greeting", "hi"); + let writer = Arc::new(RecordingConfigWriter::default()); + let composite = CompositeConfigStore::new(reader, writer.clone()); + // Act: read + write. + let read = composite .get(&StoreName::from("trusted_server_config"), "greeting") - .expect("should read via edgezero registry"); + .expect("should read via EdgeZero reader"); + composite + .put(&StoreId::from("trusted_server_config"), "current-kid", "kid-1") + .expect("should delegate write"); // Assert - assert_eq!(value, "hi", "should return the EdgeZero-backed value"); + assert_eq!(read, "hi", "read should come from the EdgeZero reader"); + assert_eq!( + writer.puts.lock().expect("lock").as_slice(), + &[("current-kid".to_owned(), "kid-1".to_owned())], + "write should delegate to the management-path writer", + ); } ``` -- [ ] **Step 2: Run the test to verify it fails** +- [ ] **Step 2: Run to verify it fails** -Run: `cargo test-fastly runtime_services_config_read_resolves_via_edgezero_registry` -Expected: FAIL (no such constructor / still uses the old read path). +Run: `cargo test-fastly composite_config_reads_edgezero_and_writes_delegate` +Expected: FAIL (module does not exist). -- [ ] **Step 3: Implement the read bridge** +- [ ] **Step 3: Implement `composite.rs`** -Add an EdgeZero-backed `PlatformConfigStore`/`PlatformSecretStore` read adapter in `platform/` that wraps a `ConfigStoreHandle`/`BoundSecretStore` resolved from the registry, mapping `edgezero_core` errors → `PlatformError`. Route `RuntimeServices::config_store()/secret_store()` reads through it; keep `put`/`create`/`delete` delegating to the existing write impl (per D6-a). Reads use `block_on` on the async EdgeZero handle (mirrors `storage/kv_store.rs`). +Reads call the EdgeZero handle via `futures::executor::block_on` (mirror `storage/kv_store.rs`), mapping `edgezero_core` errors → `PlatformError`. Writes forward to `writer`. Repeat for `CompositeSecretStore`. Add `RecordingConfigWriter`/`fixed_config_handle` test helpers. -- [ ] **Step 4: Run the test to verify it passes** +- [ ] **Step 4: Run to verify it passes** -Run: `cargo test-fastly runtime_services_config_read_resolves_via_edgezero_registry` +Run: `cargo test-fastly composite_config_reads_edgezero_and_writes_delegate` Expected: PASS. - [ ] **Step 5: Commit** ```bash git add crates/trusted-server-core/src/platform/ -git commit -m "Resolve RuntimeServices config/secret reads via EdgeZero registries" +git commit -m "Add composite config/secret store: EdgeZero reads, management-path writes" ``` --- -## Task 4: Wire registries in the standard adapters (Axum, Cloudflare, Spin) +## Task 4: Migrate Fastly + Axum BOOT config read to EdgeZero (before deleting bespoke impls) -These adapters use `dispatch_with_registries`; ensure `Config`/`Secret` registries are built from `[stores.*]` metadata and reach `build_runtime_services`. +`build_state()` loads `Settings` at boot via `get_settings_from_config_store(&FastlyPlatformConfigStore, …)` / `&AxumPlatformConfigStore` — **before** any request context. Migrate the boot read to an EdgeZero-backed boot reader so the bespoke impls can be deleted later (Task 8) without breaking boot. (P-BOOT option a for Fastly/Axum: `ConfigStore` opens at boot.) **Files:** -- Modify: `crates/trusted-server-adapter-axum/src/platform.rs`, `.../adapter-cloudflare/src/platform.rs`, `.../adapter-spin/src/platform.rs` -- Test: each adapter's route tests +- Modify: `crates/trusted-server-adapter-fastly/src/app.rs:161` (`load_settings_from_config_store`) +- Modify: `crates/trusted-server-adapter-axum/src/app.rs:54` (`build_state`) +- Modify: `crates/trusted-server-core/src/settings_data.rs` (accept an EdgeZero `ConfigStoreHandle` reader) +- Test: `crates/trusted-server-adapter-fastly/src/app.rs` (`#[cfg(test)]`), Axum equivalent **Interfaces:** -- Consumes: Task 3's read bridge; `StoresMetadata` from `Hooks::stores()`. -- Produces: `RuntimeServices` on these adapters whose reads flow through EdgeZero registries. +- Consumes: `edgezero_core` Fastly/Axum `ConfigStore` open primitives; Task 3 nothing (boot read is direct). +- Produces: `get_settings_from_config_store` taking `&ConfigStoreHandle` (EdgeZero) instead of `&dyn PlatformConfigStore`. -- [ ] **Step 1: Write a failing route test (per adapter) that reads a config/secret value through a handler** +- [ ] **Step 1: Write a failing boot test (Fastly)** asserting `load_settings_from_config_store()` returns parsed `Settings` when the EdgeZero Fastly config store holds the blob (use the EdgeZero test store / a seeded local config store). -For Axum, add a test hitting a route whose handler reads a known config value; assert 200 + expected body. (Mirror existing `adapter-axum` route tests.) +Run: `cargo test-fastly boot_config_loads_via_edgezero` → Expected: FAIL. -- [ ] **Step 2: Run to verify it fails** +- [ ] **Step 2: Re-type `get_settings_from_config_store`** to take an EdgeZero `ConfigStoreHandle`; open the EdgeZero `FastlyConfigStore` at boot in `load_settings_from_config_store`, and the EdgeZero Axum config store in Axum `build_state`. + +- [ ] **Step 3: Run to verify pass** (Fastly + Axum) + +Run: `cargo test-fastly boot_config_loads_via_edgezero && cargo test-axum boot_config_loads_via_edgezero` +Expected: PASS. + +- [ ] **Step 4: Commit** + +```bash +git add crates/trusted-server-adapter-fastly/src/app.rs crates/trusted-server-adapter-axum/src/app.rs crates/trusted-server-core/src/settings_data.rs +git commit -m "Load boot config via EdgeZero config store on Fastly and Axum" +``` + +--- -Run: `cargo test-axum ` -Expected: FAIL. +## Task 5: Wire request registries in Axum, Cloudflare, Spin; RuntimeServices uses the composite -- [ ] **Step 3: Build `Config`/`Secret` registries in each adapter's `build_runtime_services`** +These adapters use EdgeZero `dispatch_with_registries` (registries already inserted into extensions). Build `RuntimeServices` config/secret from `CompositeConfigStore`/`CompositeSecretStore` (reader from the request registry; writer = the existing per-adapter write impl). -Use the EdgeZero registries the adapter's `dispatch_with_registries` already inserts into request extensions; construct `RuntimeServices` via Task 3's bridge instead of the old `Platform*Store` read impls. +**Files:** +- Modify: `crates/trusted-server-adapter-{axum,cloudflare,spin}/src/platform.rs` (`build_runtime_services`) +- Test: `crates/trusted-server-adapter-axum/src/app.rs` route tests (+ cloudflare/spin equivalents) + +**Interfaces:** +- Consumes: Task 3 composite; `ConfigRegistry`/`SecretRegistry` from request extensions. +- Produces: `RuntimeServices` whose reads flow through EdgeZero, writes through the composite writer. + +- [ ] **Step 1: Write a failing Axum route test** — `GET /.well-known/trusted-server.json` returns the JWKS/discovery document read from the config store. Name: `discovery_reads_jwks_from_edgezero_config_store` in the Axum app test module. Seed the Axum config registry with a JWKS entry fixture; assert `200` + the JWKS `kid` in the body. -- [ ] **Step 4: Run per-adapter tests to verify pass** +Run: `cargo test-axum discovery_reads_jwks_from_edgezero_config_store` → Expected: FAIL. + +- [ ] **Step 2: Build `RuntimeServices` via the composite** in each adapter's `build_runtime_services`, resolving the reader from the request `ConfigRegistry`/`SecretRegistry` and keeping the existing writer. + +- [ ] **Step 3: Run to verify pass** (all three) Run: `cargo test-axum && cargo test-cloudflare && cargo test-spin` Expected: PASS. -- [ ] **Step 5: Commit** +- [ ] **Step 4: Commit** ```bash git add crates/trusted-server-adapter-axum crates/trusted-server-adapter-cloudflare crates/trusted-server-adapter-spin -git commit -m "Wire EdgeZero config/secret registries in Axum, Cloudflare, and Spin adapters" +git commit -m "Build RuntimeServices via composite store in Axum, Cloudflare, and Spin" ``` --- -## Task 5: Fastly-specific registry injection into the custom `oneshot` path +## Task 6: Local Fastly registry builders + injection into the custom `oneshot` path -Fastly bypasses `dispatch_with_registries` (inserts only a `ConfigStoreHandle` before `app.router().oneshot()`). Add explicit `Config`/`Secret`/`Kv` registry construction + insertion compatible with that path. +EdgeZero's Fastly `dispatch_with_registries` and its registry builders are `pub(crate)` (verified in the pinned checkout), so trusted-server must build the registries **locally** and insert them into the request extensions before `app.router().oneshot()`. (Alternative: an upstream EdgeZero public builder — tracked as **R11**; not assumed here.) **Files:** -- Modify: `crates/trusted-server-adapter-fastly/src/main.rs` (the `oneshot` dispatch block, ~L470–490) -- Modify: `crates/trusted-server-adapter-fastly/src/platform.rs` (registry builders) -- Test: `crates/trusted-server-adapter-fastly/src/route_tests.rs` +- Create: `crates/trusted-server-adapter-fastly/src/registries.rs` (`build_config_registry`, `build_secret_registry`, `build_kv_registry`) +- Modify: `crates/trusted-server-adapter-fastly/src/main.rs:477` (the `oneshot` dispatch block) +- Test: `crates/trusted-server-adapter-fastly/src/registries.rs` (`#[cfg(test)]`) + a route test **Interfaces:** -- Consumes: Task 3 bridge; the same registry builders the standard path uses. -- Produces: Fastly requests whose extensions carry `ConfigRegistry`/`SecretRegistry`/`KvRegistry`, resolvable by `RuntimeServices`. +- Consumes: `StoresMetadata` (from `Hooks::stores()`), `EnvConfig`, EdgeZero `FastlyConfigStore`/`FastlyKvStore`/`FastlySecretStore` open primitives, `StoreRegistry::from_parts`. +- Produces: `build_config_registry(&StoresMetadata, &EnvConfig) -> ConfigRegistry` (+ `_secret_/_kv_` variants) matching EdgeZero's per-id name resolution (`EDGEZERO__STORES______NAME`). -- [ ] **Step 1: Write a failing Fastly route test** that exercises a handler reading a config value and asserts the EdgeZero-backed value is returned. +- [ ] **Step 1: Write a failing builder test** — `build_config_registry` yields a registry whose `default()` resolves and whose declared non-default ids resolve; unknown id → `None`. -Run: `cargo test-fastly ` → Expected: FAIL. +Run: `cargo test-fastly build_config_registry_resolves_declared_ids` → Expected: FAIL. -- [ ] **Step 2: Build + insert the registries before `oneshot`** +- [ ] **Step 2: Implement the three builders** in `registries.rs` (iterate `StoreMetadata.ids`, resolve platform name via `EnvConfig::store_name(kind, id)`, open the EdgeZero store, assemble `StoreRegistry::from_parts`). -In `main.rs`, replace the lone `core_req.extensions_mut().insert(config_store)` with construction of `ConfigRegistry`/`SecretRegistry`/`KvRegistry` (from `[stores.*]` metadata + `EnvConfig`) and insert each into `core_req.extensions_mut()`, preserving the existing `client_info`/`device_signals` inserts. +- [ ] **Step 3: Insert registries in the oneshot block** — replace the lone `core_req.extensions_mut().insert(config_store)` at `main.rs:477` with inserts of `ConfigRegistry`/`SecretRegistry`/`KvRegistry` (built via Step 2), preserving the existing `client_info`/`device_signals` inserts. -- [ ] **Step 3: Run to verify pass** +- [ ] **Step 4: Write a failing Fastly route test** — `GET /.well-known/trusted-server.json` via the EdgeZero `oneshot` path returns the JWKS doc read through the injected `ConfigRegistry`. Name: `oneshot_discovery_reads_jwks_via_registry` (mirror the `StubJwksConfigStore`/`JWKS_CONFIG_STORE_NAME` pattern in `route_tests.rs`, but drive the EdgeZero path, not `route_request`). -Run: `cargo test-fastly ` → Expected: PASS. +Run: `cargo test-fastly oneshot_discovery_reads_jwks_via_registry` → Expected: FAIL then PASS after Steps 2–3. -- [ ] **Step 4: Full Fastly suite + parity + commit** +- [ ] **Step 5: Fastly suite + parity + commit** Run: `cargo test-fastly && cargo test --manifest-path crates/trusted-server-integration-tests/Cargo.toml --test parity` -Expected: PASS. ```bash git add crates/trusted-server-adapter-fastly -git commit -m "Inject EdgeZero registries into the Fastly custom oneshot dispatch path" +git commit -m "Add local Fastly registry builders and inject them into the oneshot dispatch" ``` --- -## Task 6: Delete the duplicated Fastly chunk resolver +## Task 7: Delete the duplicated Fastly chunk resolver -`settings_data.rs`'s `FastlyChunkPointer` resolver duplicates EdgeZero's `FastlyConfigStore` chunk handling. With reads flowing through EdgeZero (Task 3–5), collapse `get_settings_from_config_store` to a plain `ConfigStore::get` + `settings_from_config_blob`. +With reads via EdgeZero (`FastlyConfigStore` reassembles chunks transparently), collapse `get_settings_from_config_store` and drop the local resolver. **Files:** - Modify: `crates/trusted-server-core/src/settings_data.rs` - Test: `crates/trusted-server-core/src/settings_data.rs` (`#[cfg(test)]`) -**Interfaces:** -- Consumes: EdgeZero-backed config read (Task 3). -- Produces: a chunk-free `get_settings_from_config_store`. - -- [ ] **Step 1: Confirm the existing multi-chunk test now passes against the EdgeZero-resolved value** (EdgeZero's `FastlyConfigStore` reassembles chunks). If a `settings_data` test asserts the local resolver's behavior, rewrite it to assert the blob is read + parsed, not chunk-reassembled. - -- [ ] **Step 2: Delete `FastlyChunkPointer`, `FastlyChunkRef`, `resolve_fastly_chunk_pointer`, `sha256_hex`, and the chunk constants**; collapse `get_settings_from_config_store` to `ConfigStore::get` + `settings_from_config_blob`. +- [ ] **Step 1: Rewrite/keep the settings_data test** to assert the blob is read + parsed (not locally chunk-reassembled) — EdgeZero owns reassembly now. -- [ ] **Step 3: Run tests to verify pass** +- [ ] **Step 2: Delete `FastlyChunkPointer`, `FastlyChunkRef`, `resolve_fastly_chunk_pointer`, `sha256_hex`, and the chunk constants;** collapse `get_settings_from_config_store` to `ConfigStore::get` + `settings_from_config_blob`. -Run: `cargo test-fastly && cargo test-axum && cargo test-cloudflare && cargo test-spin` -Expected: PASS. +- [ ] **Step 3: Run tests** — `cargo test-fastly && cargo test-axum && cargo test-cloudflare && cargo test-spin` → PASS. - [ ] **Step 4: Commit** ```bash git add crates/trusted-server-core/src/settings_data.rs -git commit -m "Delete duplicated Fastly config-chunk resolver; read via EdgeZero FastlyConfigStore" +git commit -m "Delete duplicated Fastly config-chunk resolver; rely on EdgeZero FastlyConfigStore" ``` --- -## Task 7: Retire the per-adapter config/secret READ impls; keep the write path (D6-a) +## Task 8: Retire per-adapter config/secret READ impls; keep the write path (D6-a) -Delete the four `platform.rs` config/secret **read** implementations now that reads flow through EdgeZero. Keep the write-capable path (`management_api.rs` + `put`/`create`/`delete`) per D6-a, or execute D6-b/c if Task 1 chose it. +Now that all reads (boot + request, all adapters) flow through EdgeZero, delete the config/secret **read** implementations. Keep the **write** methods + `management_api.rs` (D6-a). Update the legacy `route_tests.rs` stubs that construct `RuntimeServices` from bespoke read stores. **Files:** - Modify: `crates/trusted-server-adapter-{fastly,axum,cloudflare,spin}/src/platform.rs` -- Modify (only if D6-b chosen): `crates/trusted-server-adapter-fastly/src/management_api.rs`, request-signing endpoints - -**Interfaces:** -- Consumes: Tasks 3–5. -- Produces: adapters with no bespoke config/secret **read** impls; write path intact per D6. +- Modify: `crates/trusted-server-adapter-fastly/src/route_tests.rs` (update stubs to the composite/registry shape) -- [ ] **Step 1: Delete the config/secret read impls** (`FastlyPlatformConfigStore::get`, `AxumPlatformConfigStore`, `NoopConfigStore`, Cloudflare/Spin equivalents, and secret read impls) that are now unused after Tasks 4–5. Keep the write impls (D6-a). +- [ ] **Step 1: Delete the config/secret read impls** now unused after Tasks 4–6 (`FastlyPlatformConfigStore::get`, `AxumPlatformConfigStore`, `NoopConfigStore`, Cloudflare/Spin equivalents, secret read impls). Keep the write impls + `management_api.rs`. -- [ ] **Step 2: If Task 1 chose D6-b** (move rotation to ops/CLI): delete `management_api.rs` and the `put`/`create`/`delete` trait methods; move `KeyRotationManager` writes behind a `ts keys` CLI command using EdgeZero provisioning; make the runtime endpoints return `501`/redirect per the ops decision. **Otherwise skip this step.** +- [ ] **Step 2: Update `route_tests.rs`** — the stub stores (`StubJwksConfigStore`, etc.) and `RuntimeServices` construction move to the composite/registry shape (reader = a fixed EdgeZero handle, writer = a recording stub). Keep coverage of the write path (`put`/`create`/`delete`) so key-rotation delegation stays tested. -- [ ] **Step 3: Run the full CI gate** +- [ ] **Step 3: Full CI gate** Run: `cargo fmt --all -- --check && cargo clippy-fastly && cargo clippy-axum && cargo clippy-cloudflare && cargo clippy-spin-native && cargo clippy-spin-wasm && cargo test-fastly && cargo test-axum && cargo test-cloudflare && cargo test-spin && cargo test --manifest-path crates/trusted-server-integration-tests/Cargo.toml --test parity` -Expected: PASS. Key rotation/delete still works (per D6 resolution). +Expected: PASS. **Key rotation/delete still works** (composite writer path). - [ ] **Step 4: Commit** ```bash git add crates/trusted-server-adapter-* -git commit -m "Retire per-adapter config/secret read impls; reads flow through EdgeZero registries" +git commit -m "Retire per-adapter config/secret read impls; reads via EdgeZero, writes via composite" ``` --- ## Task 1 Output (filled in during execution) -_D5 store-id map and D6 decision are recorded here by Task 1 before Tasks 2+ run._ +_Kind-partitioned D5 map and the confirmed D6-a decision are recorded here by Task 1 before Tasks 2+ run._ --- -## Notes on scope and gating +## Scope, gating, and follow-ups -- **Blocked-until-decided:** Tasks 3–7 assume D6-a (keep the write path). If Task 1 selects D6-b, Task 7 Step 2 activates and the `management_api.rs` deletion (spec ledger, conditional) proceeds; if D6-c, add an upstream-EdgeZero prerequisite task before Task 7. -- **Not in this phase:** `RuntimeServices` full removal (Phase 4), `include_str!` config removal on Cloudflare/Spin (Phase 2), `from_toml_and_env`/`config`-dep removal (Phase 2), `Redacted` / secret externalization (Phase 3). -- **No dependency on edgezero #305** — Phase 1 uses only the already-shipped EdgeZero store APIs. +- **D6-a locked.** Runtime key-rotation writes stay on the management path via the composite. If Task 1 selects D6-b/c, this plan **stops after Task 1**; a separate `key-rotation-ops-migration` plan handles the admin-surface change. +- **R11 (open):** whether EdgeZero should expose a **public** registry-builder helper (so Fastly need not maintain local builders, Task 6). Decide with the edgezero maintainer; not assumed here. +- **Not in this phase:** `RuntimeServices` removal (Phase 4); Cloudflare/Spin `include_str!`/side-channel config removal (Phase 2); `from_toml_and_env` + `config` dep (Phase 2); `Redacted` / secret externalization (Phase 3); `management_api.rs` deletion (only under a future D6-b). +- **No dependency on edgezero #305** — Phase 1 uses shipped EdgeZero store APIs only. diff --git a/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md b/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md index a9a494a9..ffdca6c5 100644 --- a/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md +++ b/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md @@ -85,7 +85,7 @@ Phase 1 (stores) ──> Phase 2 (config) ──> Phase 3 (secrets) Phase 4 (e **D4 — One typed `Settings` as the AppConfig root.** Replace `TrustedServerAppConfig` (wrapper with empty `SECRET_FIELDS`) by deriving `AppConfig` directly on `Settings`, with `#[secret]` on the real secret fields (Phase 3). Removes a transitional indirection. -**D5 — Reconcile ALL runtime store ids with `edgezero.toml`.** Not just the app-config blob: EdgeZero's registry lookup is **strict** (unknown id → `None`), so every logical store id any config field or call site names at runtime must appear in `edgezero.toml` `[stores.config]`/`[stores.secrets]` `ids`. Today `edgezero.toml` declares only `trusted_server_config` / `trusted_server_secrets`, while config references `app_config`, `secrets`, JWKS/config-list stores, DataDome `ts_secrets`, S3 secret store, etc. Phase 1 must either (a) declare all these ids in `edgezero.toml` and map each to a platform store via `EDGEZERO__STORES__*__NAME`, or (b) collapse them onto the declared defaults and update every config field + fixture. Recommendation: the app-config blob lives in `trusted_server_config` under key `app_config` (`CONFIG_BLOB_KEY`); collapse incidental ids onto the declared defaults where semantically identical, and declare the genuinely-separate ones (e.g. JWKS store). The full id inventory is the Phase 1 plan's task 1. +**D5 — Reconcile ALL runtime store ids with `edgezero.toml`, kind by kind.** Not just the app-config blob, and **not just config/secrets**: EdgeZero's registry lookup is **strict** (unknown id → `None`), so every logical store id any config field or call site names at runtime must appear in `edgezero.toml` under the **correct kind** — `[stores.kv]`, `[stores.config]`, or `[stores.secrets]`. Reconciliation is kind-aware (`Settings::referenced_store_ids_by_kind()`): e.g. `ec.ec_store` (`ec_identity_store`), consent/creative/counter/opid stores are **KV** ids; `app_config` + the JWKS store are **config** ids; `secrets`, `signing_keys`, DataDome `ts_secrets`, the S3 secret store are **secret** ids. Today `edgezero.toml` declares only one id per kind. Phase 1 must either (a) declare all these ids in `edgezero.toml` and map each to a platform store via `EDGEZERO__STORES__*__NAME`, or (b) collapse them onto the declared defaults and update every config field + fixture. Recommendation: the app-config blob lives in `trusted_server_config` under key `app_config` (`CONFIG_BLOB_KEY`); collapse incidental ids onto the declared defaults where semantically identical, and declare the genuinely-separate ones (e.g. JWKS store). The full id inventory is the Phase 1 plan's task 1. **D6 — Runtime write path for request-signing key rotation.** EdgeZero `ConfigStore`/`SecretStore` are **read-only** at runtime; writes go through provisioning (author/ops time). But `KeyRotationManager` writes+deletes config (JWKS) and secrets (private keys) **at request time** via `/_ts/admin/keys/{rotate,deactivate,delete}`, backed by `management_api.rs`. Three resolutions: - **(a) Keep a write-capable admin abstraction** — retain the trusted-server `put`/`create`/`delete` traits + `management_api.rs` for the admin write path only; EdgeZero read-only stores serve the read path. Least disruptive; leaves a non-EdgeZero write path (so "completely on EdgeZero" is not literally met for admin writes). @@ -129,10 +129,10 @@ Recommendation: **(a)** where the adapter env is available at boot (Cloudflare/S > **Plan ordering (per review):** the Phase 1 plan does NOT start with deletions. Task 1 is a **store-capability inventory** — enumerate every runtime store id and every read vs write call site — and a **decision on D6** (runtime writes). Deletions come only after the write path is settled. **Changes:** -- **Reads:** replace the `PlatformConfigStore`/`PlatformSecretStore` **read** methods and the `RuntimeServices` config/secret fields with EdgeZero `ConfigStoreHandle` / `BoundSecretStore` resolved from the per-request registries (`ConfigRegistry`/`SecretRegistry`), matching KV. Migrate read consumers: `proxy.rs` (S3), `request_signing/{signing,rotation}.rs` (reads), `integrations/datadome/{protection,protection_scope}.rs`. +- **Reads (runtime + boot):** route `PlatformConfigStore`/`PlatformSecretStore` **reads** and the `RuntimeServices` config/secret fields through EdgeZero handles resolved from the per-request registries, matching KV — via a **composite store** (reads → EdgeZero, writes → existing management path; see D6-a). Also migrate the **boot-time** config load: Fastly `load_settings_from_config_store()` and Axum `build_state()` read `Settings` at boot through `&FastlyPlatformConfigStore` / `&AxumPlatformConfigStore` **before** request context exists, so those must move to a boot-time EdgeZero config read *before* the bespoke read impls are deleted. Migrate read consumers: `proxy.rs` (S3), `request_signing/{signing,rotation}.rs` (reads), `integrations/datadome/{protection,protection_scope}.rs`. - **Writes (D6):** `KeyRotationManager` writes+deletes **config and secrets at request time** (`store_private_key`/`store_public_jwk`/`delete_key` for `/_ts/admin/keys/rotate` + deactivate/delete). EdgeZero `ConfigStore`/`SecretStore` are **read-only by design**. So `management_api.rs` **cannot be deleted in Phase 1 as originally written**. Resolve per D6 before touching it. - **Store-id reconciliation (D5, expanded):** every runtime store id referenced by config must be declared in `edgezero.toml` `[stores.config]`/`[stores.secrets]` `ids` or strict registry lookup returns `None`. Reconcile at least: the app-config blob store, `request_signing.config_store_id` (`app_config` today) + `secret_store_id` (`secrets` today), the JWKS/config-list store, DataDome config-list + secret stores (`ts_secrets`), the S3 secret store, and all `trusted-server.example.toml` + integration/test fixtures. -- **Fastly registry injection (ties to P0-C):** Fastly's custom `oneshot` path (§1) currently inserts only a `ConfigStoreHandle`, not registries via `dispatch_with_registries`. Phase 1 must add explicit `Kv`/`Config`/`Secret` registry construction + insertion into extensions compatible with that custom path (not just the standard dispatch helper the other adapters use). +- **Fastly registry injection (ties to P0-C):** Fastly's custom `oneshot` path (§1) currently inserts only a `ConfigStoreHandle`, not registries via `dispatch_with_registries`. EdgeZero's `dispatch_with_registries` and its registry builders are **`pub(crate)`** (verified in the pinned checkout), so trusted-server must build the registries **locally** (from `StoresMetadata` + `EnvConfig` + the EdgeZero Fastly store open primitives) and insert them into extensions before `oneshot` — or an EdgeZero public builder must be added upstream (**R11**). - Delete the 4× per-adapter `platform.rs` config/secret **read** impls; adapters build registries from `[stores.*]` metadata (via `dispatch_with_registries` on Axum/Cloudflare/Spin, via the Fastly-specific injection above). - Delete `settings_data.rs`'s `FastlyChunkPointer` resolver — EdgeZero's `FastlyConfigStore` resolves chunks transparently. `get_settings_from_config_store` collapses to `ConfigStore::get` + `settings_from_config_blob`. @@ -251,7 +251,8 @@ Two consequences: (1) edgezero #305 **must** ship `ArrayEach` + `Option` | R7 | P0-C: upstream a header-preserving Fastly dispatch, or keep a permanent Fastly dispatch shim? | Decide with edgezero maintainer (§4a); gates the Fastly end-state and Phase 5. | | R8 | P-BOOT: boot-time store handle (a) vs lazy cached first-request load (b), per adapter? | Phase 2 plan (§4a). | | R9 | D5: reconcile **all** runtime store ids (`app_config`, `secrets`, JWKS, DataDome `ts_secrets`, S3, fixtures) with `edgezero.toml` — strict lookup fails otherwise. | Phase 1 plan task 1. | -| R10 | D6: runtime write path for key rotation — keep write-capable admin abstraction (a), move to ops/CLI (b), or upstream an EdgeZero write API (c)? | **Blocks Phase 1 deletions.** Decide before deleting `management_api.rs`. | +| R10 | D6: runtime write path for key rotation — keep write-capable admin abstraction (a), move to ops/CLI (b), or upstream an EdgeZero write API (c)? | **Blocks Phase 1 deletions.** Phase 1 plan locks to **(a)**; (b)/(c) → separate plan. | +| R11 | Should EdgeZero expose a **public** Fastly registry-builder helper (so trusted-server need not maintain local builders)? | Decide with edgezero maintainer; Phase 1 plan uses local builders (Task 6) meanwhile. | | R2 | `StoreName` vs `StoreId` split — still needed after `management_api.rs` deletion? | Phase 1; drop if only the CLI provision path used it. | | R3 | EC identity API + Fastly rate limiter are Fastly-only today | Out of scope here; note as a portability follow-up (not blocking). | | R4 | Cloudflare/Spin boot-time secret-store access for D3 | Confirm in Phase 3 scoping. | From bbe5d67fe62844feaad7621c748af41fb03ff551 Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Thu, 2 Jul 2026 22:51:37 -0700 Subject: [PATCH 08/30] Revise Phase 1 plan per review 4: registry-backed composite + concrete tests - Task 3: composite holds the whole ConfigRegistry/SecretRegistry and resolves named(store_name) per read (multi-store); strict unknown-id error; test uses 2 ids - Task 1/spec: correct KV ids to ec_store + consent_store (creative/counter/opid are fastly.toml platform stores, not Settings logical ids) - Task 4: core-level loader test with InMemoryConfigStore + ConfigStoreHandle::new - Task 5: add non-default config-id (JWKS) + non-default secret-id (DataDome, S3) tests - Task 6: local EnvConfig runtime-dictionary reader (EdgeZero helper private); R12 - Task 8: tests build registries with >=2 ids and assert unknown id errors strictly - Spec D5/Phase 1: kind-aware incl [stores.kv]; R9 mentions ec_identity_store; R12 --- ...07-02-edgezero-store-registry-migration.md | 116 ++++++++++++------ ...26-07-02-edgezero-full-migration-design.md | 5 +- 2 files changed, 80 insertions(+), 41 deletions(-) diff --git a/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md b/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md index e4b0384c..8a3905ed 100644 --- a/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md +++ b/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md @@ -41,15 +41,15 @@ Deliverable: a **decision record** appended to "Task 1 Output" that Tasks 2+ con Run: ```bash cd /Users/ag/projects/iab/trusted-server/.claude/worktrees/edgezero-migration-spec -# KV ids -rg -n 'ec_store|consent_store|creative_store|ec_identity_store|counter_store|opid_store' crates/trusted-server-core/src/settings.rs trusted-server.example.toml +# KV ids (logical ids referenced by Settings — NOT fastly.toml platform stores) +rg -n 'ec_store|consent_store' crates/trusted-server-core/src/settings.rs crates/trusted-server-core/src/consent_config.rs trusted-server.example.toml # config ids rg -n 'config_store_id|jwks|JWKS_CONFIG_STORE_NAME|"app_config"|config_store\s*=' crates/trusted-server-core trusted-server.example.toml # secret ids rg -n 'secret_store_id|secret_store\s*=|"secrets"|ts_secrets|signing_keys|SIGNING_SECRET_STORE_NAME' crates/trusted-server-core trusted-server.example.toml rg -n '\[stores\.' edgezero.toml ``` -Expected: KV ids include `ec_identity_store` (from `ec.ec_store`), consent/creative/counter/opid stores; config ids include `app_config` + the JWKS store (`JWKS_CONFIG_STORE_NAME`); secret ids include `secrets`, `signing_keys`, DataDome `ts_secrets`, the S3 secret store — versus `edgezero.toml` declaring only `trusted_server_kv`/`trusted_server_config`/`trusted_server_secrets`. +Expected (verified): **KV** ids = `ec.ec_store` (`ec_identity_store`, `settings.rs:452`) + `consent.consent_store` (`consent_config.rs:80`); **config** ids = `app_config` (`request_signing.config_store_id`) + the JWKS store (`JWKS_CONFIG_STORE_NAME`); **secret** ids = `secrets` (`request_signing.secret_store_id`), DataDome `ts_secrets`, the S3 secret store, `signing_keys` (`SIGNING_SECRET_STORE_NAME`) — versus `edgezero.toml` declaring only one id per kind. NOTE: `creative_store`/`counter_store`/`opid_store` appear only in `fastly.toml` as **platform** store declarations; they are not logical ids in `Settings` and are out of scope for D5 reconciliation. Confirm this during the run. - [ ] **Step 2: Enumerate runtime WRITE sites** @@ -139,9 +139,9 @@ git commit -m "Declare kv/config/secret store ids in edgezero.toml and reconcile --- -## Task 3: Composite read/write store bridge (reads → EdgeZero, writes → management path) +## Task 3: Registry-backed composite store (reads → EdgeZero registry by store_name, writes → management path) -Concrete D6-a mechanism. Introduce a composite that implements `PlatformConfigStore`/`PlatformSecretStore` by routing **reads** to an EdgeZero-backed handle and **writes** (`put`/`create`/`delete`) to the existing management-API-backed impl (`inner_writer`). This preserves `KeyRotationManager` writes with zero call-site changes. +Concrete D6-a mechanism. The bespoke traits read **by `StoreName`** and callers use **multiple** store ids (`app_config`, JWKS, DataDome, S3, `ec_identity_store` for KV). So the composite must hold the **whole `ConfigRegistry`/`SecretRegistry`** (not a single handle) and resolve `named(store_name)` on each read; writes (`put`/`create`/`delete`) delegate to the existing management-API-backed writer. Preserves `KeyRotationManager` writes with zero call-site changes. **Files:** - Create: `crates/trusted-server-core/src/platform/composite.rs` (`CompositeConfigStore`, `CompositeSecretStore`) @@ -149,32 +149,38 @@ Concrete D6-a mechanism. Introduce a composite that implements `PlatformConfigSt - Test: `crates/trusted-server-core/src/platform/composite.rs` (`#[cfg(test)]`) **Interfaces:** -- Consumes: `edgezero_core::config_store::ConfigStoreHandle`, `edgezero_core::store_registry::BoundSecretStore`, an `Arc`/`Arc` writer. +- Consumes: `edgezero_core::store_registry::{ConfigRegistry, SecretRegistry}`, an `Arc`/`Arc` writer. - Produces: - - `CompositeConfigStore::new(reader: ConfigStoreHandle, writer: Arc) -> Self` implementing `PlatformConfigStore` (get→reader, put/delete→writer). - - `CompositeSecretStore::new(reader: BoundSecretStore, writer: Arc) -> Self` implementing `PlatformSecretStore` (get_bytes→reader, create/delete→writer). + - `CompositeConfigStore::new(reader: ConfigRegistry, writer: Arc) -> Self` implementing `PlatformConfigStore`: `get(store_name, key)` → `reader.named(store_name.as_str()).ok_or(PlatformError::ConfigStore)?.get(key)`; `put`/`delete` → `writer`. + - `CompositeSecretStore::new(reader: SecretRegistry, writer: Arc) -> Self` implementing `PlatformSecretStore`: `get_bytes(store_name, key)` → `reader.named(store_name.as_str()).ok_or(PlatformError::SecretStore)?.get_bytes(key)`; `create`/`delete` → `writer`. A store_name not in the registry is a hard error (strict), not a silent fallback. -- [ ] **Step 1: Write the failing test — read via EdgeZero, write delegates to writer** +- [ ] **Step 1: Write the failing test — reads resolve the NAMED store; unknown store errors; writes delegate** ```rust #[test] -fn composite_config_reads_edgezero_and_writes_delegate() { - // Arrange: an EdgeZero reader returning "hi"; a recording writer. - let reader = fixed_config_handle("greeting", "hi"); +fn composite_config_reads_named_store_and_writes_delegate() { + // Arrange: a ConfigRegistry with TWO ids (default `trusted_server_config`, non-default `jwks_store`). + let reader = config_registry(&[ + ("trusted_server_config", "current-kid", "kid-1"), + ("jwks_store", "kid-1", "{\"kty\":\"OKP\"}"), + ], "trusted_server_config"); let writer = Arc::new(RecordingConfigWriter::default()); let composite = CompositeConfigStore::new(reader, writer.clone()); - // Act: read + write. - let read = composite - .get(&StoreName::from("trusted_server_config"), "greeting") - .expect("should read via EdgeZero reader"); + // Act + Assert: non-default store resolves. + let jwk = composite + .get(&StoreName::from("jwks_store"), "kid-1") + .expect("should read from the non-default jwks_store"); + assert_eq!(jwk, "{\"kty\":\"OKP\"}"); + // Unknown store id is a strict error, not a fallback to default. + let err = composite.get(&StoreName::from("nope"), "kid-1").expect_err("unknown store must error"); + assert!(matches!(err.current_context(), PlatformError::ConfigStore), "unknown id -> ConfigStore error"); + // Write delegates to the management-path writer. composite - .put(&StoreId::from("trusted_server_config"), "current-kid", "kid-1") + .put(&StoreId::from("jwks_store"), "current-kid", "kid-2") .expect("should delegate write"); - // Assert - assert_eq!(read, "hi", "read should come from the EdgeZero reader"); assert_eq!( writer.puts.lock().expect("lock").as_slice(), - &[("current-kid".to_owned(), "kid-1".to_owned())], + &[("current-kid".to_owned(), "kid-2".to_owned())], "write should delegate to the management-path writer", ); } @@ -182,23 +188,23 @@ fn composite_config_reads_edgezero_and_writes_delegate() { - [ ] **Step 2: Run to verify it fails** -Run: `cargo test-fastly composite_config_reads_edgezero_and_writes_delegate` +Run: `cargo test-fastly composite_config_reads_named_store_and_writes_delegate` Expected: FAIL (module does not exist). - [ ] **Step 3: Implement `composite.rs`** -Reads call the EdgeZero handle via `futures::executor::block_on` (mirror `storage/kv_store.rs`), mapping `edgezero_core` errors → `PlatformError`. Writes forward to `writer`. Repeat for `CompositeSecretStore`. Add `RecordingConfigWriter`/`fixed_config_handle` test helpers. +`get`/`get_bytes` resolve `reader.named(store_name.as_str())` (strict — `None` → `PlatformError`), then call the bound handle via `futures::executor::block_on` (mirror `storage/kv_store.rs`), mapping `edgezero_core` errors → `PlatformError`. `put`/`create`/`delete` forward to `writer`. Add `config_registry(entries, default)` / `secret_registry(...)` / `RecordingConfigWriter` test helpers that build a real `StoreRegistry` from in-memory EdgeZero stores. - [ ] **Step 4: Run to verify it passes** -Run: `cargo test-fastly composite_config_reads_edgezero_and_writes_delegate` +Run: `cargo test-fastly composite_config_reads_named_store_and_writes_delegate` Expected: PASS. - [ ] **Step 5: Commit** ```bash git add crates/trusted-server-core/src/platform/ -git commit -m "Add composite config/secret store: EdgeZero reads, management-path writes" +git commit -m "Add registry-backed composite store: EdgeZero reads by store_name, management-path writes" ``` --- @@ -217,11 +223,26 @@ git commit -m "Add composite config/secret store: EdgeZero reads, management-pat - Consumes: `edgezero_core` Fastly/Axum `ConfigStore` open primitives; Task 3 nothing (boot read is direct). - Produces: `get_settings_from_config_store` taking `&ConfigStoreHandle` (EdgeZero) instead of `&dyn PlatformConfigStore`. -- [ ] **Step 1: Write a failing boot test (Fastly)** asserting `load_settings_from_config_store()` returns parsed `Settings` when the EdgeZero Fastly config store holds the blob (use the EdgeZero test store / a seeded local config store). +- [ ] **Step 1: Write a failing CORE-level test** for the re-typed loader (deterministic, no adapter/Viceroy). In `crates/trusted-server-core/src/settings_data.rs` `#[cfg(test)]`, build an in-memory EdgeZero store and assert the loader parses the blob: + +```rust +#[test] +fn get_settings_reads_blob_via_edgezero_handle() { + // Arrange: an EdgeZero ConfigStoreHandle over an in-memory store holding the blob envelope. + let blob = blob_envelope_json(include_str!("../../../trusted-server.example.toml")); + let handle = ConfigStoreHandle::new(Arc::new(InMemoryConfigStore::with(&[("app_config", &blob)]))); + // Act + let settings = get_settings_from_config_store(&handle, "app_config") + .expect("should parse settings from the EdgeZero-read blob"); + // Assert + assert!(settings.ec.ec_store.is_some(), "should deserialize the example config"); +} +``` +(`InMemoryConfigStore` is a local test double implementing `edgezero_core::config_store::ConfigStore`; `blob_envelope_json` wraps the TOML→JSON in a `BlobEnvelope`. Add both to the test module.) -Run: `cargo test-fastly boot_config_loads_via_edgezero` → Expected: FAIL. +Run: `cargo test-fastly get_settings_reads_blob_via_edgezero_handle` → Expected: FAIL. -- [ ] **Step 2: Re-type `get_settings_from_config_store`** to take an EdgeZero `ConfigStoreHandle`; open the EdgeZero `FastlyConfigStore` at boot in `load_settings_from_config_store`, and the EdgeZero Axum config store in Axum `build_state`. +- [ ] **Step 2: Re-type `get_settings_from_config_store`** to `(&ConfigStoreHandle, key: &str)`; in Fastly `load_settings_from_config_store()` open the EdgeZero `FastlyConfigStore` at boot (`ConfigStore::open`/adapter constructor) and wrap in a `ConfigStoreHandle`; in Axum `build_state()` open the EdgeZero Axum config store. The adapter-level boot wiring is exercised by each adapter's existing `build_state` test path (no new Viceroy test needed — the core test above covers the parse logic). - [ ] **Step 3: Run to verify pass** (Fastly + Axum) @@ -249,16 +270,19 @@ These adapters use EdgeZero `dispatch_with_registries` (registries already inser - Consumes: Task 3 composite; `ConfigRegistry`/`SecretRegistry` from request extensions. - Produces: `RuntimeServices` whose reads flow through EdgeZero, writes through the composite writer. -- [ ] **Step 1: Write a failing Axum route test** — `GET /.well-known/trusted-server.json` returns the JWKS/discovery document read from the config store. Name: `discovery_reads_jwks_from_edgezero_config_store` in the Axum app test module. Seed the Axum config registry with a JWKS entry fixture; assert `200` + the JWKS `kid` in the body. +- [ ] **Step 1: Write failing Axum tests covering the default AND a non-default config id AND a non-default secret id** (in the Axum app test module): + - `discovery_reads_jwks_from_nondefault_config_store` — `GET /.well-known/trusted-server.json`: seed the Axum `ConfigRegistry` with two ids (`trusted_server_config` default + the JWKS store id); assert `200` + the JWKS `kid` in the body (proves non-default **config** id resolution). + - `datadome_reads_secret_from_nondefault_secret_store` — a request to the DataDome integration route: seed the `SecretRegistry` with two ids (default + `ts_secrets`) and the DataDome server-side key under `ts_secrets`; assert the handler reads it (proves non-default **secret** id resolution). + - `first_party_proxy_reads_s3_secret` — `GET /first-party/proxy` for an S3-auth asset route: seed the S3 secret id; assert the SigV4 path obtains the secret (proves the S3 secret read). -Run: `cargo test-axum discovery_reads_jwks_from_edgezero_config_store` → Expected: FAIL. +Run: `cargo test-axum discovery_reads_jwks_from_nondefault_config_store datadome_reads_secret_from_nondefault_secret_store first_party_proxy_reads_s3_secret` → Expected: FAIL. -- [ ] **Step 2: Build `RuntimeServices` via the composite** in each adapter's `build_runtime_services`, resolving the reader from the request `ConfigRegistry`/`SecretRegistry` and keeping the existing writer. +- [ ] **Step 2: Build `RuntimeServices` via the composite** in each adapter's `build_runtime_services`, passing the whole request `ConfigRegistry`/`SecretRegistry` as the composite reader (Task 3) and keeping the existing writer. - [ ] **Step 3: Run to verify pass** (all three) Run: `cargo test-axum && cargo test-cloudflare && cargo test-spin` -Expected: PASS. +Expected: PASS. (Cloudflare/Spin reuse the same composite; their route tests assert the default-id read at minimum.) - [ ] **Step 4: Commit** @@ -280,21 +304,35 @@ EdgeZero's Fastly `dispatch_with_registries` and its registry builders are `pub( **Interfaces:** - Consumes: `StoresMetadata` (from `Hooks::stores()`), `EnvConfig`, EdgeZero `FastlyConfigStore`/`FastlyKvStore`/`FastlySecretStore` open primitives, `StoreRegistry::from_parts`. -- Produces: `build_config_registry(&StoresMetadata, &EnvConfig) -> ConfigRegistry` (+ `_secret_/_kv_` variants) matching EdgeZero's per-id name resolution (`EDGEZERO__STORES______NAME`). +- Produces: `local_env_config() -> EnvConfig` (Fastly runtime-dictionary reader, see Step 1); `build_config_registry(&StoresMetadata, &EnvConfig) -> ConfigRegistry` (+ `_secret_/_kv_` variants) matching EdgeZero's per-id name resolution (`EDGEZERO__STORES______NAME`). + +- [ ] **Step 1: Build a local `EnvConfig` reader (EdgeZero's is private).** Fastly Compute has no `std::env`; EdgeZero reads `EDGEZERO__*` from a Fastly Config Store (`env_config_from_runtime_dictionary`), which is **private** in the pinned dep (R12). Write `local_env_config()` in `registries.rs` that opens the `edgezero_runtime_env` Fastly Config Store, iterates its entries, and calls `EnvConfig::from_vars(...)`. If R11/R12 resolves by exposing a public EdgeZero helper, delete this and call that instead. + +```rust +// registries.rs +fn local_env_config() -> EnvConfig { + // Mirror EdgeZero's runtime-dictionary reader: read the well-known + // Fastly Config Store into (key,value) pairs, then EnvConfig::from_vars. + match fastly::ConfigStore::try_open("edgezero_runtime_env") { + Ok(store) => EnvConfig::from_vars(store.iter().map(|(k, v)| (k, v))), + Err(_) => EnvConfig::default(), + } +} +``` -- [ ] **Step 1: Write a failing builder test** — `build_config_registry` yields a registry whose `default()` resolves and whose declared non-default ids resolve; unknown id → `None`. +- [ ] **Step 2: Write a failing builder test** — `build_config_registry` yields a registry whose `default()` resolves and whose declared non-default id (e.g. `jwks_store`) resolves; unknown id → `None`. Name: `build_config_registry_resolves_declared_ids`. Run: `cargo test-fastly build_config_registry_resolves_declared_ids` → Expected: FAIL. -- [ ] **Step 2: Implement the three builders** in `registries.rs` (iterate `StoreMetadata.ids`, resolve platform name via `EnvConfig::store_name(kind, id)`, open the EdgeZero store, assemble `StoreRegistry::from_parts`). +- [ ] **Step 3: Implement the three builders** in `registries.rs` (iterate `StoreMetadata.ids`, resolve platform name via `EnvConfig::store_name(kind, id)`, open the EdgeZero store, assemble `StoreRegistry::from_parts`), using `local_env_config()` from Step 1. -- [ ] **Step 3: Insert registries in the oneshot block** — replace the lone `core_req.extensions_mut().insert(config_store)` at `main.rs:477` with inserts of `ConfigRegistry`/`SecretRegistry`/`KvRegistry` (built via Step 2), preserving the existing `client_info`/`device_signals` inserts. +- [ ] **Step 4: Insert registries in the oneshot block** — replace the lone `core_req.extensions_mut().insert(config_store)` at `main.rs:477` with inserts of `ConfigRegistry`/`SecretRegistry`/`KvRegistry` (built via Step 3), preserving the existing `client_info`/`device_signals` inserts. -- [ ] **Step 4: Write a failing Fastly route test** — `GET /.well-known/trusted-server.json` via the EdgeZero `oneshot` path returns the JWKS doc read through the injected `ConfigRegistry`. Name: `oneshot_discovery_reads_jwks_via_registry` (mirror the `StubJwksConfigStore`/`JWKS_CONFIG_STORE_NAME` pattern in `route_tests.rs`, but drive the EdgeZero path, not `route_request`). +- [ ] **Step 5: Write a failing Fastly route test** — `GET /.well-known/trusted-server.json` via the EdgeZero `oneshot` path returns the JWKS doc read through the injected `ConfigRegistry` (built with default + `jwks_store` ids). Name: `oneshot_discovery_reads_jwks_via_registry` (mirror the `StubJwksConfigStore`/`JWKS_CONFIG_STORE_NAME` pattern in `route_tests.rs`, but drive the EdgeZero path, not `route_request`). -Run: `cargo test-fastly oneshot_discovery_reads_jwks_via_registry` → Expected: FAIL then PASS after Steps 2–3. +Run: `cargo test-fastly oneshot_discovery_reads_jwks_via_registry` → Expected: FAIL then PASS after Steps 3–4. -- [ ] **Step 5: Fastly suite + parity + commit** +- [ ] **Step 6: Fastly suite + parity + commit** Run: `cargo test-fastly && cargo test --manifest-path crates/trusted-server-integration-tests/Cargo.toml --test parity` ```bash @@ -337,7 +375,7 @@ Now that all reads (boot + request, all adapters) flow through EdgeZero, delete - [ ] **Step 1: Delete the config/secret read impls** now unused after Tasks 4–6 (`FastlyPlatformConfigStore::get`, `AxumPlatformConfigStore`, `NoopConfigStore`, Cloudflare/Spin equivalents, secret read impls). Keep the write impls + `management_api.rs`. -- [ ] **Step 2: Update `route_tests.rs`** — the stub stores (`StubJwksConfigStore`, etc.) and `RuntimeServices` construction move to the composite/registry shape (reader = a fixed EdgeZero handle, writer = a recording stub). Keep coverage of the write path (`put`/`create`/`delete`) so key-rotation delegation stays tested. +- [ ] **Step 2: Update `route_tests.rs`** — the stub stores (`StubJwksConfigStore`, etc.) and `RuntimeServices` construction move to the composite/registry shape: build the composite reader from a real `ConfigRegistry`/`SecretRegistry` with **at least two ids** (default + a non-default such as `jwks_store`/`ts_secrets`), and assert an **unknown store id resolves strictly to an error** (not a silent fallback to default). Writer = a recording stub; keep coverage of the write path (`put`/`create`/`delete`) so key-rotation delegation stays tested. - [ ] **Step 3: Full CI gate** diff --git a/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md b/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md index ffdca6c5..899f4974 100644 --- a/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md +++ b/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md @@ -131,7 +131,7 @@ Recommendation: **(a)** where the adapter env is available at boot (Cloudflare/S **Changes:** - **Reads (runtime + boot):** route `PlatformConfigStore`/`PlatformSecretStore` **reads** and the `RuntimeServices` config/secret fields through EdgeZero handles resolved from the per-request registries, matching KV — via a **composite store** (reads → EdgeZero, writes → existing management path; see D6-a). Also migrate the **boot-time** config load: Fastly `load_settings_from_config_store()` and Axum `build_state()` read `Settings` at boot through `&FastlyPlatformConfigStore` / `&AxumPlatformConfigStore` **before** request context exists, so those must move to a boot-time EdgeZero config read *before* the bespoke read impls are deleted. Migrate read consumers: `proxy.rs` (S3), `request_signing/{signing,rotation}.rs` (reads), `integrations/datadome/{protection,protection_scope}.rs`. - **Writes (D6):** `KeyRotationManager` writes+deletes **config and secrets at request time** (`store_private_key`/`store_public_jwk`/`delete_key` for `/_ts/admin/keys/rotate` + deactivate/delete). EdgeZero `ConfigStore`/`SecretStore` are **read-only by design**. So `management_api.rs` **cannot be deleted in Phase 1 as originally written**. Resolve per D6 before touching it. -- **Store-id reconciliation (D5, expanded):** every runtime store id referenced by config must be declared in `edgezero.toml` `[stores.config]`/`[stores.secrets]` `ids` or strict registry lookup returns `None`. Reconcile at least: the app-config blob store, `request_signing.config_store_id` (`app_config` today) + `secret_store_id` (`secrets` today), the JWKS/config-list store, DataDome config-list + secret stores (`ts_secrets`), the S3 secret store, and all `trusted-server.example.toml` + integration/test fixtures. +- **Store-id reconciliation (D5, expanded, kind-aware):** every runtime store id referenced by config must be declared in `edgezero.toml` under the **correct kind** — `[stores.kv]`, `[stores.config]`, or `[stores.secrets]` — or strict registry lookup returns `None`. Reconcile at least: **KV** — `ec.ec_store` (`ec_identity_store`), `consent.consent_store`; **config** — the app-config blob store, `request_signing.config_store_id` (`app_config` today), the JWKS/config-list store; **secrets** — `request_signing.secret_store_id` (`secrets` today), DataDome secret store (`ts_secrets`), the S3 secret store — plus all `trusted-server.example.toml` + integration/test fixtures. (`creative_store`/`counter_store`/`opid_store` in `fastly.toml` are platform store declarations, **not** logical ids referenced by `Settings`.) - **Fastly registry injection (ties to P0-C):** Fastly's custom `oneshot` path (§1) currently inserts only a `ConfigStoreHandle`, not registries via `dispatch_with_registries`. EdgeZero's `dispatch_with_registries` and its registry builders are **`pub(crate)`** (verified in the pinned checkout), so trusted-server must build the registries **locally** (from `StoresMetadata` + `EnvConfig` + the EdgeZero Fastly store open primitives) and insert them into extensions before `oneshot` — or an EdgeZero public builder must be added upstream (**R11**). - Delete the 4× per-adapter `platform.rs` config/secret **read** impls; adapters build registries from `[stores.*]` metadata (via `dispatch_with_registries` on Axum/Cloudflare/Spin, via the Fastly-specific injection above). - Delete `settings_data.rs`'s `FastlyChunkPointer` resolver — EdgeZero's `FastlyConfigStore` resolves chunks transparently. `get_settings_from_config_store` collapses to `ConfigStore::get` + `settings_from_config_blob`. @@ -250,7 +250,8 @@ Two consequences: (1) edgezero #305 **must** ship `ArrayEach` + `Option` | R1 | Do any `Settings` secrets live inside **arrays**? | **Resolved: yes** (`ec.partners[].api_token`, `handlers[].password`) + optional (`ts_pull_token`). edgezero #305 must ship `ArrayEach` + `Option` (see §5 Phase 3 inventory + Phase 0 note). | | R7 | P0-C: upstream a header-preserving Fastly dispatch, or keep a permanent Fastly dispatch shim? | Decide with edgezero maintainer (§4a); gates the Fastly end-state and Phase 5. | | R8 | P-BOOT: boot-time store handle (a) vs lazy cached first-request load (b), per adapter? | Phase 2 plan (§4a). | -| R9 | D5: reconcile **all** runtime store ids (`app_config`, `secrets`, JWKS, DataDome `ts_secrets`, S3, fixtures) with `edgezero.toml` — strict lookup fails otherwise. | Phase 1 plan task 1. | +| R9 | D5: reconcile **all** runtime store ids **by kind** — KV (`ec_identity_store`, `consent_store`), config (`app_config`, JWKS), secrets (`secrets`, DataDome `ts_secrets`, S3) + fixtures — with `edgezero.toml`; strict lookup fails otherwise. | Phase 1 plan task 1. | +| R12 | Fastly `EnvConfig` reader (`env_config_from_runtime_dictionary`) is **private** upstream; Fastly has no `std::env`. Build a local runtime-dictionary reader, or make R11's public helper a prerequisite. | Phase 1 plan Task 6. | | R10 | D6: runtime write path for key rotation — keep write-capable admin abstraction (a), move to ops/CLI (b), or upstream an EdgeZero write API (c)? | **Blocks Phase 1 deletions.** Phase 1 plan locks to **(a)**; (b)/(c) → separate plan. | | R11 | Should EdgeZero expose a **public** Fastly registry-builder helper (so trusted-server need not maintain local builders)? | Decide with edgezero maintainer; Phase 1 plan uses local builders (Task 6) meanwhile. | | R2 | `StoreName` vs `StoreId` split — still needed after `management_api.rs` deletion? | Phase 1; drop if only the CLI provision path used it. | From 1e57ad7fc63ea42b56ac2d185725a48ff91c903c Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Thu, 2 Jul 2026 23:08:23 -0700 Subject: [PATCH 09/30] Add D7 config-store-only hard cutoff; fix review-5 findings - D7: runtime app-config is config-store-only; NO runtime env vars. ts config push reads env at push time and bakes resolved values into the blob. Stores open by logical id (name == id); no runtime EDGEZERO__STORES__*__NAME read. - Task 6: drop local_env_config entirely (D7) - open stores by logical id; resolves the fastly::ConfigStore-has-no-iter and private-helper findings (R12) - creative_store IS a Settings KV id (deprecated); include it, exclude counter_store/opid_store (Fastly-adapter constants) - Task 2/spec: tighten kind-aware KV inventory (ec_store, consent_store, creative_store) - Task 4: run the actual core test name; Task 5: one filter per cargo test invocation --- ...07-02-edgezero-store-registry-migration.md | 45 +++++++++---------- ...26-07-02-edgezero-full-migration-design.md | 15 +++++-- 2 files changed, 32 insertions(+), 28 deletions(-) diff --git a/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md b/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md index 8a3905ed..8b895198 100644 --- a/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md +++ b/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md @@ -41,15 +41,15 @@ Deliverable: a **decision record** appended to "Task 1 Output" that Tasks 2+ con Run: ```bash cd /Users/ag/projects/iab/trusted-server/.claude/worktrees/edgezero-migration-spec -# KV ids (logical ids referenced by Settings — NOT fastly.toml platform stores) -rg -n 'ec_store|consent_store' crates/trusted-server-core/src/settings.rs crates/trusted-server-core/src/consent_config.rs trusted-server.example.toml +# KV ids (logical ids referenced by Settings — NOT Fastly-only platform stores) +rg -n 'ec_store|consent_store|creative_store' crates/trusted-server-core/src/settings.rs crates/trusted-server-core/src/consent_config.rs crates/trusted-server-core/src/auction_config_types.rs trusted-server.example.toml # config ids rg -n 'config_store_id|jwks|JWKS_CONFIG_STORE_NAME|"app_config"|config_store\s*=' crates/trusted-server-core trusted-server.example.toml # secret ids rg -n 'secret_store_id|secret_store\s*=|"secrets"|ts_secrets|signing_keys|SIGNING_SECRET_STORE_NAME' crates/trusted-server-core trusted-server.example.toml rg -n '\[stores\.' edgezero.toml ``` -Expected (verified): **KV** ids = `ec.ec_store` (`ec_identity_store`, `settings.rs:452`) + `consent.consent_store` (`consent_config.rs:80`); **config** ids = `app_config` (`request_signing.config_store_id`) + the JWKS store (`JWKS_CONFIG_STORE_NAME`); **secret** ids = `secrets` (`request_signing.secret_store_id`), DataDome `ts_secrets`, the S3 secret store, `signing_keys` (`SIGNING_SECRET_STORE_NAME`) — versus `edgezero.toml` declaring only one id per kind. NOTE: `creative_store`/`counter_store`/`opid_store` appear only in `fastly.toml` as **platform** store declarations; they are not logical ids in `Settings` and are out of scope for D5 reconciliation. Confirm this during the run. +Expected (verified): **KV** ids = `ec.ec_store` (`ec_identity_store`, `settings.rs:452`), `consent.consent_store` (`consent_config.rs:80`), and `auction.creative_store` (`auction_config_types.rs:28`, default `"creative_store"`, **deprecated** — creatives are delivered inline); **config** ids = `app_config` (`request_signing.config_store_id`) + the JWKS store (`JWKS_CONFIG_STORE_NAME`); **secret** ids = `secrets` (`request_signing.secret_store_id`), DataDome `ts_secrets`, the S3 secret store, `signing_keys` (`SIGNING_SECRET_STORE_NAME`) — versus `edgezero.toml` declaring only one id per kind. NOTE: `counter_store` (`RATE_COUNTER_NAME` in the Fastly `rate_limiter.rs`) and `opid_store` are **Fastly-only** platform stores, not `Settings` logical ids — out of scope for D5. `creative_store` **is** a `Settings` id: declare it in `[stores.kv]` (deprecated) so strict lookup can't fail, and flag it for removal in a later phase. - [ ] **Step 2: Enumerate runtime WRITE sites** @@ -120,7 +120,7 @@ Expected: FAIL — `ec_identity_store` (kv), `app_config`/JWKS (config), `secret - [ ] **Step 3: Implement `referenced_store_ids_by_kind()` + manifest helper** -Add the `ReferencedStoreIds` struct + method returning KV ids (`ec.ec_store`, consent/creative/counter/opid), config ids (`request_signing.config_store_id`, `JWKS_CONFIG_STORE_NAME`, app-config), secret ids (`request_signing.secret_store_id`, DataDome, S3, `SIGNING_SECRET_STORE_NAME`). Add test-only `declared_store_ids_by_kind_from_manifest()` parsing `edgezero.toml`. +Add the `ReferencedStoreIds` struct + method returning **KV** ids (`ec.ec_store`, `consent.consent_store`, `auction.creative_store`), **config** ids (`request_signing.config_store_id`, `JWKS_CONFIG_STORE_NAME`, app-config), **secret** ids (`request_signing.secret_store_id`, DataDome, S3, `SIGNING_SECRET_STORE_NAME`). Do **not** include `counter_store`/`opid_store` — those are Fastly-adapter constants, not `Settings` fields. Add test-only `declared_store_ids_by_kind_from_manifest()` parsing `edgezero.toml`. - [ ] **Step 4: Update `edgezero.toml` + config fields/fixtures per the Task 1 map** @@ -244,9 +244,12 @@ Run: `cargo test-fastly get_settings_reads_blob_via_edgezero_handle` → Expecte - [ ] **Step 2: Re-type `get_settings_from_config_store`** to `(&ConfigStoreHandle, key: &str)`; in Fastly `load_settings_from_config_store()` open the EdgeZero `FastlyConfigStore` at boot (`ConfigStore::open`/adapter constructor) and wrap in a `ConfigStoreHandle`; in Axum `build_state()` open the EdgeZero Axum config store. The adapter-level boot wiring is exercised by each adapter's existing `build_state` test path (no new Viceroy test needed — the core test above covers the parse logic). -- [ ] **Step 3: Run to verify pass** (Fastly + Axum) +- [ ] **Step 3: Run to verify pass** (core test + adapter boot suites) -Run: `cargo test-fastly boot_config_loads_via_edgezero && cargo test-axum boot_config_loads_via_edgezero` +Run: `cargo test-fastly get_settings_reads_blob_via_edgezero_handle` +Expected: PASS. +Then confirm the adapter boot paths still build/pass via their existing `build_state` coverage: +Run: `cargo test-fastly && cargo test-axum` Expected: PASS. - [ ] **Step 4: Commit** @@ -275,7 +278,13 @@ These adapters use EdgeZero `dispatch_with_registries` (registries already inser - `datadome_reads_secret_from_nondefault_secret_store` — a request to the DataDome integration route: seed the `SecretRegistry` with two ids (default + `ts_secrets`) and the DataDome server-side key under `ts_secrets`; assert the handler reads it (proves non-default **secret** id resolution). - `first_party_proxy_reads_s3_secret` — `GET /first-party/proxy` for an S3-auth asset route: seed the S3 secret id; assert the SigV4 path obtains the secret (proves the S3 secret read). -Run: `cargo test-axum discovery_reads_jwks_from_nondefault_config_store datadome_reads_secret_from_nondefault_secret_store first_party_proxy_reads_s3_secret` → Expected: FAIL. +Run each (one filter per `cargo test` invocation): +```bash +cargo test-axum discovery_reads_jwks_from_nondefault_config_store +cargo test-axum datadome_reads_secret_from_nondefault_secret_store +cargo test-axum first_party_proxy_reads_s3_secret +``` +Expected: FAIL (all three). - [ ] **Step 2: Build `RuntimeServices` via the composite** in each adapter's `build_runtime_services`, passing the whole request `ConfigRegistry`/`SecretRegistry` as the composite reader (Task 3) and keeping the existing writer. @@ -303,28 +312,16 @@ EdgeZero's Fastly `dispatch_with_registries` and its registry builders are `pub( - Test: `crates/trusted-server-adapter-fastly/src/registries.rs` (`#[cfg(test)]`) + a route test **Interfaces:** -- Consumes: `StoresMetadata` (from `Hooks::stores()`), `EnvConfig`, EdgeZero `FastlyConfigStore`/`FastlyKvStore`/`FastlySecretStore` open primitives, `StoreRegistry::from_parts`. -- Produces: `local_env_config() -> EnvConfig` (Fastly runtime-dictionary reader, see Step 1); `build_config_registry(&StoresMetadata, &EnvConfig) -> ConfigRegistry` (+ `_secret_/_kv_` variants) matching EdgeZero's per-id name resolution (`EDGEZERO__STORES______NAME`). - -- [ ] **Step 1: Build a local `EnvConfig` reader (EdgeZero's is private).** Fastly Compute has no `std::env`; EdgeZero reads `EDGEZERO__*` from a Fastly Config Store (`env_config_from_runtime_dictionary`), which is **private** in the pinned dep (R12). Write `local_env_config()` in `registries.rs` that opens the `edgezero_runtime_env` Fastly Config Store, iterates its entries, and calls `EnvConfig::from_vars(...)`. If R11/R12 resolves by exposing a public EdgeZero helper, delete this and call that instead. +- Consumes: `StoresMetadata` (from `Hooks::stores()`), EdgeZero `FastlyConfigStore`/`FastlyKvStore`/`FastlySecretStore` open primitives, `StoreRegistry::from_parts`. +- Produces: `build_config_registry(&StoresMetadata) -> ConfigRegistry` (+ `_secret_/_kv_` variants) that opens each declared store **by its logical id** (per **D7** hard cutoff — no runtime env/dictionary read; platform store name == logical id). -```rust -// registries.rs -fn local_env_config() -> EnvConfig { - // Mirror EdgeZero's runtime-dictionary reader: read the well-known - // Fastly Config Store into (key,value) pairs, then EnvConfig::from_vars. - match fastly::ConfigStore::try_open("edgezero_runtime_env") { - Ok(store) => EnvConfig::from_vars(store.iter().map(|(k, v)| (k, v))), - Err(_) => EnvConfig::default(), - } -} -``` +- [ ] **Step 1: (D7) No runtime env reader.** Per D7 the runtime does **not** read `EDGEZERO__STORES__*__NAME` — stores are opened by **logical id**. This deletes the need for a Fastly runtime-dictionary `EnvConfig` reader (and sidesteps that `fastly::ConfigStore` has no `iter()` and EdgeZero's reader is private). If a deployment ever needs to remap a physical store name, that is handled at provisioning time, not here. No code in this step; it records the design constraint the builders follow. -- [ ] **Step 2: Write a failing builder test** — `build_config_registry` yields a registry whose `default()` resolves and whose declared non-default id (e.g. `jwks_store`) resolves; unknown id → `None`. Name: `build_config_registry_resolves_declared_ids`. +- [ ] **Step 2: Write a failing builder test** — `build_config_registry` opens each declared id by name and yields a registry whose `default()` resolves and whose declared non-default id (`jwks_store`) resolves; an id **not** in `StoresMetadata` is absent (`named("nope").is_none()`). Name: `build_config_registry_resolves_declared_ids`. Run: `cargo test-fastly build_config_registry_resolves_declared_ids` → Expected: FAIL. -- [ ] **Step 3: Implement the three builders** in `registries.rs` (iterate `StoreMetadata.ids`, resolve platform name via `EnvConfig::store_name(kind, id)`, open the EdgeZero store, assemble `StoreRegistry::from_parts`), using `local_env_config()` from Step 1. +- [ ] **Step 3: Implement the three builders** in `registries.rs`: iterate `StoreMetadata.ids`, open the EdgeZero Fastly store **by the logical id** (`FastlyConfigStore`/`FastlyKvStore`/`FastlySecretStore` open primitive), and assemble `StoreRegistry::from_parts(by_id, default_id)`. No `EnvConfig`, no runtime dictionary. - [ ] **Step 4: Insert registries in the oneshot block** — replace the lone `core_req.extensions_mut().insert(config_store)` at `main.rs:477` with inserts of `ConfigRegistry`/`SecretRegistry`/`KvRegistry` (built via Step 3), preserving the existing `client_info`/`device_signals` inserts. diff --git a/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md b/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md index 899f4974..6ed603cd 100644 --- a/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md +++ b/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md @@ -95,6 +95,13 @@ Phase 1 (stores) ──> Phase 2 (config) ──> Phase 3 (secrets) Phase 4 (e --- +**D7 — Runtime app-config is config-store-only; no runtime environment variables (hard cutoff).** At runtime, every adapter builds `Settings` **solely** from the config-store blob: open store → verify envelope → deserialize (+ secret walk) → validate. There is **no runtime environment-variable overlay** for app config on any adapter. All env-var influence on config content happens at **push time**: `ts config push` (which runs on a host that has the env vars) reads them, applies the AppConfig env overlay, and bakes the resolved values into the signed blob. Consequences: +- Delete `Settings::from_toml_and_env` and the `TRUSTED_SERVER__*` overlay entirely (Phase 2) — not merely unused, **forbidden** at runtime. +- The AppConfig loader's `env_overlay` is a **push-time-only** option; runtime never applies it. +- **Store-name binding follows the same spirit:** a logical store id resolves to the **platform store of the same name by default** (EdgeZero's `store_name` fallback), so the runtime does **not** need to read `EDGEZERO__STORES__*__NAME` env/dictionary to open stores. Adapters (incl. Fastly's custom path) open stores by **logical id**. If a deployment ever needs a different physical name, that is a **provisioning/manifest** concern resolved at deploy time, never a runtime env read. (This removes the need for a Fastly runtime-dictionary `EnvConfig` reader — see Phase 1 Task 6.) + +--- + ## 4a. Prerequisites (must resolve before or during Phase 1/2) These are not trusted-server refactors; they are EdgeZero-adapter capability gaps or up-front decisions that gate the phases. @@ -131,7 +138,7 @@ Recommendation: **(a)** where the adapter env is available at boot (Cloudflare/S **Changes:** - **Reads (runtime + boot):** route `PlatformConfigStore`/`PlatformSecretStore` **reads** and the `RuntimeServices` config/secret fields through EdgeZero handles resolved from the per-request registries, matching KV — via a **composite store** (reads → EdgeZero, writes → existing management path; see D6-a). Also migrate the **boot-time** config load: Fastly `load_settings_from_config_store()` and Axum `build_state()` read `Settings` at boot through `&FastlyPlatformConfigStore` / `&AxumPlatformConfigStore` **before** request context exists, so those must move to a boot-time EdgeZero config read *before* the bespoke read impls are deleted. Migrate read consumers: `proxy.rs` (S3), `request_signing/{signing,rotation}.rs` (reads), `integrations/datadome/{protection,protection_scope}.rs`. - **Writes (D6):** `KeyRotationManager` writes+deletes **config and secrets at request time** (`store_private_key`/`store_public_jwk`/`delete_key` for `/_ts/admin/keys/rotate` + deactivate/delete). EdgeZero `ConfigStore`/`SecretStore` are **read-only by design**. So `management_api.rs` **cannot be deleted in Phase 1 as originally written**. Resolve per D6 before touching it. -- **Store-id reconciliation (D5, expanded, kind-aware):** every runtime store id referenced by config must be declared in `edgezero.toml` under the **correct kind** — `[stores.kv]`, `[stores.config]`, or `[stores.secrets]` — or strict registry lookup returns `None`. Reconcile at least: **KV** — `ec.ec_store` (`ec_identity_store`), `consent.consent_store`; **config** — the app-config blob store, `request_signing.config_store_id` (`app_config` today), the JWKS/config-list store; **secrets** — `request_signing.secret_store_id` (`secrets` today), DataDome secret store (`ts_secrets`), the S3 secret store — plus all `trusted-server.example.toml` + integration/test fixtures. (`creative_store`/`counter_store`/`opid_store` in `fastly.toml` are platform store declarations, **not** logical ids referenced by `Settings`.) +- **Store-id reconciliation (D5, expanded, kind-aware):** every runtime store id referenced by config must be declared in `edgezero.toml` under the **correct kind** — `[stores.kv]`, `[stores.config]`, or `[stores.secrets]` — or strict registry lookup returns `None`. Reconcile at least: **KV** — `ec.ec_store` (`ec_identity_store`), `consent.consent_store`, `auction.creative_store` (deprecated — creatives are inline — but still a `Settings` field, so declare it to keep strict lookup safe); **config** — the app-config blob store, `request_signing.config_store_id` (`app_config` today), the JWKS/config-list store; **secrets** — `request_signing.secret_store_id` (`secrets` today), DataDome secret store (`ts_secrets`), the S3 secret store — plus all `trusted-server.example.toml` + integration/test fixtures. (`counter_store`/`opid_store` are **Fastly-adapter** constants (rate limiter / opid), **not** `Settings` logical ids, and are out of scope.) - **Fastly registry injection (ties to P0-C):** Fastly's custom `oneshot` path (§1) currently inserts only a `ConfigStoreHandle`, not registries via `dispatch_with_registries`. EdgeZero's `dispatch_with_registries` and its registry builders are **`pub(crate)`** (verified in the pinned checkout), so trusted-server must build the registries **locally** (from `StoresMetadata` + `EnvConfig` + the EdgeZero Fastly store open primitives) and insert them into extensions before `oneshot` — or an EdgeZero public builder must be added upstream (**R11**). - Delete the 4× per-adapter `platform.rs` config/secret **read** impls; adapters build registries from `[stores.*]` metadata (via `dispatch_with_registries` on Axum/Cloudflare/Spin, via the Fastly-specific injection above). - Delete `settings_data.rs`'s `FastlyChunkPointer` resolver — EdgeZero's `FastlyConfigStore` resolves chunks transparently. `get_settings_from_config_store` collapses to `ConfigStore::get` + `settings_from_config_blob`. @@ -149,7 +156,7 @@ Recommendation: **(a)** where the adapter env is available at boot (Cloudflare/S **Changes:** - Derive `AppConfig` on the config root (interim: still `TrustedServerAppConfig` until Phase 3 collapses it onto `Settings`) so all adapters use the same store-load path. - **Cloudflare** (`adapter-cloudflare/src/app.rs`) and **Spin** (`adapter-spin/src/app.rs`): replace startup config sourcing (Cloudflare's `TRUSTED_SERVER_CONFIG` env side-channel + the native `include_str!` fallback; Spin's baked example TOML) with a **boot-time config-store read** per **P-BOOT (§4a)**. `build_state()` obtains a config-store handle from the adapter env passed to `run_app` (option a) or defers to a lazy first-request cached load (option b). This is the load-bearing detail — settle the mechanism in the Phase 2 plan. Seed each platform's config store (`wrangler.toml` / `runtime-config.toml` / `fastly.toml` local blocks) with the pushed blob under the D5 store id/key. -- Delete `Settings::from_toml_and_env`, `ENVIRONMENT_VARIABLE_PREFIX/SEPARATOR`, and the `config` **dev-dependency**. Any remaining env overlay uses EdgeZero's `EDGEZERO__*` / AppConfig `__…` layers. +- Delete `Settings::from_toml_and_env`, `ENVIRONMENT_VARIABLE_PREFIX/SEPARATOR`, and the `config` **dev-dependency**. Per **D7 (hard cutoff)** there is **no runtime env overlay** — runtime reads the blob only. Env-var influence on config happens exclusively at **push time** via `ts config push` (AppConfig `env_overlay` applied there, then baked into the blob). **Deletions:** both `include_str!` config paths, `from_toml_and_env`, `config` crate dep. **Acceptance:** Cloudflare + Spin serve with store-loaded config (no baked TOML); `ts config push` blob is the single source on all four adapters; tests green. @@ -251,9 +258,9 @@ Two consequences: (1) edgezero #305 **must** ship `ArrayEach` + `Option` | R7 | P0-C: upstream a header-preserving Fastly dispatch, or keep a permanent Fastly dispatch shim? | Decide with edgezero maintainer (§4a); gates the Fastly end-state and Phase 5. | | R8 | P-BOOT: boot-time store handle (a) vs lazy cached first-request load (b), per adapter? | Phase 2 plan (§4a). | | R9 | D5: reconcile **all** runtime store ids **by kind** — KV (`ec_identity_store`, `consent_store`), config (`app_config`, JWKS), secrets (`secrets`, DataDome `ts_secrets`, S3) + fixtures — with `edgezero.toml`; strict lookup fails otherwise. | Phase 1 plan task 1. | -| R12 | Fastly `EnvConfig` reader (`env_config_from_runtime_dictionary`) is **private** upstream; Fastly has no `std::env`. Build a local runtime-dictionary reader, or make R11's public helper a prerequisite. | Phase 1 plan Task 6. | +| R12 | Fastly `EnvConfig` reader is private / `fastly::ConfigStore` has no `iter()`. | **Resolved by D7** — runtime opens stores by logical id; no store-name env/dictionary read; no local `EnvConfig` reader needed. | | R10 | D6: runtime write path for key rotation — keep write-capable admin abstraction (a), move to ops/CLI (b), or upstream an EdgeZero write API (c)? | **Blocks Phase 1 deletions.** Phase 1 plan locks to **(a)**; (b)/(c) → separate plan. | -| R11 | Should EdgeZero expose a **public** Fastly registry-builder helper (so trusted-server need not maintain local builders)? | Decide with edgezero maintainer; Phase 1 plan uses local builders (Task 6) meanwhile. | +| R11 | Should EdgeZero expose a **public** Fastly registry-builder helper? | Lower priority under D7 — local builders open by logical id and need only public store constructors. Decide with edgezero maintainer if convenient. | | R2 | `StoreName` vs `StoreId` split — still needed after `management_api.rs` deletion? | Phase 1; drop if only the CLI provision path used it. | | R3 | EC identity API + Fastly rate limiter are Fastly-only today | Out of scope here; note as a portability follow-up (not blocking). | | R4 | Cloudflare/Spin boot-time secret-store access for D3 | Confirm in Phase 3 scoping. | From 938c761c4e9c7f8037d8cb9331f59a8d1ceaa27d Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Thu, 2 Jul 2026 23:19:08 -0700 Subject: [PATCH 10/30] Point edgezero dependencies at PR #306 branch (State + with_state) Switch the six edgezero git deps from branch main to worktree-state-nested-secrets-spec-review (stackpop/edgezero#306, stacked on #300) to pick up the Phase 0 State extractor work. cargo check-axum passes. --- Cargo.lock | 111 +++++++++-------------------------------------------- Cargo.toml | 12 +++--- 2 files changed, 25 insertions(+), 98 deletions(-) diff --git a/Cargo.lock b/Cargo.lock index c7716783..6b68ad21 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -766,7 +766,7 @@ version = "3.1.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "faf9468729b8cbcea668e36183cb69d317348c2e08e994829fb56ebfdfbaac34" dependencies = [ - "windows-sys 0.61.2", + "windows-sys 0.48.0", ] [[package]] @@ -1397,7 +1397,7 @@ dependencies = [ [[package]] name = "edgezero-adapter" version = "0.1.0" -source = "git+https://github.com/stackpop/edgezero?branch=main#42843b1c3934fab32f99cc76ddd5881f421cccc7" +source = "git+https://github.com/stackpop/edgezero?branch=worktree-state-nested-secrets-spec-review#6ebc29a58df179e1c61e15cc2a54edae927a3f2f" dependencies = [ "toml", ] @@ -1405,7 +1405,7 @@ dependencies = [ [[package]] name = "edgezero-adapter-axum" version = "0.1.0" -source = "git+https://github.com/stackpop/edgezero?branch=main#42843b1c3934fab32f99cc76ddd5881f421cccc7" +source = "git+https://github.com/stackpop/edgezero?branch=worktree-state-nested-secrets-spec-review#6ebc29a58df179e1c61e15cc2a54edae927a3f2f" dependencies = [ "anyhow", "async-trait", @@ -1433,7 +1433,7 @@ dependencies = [ [[package]] name = "edgezero-adapter-cloudflare" version = "0.1.0" -source = "git+https://github.com/stackpop/edgezero?branch=main#42843b1c3934fab32f99cc76ddd5881f421cccc7" +source = "git+https://github.com/stackpop/edgezero?branch=worktree-state-nested-secrets-spec-review#6ebc29a58df179e1c61e15cc2a54edae927a3f2f" dependencies = [ "anyhow", "async-trait", @@ -1456,7 +1456,7 @@ dependencies = [ [[package]] name = "edgezero-adapter-fastly" version = "0.1.0" -source = "git+https://github.com/stackpop/edgezero?branch=main#42843b1c3934fab32f99cc76ddd5881f421cccc7" +source = "git+https://github.com/stackpop/edgezero?branch=worktree-state-nested-secrets-spec-review#6ebc29a58df179e1c61e15cc2a54edae927a3f2f" dependencies = [ "anyhow", "async-stream", @@ -1485,7 +1485,7 @@ dependencies = [ [[package]] name = "edgezero-adapter-spin" version = "0.1.0" -source = "git+https://github.com/stackpop/edgezero?branch=main#42843b1c3934fab32f99cc76ddd5881f421cccc7" +source = "git+https://github.com/stackpop/edgezero?branch=worktree-state-nested-secrets-spec-review#6ebc29a58df179e1c61e15cc2a54edae927a3f2f" dependencies = [ "anyhow", "async-trait", @@ -1512,7 +1512,7 @@ dependencies = [ [[package]] name = "edgezero-cli" version = "0.1.0" -source = "git+https://github.com/stackpop/edgezero?branch=main#42843b1c3934fab32f99cc76ddd5881f421cccc7" +source = "git+https://github.com/stackpop/edgezero?branch=worktree-state-nested-secrets-spec-review#6ebc29a58df179e1c61e15cc2a54edae927a3f2f" dependencies = [ "chrono", "clap", @@ -1537,7 +1537,7 @@ dependencies = [ [[package]] name = "edgezero-core" version = "0.1.0" -source = "git+https://github.com/stackpop/edgezero?branch=main#42843b1c3934fab32f99cc76ddd5881f421cccc7" +source = "git+https://github.com/stackpop/edgezero?branch=worktree-state-nested-secrets-spec-review#6ebc29a58df179e1c61e15cc2a54edae927a3f2f" dependencies = [ "anyhow", "async-compression", @@ -1568,12 +1568,13 @@ dependencies = [ [[package]] name = "edgezero-macros" version = "0.1.0" -source = "git+https://github.com/stackpop/edgezero?branch=main#42843b1c3934fab32f99cc76ddd5881f421cccc7" +source = "git+https://github.com/stackpop/edgezero?branch=worktree-state-nested-secrets-spec-review#6ebc29a58df179e1c61e15cc2a54edae927a3f2f" dependencies = [ "log", "proc-macro2", "quote", "serde", + "serde_json", "syn 2.0.118", "toml", "validator", @@ -1675,7 +1676,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb" dependencies = [ "libc", - "windows-sys 0.61.2", + "windows-sys 0.59.0", ] [[package]] @@ -2551,7 +2552,7 @@ checksum = "3640c1c38b8e4e43584d8df18be5fc6b0aa314ce6ebf51b53313d4306cca8e46" dependencies = [ "hermit-abi", "libc", - "windows-sys 0.61.2", + "windows-sys 0.59.0", ] [[package]] @@ -3564,7 +3565,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "b570b25f7617e43d59005d0990ccb79e950a423952cea19671b7a876da390adf" dependencies = [ "anyhow", - "itertools 0.13.0", + "itertools 0.10.5", "proc-macro2", "quote", "syn 2.0.118", @@ -3648,7 +3649,7 @@ dependencies = [ "once_cell", "socket2", "tracing", - "windows-sys 0.60.2", + "windows-sys 0.59.0", ] [[package]] @@ -4010,7 +4011,7 @@ dependencies = [ "errno", "libc", "linux-raw-sys", - "windows-sys 0.61.2", + "windows-sys 0.59.0", ] [[package]] @@ -4078,7 +4079,7 @@ dependencies = [ "security-framework", "security-framework-sys", "webpki-root-certs", - "windows-sys 0.61.2", + "windows-sys 0.59.0", ] [[package]] @@ -4726,7 +4727,7 @@ dependencies = [ "getrandom 0.4.3", "once_cell", "rustix", - "windows-sys 0.61.2", + "windows-sys 0.59.0", ] [[package]] @@ -5801,7 +5802,7 @@ version = "0.1.11" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "c2a7b1c03c876122aa43f3020e6c3c3ee5c05081c9a00739faf7503aeba10d22" dependencies = [ - "windows-sys 0.61.2", + "windows-sys 0.48.0", ] [[package]] @@ -5907,15 +5908,6 @@ dependencies = [ "windows-targets 0.52.6", ] -[[package]] -name = "windows-sys" -version = "0.60.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f2f500e4d28234f72040990ec9d39e3a6b950f9f22d3dba18416c35882612bcb" -dependencies = [ - "windows-targets 0.53.5", -] - [[package]] name = "windows-sys" version = "0.61.2" @@ -5949,30 +5941,13 @@ dependencies = [ "windows_aarch64_gnullvm 0.52.6", "windows_aarch64_msvc 0.52.6", "windows_i686_gnu 0.52.6", - "windows_i686_gnullvm 0.52.6", + "windows_i686_gnullvm", "windows_i686_msvc 0.52.6", "windows_x86_64_gnu 0.52.6", "windows_x86_64_gnullvm 0.52.6", "windows_x86_64_msvc 0.52.6", ] -[[package]] -name = "windows-targets" -version = "0.53.5" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "4945f9f551b88e0d65f3db0bc25c33b8acea4d9e41163edf90dcd0b19f9069f3" -dependencies = [ - "windows-link", - "windows_aarch64_gnullvm 0.53.1", - "windows_aarch64_msvc 0.53.1", - "windows_i686_gnu 0.53.1", - "windows_i686_gnullvm 0.53.1", - "windows_i686_msvc 0.53.1", - "windows_x86_64_gnu 0.53.1", - "windows_x86_64_gnullvm 0.53.1", - "windows_x86_64_msvc 0.53.1", -] - [[package]] name = "windows_aarch64_gnullvm" version = "0.48.5" @@ -5985,12 +5960,6 @@ version = "0.52.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "32a4622180e7a0ec044bb555404c800bc9fd9ec262ec147edd5989ccd0c02cd3" -[[package]] -name = "windows_aarch64_gnullvm" -version = "0.53.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "a9d8416fa8b42f5c947f8482c43e7d89e73a173cead56d044f6a56104a6d1b53" - [[package]] name = "windows_aarch64_msvc" version = "0.48.5" @@ -6003,12 +5972,6 @@ version = "0.52.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "09ec2a7bb152e2252b53fa7803150007879548bc709c039df7627cabbd05d469" -[[package]] -name = "windows_aarch64_msvc" -version = "0.53.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "b9d782e804c2f632e395708e99a94275910eb9100b2114651e04744e9b125006" - [[package]] name = "windows_i686_gnu" version = "0.48.5" @@ -6021,24 +5984,12 @@ version = "0.52.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "8e9b5ad5ab802e97eb8e295ac6720e509ee4c243f69d781394014ebfe8bbfa0b" -[[package]] -name = "windows_i686_gnu" -version = "0.53.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "960e6da069d81e09becb0ca57a65220ddff016ff2d6af6a223cf372a506593a3" - [[package]] name = "windows_i686_gnullvm" version = "0.52.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "0eee52d38c090b3caa76c563b86c3a4bd71ef1a819287c19d586d7334ae8ed66" -[[package]] -name = "windows_i686_gnullvm" -version = "0.53.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "fa7359d10048f68ab8b09fa71c3daccfb0e9b559aed648a8f95469c27057180c" - [[package]] name = "windows_i686_msvc" version = "0.48.5" @@ -6051,12 +6002,6 @@ version = "0.52.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "240948bc05c5e7c6dabba28bf89d89ffce3e303022809e73deaefe4f6ec56c66" -[[package]] -name = "windows_i686_msvc" -version = "0.53.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "1e7ac75179f18232fe9c285163565a57ef8d3c89254a30685b57d83a38d326c2" - [[package]] name = "windows_x86_64_gnu" version = "0.48.5" @@ -6069,12 +6014,6 @@ version = "0.52.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "147a5c80aabfbf0c7d901cb5895d1de30ef2907eb21fbbab29ca94c5b08b1a78" -[[package]] -name = "windows_x86_64_gnu" -version = "0.53.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "9c3842cdd74a865a8066ab39c8a7a473c0778a3f29370b5fd6b4b9aa7df4a499" - [[package]] name = "windows_x86_64_gnullvm" version = "0.48.5" @@ -6087,12 +6026,6 @@ version = "0.52.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "24d5b23dc417412679681396f2b49f3de8c1473deb516bd34410872eff51ed0d" -[[package]] -name = "windows_x86_64_gnullvm" -version = "0.53.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "0ffa179e2d07eee8ad8f57493436566c7cc30ac536a3379fdf008f47f6bb7ae1" - [[package]] name = "windows_x86_64_msvc" version = "0.48.5" @@ -6105,12 +6038,6 @@ version = "0.52.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "589f6da84c646204747d1270a2a5661ea66ed1cced2631d546fdfb155959f9ec" -[[package]] -name = "windows_x86_64_msvc" -version = "0.53.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d6bbff5f0aada427a1e5a6da5f1f98158182f26556f345ac9e04d36d0ebed650" - [[package]] name = "winnow" version = "0.7.15" diff --git a/Cargo.toml b/Cargo.toml index 3d139333..2faeee12 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -43,12 +43,12 @@ criterion = { version = "0.5", default-features = false, features = ["cargo_benc derive_more = { version = "2.0", features = ["display", "error"] } directories = "5" ed25519-dalek = { version = "2.2", features = ["rand_core"] } -edgezero-adapter-axum = { git = "https://github.com/stackpop/edgezero", branch = "main", default-features = false } -edgezero-adapter-cloudflare = { git = "https://github.com/stackpop/edgezero", branch = "main", default-features = false } -edgezero-adapter-fastly = { git = "https://github.com/stackpop/edgezero", branch = "main", default-features = false } -edgezero-adapter-spin = { git = "https://github.com/stackpop/edgezero", branch = "main", default-features = false } -edgezero-cli = { git = "https://github.com/stackpop/edgezero", branch = "main" } -edgezero-core = { git = "https://github.com/stackpop/edgezero", branch = "main", default-features = false } +edgezero-adapter-axum = { git = "https://github.com/stackpop/edgezero", branch = "worktree-state-nested-secrets-spec-review", default-features = false } +edgezero-adapter-cloudflare = { git = "https://github.com/stackpop/edgezero", branch = "worktree-state-nested-secrets-spec-review", default-features = false } +edgezero-adapter-fastly = { git = "https://github.com/stackpop/edgezero", branch = "worktree-state-nested-secrets-spec-review", default-features = false } +edgezero-adapter-spin = { git = "https://github.com/stackpop/edgezero", branch = "worktree-state-nested-secrets-spec-review", default-features = false } +edgezero-cli = { git = "https://github.com/stackpop/edgezero", branch = "worktree-state-nested-secrets-spec-review" } +edgezero-core = { git = "https://github.com/stackpop/edgezero", branch = "worktree-state-nested-secrets-spec-review", default-features = false } env_logger = "0.11" error-stack = "0.6" fastly = "0.12" From 7e914df174ffde27c3d07ba3c9b27600033f8ce3 Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Thu, 2 Jul 2026 23:52:44 -0700 Subject: [PATCH 11/30] Amend spec/plan per review 6 - DataDome IP-CIDR config store (datadome-ip-bypass) added to store inventory - Explicit app-config decision: store id trusted_server_config, key app_config; rename DEFAULT_CONFIG_STORE_ID + repoint request_signing.config_store_id - Task 3: ConfigRegistry::named returns ConfigStoreBinding -> use binding.handle.get; map EdgeZero Ok(None)/Err to PlatformError - Task 6: exact builder signatures (KV Result,FastlyError>, config/secret Option<..>) + missing-store/open-failure policy - Task 5: DataDome secret read tested on a protected NON-integration route - StoreName reconciled to logical read id (D7); doc + call-site audit step - Task 4: Axum boot reads .edgezero/local-config-trusted_server_config.json, no env override --- ...07-02-edgezero-store-registry-migration.md | 41 +++++++++++-------- ...26-07-02-edgezero-full-migration-design.md | 6 ++- 2 files changed, 30 insertions(+), 17 deletions(-) diff --git a/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md b/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md index 8b895198..11b2998a 100644 --- a/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md +++ b/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md @@ -43,13 +43,15 @@ Run: cd /Users/ag/projects/iab/trusted-server/.claude/worktrees/edgezero-migration-spec # KV ids (logical ids referenced by Settings — NOT Fastly-only platform stores) rg -n 'ec_store|consent_store|creative_store' crates/trusted-server-core/src/settings.rs crates/trusted-server-core/src/consent_config.rs crates/trusted-server-core/src/auction_config_types.rs trusted-server.example.toml -# config ids -rg -n 'config_store_id|jwks|JWKS_CONFIG_STORE_NAME|"app_config"|config_store\s*=' crates/trusted-server-core trusted-server.example.toml +# config ids (incl. DataDome IP-CIDR config store) +rg -n 'config_store_id|jwks|JWKS_CONFIG_STORE_NAME|"app_config"|config_store\s*=|datadome-ip-bypass|default_ip_cidr_source_store' crates/trusted-server-core trusted-server.example.toml # secret ids rg -n 'secret_store_id|secret_store\s*=|"secrets"|ts_secrets|signing_keys|SIGNING_SECRET_STORE_NAME' crates/trusted-server-core trusted-server.example.toml rg -n '\[stores\.' edgezero.toml ``` -Expected (verified): **KV** ids = `ec.ec_store` (`ec_identity_store`, `settings.rs:452`), `consent.consent_store` (`consent_config.rs:80`), and `auction.creative_store` (`auction_config_types.rs:28`, default `"creative_store"`, **deprecated** — creatives are delivered inline); **config** ids = `app_config` (`request_signing.config_store_id`) + the JWKS store (`JWKS_CONFIG_STORE_NAME`); **secret** ids = `secrets` (`request_signing.secret_store_id`), DataDome `ts_secrets`, the S3 secret store, `signing_keys` (`SIGNING_SECRET_STORE_NAME`) — versus `edgezero.toml` declaring only one id per kind. NOTE: `counter_store` (`RATE_COUNTER_NAME` in the Fastly `rate_limiter.rs`) and `opid_store` are **Fastly-only** platform stores, not `Settings` logical ids — out of scope for D5. `creative_store` **is** a `Settings` id: declare it in `[stores.kv]` (deprecated) so strict lookup can't fail, and flag it for removal in a later phase. +Expected (verified): **KV** ids = `ec.ec_store` (`ec_identity_store`, `settings.rs:452`), `consent.consent_store` (`consent_config.rs:80`), and `auction.creative_store` (`auction_config_types.rs:28`, default `"creative_store"`, **deprecated** — creatives are delivered inline); **config** ids = the app-config blob store (**store id `trusted_server_config`**, see D5 rule below), `request_signing.config_store_id`, the JWKS store (`JWKS_CONFIG_STORE_NAME`), and **DataDome's IP-CIDR config store** (`ProtectionIpCidrSourceConfig.config_store`, default `datadome-ip-bypass`, `protection_scope.rs:165`); **secret** ids = `secrets` (`request_signing.secret_store_id`), DataDome `ts_secrets`, the S3 secret store, `signing_keys` (`SIGNING_SECRET_STORE_NAME`) — versus `edgezero.toml` declaring only one id per kind. NOTE: `counter_store` (`RATE_COUNTER_NAME` in the Fastly `rate_limiter.rs`) and `opid_store` are **Fastly-only** platform stores, not `Settings` logical ids — out of scope for D5. `creative_store` **is** a `Settings` id: declare it in `[stores.kv]` (deprecated) so strict lookup can't fail, and flag it for removal in a later phase. + + **D5 app-config store-id/key decision (record in Task 1 Output):** the app-config blob → config **store id `trusted_server_config`**, blob **key `app_config`** (`CONFIG_BLOB_KEY`). This changes `settings_data.rs::DEFAULT_CONFIG_STORE_ID` from `"app_config"` to `"trusted_server_config"` (it is currently a *store id*, `settings_data.rs:11`) and repoints `request_signing.config_store_id` in `trusted-server.example.toml`/fixtures to `trusted_server_config`; `app_config` survives only as the blob **key**. - [ ] **Step 2: Enumerate runtime WRITE sites** @@ -120,7 +122,9 @@ Expected: FAIL — `ec_identity_store` (kv), `app_config`/JWKS (config), `secret - [ ] **Step 3: Implement `referenced_store_ids_by_kind()` + manifest helper** -Add the `ReferencedStoreIds` struct + method returning **KV** ids (`ec.ec_store`, `consent.consent_store`, `auction.creative_store`), **config** ids (`request_signing.config_store_id`, `JWKS_CONFIG_STORE_NAME`, app-config), **secret** ids (`request_signing.secret_store_id`, DataDome, S3, `SIGNING_SECRET_STORE_NAME`). Do **not** include `counter_store`/`opid_store` — those are Fastly-adapter constants, not `Settings` fields. Add test-only `declared_store_ids_by_kind_from_manifest()` parsing `edgezero.toml`. +Add the `ReferencedStoreIds` struct + method returning **KV** ids (`ec.ec_store`, `consent.consent_store`, `auction.creative_store`), **config** ids (`request_signing.config_store_id`, `JWKS_CONFIG_STORE_NAME`, the app-config store id, and **every `ProtectionIpCidrSourceConfig.config_store`** from DataDome scopes — default `datadome-ip-bypass`), **secret** ids (`request_signing.secret_store_id`, DataDome `ts_secrets`, S3, `SIGNING_SECRET_STORE_NAME`). Do **not** include `counter_store`/`opid_store` — those are Fastly-adapter constants, not `Settings` fields. Add test-only `declared_store_ids_by_kind_from_manifest()` parsing `edgezero.toml`. + +Also apply the **D5 store-id rename** here: set `settings_data.rs::DEFAULT_CONFIG_STORE_ID = "trusted_server_config"`, repoint `request_signing.config_store_id` to `trusted_server_config` in `trusted-server.example.toml` + the integration fixture, and declare `datadome-ip-bypass` + JWKS + `trusted_server_config` under `[stores.config]` in `edgezero.toml`. - [ ] **Step 4: Update `edgezero.toml` + config fields/fixtures per the Task 1 map** @@ -149,10 +153,10 @@ Concrete D6-a mechanism. The bespoke traits read **by `StoreName`** and callers - Test: `crates/trusted-server-core/src/platform/composite.rs` (`#[cfg(test)]`) **Interfaces:** -- Consumes: `edgezero_core::store_registry::{ConfigRegistry, SecretRegistry}`, an `Arc`/`Arc` writer. +- Consumes: `edgezero_core::store_registry::{ConfigRegistry, SecretRegistry, ConfigStoreBinding, BoundSecretStore}`, an `Arc`/`Arc` writer. - Produces: - - `CompositeConfigStore::new(reader: ConfigRegistry, writer: Arc) -> Self` implementing `PlatformConfigStore`: `get(store_name, key)` → `reader.named(store_name.as_str()).ok_or(PlatformError::ConfigStore)?.get(key)`; `put`/`delete` → `writer`. - - `CompositeSecretStore::new(reader: SecretRegistry, writer: Arc) -> Self` implementing `PlatformSecretStore`: `get_bytes(store_name, key)` → `reader.named(store_name.as_str()).ok_or(PlatformError::SecretStore)?.get_bytes(key)`; `create`/`delete` → `writer`. A store_name not in the registry is a hard error (strict), not a silent fallback. + - `CompositeConfigStore::new(reader: ConfigRegistry, writer: Arc) -> Self` implementing `PlatformConfigStore`. **`ConfigRegistry::named(id)` returns `Option`, not a handle** — so `get(store_name, key)` = resolve `binding = reader.named(store_name.as_str()).ok_or(PlatformError::ConfigStore)?`, then `block_on(binding.handle.get(key))`. EdgeZero `ConfigStore::get` returns `Result, ConfigStoreError>`; the bespoke `PlatformConfigStore::get` returns `Result`, so map `Ok(None)` → `PlatformError::ConfigStore` (missing key) and `Err(ConfigStoreError::*)` → `PlatformError::ConfigStore`. `put`/`delete` → `writer`. + - `CompositeSecretStore::new(reader: SecretRegistry, writer: Arc) -> Self` implementing `PlatformSecretStore`: `get_bytes(store_name, key)` = `reader.named(store_name.as_str()).ok_or(PlatformError::SecretStore)?` → `block_on(bound.get_bytes(key))` (here `named` yields a `BoundSecretStore` which **does** have `get_bytes`); map `Ok(None)`/`Err` → `PlatformError::SecretStore`. `create`/`delete` → `writer`. A store_name not in the registry is a hard error (strict), not a silent fallback. - [ ] **Step 1: Write the failing test — reads resolve the NAMED store; unknown store errors; writes delegate** @@ -193,18 +197,20 @@ Expected: FAIL (module does not exist). - [ ] **Step 3: Implement `composite.rs`** -`get`/`get_bytes` resolve `reader.named(store_name.as_str())` (strict — `None` → `PlatformError`), then call the bound handle via `futures::executor::block_on` (mirror `storage/kv_store.rs`), mapping `edgezero_core` errors → `PlatformError`. `put`/`create`/`delete` forward to `writer`. Add `config_registry(entries, default)` / `secret_registry(...)` / `RecordingConfigWriter` test helpers that build a real `StoreRegistry` from in-memory EdgeZero stores. +`get` resolves `reader.named(store_name)` → `ConfigStoreBinding`, then `block_on(binding.handle.get(key))`; `get_bytes` resolves `reader.named(store_name)` → `BoundSecretStore`, then `block_on(bound.get_bytes(key))` (mirror `storage/kv_store.rs`). Strict: `named` returning `None` → `PlatformError`; EdgeZero `Ok(None)`/`Err` → `PlatformError`. `put`/`create`/`delete` forward to `writer`. Add `config_registry(entries, default)` / `secret_registry(...)` / `RecordingConfigWriter` test helpers that build a real `StoreRegistry` from in-memory EdgeZero stores (config entries wrapped as `ConfigStoreBinding { handle, default_key }`). - [ ] **Step 4: Run to verify it passes** Run: `cargo test-fastly composite_config_reads_named_store_and_writes_delegate` Expected: PASS. -- [ ] **Step 5: Commit** +- [ ] **Step 5: Reconcile `StoreName` semantics (D7).** `platform/types.rs::StoreName` is documented as an "edge-visible **platform** name". The composite now resolves `registry.named(store_name.as_str())` by **logical id**, so `StoreName` for reads must carry the **logical store id**. Update the `StoreName` doc comment to say "logical runtime store id" for reads, and audit read call sites (`request_signing/{signing,rotation}.rs`, `proxy.rs`, `integrations/datadome/{protection,protection_scope}.rs`) to confirm they pass **logical ids** (`trusted_server_config`, `jwks_store`, `ts_secrets`, `datadome-ip-bypass`, …), not physical platform names. No functional change if ids already equal names (D7 convention), but the doc + audit prevent implementers from passing physical names into logical registries. + +- [ ] **Step 6: Commit** ```bash git add crates/trusted-server-core/src/platform/ -git commit -m "Add registry-backed composite store: EdgeZero reads by store_name, management-path writes" +git commit -m "Add registry-backed composite store; document StoreName as logical read id" ``` --- @@ -242,7 +248,7 @@ fn get_settings_reads_blob_via_edgezero_handle() { Run: `cargo test-fastly get_settings_reads_blob_via_edgezero_handle` → Expected: FAIL. -- [ ] **Step 2: Re-type `get_settings_from_config_store`** to `(&ConfigStoreHandle, key: &str)`; in Fastly `load_settings_from_config_store()` open the EdgeZero `FastlyConfigStore` at boot (`ConfigStore::open`/adapter constructor) and wrap in a `ConfigStoreHandle`; in Axum `build_state()` open the EdgeZero Axum config store. The adapter-level boot wiring is exercised by each adapter's existing `build_state` test path (no new Viceroy test needed — the core test above covers the parse logic). +- [ ] **Step 2: Re-type `get_settings_from_config_store`** to `(&ConfigStoreHandle, key: &str)`, called with **store id `trusted_server_config`, key `app_config`** (D5). In Fastly `load_settings_from_config_store()` open the EdgeZero `FastlyConfigStore` for `trusted_server_config` at boot and wrap in a `ConfigStoreHandle`. In Axum `build_state()` open the EdgeZero Axum config store, which reads **`.edgezero/local-config-trusted_server_config.json`** (`edgezero-adapter-axum/src/config_store.rs` — id-scoped local file); do **not** apply any env-key override (D7 — runtime is config-store-only). The adapter-level boot wiring is exercised by each adapter's existing `build_state` test path (no new Viceroy test needed — the core test above covers the parse logic). - [ ] **Step 3: Run to verify pass** (core test + adapter boot suites) @@ -275,7 +281,7 @@ These adapters use EdgeZero `dispatch_with_registries` (registries already inser - [ ] **Step 1: Write failing Axum tests covering the default AND a non-default config id AND a non-default secret id** (in the Axum app test module): - `discovery_reads_jwks_from_nondefault_config_store` — `GET /.well-known/trusted-server.json`: seed the Axum `ConfigRegistry` with two ids (`trusted_server_config` default + the JWKS store id); assert `200` + the JWKS `kid` in the body (proves non-default **config** id resolution). - - `datadome_reads_secret_from_nondefault_secret_store` — a request to the DataDome integration route: seed the `SecretRegistry` with two ids (default + `ts_secrets`) and the DataDome server-side key under `ts_secrets`; assert the handler reads it (proves non-default **secret** id resolution). + - `datadome_reads_secret_from_nondefault_secret_store` — a request to a **protected non-integration** route (the DataDome server-side key is read during protected-request *filtering* in `protection.rs`, which **skips** the `/integrations/datadome/*` routes — see `protection.rs:110`), so drive a publisher/first-party route in DataDome's protection scope. Seed the `SecretRegistry` with two ids (default + `ts_secrets`) and the server-side key under `ts_secrets`; assert the filter reads it (proves non-default **secret** id resolution). - `first_party_proxy_reads_s3_secret` — `GET /first-party/proxy` for an S3-auth asset route: seed the S3 secret id; assert the SigV4 path obtains the secret (proves the S3 secret read). Run each (one filter per `cargo test` invocation): @@ -312,8 +318,11 @@ EdgeZero's Fastly `dispatch_with_registries` and its registry builders are `pub( - Test: `crates/trusted-server-adapter-fastly/src/registries.rs` (`#[cfg(test)]`) + a route test **Interfaces:** -- Consumes: `StoresMetadata` (from `Hooks::stores()`), EdgeZero `FastlyConfigStore`/`FastlyKvStore`/`FastlySecretStore` open primitives, `StoreRegistry::from_parts`. -- Produces: `build_config_registry(&StoresMetadata) -> ConfigRegistry` (+ `_secret_/_kv_` variants) that opens each declared store **by its logical id** (per **D7** hard cutoff — no runtime env/dictionary read; platform store name == logical id). +- Consumes: `StoresMetadata` (from `Hooks::stores()`), EdgeZero `FastlyConfigStore`/`FastlyKvStore`/`FastlySecretStore` open primitives, `StoreRegistry::from_parts` (which returns **`Option`** — `None` when the default id is absent from `by_id`). +- Produces (signatures mirror EdgeZero's own private Fastly builders): + - `build_kv_registry(&StoresMetadata) -> Result, FastlyError>` — KV store `open` can fail (→ `Err`); a metadata with no KV stores or a missing default → `Ok(None)`. + - `build_config_registry(&StoresMetadata) -> Option` and `build_secret_registry(&StoresMetadata) -> Option` — `None` when the kind is undeclared or the default id can't be assembled. + - **Failure policy:** each opens every declared id **by logical id** (D7). If a *declared* store fails to open (KV `Err`) propagate it to the request as an error; if the *default* id is missing, `from_parts` yields `None` → the registry is not inserted → the strict extractor later returns `None` → the handler surfaces a 500 (no silent fallback). This matches EdgeZero's `dispatch_with_registries` behavior for the other adapters. - [ ] **Step 1: (D7) No runtime env reader.** Per D7 the runtime does **not** read `EDGEZERO__STORES__*__NAME` — stores are opened by **logical id**. This deletes the need for a Fastly runtime-dictionary `EnvConfig` reader (and sidesteps that `fastly::ConfigStore` has no `iter()` and EdgeZero's reader is private). If a deployment ever needs to remap a physical store name, that is handled at provisioning time, not here. No code in this step; it records the design constraint the builders follow. @@ -321,9 +330,9 @@ EdgeZero's Fastly `dispatch_with_registries` and its registry builders are `pub( Run: `cargo test-fastly build_config_registry_resolves_declared_ids` → Expected: FAIL. -- [ ] **Step 3: Implement the three builders** in `registries.rs`: iterate `StoreMetadata.ids`, open the EdgeZero Fastly store **by the logical id** (`FastlyConfigStore`/`FastlyKvStore`/`FastlySecretStore` open primitive), and assemble `StoreRegistry::from_parts(by_id, default_id)`. No `EnvConfig`, no runtime dictionary. +- [ ] **Step 3: Implement the three builders** in `registries.rs` with the signatures above: iterate `StoreMetadata.ids`, open the EdgeZero Fastly store **by the logical id** (`FastlyConfigStore`/`FastlyKvStore`/`FastlySecretStore` open primitive), collect into a `BTreeMap`, and `StoreRegistry::from_parts(by_id, default_id.to_owned())` (propagating the KV `open` error in `build_kv_registry`). No `EnvConfig`, no runtime dictionary. -- [ ] **Step 4: Insert registries in the oneshot block** — replace the lone `core_req.extensions_mut().insert(config_store)` at `main.rs:477` with inserts of `ConfigRegistry`/`SecretRegistry`/`KvRegistry` (built via Step 3), preserving the existing `client_info`/`device_signals` inserts. +- [ ] **Step 4: Insert registries in the oneshot block** — replace the lone `core_req.extensions_mut().insert(config_store)` at `main.rs:477`: build the three registries via Step 3 (propagate `build_kv_registry`'s `FastlyError` into the dispatch's `Result`), and `if let Some(reg) = ...` insert each into `core_req.extensions_mut()`, preserving the existing `client_info`/`device_signals` inserts. - [ ] **Step 5: Write a failing Fastly route test** — `GET /.well-known/trusted-server.json` via the EdgeZero `oneshot` path returns the JWKS doc read through the injected `ConfigRegistry` (built with default + `jwks_store` ids). Name: `oneshot_discovery_reads_jwks_via_registry` (mirror the `StubJwksConfigStore`/`JWKS_CONFIG_STORE_NAME` pattern in `route_tests.rs`, but drive the EdgeZero path, not `route_request`). diff --git a/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md b/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md index 6ed603cd..70c4f1f1 100644 --- a/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md +++ b/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md @@ -138,7 +138,11 @@ Recommendation: **(a)** where the adapter env is available at boot (Cloudflare/S **Changes:** - **Reads (runtime + boot):** route `PlatformConfigStore`/`PlatformSecretStore` **reads** and the `RuntimeServices` config/secret fields through EdgeZero handles resolved from the per-request registries, matching KV — via a **composite store** (reads → EdgeZero, writes → existing management path; see D6-a). Also migrate the **boot-time** config load: Fastly `load_settings_from_config_store()` and Axum `build_state()` read `Settings` at boot through `&FastlyPlatformConfigStore` / `&AxumPlatformConfigStore` **before** request context exists, so those must move to a boot-time EdgeZero config read *before* the bespoke read impls are deleted. Migrate read consumers: `proxy.rs` (S3), `request_signing/{signing,rotation}.rs` (reads), `integrations/datadome/{protection,protection_scope}.rs`. - **Writes (D6):** `KeyRotationManager` writes+deletes **config and secrets at request time** (`store_private_key`/`store_public_jwk`/`delete_key` for `/_ts/admin/keys/rotate` + deactivate/delete). EdgeZero `ConfigStore`/`SecretStore` are **read-only by design**. So `management_api.rs` **cannot be deleted in Phase 1 as originally written**. Resolve per D6 before touching it. -- **Store-id reconciliation (D5, expanded, kind-aware):** every runtime store id referenced by config must be declared in `edgezero.toml` under the **correct kind** — `[stores.kv]`, `[stores.config]`, or `[stores.secrets]` — or strict registry lookup returns `None`. Reconcile at least: **KV** — `ec.ec_store` (`ec_identity_store`), `consent.consent_store`, `auction.creative_store` (deprecated — creatives are inline — but still a `Settings` field, so declare it to keep strict lookup safe); **config** — the app-config blob store, `request_signing.config_store_id` (`app_config` today), the JWKS/config-list store; **secrets** — `request_signing.secret_store_id` (`secrets` today), DataDome secret store (`ts_secrets`), the S3 secret store — plus all `trusted-server.example.toml` + integration/test fixtures. (`counter_store`/`opid_store` are **Fastly-adapter** constants (rate limiter / opid), **not** `Settings` logical ids, and are out of scope.) +- **Store-id reconciliation (D5, expanded, kind-aware):** every runtime store id referenced by config must be declared in `edgezero.toml` under the **correct kind** — `[stores.kv]`, `[stores.config]`, or `[stores.secrets]` — or strict registry lookup returns `None`. Reconcile at least: **KV** — `ec.ec_store` (`ec_identity_store`), `consent.consent_store`, `auction.creative_store` (deprecated — creatives are inline — but still a `Settings` field, so declare it to keep strict lookup safe); **config** — the **app-config blob store** (see the store-id/key rule below), `request_signing.config_store_id` (`app_config` today), the JWKS/config-list store, and **DataDome's IP-CIDR config store** (`ProtectionIpCidrSourceConfig.config_store`, default `datadome-ip-bypass`, `protection_scope.rs`); **secrets** — `request_signing.secret_store_id` (`secrets` today), DataDome secret store (`ts_secrets`), the S3 secret store — plus all `trusted-server.example.toml` + integration/test fixtures. (`counter_store`/`opid_store` are **Fastly-adapter** constants (rate limiter / opid), **not** `Settings` logical ids, and are out of scope.) + + **App-config store-id/key rule (explicit):** the app-config blob lives in the config **store id `trusted_server_config`** (the `edgezero.toml` default) under **blob key `app_config`** (`CONFIG_BLOB_KEY`). Phase 1 changes the current `settings_data.rs::DEFAULT_CONFIG_STORE_ID = "app_config"` (a *store id*) to `trusted_server_config`, and repoints `request_signing.config_store_id` accordingly, keeping `app_config` only as the blob **key**. + + **`StoreName` semantics (D7):** `platform/types.rs::StoreName` is documented as an "edge-visible platform name". Under D7 runtime reads resolve through the registry by **logical id** (`registry.named(id)`), so for reads `StoreName` now carries the **logical store id** (== platform name by default). Phase 1 updates the `StoreName` doc and audits read call sites to pass logical ids, not physical platform names. - **Fastly registry injection (ties to P0-C):** Fastly's custom `oneshot` path (§1) currently inserts only a `ConfigStoreHandle`, not registries via `dispatch_with_registries`. EdgeZero's `dispatch_with_registries` and its registry builders are **`pub(crate)`** (verified in the pinned checkout), so trusted-server must build the registries **locally** (from `StoresMetadata` + `EnvConfig` + the EdgeZero Fastly store open primitives) and insert them into extensions before `oneshot` — or an EdgeZero public builder must be added upstream (**R11**). - Delete the 4× per-adapter `platform.rs` config/secret **read** impls; adapters build registries from `[stores.*]` metadata (via `dispatch_with_registries` on Axum/Cloudflare/Spin, via the Fastly-specific injection above). - Delete `settings_data.rs`'s `FastlyChunkPointer` resolver — EdgeZero's `FastlyConfigStore` resolves chunks transparently. `get_settings_from_config_store` collapses to `ConfigStore::get` + `settings_from_config_blob`. From a4e5b1b5d2275fc417252c881d262defd422074b Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Fri, 3 Jul 2026 00:31:05 -0700 Subject: [PATCH 12/30] Amend spec/plan per review 7 (wiring + trait split + store-id fixes) Blockers: - Hooks::stores() is not overridden on any adapter (empty StoresMetadata); add Task 2 Step 5 to wire stores() from edgezero.toml on all four adapters - Axum uses routes()+AxumDevServer, not run_app; Task 5 Step 0 switches Axum to edgezero_adapter_axum::run_app::() - request-signing reads use jwks_store/signing_keys but writes use config_store_id/secret_store_id; fix example/fixtures to jwks_store/signing_keys (NOT app_config/secrets); only the app-config store renames to trusted_server_config - PlatformConfigStore/SecretStore mix read+write; Task 3 Step 0 splits write-only PlatformConfigWriter/PlatformSecretWriter so Task 8 can drop reads and compile High/Medium: - D7 softened: EdgeZero builders read EnvConfig but fall back to logical id; we set none - Task 2 Step 6: declare stores in fastly.toml/wrangler.toml/spin.toml/Axum local files - Task 2 test parameterized over example + fixture + all-store-refs config - CI gate adds cargo check-cloudflare + check-spin (wasm surfaces) - Fix stale spec D5/R9 wording --- ...07-02-edgezero-store-registry-migration.md | 88 +++++++++++-------- ...26-07-02-edgezero-full-migration-design.md | 10 ++- 2 files changed, 59 insertions(+), 39 deletions(-) diff --git a/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md b/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md index 11b2998a..48a06d2d 100644 --- a/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md +++ b/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md @@ -17,7 +17,7 @@ - No `unwrap()` in production (`expect("should …")`); no `println!`/`eprintln!` (use `log`). - No wildcard imports (except `use super::*` in `#[cfg(test)]`); no imports inside functions. - Commits: sentence case, imperative, no semantic prefixes, no `Co-Authored-By`/AI footers. -- CI gate before PR: `cargo fmt --all -- --check`; `cargo clippy-{fastly,axum,cloudflare,spin-native,spin-wasm}`; `cargo test-{fastly,axum,cloudflare,spin}`; `cargo test --manifest-path crates/trusted-server-integration-tests/Cargo.toml --test parity`. +- CI gate before PR: `cargo fmt --all -- --check`; `cargo clippy-{fastly,axum,cloudflare,spin-native,spin-wasm}`; **`cargo check-cloudflare` + `cargo check-spin`** (wasm-target surfaces — `test-cloudflare`/`test-spin` are native and do **not** compile the wasm runtime paths); `cargo test-{fastly,axum,cloudflare,spin}`; `cargo test --manifest-path crates/trusted-server-integration-tests/Cargo.toml --test parity`. - **Every task leaves all four adapters building and green.** - **EdgeZero `ConfigStore`/`SecretStore` are read-only.** Runtime writes stay on the management path (D6-a). - **Registry lookup is strict:** an unknown logical id yields `None`. Every id any config field names — in **any** kind (kv/config/secrets) — must be declared in `edgezero.toml`. @@ -51,7 +51,9 @@ rg -n '\[stores\.' edgezero.toml ``` Expected (verified): **KV** ids = `ec.ec_store` (`ec_identity_store`, `settings.rs:452`), `consent.consent_store` (`consent_config.rs:80`), and `auction.creative_store` (`auction_config_types.rs:28`, default `"creative_store"`, **deprecated** — creatives are delivered inline); **config** ids = the app-config blob store (**store id `trusted_server_config`**, see D5 rule below), `request_signing.config_store_id`, the JWKS store (`JWKS_CONFIG_STORE_NAME`), and **DataDome's IP-CIDR config store** (`ProtectionIpCidrSourceConfig.config_store`, default `datadome-ip-bypass`, `protection_scope.rs:165`); **secret** ids = `secrets` (`request_signing.secret_store_id`), DataDome `ts_secrets`, the S3 secret store, `signing_keys` (`SIGNING_SECRET_STORE_NAME`) — versus `edgezero.toml` declaring only one id per kind. NOTE: `counter_store` (`RATE_COUNTER_NAME` in the Fastly `rate_limiter.rs`) and `opid_store` are **Fastly-only** platform stores, not `Settings` logical ids — out of scope for D5. `creative_store` **is** a `Settings` id: declare it in `[stores.kv]` (deprecated) so strict lookup can't fail, and flag it for removal in a later phase. - **D5 app-config store-id/key decision (record in Task 1 Output):** the app-config blob → config **store id `trusted_server_config`**, blob **key `app_config`** (`CONFIG_BLOB_KEY`). This changes `settings_data.rs::DEFAULT_CONFIG_STORE_ID` from `"app_config"` to `"trusted_server_config"` (it is currently a *store id*, `settings_data.rs:11`) and repoints `request_signing.config_store_id` in `trusted-server.example.toml`/fixtures to `trusted_server_config`; `app_config` survives only as the blob **key**. + **D5 app-config store-id/key decision (record in Task 1 Output):** the app-config blob → config **store id `trusted_server_config`**, blob **key `app_config`** (`CONFIG_BLOB_KEY`). This changes only `settings_data.rs::DEFAULT_CONFIG_STORE_ID` from `"app_config"` to `"trusted_server_config"` (it is currently a *store id*, `settings_data.rs:11`); `app_config` survives only as the blob **key**. + + **Request-signing store ids (do NOT point at app-config):** request signing reads use hard-coded `JWKS_CONFIG_STORE_NAME = "jwks_store"` (config) + `SIGNING_SECRET_STORE_NAME = "signing_keys"` (secret); writes use `request_signing.config_store_id`/`secret_store_id`. Today the example sets these to `"app_config"`/`"secrets"` — which sends **writes to a different store than reads**. Fix: set `request_signing.config_store_id = "jwks_store"` and `secret_store_id = "signing_keys"` in `trusted-server.example.toml` + fixtures, and declare `jwks_store` (config) + `signing_keys` (secret) as logical ids in `edgezero.toml`. (Under the composite, reads resolve `registry.named("jwks_store")`; writes go to the same store via the writer/management id.) - [ ] **Step 2: Enumerate runtime WRITE sites** @@ -89,55 +91,64 @@ git commit -m "Record Phase 1 kind-aware store-id map and confirm D6-a" - Consumes: Task 1 map. - Produces: `Settings::referenced_store_ids_by_kind() -> ReferencedStoreIds { kv: BTreeSet, config: BTreeSet, secrets: BTreeSet }`; an `edgezero.toml` whose per-kind `ids` are supersets. -- [ ] **Step 1: Write the failing test** +- [ ] **Step 1: Write the failing test (parameterized over multiple configs)** -Add to `settings.rs` under `#[cfg(test)]`: +Cover the example config, the integration fixture, AND a purpose-built config that exercises every store-backed field (DataDome IP-CIDR sources, S3 auth, request-signing) so optional/targeted settings can't escape coverage. Add to `settings.rs` under `#[cfg(test)]`: ```rust -#[test] -fn every_referenced_store_id_is_declared_by_kind() { - let settings = Settings::from_toml(include_str!("../../../trusted-server.example.toml")) - .expect("should parse example config"); +fn assert_all_ids_declared(config_toml: &str, label: &str) { + let settings = Settings::from_toml(config_toml).unwrap_or_else(|e| panic!("{label} should parse: {e}")); let referenced = settings.referenced_store_ids_by_kind(); let declared = declared_store_ids_by_kind_from_manifest(); // reads edgezero.toml - for (kind, ids) in [ - ("kv", &referenced.kv), - ("config", &referenced.config), - ("secrets", &referenced.secrets), - ] { - let declared_for_kind = declared.for_kind(kind); + for (kind, ids) in [("kv", &referenced.kv), ("config", &referenced.config), ("secrets", &referenced.secrets)] { for id in ids { assert!( - declared_for_kind.contains(id), - "{kind} store id `{id}` referenced by Settings is not declared in edgezero.toml", + declared.for_kind(kind).contains(id), + "[{label}] {kind} store id `{id}` referenced by Settings is not declared in edgezero.toml", ); } } } + +#[test] +fn every_referenced_store_id_is_declared_by_kind() { + assert_all_ids_declared(include_str!("../../../trusted-server.example.toml"), "example"); + assert_all_ids_declared( + include_str!("../../trusted-server-integration-tests/fixtures/configs/trusted-server.integration.toml"), + "integration-fixture", + ); + // Purpose-built config exercising DataDome IP-CIDR, S3, and request-signing store refs. + assert_all_ids_declared(include_str!("../testdata/all-store-refs.toml"), "all-store-refs"); +} ``` +(Create `crates/trusted-server-core/src/testdata/all-store-refs.toml` populating every store-id field with a declared id.) - [ ] **Step 2: Run to verify it fails** Run: `cargo test-fastly every_referenced_store_id_is_declared_by_kind` -Expected: FAIL — `ec_identity_store` (kv), `app_config`/JWKS (config), `secrets`/`ts_secrets` (secrets) referenced but not declared. +Expected: FAIL — `ec_identity_store`/`consent_store`/`creative_store` (kv), `jwks_store`/`datadome-ip-bypass`/`trusted_server_config` (config), `signing_keys`/`ts_secrets` (secrets) referenced but not declared. - [ ] **Step 3: Implement `referenced_store_ids_by_kind()` + manifest helper** -Add the `ReferencedStoreIds` struct + method returning **KV** ids (`ec.ec_store`, `consent.consent_store`, `auction.creative_store`), **config** ids (`request_signing.config_store_id`, `JWKS_CONFIG_STORE_NAME`, the app-config store id, and **every `ProtectionIpCidrSourceConfig.config_store`** from DataDome scopes — default `datadome-ip-bypass`), **secret** ids (`request_signing.secret_store_id`, DataDome `ts_secrets`, S3, `SIGNING_SECRET_STORE_NAME`). Do **not** include `counter_store`/`opid_store` — those are Fastly-adapter constants, not `Settings` fields. Add test-only `declared_store_ids_by_kind_from_manifest()` parsing `edgezero.toml`. +Add the `ReferencedStoreIds` struct + method returning **KV** ids (`ec.ec_store`, `consent.consent_store`, `auction.creative_store`), **config** ids (`request_signing.config_store_id`, the app-config store id, and **every `ProtectionIpCidrSourceConfig.config_store`** from DataDome scopes — default `datadome-ip-bypass`), **secret** ids (`request_signing.secret_store_id`, DataDome `ts_secrets`, S3). Do **not** include `counter_store`/`opid_store`. Add test-only `declared_store_ids_by_kind_from_manifest()` parsing `edgezero.toml`. + +Apply the **D5 renames**: set `settings_data.rs::DEFAULT_CONFIG_STORE_ID = "trusted_server_config"`; set `request_signing.config_store_id = "jwks_store"` and `secret_store_id = "signing_keys"` in `trusted-server.example.toml` + fixtures (they must match the read constants — **not** `app_config`/`secrets`). -Also apply the **D5 store-id rename** here: set `settings_data.rs::DEFAULT_CONFIG_STORE_ID = "trusted_server_config"`, repoint `request_signing.config_store_id` to `trusted_server_config` in `trusted-server.example.toml` + the integration fixture, and declare `datadome-ip-bypass` + JWKS + `trusted_server_config` under `[stores.config]` in `edgezero.toml`. +- [ ] **Step 4: Declare every id in `edgezero.toml`** — `[stores.kv]` = `trusted_server_kv`, `ec_identity_store`, `consent_store`, `creative_store`; `[stores.config]` = `trusted_server_config`, `jwks_store`, `datadome-ip-bypass`; `[stores.secrets]` = `trusted_server_secrets`, `signing_keys`, `ts_secrets`, and the S3 secret id. (Names double as the platform store names under D7.) -- [ ] **Step 4: Update `edgezero.toml` + config fields/fixtures per the Task 1 map** +- [ ] **Step 5: Wire `Hooks::stores()` on all four adapters (Blocker — metadata is not wired today).** Each `impl Hooks for TrustedServerApp` currently overrides only `routes()`; the default `stores()` returns **empty** `StoresMetadata`, so no registries can be built from it. Add `fn stores() -> StoresMetadata` returning the `[stores.*]` metadata, generated once from `edgezero.toml`. Prefer a single shared `const`/fn in `trusted-server-core` (e.g. `pub fn stores_metadata() -> StoresMetadata`) that all four adapters return, so the ids live in one place. Verify against `edgezero_core::app::StoresMetadata`/`StoreMetadata` shape. -- [ ] **Step 5: Run to verify pass** +- [ ] **Step 6: Declare the stores in every PLATFORM manifest (Blocker — local resources missing).** D7 requires each logical id to be openable as a real platform store. Add the new ids to: `fastly.toml` (`[local_server.kv_stores]`/`[local_server.config_stores]`/`[local_server.secret_stores]` + the production service store bindings), `crates/trusted-server-adapter-cloudflare/wrangler.toml` (`[[kv_namespaces]]` + config/secret bindings), `crates/trusted-server-adapter-spin/spin.toml` (`key_value_stores` + variables/config), and the Axum local files `.edgezero/local-config-.json` / KV redb defaults. Cross-check each existing manifest — some planned ids (`jwks_store`, `datadome-ip-bypass`, `signing_keys`) may already be partially declared; add the missing ones. + +- [ ] **Step 7: Run to verify pass + full adapter suites + wasm checks** Run: `cargo test-fastly every_referenced_store_id_is_declared_by_kind` +Then: `cargo test-fastly && cargo test-axum && cargo test-cloudflare && cargo test-spin && cargo check-cloudflare && cargo check-spin` Expected: PASS. -- [ ] **Step 6: Full adapter tests + commit** +- [ ] **Step 8: Commit** -Run: `cargo test-fastly && cargo test-axum && cargo test-cloudflare && cargo test-spin` ```bash -git add edgezero.toml trusted-server.example.toml crates/trusted-server-integration-tests/fixtures crates/trusted-server-core/src/settings.rs +git add edgezero.toml fastly.toml trusted-server.example.toml crates/trusted-server-adapter-*/{wrangler.toml,spin.toml} crates/trusted-server-integration-tests/fixtures crates/trusted-server-core/src git commit -m "Declare kv/config/secret store ids in edgezero.toml and reconcile config fields" ``` @@ -148,15 +159,19 @@ git commit -m "Declare kv/config/secret store ids in edgezero.toml and reconcile Concrete D6-a mechanism. The bespoke traits read **by `StoreName`** and callers use **multiple** store ids (`app_config`, JWKS, DataDome, S3, `ec_identity_store` for KV). So the composite must hold the **whole `ConfigRegistry`/`SecretRegistry`** (not a single handle) and resolve `named(store_name)` on each read; writes (`put`/`create`/`delete`) delegate to the existing management-API-backed writer. Preserves `KeyRotationManager` writes with zero call-site changes. **Files:** +- Modify: `crates/trusted-server-core/src/platform/traits.rs` (split write-only traits) - Create: `crates/trusted-server-core/src/platform/composite.rs` (`CompositeConfigStore`, `CompositeSecretStore`) -- Modify: `crates/trusted-server-core/src/platform/mod.rs` (export composite) +- Modify: `crates/trusted-server-core/src/platform/mod.rs` (export composite + writer traits) - Test: `crates/trusted-server-core/src/platform/composite.rs` (`#[cfg(test)]`) **Interfaces:** -- Consumes: `edgezero_core::store_registry::{ConfigRegistry, SecretRegistry, ConfigStoreBinding, BoundSecretStore}`, an `Arc`/`Arc` writer. +- Consumes: `edgezero_core::store_registry::{ConfigRegistry, SecretRegistry, ConfigStoreBinding, BoundSecretStore}`, write-only `Arc`/`Arc`. - Produces: - - `CompositeConfigStore::new(reader: ConfigRegistry, writer: Arc) -> Self` implementing `PlatformConfigStore`. **`ConfigRegistry::named(id)` returns `Option`, not a handle** — so `get(store_name, key)` = resolve `binding = reader.named(store_name.as_str()).ok_or(PlatformError::ConfigStore)?`, then `block_on(binding.handle.get(key))`. EdgeZero `ConfigStore::get` returns `Result, ConfigStoreError>`; the bespoke `PlatformConfigStore::get` returns `Result`, so map `Ok(None)` → `PlatformError::ConfigStore` (missing key) and `Err(ConfigStoreError::*)` → `PlatformError::ConfigStore`. `put`/`delete` → `writer`. - - `CompositeSecretStore::new(reader: SecretRegistry, writer: Arc) -> Self` implementing `PlatformSecretStore`: `get_bytes(store_name, key)` = `reader.named(store_name.as_str()).ok_or(PlatformError::SecretStore)?` → `block_on(bound.get_bytes(key))` (here `named` yields a `BoundSecretStore` which **does** have `get_bytes`); map `Ok(None)`/`Err` → `PlatformError::SecretStore`. `create`/`delete` → `writer`. A store_name not in the registry is a hard error (strict), not a silent fallback. + - New **write-only** traits `PlatformConfigWriter { put; delete }` and `PlatformSecretWriter { create; delete }` (extracted from the read+write `PlatformConfigStore`/`PlatformSecretStore`). This is what lets Task 8 delete the per-adapter **read** impls while keeping the writer object — the writer no longer needs `get`/`get_bytes`. + - `CompositeConfigStore::new(reader: ConfigRegistry, writer: Arc) -> Self` implementing the full read+write `PlatformConfigStore`. **`ConfigRegistry::named(id)` returns `Option`, not a handle** — so `get(store_name, key)` = resolve `binding = reader.named(store_name.as_str()).ok_or(PlatformError::ConfigStore)?`, then `block_on(binding.handle.get(key))`. EdgeZero `ConfigStore::get` returns `Result, ConfigStoreError>`; the bespoke `get` returns `Result`, so map `Ok(None)`/`Err(ConfigStoreError::*)` → `PlatformError::ConfigStore`. `put`/`delete` → `writer`. + - `CompositeSecretStore::new(reader: SecretRegistry, writer: Arc) -> Self` implementing `PlatformSecretStore`: `get_bytes(store_name, key)` = `reader.named(store_name.as_str()).ok_or(PlatformError::SecretStore)?` → `block_on(bound.get_bytes(key))`; map `Ok(None)`/`Err` → `PlatformError::SecretStore`. `create`/`delete` → `writer`. A store_name not in the registry is a hard error (strict), not a silent fallback. + +- [ ] **Step 0: Split write-only traits.** In `traits.rs`, define `PlatformConfigWriter` (`put`, `delete`) and `PlatformSecretWriter` (`create`, `delete`). Keep `PlatformConfigStore`/`PlatformSecretStore` as the read+write surface `RuntimeServices` exposes. This split is the prerequisite that makes Task 8's "delete reads, keep writes" compile. Run `cargo check-axum` to confirm the split compiles before proceeding. - [ ] **Step 1: Write the failing test — reads resolve the NAMED store; unknown store errors; writes delegate** @@ -267,14 +282,17 @@ git commit -m "Load boot config via EdgeZero config store on Fastly and Axum" --- -## Task 5: Wire request registries in Axum, Cloudflare, Spin; RuntimeServices uses the composite +## Task 5: Switch adapters to EdgeZero's registry-aware entry; RuntimeServices uses the composite -These adapters use EdgeZero `dispatch_with_registries` (registries already inserted into extensions). Build `RuntimeServices` config/secret from `CompositeConfigStore`/`CompositeSecretStore` (reader from the request registry; writer = the existing per-adapter write impl). +**Blocker addressed:** Axum today calls `TrustedServerApp::routes()` + `AxumDevServer::with_config(...)` (`adapter-axum/src/main.rs:23`) — which never builds registries. This task switches Axum to EdgeZero's registry-aware `run_app::()` (`edgezero_adapter_axum::dev_server::run_app`, which builds registries from `Hooks::stores()` — now wired in Task 2 Step 5). Cloudflare and Spin already dispatch via EdgeZero `run_app`; confirm their `run_app` builds registries once `stores()` is wired. Then build `RuntimeServices` config/secret from `CompositeConfigStore`/`CompositeSecretStore` (reader from the request registry; writer = the per-adapter write impl). Store-name binding uses EdgeZero's `EnvConfig` fallback-to-logical-id (D7 — we set no `EDGEZERO__STORES__*__NAME`). **Files:** -- Modify: `crates/trusted-server-adapter-{axum,cloudflare,spin}/src/platform.rs` (`build_runtime_services`) +- Modify: `crates/trusted-server-adapter-axum/src/main.rs` (switch to `run_app::()`) +- Modify: `crates/trusted-server-adapter-{axum,cloudflare,spin}/src/platform.rs` (`build_runtime_services` → composite) - Test: `crates/trusted-server-adapter-axum/src/app.rs` route tests (+ cloudflare/spin equivalents) +- [ ] **Step 0: Switch Axum `main.rs` to `run_app::()`.** Replace the `TrustedServerApp::routes()` + `AxumDevServer::with_config(router, config).run()` path with `edgezero_adapter_axum::run_app::()` (preserving bind-address/port behavior via the config surface `run_app` exposes, or `DevServer::run` with `.with_*` registry wiring). Confirm the dev server still binds the configured address. Run `cargo run -p trusted-server-adapter-axum` locally to sanity-check it boots. + **Interfaces:** - Consumes: Task 3 composite; `ConfigRegistry`/`SecretRegistry` from request extensions. - Produces: `RuntimeServices` whose reads flow through EdgeZero, writes through the composite writer. @@ -292,7 +310,7 @@ cargo test-axum first_party_proxy_reads_s3_secret ``` Expected: FAIL (all three). -- [ ] **Step 2: Build `RuntimeServices` via the composite** in each adapter's `build_runtime_services`, passing the whole request `ConfigRegistry`/`SecretRegistry` as the composite reader (Task 3) and keeping the existing writer. +- [ ] **Step 2: Build `RuntimeServices` via the composite** in each adapter's `build_runtime_services`, passing the whole request `ConfigRegistry`/`SecretRegistry` as the composite reader (Task 3) and the per-adapter **write-only** impl (`PlatformConfigWriter`/`PlatformSecretWriter`, Task 3 Step 0 / Task 8) as the writer. - [ ] **Step 3: Run to verify pass** (all three) @@ -373,19 +391,19 @@ git commit -m "Delete duplicated Fastly config-chunk resolver; rely on EdgeZero ## Task 8: Retire per-adapter config/secret READ impls; keep the write path (D6-a) -Now that all reads (boot + request, all adapters) flow through EdgeZero, delete the config/secret **read** implementations. Keep the **write** methods + `management_api.rs` (D6-a). Update the legacy `route_tests.rs` stubs that construct `RuntimeServices` from bespoke read stores. +Now that all reads (boot + request, all adapters) flow through EdgeZero, delete the config/secret **read** implementations. The per-adapter management impls become **write-only** (`PlatformConfigWriter`/`PlatformSecretWriter` from Task 3 Step 0) + `management_api.rs` (D6-a). Update the legacy `route_tests.rs` stubs that construct `RuntimeServices` from bespoke read stores. **Files:** - Modify: `crates/trusted-server-adapter-{fastly,axum,cloudflare,spin}/src/platform.rs` - Modify: `crates/trusted-server-adapter-fastly/src/route_tests.rs` (update stubs to the composite/registry shape) -- [ ] **Step 1: Delete the config/secret read impls** now unused after Tasks 4–6 (`FastlyPlatformConfigStore::get`, `AxumPlatformConfigStore`, `NoopConfigStore`, Cloudflare/Spin equivalents, secret read impls). Keep the write impls + `management_api.rs`. +- [ ] **Step 1: Convert per-adapter config/secret impls to write-only.** The old impls implemented the read+write `PlatformConfigStore`/`PlatformSecretStore` (`FastlyPlatformConfigStore` with `get`+`put`+`delete`, `AxumPlatformConfigStore`, `NoopConfigStore`, Cloudflare/Spin equivalents, secret impls). Now that the composite serves reads, **re-implement them as `PlatformConfigWriter`/`PlatformSecretWriter`** (drop `get`/`get_bytes`; keep `put`/`create`/`delete`). This compiles only because Task 3 Step 0 split the traits — otherwise deleting `get` from a `PlatformConfigStore` impl is a trait-incompleteness error. Keep `management_api.rs`. - [ ] **Step 2: Update `route_tests.rs`** — the stub stores (`StubJwksConfigStore`, etc.) and `RuntimeServices` construction move to the composite/registry shape: build the composite reader from a real `ConfigRegistry`/`SecretRegistry` with **at least two ids** (default + a non-default such as `jwks_store`/`ts_secrets`), and assert an **unknown store id resolves strictly to an error** (not a silent fallback to default). Writer = a recording stub; keep coverage of the write path (`put`/`create`/`delete`) so key-rotation delegation stays tested. - [ ] **Step 3: Full CI gate** -Run: `cargo fmt --all -- --check && cargo clippy-fastly && cargo clippy-axum && cargo clippy-cloudflare && cargo clippy-spin-native && cargo clippy-spin-wasm && cargo test-fastly && cargo test-axum && cargo test-cloudflare && cargo test-spin && cargo test --manifest-path crates/trusted-server-integration-tests/Cargo.toml --test parity` +Run: `cargo fmt --all -- --check && cargo clippy-fastly && cargo clippy-axum && cargo clippy-cloudflare && cargo clippy-spin-native && cargo clippy-spin-wasm && cargo check-cloudflare && cargo check-spin && cargo test-fastly && cargo test-axum && cargo test-cloudflare && cargo test-spin && cargo test --manifest-path crates/trusted-server-integration-tests/Cargo.toml --test parity` Expected: PASS. **Key rotation/delete still works** (composite writer path). - [ ] **Step 4: Commit** diff --git a/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md b/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md index 70c4f1f1..0739c7c9 100644 --- a/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md +++ b/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md @@ -85,7 +85,7 @@ Phase 1 (stores) ──> Phase 2 (config) ──> Phase 3 (secrets) Phase 4 (e **D4 — One typed `Settings` as the AppConfig root.** Replace `TrustedServerAppConfig` (wrapper with empty `SECRET_FIELDS`) by deriving `AppConfig` directly on `Settings`, with `#[secret]` on the real secret fields (Phase 3). Removes a transitional indirection. -**D5 — Reconcile ALL runtime store ids with `edgezero.toml`, kind by kind.** Not just the app-config blob, and **not just config/secrets**: EdgeZero's registry lookup is **strict** (unknown id → `None`), so every logical store id any config field or call site names at runtime must appear in `edgezero.toml` under the **correct kind** — `[stores.kv]`, `[stores.config]`, or `[stores.secrets]`. Reconciliation is kind-aware (`Settings::referenced_store_ids_by_kind()`): e.g. `ec.ec_store` (`ec_identity_store`), consent/creative/counter/opid stores are **KV** ids; `app_config` + the JWKS store are **config** ids; `secrets`, `signing_keys`, DataDome `ts_secrets`, the S3 secret store are **secret** ids. Today `edgezero.toml` declares only one id per kind. Phase 1 must either (a) declare all these ids in `edgezero.toml` and map each to a platform store via `EDGEZERO__STORES__*__NAME`, or (b) collapse them onto the declared defaults and update every config field + fixture. Recommendation: the app-config blob lives in `trusted_server_config` under key `app_config` (`CONFIG_BLOB_KEY`); collapse incidental ids onto the declared defaults where semantically identical, and declare the genuinely-separate ones (e.g. JWKS store). The full id inventory is the Phase 1 plan's task 1. +**D5 — Reconcile ALL runtime store ids with `edgezero.toml`, kind by kind.** Not just the app-config blob, and **not just config/secrets**: EdgeZero's registry lookup is **strict** (unknown id → `None`), so every logical store id any config field or call site names at runtime must appear in `edgezero.toml` under the **correct kind** — `[stores.kv]`, `[stores.config]`, or `[stores.secrets]`. Reconciliation is kind-aware (`Settings::referenced_store_ids_by_kind()`): **KV** ids = `ec.ec_store` (`ec_identity_store`), `consent.consent_store`, `auction.creative_store` (deprecated); **config** ids = the app-config store `trusted_server_config` (key `app_config`), `jwks_store`, `datadome-ip-bypass`; **secret** ids = `signing_keys`, DataDome `ts_secrets`, the S3 secret store. (`counter_store`/`opid_store` are Fastly-adapter constants, not `Settings` ids.) Every declared id must map to a real platform store, opened **by logical id** (D7 — no `EDGEZERO__STORES__*__NAME`). The app-config-store rename and the request-signing store-id fix are detailed just below; the full inventory is the Phase 1 plan's Task 1. **D6 — Runtime write path for request-signing key rotation.** EdgeZero `ConfigStore`/`SecretStore` are **read-only** at runtime; writes go through provisioning (author/ops time). But `KeyRotationManager` writes+deletes config (JWKS) and secrets (private keys) **at request time** via `/_ts/admin/keys/{rotate,deactivate,delete}`, backed by `management_api.rs`. Three resolutions: - **(a) Keep a write-capable admin abstraction** — retain the trusted-server `put`/`create`/`delete` traits + `management_api.rs` for the admin write path only; EdgeZero read-only stores serve the read path. Least disruptive; leaves a non-EdgeZero write path (so "completely on EdgeZero" is not literally met for admin writes). @@ -98,7 +98,7 @@ Phase 1 (stores) ──> Phase 2 (config) ──> Phase 3 (secrets) Phase 4 (e **D7 — Runtime app-config is config-store-only; no runtime environment variables (hard cutoff).** At runtime, every adapter builds `Settings` **solely** from the config-store blob: open store → verify envelope → deserialize (+ secret walk) → validate. There is **no runtime environment-variable overlay** for app config on any adapter. All env-var influence on config content happens at **push time**: `ts config push` (which runs on a host that has the env vars) reads them, applies the AppConfig env overlay, and bakes the resolved values into the signed blob. Consequences: - Delete `Settings::from_toml_and_env` and the `TRUSTED_SERVER__*` overlay entirely (Phase 2) — not merely unused, **forbidden** at runtime. - The AppConfig loader's `env_overlay` is a **push-time-only** option; runtime never applies it. -- **Store-name binding follows the same spirit:** a logical store id resolves to the **platform store of the same name by default** (EdgeZero's `store_name` fallback), so the runtime does **not** need to read `EDGEZERO__STORES__*__NAME` env/dictionary to open stores. Adapters (incl. Fastly's custom path) open stores by **logical id**. If a deployment ever needs a different physical name, that is a **provisioning/manifest** concern resolved at deploy time, never a runtime env read. (This removes the need for a Fastly runtime-dictionary `EnvConfig` reader — see Phase 1 Task 6.) +- **Store-name binding follows the same spirit:** a logical store id resolves to the **platform store of the same name by default** — via EdgeZero's existing `store_name` **fallback to the logical id when the env var is unset**. So we **provision no `EDGEZERO__STORES__*__NAME` values**, and every adapter opens stores by logical id. Note the mechanism, not the absolute: EdgeZero's Axum/Cloudflare/Spin builders still *call* `EnvConfig` internally — but with nothing set they fall back to the logical id, which is D7-compliant with **no EdgeZero change**. Only Fastly's *custom* dispatch path (which bypasses EdgeZero's builders) needs trusted-server-local by-id builders (Phase 1 Task 6). If a deployment ever needs a different physical name, that is a **provisioning/manifest** concern at deploy time, never a runtime env read. --- @@ -140,7 +140,9 @@ Recommendation: **(a)** where the adapter env is available at boot (Cloudflare/S - **Writes (D6):** `KeyRotationManager` writes+deletes **config and secrets at request time** (`store_private_key`/`store_public_jwk`/`delete_key` for `/_ts/admin/keys/rotate` + deactivate/delete). EdgeZero `ConfigStore`/`SecretStore` are **read-only by design**. So `management_api.rs` **cannot be deleted in Phase 1 as originally written**. Resolve per D6 before touching it. - **Store-id reconciliation (D5, expanded, kind-aware):** every runtime store id referenced by config must be declared in `edgezero.toml` under the **correct kind** — `[stores.kv]`, `[stores.config]`, or `[stores.secrets]` — or strict registry lookup returns `None`. Reconcile at least: **KV** — `ec.ec_store` (`ec_identity_store`), `consent.consent_store`, `auction.creative_store` (deprecated — creatives are inline — but still a `Settings` field, so declare it to keep strict lookup safe); **config** — the **app-config blob store** (see the store-id/key rule below), `request_signing.config_store_id` (`app_config` today), the JWKS/config-list store, and **DataDome's IP-CIDR config store** (`ProtectionIpCidrSourceConfig.config_store`, default `datadome-ip-bypass`, `protection_scope.rs`); **secrets** — `request_signing.secret_store_id` (`secrets` today), DataDome secret store (`ts_secrets`), the S3 secret store — plus all `trusted-server.example.toml` + integration/test fixtures. (`counter_store`/`opid_store` are **Fastly-adapter** constants (rate limiter / opid), **not** `Settings` logical ids, and are out of scope.) - **App-config store-id/key rule (explicit):** the app-config blob lives in the config **store id `trusted_server_config`** (the `edgezero.toml` default) under **blob key `app_config`** (`CONFIG_BLOB_KEY`). Phase 1 changes the current `settings_data.rs::DEFAULT_CONFIG_STORE_ID = "app_config"` (a *store id*) to `trusted_server_config`, and repoints `request_signing.config_store_id` accordingly, keeping `app_config` only as the blob **key**. + **App-config store-id/key rule (explicit):** the app-config blob lives in the config **store id `trusted_server_config`** (the `edgezero.toml` default) under **blob key `app_config`** (`CONFIG_BLOB_KEY`). Phase 1 changes only the **app-config** store id — `settings_data.rs::DEFAULT_CONFIG_STORE_ID` from `"app_config"` to `"trusted_server_config"` — keeping `app_config` as the blob **key**. + + **Do NOT repoint request-signing at the app-config store.** Request signing uses its **own** stores: reads use the hard-coded runtime names `JWKS_CONFIG_STORE_NAME = "jwks_store"` (config) and `SIGNING_SECRET_STORE_NAME = "signing_keys"` (secret); writes use `request_signing.config_store_id`/`secret_store_id`, which are the **management-API identifiers of those same jwks/signing stores** (Fastly separates runtime name from management id). Today `trusted-server.example.toml` sets `config_store_id = "app_config"` / `secret_store_id = "secrets"`, which points **writes at the wrong store** (writes JWKS state into app-config while reads look in `jwks_store`). Phase 1 fixes this: set `request_signing.config_store_id` to the **jwks store**'s id and `secret_store_id` to the **signing_keys** store's id, so reads and writes target the same store. Declare `jwks_store` (config) and `signing_keys` (secret) as their own logical ids in `edgezero.toml`. **`StoreName` semantics (D7):** `platform/types.rs::StoreName` is documented as an "edge-visible platform name". Under D7 runtime reads resolve through the registry by **logical id** (`registry.named(id)`), so for reads `StoreName` now carries the **logical store id** (== platform name by default). Phase 1 updates the `StoreName` doc and audits read call sites to pass logical ids, not physical platform names. - **Fastly registry injection (ties to P0-C):** Fastly's custom `oneshot` path (§1) currently inserts only a `ConfigStoreHandle`, not registries via `dispatch_with_registries`. EdgeZero's `dispatch_with_registries` and its registry builders are **`pub(crate)`** (verified in the pinned checkout), so trusted-server must build the registries **locally** (from `StoresMetadata` + `EnvConfig` + the EdgeZero Fastly store open primitives) and insert them into extensions before `oneshot` — or an EdgeZero public builder must be added upstream (**R11**). @@ -261,7 +263,7 @@ Two consequences: (1) edgezero #305 **must** ship `ArrayEach` + `Option` | R1 | Do any `Settings` secrets live inside **arrays**? | **Resolved: yes** (`ec.partners[].api_token`, `handlers[].password`) + optional (`ts_pull_token`). edgezero #305 must ship `ArrayEach` + `Option` (see §5 Phase 3 inventory + Phase 0 note). | | R7 | P0-C: upstream a header-preserving Fastly dispatch, or keep a permanent Fastly dispatch shim? | Decide with edgezero maintainer (§4a); gates the Fastly end-state and Phase 5. | | R8 | P-BOOT: boot-time store handle (a) vs lazy cached first-request load (b), per adapter? | Phase 2 plan (§4a). | -| R9 | D5: reconcile **all** runtime store ids **by kind** — KV (`ec_identity_store`, `consent_store`), config (`app_config`, JWKS), secrets (`secrets`, DataDome `ts_secrets`, S3) + fixtures — with `edgezero.toml`; strict lookup fails otherwise. | Phase 1 plan task 1. | +| R9 | D5: reconcile **all** runtime store ids **by kind** with `edgezero.toml` (strict lookup fails otherwise) — KV: `ec_identity_store`, `consent_store`, `creative_store`; config: `trusted_server_config` (app-config blob, key `app_config`), `jwks_store`, `datadome-ip-bypass`; secrets: `signing_keys`, `ts_secrets`, S3. Request signing uses `jwks_store`/`signing_keys` (not the app-config store). | Phase 1 plan task 1/2. | | R12 | Fastly `EnvConfig` reader is private / `fastly::ConfigStore` has no `iter()`. | **Resolved by D7** — runtime opens stores by logical id; no store-name env/dictionary read; no local `EnvConfig` reader needed. | | R10 | D6: runtime write path for key rotation — keep write-capable admin abstraction (a), move to ops/CLI (b), or upstream an EdgeZero write API (c)? | **Blocks Phase 1 deletions.** Phase 1 plan locks to **(a)**; (b)/(c) → separate plan. | | R11 | Should EdgeZero expose a **public** Fastly registry-builder helper? | Lower priority under D7 — local builders open by logical id and need only public store constructors. Decide with edgezero maintainer if convenient. | From b16a254ddee4519f41832ff90962f1c12f846bea Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Fri, 3 Jul 2026 08:21:57 -0700 Subject: [PATCH 13/30] Amend plan per review 8 (precision, no new blockers) - Axum: exact path = keep AxumDevServer::with_config + .with_{config,kv,secret}_registry (dev_server::run_app drops custom PORT/axum.toml); add adapter-axum registry builders - Task 5: extract whole registry via ctx.request().extensions().get::() (RequestContext has no whole-registry getter; per-id accessors would wire only default) - Task 2: enumerate exact file paths; replace brace-glob git add that would hit non-existent adapter manifest paths - Spec: state requirement that Fastly management id == runtime logical id for jwks_store/signing_keys (or supply a mapping, out of Phase 1 scope) - Task 3: composite writer test asserts (StoreId, key, value) forwarding (D6-a risk) - Minor: expect("should ...") convention; Task 7 local verify adds check-cloudflare/spin --- ...07-02-edgezero-store-registry-migration.md | 51 ++++++++++++++----- ...26-07-02-edgezero-full-migration-design.md | 2 + 2 files changed, 41 insertions(+), 12 deletions(-) diff --git a/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md b/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md index 48a06d2d..61cf25ce 100644 --- a/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md +++ b/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md @@ -82,9 +82,13 @@ git commit -m "Record Phase 1 kind-aware store-id map and confirm D6-a" ## Task 2: Declare all store ids (kv/config/secrets) in `edgezero.toml` + reconcile fields/fixtures -**Files:** -- Modify: `edgezero.toml` (`[stores.kv]`, `[stores.config]`, `[stores.secrets]` `ids`) +**Files (exact):** +- Modify: `edgezero.toml` (`[stores.kv]`/`[stores.config]`/`[stores.secrets]` `ids`) +- Modify: `crates/trusted-server-core/src/settings_data.rs` (`DEFAULT_CONFIG_STORE_ID`) - Modify: `trusted-server.example.toml`, `crates/trusted-server-integration-tests/fixtures/configs/trusted-server.integration.toml` +- Create: `crates/trusted-server-core/src/testdata/all-store-refs.toml` +- Modify (platform manifests): `fastly.toml`, `crates/trusted-server-adapter-cloudflare/wrangler.toml`, `crates/trusted-server-adapter-spin/spin.toml` (Axum uses `.edgezero/local-config-*.json` local dev files — document but do not commit machine-local state) +- Create: `crates/trusted-server-core/src/stores.rs` (shared `stores_metadata()`); Modify: each `crates/trusted-server-adapter-{fastly,axum,cloudflare,spin}/src/app.rs` (`impl Hooks` → add `fn stores()`) - Test: `crates/trusted-server-core/src/settings.rs` (`#[cfg(test)]`) **Interfaces:** @@ -148,8 +152,20 @@ Expected: PASS. - [ ] **Step 8: Commit** ```bash -git add edgezero.toml fastly.toml trusted-server.example.toml crates/trusted-server-adapter-*/{wrangler.toml,spin.toml} crates/trusted-server-integration-tests/fixtures crates/trusted-server-core/src -git commit -m "Declare kv/config/secret store ids in edgezero.toml and reconcile config fields" +git add edgezero.toml fastly.toml trusted-server.example.toml \ + crates/trusted-server-adapter-cloudflare/wrangler.toml \ + crates/trusted-server-adapter-spin/spin.toml \ + crates/trusted-server-integration-tests/fixtures/configs/trusted-server.integration.toml \ + crates/trusted-server-core/src/settings.rs \ + crates/trusted-server-core/src/settings_data.rs \ + crates/trusted-server-core/src/stores.rs \ + crates/trusted-server-core/src/lib.rs \ + crates/trusted-server-core/src/testdata/all-store-refs.toml \ + crates/trusted-server-adapter-fastly/src/app.rs \ + crates/trusted-server-adapter-axum/src/app.rs \ + crates/trusted-server-adapter-cloudflare/src/app.rs \ + crates/trusted-server-adapter-spin/src/app.rs +git commit -m "Declare kv/config/secret store ids in edgezero.toml, platform manifests, and Hooks::stores()" ``` --- @@ -193,14 +209,14 @@ fn composite_config_reads_named_store_and_writes_delegate() { // Unknown store id is a strict error, not a fallback to default. let err = composite.get(&StoreName::from("nope"), "kid-1").expect_err("unknown store must error"); assert!(matches!(err.current_context(), PlatformError::ConfigStore), "unknown id -> ConfigStore error"); - // Write delegates to the management-path writer. + // Write delegates to the management-path writer, PRESERVING the target StoreId (core D6-a risk). composite .put(&StoreId::from("jwks_store"), "current-kid", "kid-2") .expect("should delegate write"); assert_eq!( - writer.puts.lock().expect("lock").as_slice(), - &[("current-kid".to_owned(), "kid-2".to_owned())], - "write should delegate to the management-path writer", + writer.puts.lock().expect("should acquire writer lock").as_slice(), + &[("jwks_store".to_owned(), "current-kid".to_owned(), "kid-2".to_owned())], + "write must delegate to the writer with the SAME StoreId, key, and value", ); } ``` @@ -212,7 +228,7 @@ Expected: FAIL (module does not exist). - [ ] **Step 3: Implement `composite.rs`** -`get` resolves `reader.named(store_name)` → `ConfigStoreBinding`, then `block_on(binding.handle.get(key))`; `get_bytes` resolves `reader.named(store_name)` → `BoundSecretStore`, then `block_on(bound.get_bytes(key))` (mirror `storage/kv_store.rs`). Strict: `named` returning `None` → `PlatformError`; EdgeZero `Ok(None)`/`Err` → `PlatformError`. `put`/`create`/`delete` forward to `writer`. Add `config_registry(entries, default)` / `secret_registry(...)` / `RecordingConfigWriter` test helpers that build a real `StoreRegistry` from in-memory EdgeZero stores (config entries wrapped as `ConfigStoreBinding { handle, default_key }`). +`get` resolves `reader.named(store_name)` → `ConfigStoreBinding`, then `block_on(binding.handle.get(key))`; `get_bytes` resolves `reader.named(store_name)` → `BoundSecretStore`, then `block_on(bound.get_bytes(key))` (mirror `storage/kv_store.rs`). Strict: `named` returning `None` → `PlatformError`; EdgeZero `Ok(None)`/`Err` → `PlatformError`. `put`/`create`/`delete` forward to `writer`. Add `config_registry(entries, default)` / `secret_registry(...)` test helpers that build a real `StoreRegistry` from in-memory EdgeZero stores (config entries wrapped as `ConfigStoreBinding { handle, default_key }`), and a `RecordingConfigWriter` (impl `PlatformConfigWriter`) that records **`(StoreId, key, value)`** tuples so tests assert the target StoreId is preserved on delegation. - [ ] **Step 4: Run to verify it passes** @@ -291,7 +307,18 @@ git commit -m "Load boot config via EdgeZero config store on Fastly and Axum" - Modify: `crates/trusted-server-adapter-{axum,cloudflare,spin}/src/platform.rs` (`build_runtime_services` → composite) - Test: `crates/trusted-server-adapter-axum/src/app.rs` route tests (+ cloudflare/spin equivalents) -- [ ] **Step 0: Switch Axum `main.rs` to `run_app::()`.** Replace the `TrustedServerApp::routes()` + `AxumDevServer::with_config(router, config).run()` path with `edgezero_adapter_axum::run_app::()` (preserving bind-address/port behavior via the config surface `run_app` exposes, or `DevServer::run` with `.with_*` registry wiring). Confirm the dev server still binds the configured address. Run `cargo run -p trusted-server-adapter-axum` locally to sanity-check it boots. +- [ ] **Step 0: Wire registries into Axum while keeping the custom PORT behavior.** Do **not** call `dev_server::run_app` — it reads bind config only from `EDGEZERO__ADAPTER__HOST/PORT` and would drop trusted-server's `PORT`/`axum.toml` handling (`main.rs:11`, `port_from_env`). Instead keep the current `AxumDevServer::with_config(router, config)` and chain the builder's registry setters (verified present: `AxumDevServer::{with_config_registry, with_kv_registry, with_secret_registry}`): +```rust +// adapter-axum/src/main.rs +let router = TrustedServerApp::routes(); +let stores = trusted_server_core::stores_metadata(); // Task 2 Step 5 +let mut server = AxumDevServer::with_config(router, config); +if let Some(reg) = build_config_registry_axum(&stores) { server = server.with_config_registry(reg); } +if let Some(reg) = build_kv_registry_axum(&stores) { server = server.with_kv_registry(reg); } +if let Some(reg) = build_secret_registry_axum(&stores) { server = server.with_secret_registry(reg); } +server.run()?; +``` +Add `build_*_registry_axum(&StoresMetadata)` in `adapter-axum/src/registries.rs` mirroring Task 6's Fastly by-id builders but opening the EdgeZero **Axum** store primitives (`.edgezero/*` local backends). This preserves PORT/axum.toml exactly and wires registries. Run `cargo run -p trusted-server-adapter-axum` to confirm it boots on the configured port. **Interfaces:** - Consumes: Task 3 composite; `ConfigRegistry`/`SecretRegistry` from request extensions. @@ -310,7 +337,7 @@ cargo test-axum first_party_proxy_reads_s3_secret ``` Expected: FAIL (all three). -- [ ] **Step 2: Build `RuntimeServices` via the composite** in each adapter's `build_runtime_services`, passing the whole request `ConfigRegistry`/`SecretRegistry` as the composite reader (Task 3) and the per-adapter **write-only** impl (`PlatformConfigWriter`/`PlatformSecretWriter`, Task 3 Step 0 / Task 8) as the writer. +- [ ] **Step 2: Build `RuntimeServices` via the composite** in each adapter's `build_runtime_services(ctx: &RequestContext)`. **Extract the whole registry from request extensions** — `ctx.request().extensions().get::().cloned()` / `get::()` — the same way EdgeZero's `Config`/`Secrets` extractors do. Do **not** use `ctx.config_store_default()`/`config_store(id)` (those return a single bound handle and would wire only the default store). Pass the cloned registry as the composite reader (Task 3) and the per-adapter **write-only** impl (`PlatformConfigWriter`/`PlatformSecretWriter`) as the writer. If a registry is absent from extensions, that is a wiring bug (Step 0 / EdgeZero dispatch) — surface it, don't silently fall back. - [ ] **Step 3: Run to verify pass** (all three) @@ -378,7 +405,7 @@ With reads via EdgeZero (`FastlyConfigStore` reassembles chunks transparently), - [ ] **Step 2: Delete `FastlyChunkPointer`, `FastlyChunkRef`, `resolve_fastly_chunk_pointer`, `sha256_hex`, and the chunk constants;** collapse `get_settings_from_config_store` to `ConfigStore::get` + `settings_from_config_blob`. -- [ ] **Step 3: Run tests** — `cargo test-fastly && cargo test-axum && cargo test-cloudflare && cargo test-spin` → PASS. +- [ ] **Step 3: Run tests + wasm checks** — `cargo test-fastly && cargo test-axum && cargo test-cloudflare && cargo test-spin && cargo check-cloudflare && cargo check-spin` → PASS. - [ ] **Step 4: Commit** diff --git a/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md b/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md index 0739c7c9..412a7d13 100644 --- a/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md +++ b/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md @@ -144,6 +144,8 @@ Recommendation: **(a)** where the adapter env is available at boot (Cloudflare/S **Do NOT repoint request-signing at the app-config store.** Request signing uses its **own** stores: reads use the hard-coded runtime names `JWKS_CONFIG_STORE_NAME = "jwks_store"` (config) and `SIGNING_SECRET_STORE_NAME = "signing_keys"` (secret); writes use `request_signing.config_store_id`/`secret_store_id`, which are the **management-API identifiers of those same jwks/signing stores** (Fastly separates runtime name from management id). Today `trusted-server.example.toml` sets `config_store_id = "app_config"` / `secret_store_id = "secrets"`, which points **writes at the wrong store** (writes JWKS state into app-config while reads look in `jwks_store`). Phase 1 fixes this: set `request_signing.config_store_id` to the **jwks store**'s id and `secret_store_id` to the **signing_keys** store's id, so reads and writes target the same store. Declare `jwks_store` (config) and `signing_keys` (secret) as their own logical ids in `edgezero.toml`. + **Requirement (state explicitly):** because writes use `config_store_id`/`secret_store_id` as **Fastly management-API identifiers** while reads use the runtime **logical id**, Phase 1 requires — for the jwks/signing stores — that the **Fastly management resource id equals the runtime logical id** (`jwks_store`, `signing_keys`). Provision the Fastly stores with those exact names/ids. If a deployment cannot make them equal, it must supply an explicit management-id→logical-id mapping (out of Phase 1 scope); the plan assumes equality and the operator runbook must call it out. + **`StoreName` semantics (D7):** `platform/types.rs::StoreName` is documented as an "edge-visible platform name". Under D7 runtime reads resolve through the registry by **logical id** (`registry.named(id)`), so for reads `StoreName` now carries the **logical store id** (== platform name by default). Phase 1 updates the `StoreName` doc and audits read call sites to pass logical ids, not physical platform names. - **Fastly registry injection (ties to P0-C):** Fastly's custom `oneshot` path (§1) currently inserts only a `ConfigStoreHandle`, not registries via `dispatch_with_registries`. EdgeZero's `dispatch_with_registries` and its registry builders are **`pub(crate)`** (verified in the pinned checkout), so trusted-server must build the registries **locally** (from `StoresMetadata` + `EnvConfig` + the EdgeZero Fastly store open primitives) and insert them into extensions before `oneshot` — or an EdgeZero public builder must be added upstream (**R11**). - Delete the 4× per-adapter `platform.rs` config/secret **read** impls; adapters build registries from `[stores.*]` metadata (via `dispatch_with_registries` on Axum/Cloudflare/Spin, via the Fastly-specific injection above). From 408fffdf24546ed60898b296d59ad2744193fa5b Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Fri, 3 Jul 2026 08:46:34 -0700 Subject: [PATCH 14/30] Amend spec/plan per review 9 (precision + Fastly composite wiring) - Fastly build_per_request_services still read FastlyPlatformConfigStore directly, so injected registries were unused; add Task 6 Step 4b to build RuntimeServices from the composite via registries in extensions - App-config key == store id (trusted_server_config): default_config_key falls back to id and D7 forbids the __KEY env; set CONFIG_BLOB_KEY=trusted_server_config, retire the app_config key (no --key/env needed) - Task 5: rename heading to 'preserve AxumDevServer::with_config + add registries'; list adapter-axum/src/registries.rs (Step 0 already kept with_config) - Task 2 Step 6: exact per-adapter manifest mappings (CF config=KV binding, secrets=flat Worker secrets via wrangler secret put; Spin config/kv=KV labels) - Name S3 secret store id s3-auth explicitly in D5/Task 2 - Fix all-store-refs.toml include_str path (testdata/, not ../testdata/) - Writer traits Send+Sync; expect_err 'should ...' convention --- ...07-02-edgezero-store-registry-migration.md | 38 +++++++++++-------- ...26-07-02-edgezero-full-migration-design.md | 4 +- 2 files changed, 25 insertions(+), 17 deletions(-) diff --git a/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md b/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md index 61cf25ce..3be20092 100644 --- a/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md +++ b/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md @@ -51,7 +51,7 @@ rg -n '\[stores\.' edgezero.toml ``` Expected (verified): **KV** ids = `ec.ec_store` (`ec_identity_store`, `settings.rs:452`), `consent.consent_store` (`consent_config.rs:80`), and `auction.creative_store` (`auction_config_types.rs:28`, default `"creative_store"`, **deprecated** — creatives are delivered inline); **config** ids = the app-config blob store (**store id `trusted_server_config`**, see D5 rule below), `request_signing.config_store_id`, the JWKS store (`JWKS_CONFIG_STORE_NAME`), and **DataDome's IP-CIDR config store** (`ProtectionIpCidrSourceConfig.config_store`, default `datadome-ip-bypass`, `protection_scope.rs:165`); **secret** ids = `secrets` (`request_signing.secret_store_id`), DataDome `ts_secrets`, the S3 secret store, `signing_keys` (`SIGNING_SECRET_STORE_NAME`) — versus `edgezero.toml` declaring only one id per kind. NOTE: `counter_store` (`RATE_COUNTER_NAME` in the Fastly `rate_limiter.rs`) and `opid_store` are **Fastly-only** platform stores, not `Settings` logical ids — out of scope for D5. `creative_store` **is** a `Settings` id: declare it in `[stores.kv]` (deprecated) so strict lookup can't fail, and flag it for removal in a later phase. - **D5 app-config store-id/key decision (record in Task 1 Output):** the app-config blob → config **store id `trusted_server_config`**, blob **key `app_config`** (`CONFIG_BLOB_KEY`). This changes only `settings_data.rs::DEFAULT_CONFIG_STORE_ID` from `"app_config"` to `"trusted_server_config"` (it is currently a *store id*, `settings_data.rs:11`); `app_config` survives only as the blob **key**. + **D5 app-config store-id/key decision (record in Task 1 Output):** the app-config blob → config **store id `trusted_server_config`**, blob **key `trusted_server_config`** (key == store id, D7-consistent — see below). This changes `settings_data.rs::DEFAULT_CONFIG_STORE_ID` from `"app_config"` to `"trusted_server_config"` (`settings_data.rs:11`) **and** `config_payload.rs::CONFIG_BLOB_KEY` from `"app_config"` to `"trusted_server_config"`. Rationale: `default_config_key()` falls back to the id when no `…__KEY` env is set, and D7 forbids that runtime env — so the key must equal the id, or `ts config push` would need `--key`. The `app_config` name is fully retired. **Request-signing store ids (do NOT point at app-config):** request signing reads use hard-coded `JWKS_CONFIG_STORE_NAME = "jwks_store"` (config) + `SIGNING_SECRET_STORE_NAME = "signing_keys"` (secret); writes use `request_signing.config_store_id`/`secret_store_id`. Today the example sets these to `"app_config"`/`"secrets"` — which sends **writes to a different store than reads**. Fix: set `request_signing.config_store_id = "jwks_store"` and `secret_store_id = "signing_keys"` in `trusted-server.example.toml` + fixtures, and declare `jwks_store` (config) + `signing_keys` (secret) as logical ids in `edgezero.toml`. (Under the composite, reads resolve `registry.named("jwks_store")`; writes go to the same store via the writer/management id.) @@ -65,7 +65,7 @@ Expected: only `KeyRotationManager` in `crates/trusted-server-core/src/request_s - [ ] **Step 3: Record the kind-partitioned D5 map** -Append a table to "Task 1 Output": for each `{kv|config|secrets}` id → resolution (declare in `edgezero.toml`, or collapse onto the kind's default) → `EDGEZERO__STORES______NAME`. Spec default: app-config blob → config id `trusted_server_config` key `app_config`; JWKS → its own config id; `ec_identity_store` → kv id; collapse `secrets`→`trusted_server_secrets` where identical; declare DataDome/S3/signing as distinct secret ids. +Append a table to "Task 1 Output": for each `{kv|config|secrets}` id → resolution (declare in `edgezero.toml`, or collapse onto the kind's default) → `EDGEZERO__STORES______NAME`. Spec default: app-config blob → config id `trusted_server_config` key `trusted_server_config` (key == id); JWKS → its own config id; `ec_identity_store` → kv id; collapse `secrets`→`trusted_server_secrets` where identical; declare DataDome/S3/signing as distinct secret ids. - [ ] **Step 4: Confirm D6-a (or STOP)** @@ -121,10 +121,10 @@ fn every_referenced_store_id_is_declared_by_kind() { "integration-fixture", ); // Purpose-built config exercising DataDome IP-CIDR, S3, and request-signing store refs. - assert_all_ids_declared(include_str!("../testdata/all-store-refs.toml"), "all-store-refs"); + assert_all_ids_declared(include_str!("testdata/all-store-refs.toml"), "all-store-refs"); } ``` -(Create `crates/trusted-server-core/src/testdata/all-store-refs.toml` populating every store-id field with a declared id.) +(Create `crates/trusted-server-core/src/testdata/all-store-refs.toml` populating every store-id field with a declared id. `settings.rs` lives in `src/`, so the `include_str!` path relative to it is `testdata/all-store-refs.toml` — **not** `../testdata/…`.) - [ ] **Step 2: Run to verify it fails** @@ -137,11 +137,16 @@ Add the `ReferencedStoreIds` struct + method returning **KV** ids (`ec.ec_store` Apply the **D5 renames**: set `settings_data.rs::DEFAULT_CONFIG_STORE_ID = "trusted_server_config"`; set `request_signing.config_store_id = "jwks_store"` and `secret_store_id = "signing_keys"` in `trusted-server.example.toml` + fixtures (they must match the read constants — **not** `app_config`/`secrets`). -- [ ] **Step 4: Declare every id in `edgezero.toml`** — `[stores.kv]` = `trusted_server_kv`, `ec_identity_store`, `consent_store`, `creative_store`; `[stores.config]` = `trusted_server_config`, `jwks_store`, `datadome-ip-bypass`; `[stores.secrets]` = `trusted_server_secrets`, `signing_keys`, `ts_secrets`, and the S3 secret id. (Names double as the platform store names under D7.) +- [ ] **Step 4: Declare every id in `edgezero.toml`** — `[stores.kv]` = `trusted_server_kv`, `ec_identity_store`, `consent_store`, `creative_store`; `[stores.config]` = `trusted_server_config`, `jwks_store`, `datadome-ip-bypass`; `[stores.secrets]` = `trusted_server_secrets`, `signing_keys`, `ts_secrets`, `s3-auth`. (Names double as the platform store names under D7.) Also set `config_payload.rs::CONFIG_BLOB_KEY = "trusted_server_config"` (blob key == store id, per the D5 rule) so `ts config push`'s default key and the boot read agree with no env/`--key`. - [ ] **Step 5: Wire `Hooks::stores()` on all four adapters (Blocker — metadata is not wired today).** Each `impl Hooks for TrustedServerApp` currently overrides only `routes()`; the default `stores()` returns **empty** `StoresMetadata`, so no registries can be built from it. Add `fn stores() -> StoresMetadata` returning the `[stores.*]` metadata, generated once from `edgezero.toml`. Prefer a single shared `const`/fn in `trusted-server-core` (e.g. `pub fn stores_metadata() -> StoresMetadata`) that all four adapters return, so the ids live in one place. Verify against `edgezero_core::app::StoresMetadata`/`StoreMetadata` shape. -- [ ] **Step 6: Declare the stores in every PLATFORM manifest (Blocker — local resources missing).** D7 requires each logical id to be openable as a real platform store. Add the new ids to: `fastly.toml` (`[local_server.kv_stores]`/`[local_server.config_stores]`/`[local_server.secret_stores]` + the production service store bindings), `crates/trusted-server-adapter-cloudflare/wrangler.toml` (`[[kv_namespaces]]` + config/secret bindings), `crates/trusted-server-adapter-spin/spin.toml` (`key_value_stores` + variables/config), and the Axum local files `.edgezero/local-config-.json` / KV redb defaults. Cross-check each existing manifest — some planned ids (`jwks_store`, `datadome-ip-bypass`, `signing_keys`) may already be partially declared; add the missing ones. +- [ ] **Step 6: Declare the stores in every PLATFORM manifest (Blocker — local resources missing), per each adapter's real mapping.** D7 requires each logical id to be openable as a real platform store. The adapters map kinds to concrete resources differently — declare exactly: + - **Fastly** (`fastly.toml`): KV ids → `[[local_server.kv_stores.]]`; config ids → `[local_server.config_stores.]`; secret ids → `[local_server.secret_stores.]`. Also add the production-service store bindings for each id. + - **Cloudflare** (`wrangler.toml`): EdgeZero backs **config stores by a KV namespace binding** (`config_store.rs`) — so each **config** id (`trusted_server_config`, `jwks_store`, `datadome-ip-bypass`) gets a `[[kv_namespaces]]` binding (as does each KV id). **Secrets are flat Worker secrets**, provisioned via `wrangler secret put ` / the dashboard — **not** `wrangler.toml` bindings; document the `wrangler secret put` commands for `signing_keys`/`ts_secrets`/`s3-auth` in the operator runbook rather than in `wrangler.toml`. + - **Spin** (`spin.toml`): config **and** KV ids open **KV-store labels** (`request.rs:282`) — declare each under the component's `key_value_stores = [...]`; secrets map to Spin variables/config — declare each secret id accordingly. + - **Axum**: dev-only local files `.edgezero/local-config-.json` (config) and the redb KV default; document how to seed them, do not commit machine-local state. + Cross-check each existing manifest — some ids (`jwks_store`, `signing_keys`) are already partially declared (`request_signing/mod.rs` doc references `fastly.toml`); add only the missing ones. - [ ] **Step 7: Run to verify pass + full adapter suites + wasm checks** @@ -172,7 +177,7 @@ git commit -m "Declare kv/config/secret store ids in edgezero.toml, platform man ## Task 3: Registry-backed composite store (reads → EdgeZero registry by store_name, writes → management path) -Concrete D6-a mechanism. The bespoke traits read **by `StoreName`** and callers use **multiple** store ids (`app_config`, JWKS, DataDome, S3, `ec_identity_store` for KV). So the composite must hold the **whole `ConfigRegistry`/`SecretRegistry`** (not a single handle) and resolve `named(store_name)` on each read; writes (`put`/`create`/`delete`) delegate to the existing management-API-backed writer. Preserves `KeyRotationManager` writes with zero call-site changes. +Concrete D6-a mechanism. The bespoke traits read **by `StoreName`** and callers use **multiple** store ids (`trusted_server_config`, `jwks_store`, `datadome-ip-bypass`, `s3-auth`, `ts_secrets`, `ec_identity_store` for KV). So the composite must hold the **whole `ConfigRegistry`/`SecretRegistry`** (not a single handle) and resolve `named(store_name)` on each read; writes (`put`/`create`/`delete`) delegate to the existing management-API-backed writer. Preserves `KeyRotationManager` writes with zero call-site changes. **Files:** - Modify: `crates/trusted-server-core/src/platform/traits.rs` (split write-only traits) @@ -187,7 +192,7 @@ Concrete D6-a mechanism. The bespoke traits read **by `StoreName`** and callers - `CompositeConfigStore::new(reader: ConfigRegistry, writer: Arc) -> Self` implementing the full read+write `PlatformConfigStore`. **`ConfigRegistry::named(id)` returns `Option`, not a handle** — so `get(store_name, key)` = resolve `binding = reader.named(store_name.as_str()).ok_or(PlatformError::ConfigStore)?`, then `block_on(binding.handle.get(key))`. EdgeZero `ConfigStore::get` returns `Result, ConfigStoreError>`; the bespoke `get` returns `Result`, so map `Ok(None)`/`Err(ConfigStoreError::*)` → `PlatformError::ConfigStore`. `put`/`delete` → `writer`. - `CompositeSecretStore::new(reader: SecretRegistry, writer: Arc) -> Self` implementing `PlatformSecretStore`: `get_bytes(store_name, key)` = `reader.named(store_name.as_str()).ok_or(PlatformError::SecretStore)?` → `block_on(bound.get_bytes(key))`; map `Ok(None)`/`Err` → `PlatformError::SecretStore`. `create`/`delete` → `writer`. A store_name not in the registry is a hard error (strict), not a silent fallback. -- [ ] **Step 0: Split write-only traits.** In `traits.rs`, define `PlatformConfigWriter` (`put`, `delete`) and `PlatformSecretWriter` (`create`, `delete`). Keep `PlatformConfigStore`/`PlatformSecretStore` as the read+write surface `RuntimeServices` exposes. This split is the prerequisite that makes Task 8's "delete reads, keep writes" compile. Run `cargo check-axum` to confirm the split compiles before proceeding. +- [ ] **Step 0: Split write-only traits.** In `traits.rs`, define `pub trait PlatformConfigWriter: Send + Sync { put; delete }` and `pub trait PlatformSecretWriter: Send + Sync { create; delete }` (matching the parent traits' `Send + Sync` bounds, since they're held as `Arc`). Keep `PlatformConfigStore`/`PlatformSecretStore` as the read+write surface `RuntimeServices` exposes. This split is the prerequisite that makes Task 8's "delete reads, keep writes" compile. Run `cargo check-axum` to confirm the split compiles before proceeding. - [ ] **Step 1: Write the failing test — reads resolve the NAMED store; unknown store errors; writes delegate** @@ -207,7 +212,7 @@ fn composite_config_reads_named_store_and_writes_delegate() { .expect("should read from the non-default jwks_store"); assert_eq!(jwk, "{\"kty\":\"OKP\"}"); // Unknown store id is a strict error, not a fallback to default. - let err = composite.get(&StoreName::from("nope"), "kid-1").expect_err("unknown store must error"); + let err = composite.get(&StoreName::from("nope"), "kid-1").expect_err("should error on unknown store id"); assert!(matches!(err.current_context(), PlatformError::ConfigStore), "unknown id -> ConfigStore error"); // Write delegates to the management-path writer, PRESERVING the target StoreId (core D6-a risk). composite @@ -267,9 +272,9 @@ git commit -m "Add registry-backed composite store; document StoreName as logica fn get_settings_reads_blob_via_edgezero_handle() { // Arrange: an EdgeZero ConfigStoreHandle over an in-memory store holding the blob envelope. let blob = blob_envelope_json(include_str!("../../../trusted-server.example.toml")); - let handle = ConfigStoreHandle::new(Arc::new(InMemoryConfigStore::with(&[("app_config", &blob)]))); + let handle = ConfigStoreHandle::new(Arc::new(InMemoryConfigStore::with(&[("trusted_server_config", &blob)]))); // Act - let settings = get_settings_from_config_store(&handle, "app_config") + let settings = get_settings_from_config_store(&handle, "trusted_server_config") .expect("should parse settings from the EdgeZero-read blob"); // Assert assert!(settings.ec.ec_store.is_some(), "should deserialize the example config"); @@ -279,7 +284,7 @@ fn get_settings_reads_blob_via_edgezero_handle() { Run: `cargo test-fastly get_settings_reads_blob_via_edgezero_handle` → Expected: FAIL. -- [ ] **Step 2: Re-type `get_settings_from_config_store`** to `(&ConfigStoreHandle, key: &str)`, called with **store id `trusted_server_config`, key `app_config`** (D5). In Fastly `load_settings_from_config_store()` open the EdgeZero `FastlyConfigStore` for `trusted_server_config` at boot and wrap in a `ConfigStoreHandle`. In Axum `build_state()` open the EdgeZero Axum config store, which reads **`.edgezero/local-config-trusted_server_config.json`** (`edgezero-adapter-axum/src/config_store.rs` — id-scoped local file); do **not** apply any env-key override (D7 — runtime is config-store-only). The adapter-level boot wiring is exercised by each adapter's existing `build_state` test path (no new Viceroy test needed — the core test above covers the parse logic). +- [ ] **Step 2: Re-type `get_settings_from_config_store`** to `(&ConfigStoreHandle, key: &str)`, called with **store id `trusted_server_config`, key `trusted_server_config`** (D5 — key == store id; `CONFIG_BLOB_KEY` is set to `trusted_server_config` in Task 2). In Fastly `load_settings_from_config_store()` open the EdgeZero `FastlyConfigStore` for `trusted_server_config` at boot and wrap in a `ConfigStoreHandle`. In Axum `build_state()` open the EdgeZero Axum config store, which reads **`.edgezero/local-config-trusted_server_config.json`** (`edgezero-adapter-axum/src/config_store.rs` — id-scoped local file); do **not** apply any env-key override (D7). The adapter-level boot wiring is exercised by each adapter's existing `build_state` test path (no new Viceroy test needed — the core test above covers the parse logic). - [ ] **Step 3: Run to verify pass** (core test + adapter boot suites) @@ -298,12 +303,13 @@ git commit -m "Load boot config via EdgeZero config store on Fastly and Axum" --- -## Task 5: Switch adapters to EdgeZero's registry-aware entry; RuntimeServices uses the composite +## Task 5: Preserve `AxumDevServer::with_config` + add registries; RuntimeServices via composite (Axum/Cloudflare/Spin) -**Blocker addressed:** Axum today calls `TrustedServerApp::routes()` + `AxumDevServer::with_config(...)` (`adapter-axum/src/main.rs:23`) — which never builds registries. This task switches Axum to EdgeZero's registry-aware `run_app::()` (`edgezero_adapter_axum::dev_server::run_app`, which builds registries from `Hooks::stores()` — now wired in Task 2 Step 5). Cloudflare and Spin already dispatch via EdgeZero `run_app`; confirm their `run_app` builds registries once `stores()` is wired. Then build `RuntimeServices` config/secret from `CompositeConfigStore`/`CompositeSecretStore` (reader from the request registry; writer = the per-adapter write impl). Store-name binding uses EdgeZero's `EnvConfig` fallback-to-logical-id (D7 — we set no `EDGEZERO__STORES__*__NAME`). +**Blocker addressed:** Axum today calls `TrustedServerApp::routes()` + `AxumDevServer::with_config(...)` (`adapter-axum/src/main.rs:23`) — which never builds registries. We **keep** `AxumDevServer::with_config` (to preserve the custom `PORT`/`axum.toml` behavior) and add registries via its `.with_config_registry()/.with_kv_registry()/.with_secret_registry()` builder methods (Step 0). Cloudflare and Spin already dispatch via EdgeZero `run_app`, which builds registries from `Hooks::stores()` (wired in Task 2 Step 5) — confirm they do once `stores()` exists. Then build `RuntimeServices` config/secret from `CompositeConfigStore`/`CompositeSecretStore` (reader = the whole request registry from extensions; writer = the per-adapter write-only impl). Store-name binding uses EdgeZero's `EnvConfig` fallback-to-logical-id (D7 — we set no `EDGEZERO__STORES__*__NAME`). **Files:** -- Modify: `crates/trusted-server-adapter-axum/src/main.rs` (switch to `run_app::()`) +- Modify: `crates/trusted-server-adapter-axum/src/main.rs` (keep `AxumDevServer::with_config`, chain registry setters) +- Create: `crates/trusted-server-adapter-axum/src/registries.rs` (`build_{config,kv,secret}_registry_axum(&StoresMetadata)`) - Modify: `crates/trusted-server-adapter-{axum,cloudflare,spin}/src/platform.rs` (`build_runtime_services` → composite) - Test: `crates/trusted-server-adapter-axum/src/app.rs` route tests (+ cloudflare/spin equivalents) @@ -379,6 +385,8 @@ Run: `cargo test-fastly build_config_registry_resolves_declared_ids` → Expecte - [ ] **Step 4: Insert registries in the oneshot block** — replace the lone `core_req.extensions_mut().insert(config_store)` at `main.rs:477`: build the three registries via Step 3 (propagate `build_kv_registry`'s `FastlyError` into the dispatch's `Result`), and `if let Some(reg) = ...` insert each into `core_req.extensions_mut()`, preserving the existing `client_info`/`device_signals` inserts. +- [ ] **Step 4b: Build Fastly `RuntimeServices` from the composite (else the injected registries are unused).** Fastly's `build_per_request_services` (`adapter-fastly/src/app.rs:238`) currently does `RuntimeServices::builder().config_store(Arc::new(FastlyPlatformConfigStore))…` — reading directly, ignoring the registries. Change it to extract the registries from extensions (`ctx.request().extensions().get::().cloned()` / `SecretRegistry`) and build `CompositeConfigStore`/`CompositeSecretStore` (reader = registry; writer = the Fastly write-only impl), exactly as Task 5 does for the other adapters. Without this, Steps 1–4 wire registries nothing reads. + - [ ] **Step 5: Write a failing Fastly route test** — `GET /.well-known/trusted-server.json` via the EdgeZero `oneshot` path returns the JWKS doc read through the injected `ConfigRegistry` (built with default + `jwks_store` ids). Name: `oneshot_discovery_reads_jwks_via_registry` (mirror the `StubJwksConfigStore`/`JWKS_CONFIG_STORE_NAME` pattern in `route_tests.rs`, but drive the EdgeZero path, not `route_request`). Run: `cargo test-fastly oneshot_discovery_reads_jwks_via_registry` → Expected: FAIL then PASS after Steps 3–4. diff --git a/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md b/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md index 412a7d13..a9bd6324 100644 --- a/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md +++ b/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md @@ -85,7 +85,7 @@ Phase 1 (stores) ──> Phase 2 (config) ──> Phase 3 (secrets) Phase 4 (e **D4 — One typed `Settings` as the AppConfig root.** Replace `TrustedServerAppConfig` (wrapper with empty `SECRET_FIELDS`) by deriving `AppConfig` directly on `Settings`, with `#[secret]` on the real secret fields (Phase 3). Removes a transitional indirection. -**D5 — Reconcile ALL runtime store ids with `edgezero.toml`, kind by kind.** Not just the app-config blob, and **not just config/secrets**: EdgeZero's registry lookup is **strict** (unknown id → `None`), so every logical store id any config field or call site names at runtime must appear in `edgezero.toml` under the **correct kind** — `[stores.kv]`, `[stores.config]`, or `[stores.secrets]`. Reconciliation is kind-aware (`Settings::referenced_store_ids_by_kind()`): **KV** ids = `ec.ec_store` (`ec_identity_store`), `consent.consent_store`, `auction.creative_store` (deprecated); **config** ids = the app-config store `trusted_server_config` (key `app_config`), `jwks_store`, `datadome-ip-bypass`; **secret** ids = `signing_keys`, DataDome `ts_secrets`, the S3 secret store. (`counter_store`/`opid_store` are Fastly-adapter constants, not `Settings` ids.) Every declared id must map to a real platform store, opened **by logical id** (D7 — no `EDGEZERO__STORES__*__NAME`). The app-config-store rename and the request-signing store-id fix are detailed just below; the full inventory is the Phase 1 plan's Task 1. +**D5 — Reconcile ALL runtime store ids with `edgezero.toml`, kind by kind.** Not just the app-config blob, and **not just config/secrets**: EdgeZero's registry lookup is **strict** (unknown id → `None`), so every logical store id any config field or call site names at runtime must appear in `edgezero.toml` under the **correct kind** — `[stores.kv]`, `[stores.config]`, or `[stores.secrets]`. Reconciliation is kind-aware (`Settings::referenced_store_ids_by_kind()`): **KV** ids = `ec.ec_store` (`ec_identity_store`), `consent.consent_store`, `auction.creative_store` (deprecated); **config** ids = the app-config store `trusted_server_config` (key `app_config`), `jwks_store`, `datadome-ip-bypass`; **secret** ids = `signing_keys`, DataDome `ts_secrets`, the S3 secret store `s3-auth` (`default_s3_secret_store()`). (`counter_store`/`opid_store` are Fastly-adapter constants, not `Settings` ids.) Every declared id must map to a real platform store, opened **by logical id** (D7 — no `EDGEZERO__STORES__*__NAME`). The app-config-store rename and the request-signing store-id fix are detailed just below; the full inventory is the Phase 1 plan's Task 1. **D6 — Runtime write path for request-signing key rotation.** EdgeZero `ConfigStore`/`SecretStore` are **read-only** at runtime; writes go through provisioning (author/ops time). But `KeyRotationManager` writes+deletes config (JWKS) and secrets (private keys) **at request time** via `/_ts/admin/keys/{rotate,deactivate,delete}`, backed by `management_api.rs`. Three resolutions: - **(a) Keep a write-capable admin abstraction** — retain the trusted-server `put`/`create`/`delete` traits + `management_api.rs` for the admin write path only; EdgeZero read-only stores serve the read path. Least disruptive; leaves a non-EdgeZero write path (so "completely on EdgeZero" is not literally met for admin writes). @@ -140,7 +140,7 @@ Recommendation: **(a)** where the adapter env is available at boot (Cloudflare/S - **Writes (D6):** `KeyRotationManager` writes+deletes **config and secrets at request time** (`store_private_key`/`store_public_jwk`/`delete_key` for `/_ts/admin/keys/rotate` + deactivate/delete). EdgeZero `ConfigStore`/`SecretStore` are **read-only by design**. So `management_api.rs` **cannot be deleted in Phase 1 as originally written**. Resolve per D6 before touching it. - **Store-id reconciliation (D5, expanded, kind-aware):** every runtime store id referenced by config must be declared in `edgezero.toml` under the **correct kind** — `[stores.kv]`, `[stores.config]`, or `[stores.secrets]` — or strict registry lookup returns `None`. Reconcile at least: **KV** — `ec.ec_store` (`ec_identity_store`), `consent.consent_store`, `auction.creative_store` (deprecated — creatives are inline — but still a `Settings` field, so declare it to keep strict lookup safe); **config** — the **app-config blob store** (see the store-id/key rule below), `request_signing.config_store_id` (`app_config` today), the JWKS/config-list store, and **DataDome's IP-CIDR config store** (`ProtectionIpCidrSourceConfig.config_store`, default `datadome-ip-bypass`, `protection_scope.rs`); **secrets** — `request_signing.secret_store_id` (`secrets` today), DataDome secret store (`ts_secrets`), the S3 secret store — plus all `trusted-server.example.toml` + integration/test fixtures. (`counter_store`/`opid_store` are **Fastly-adapter** constants (rate limiter / opid), **not** `Settings` logical ids, and are out of scope.) - **App-config store-id/key rule (explicit):** the app-config blob lives in the config **store id `trusted_server_config`** (the `edgezero.toml` default) under **blob key `app_config`** (`CONFIG_BLOB_KEY`). Phase 1 changes only the **app-config** store id — `settings_data.rs::DEFAULT_CONFIG_STORE_ID` from `"app_config"` to `"trusted_server_config"` — keeping `app_config` as the blob **key**. + **App-config store-id/key rule (explicit, D7-consistent):** the app-config blob lives in the config **store id `trusted_server_config`** (the `edgezero.toml` default) under **blob key `trusted_server_config`** — the key equals the store id. Why: `default_config_key()` = `EnvConfig::store_key("config", id)`, which **falls back to the id** when no `EDGEZERO__STORES__CONFIG__…__KEY` env is set — and D7 forbids that runtime env. So a separate `app_config` key would require a runtime env override (violating D7) or `ts config push --key app_config`. Instead Phase 1 sets `settings_data.rs::DEFAULT_CONFIG_STORE_ID = "trusted_server_config"` **and** changes `config_payload.rs::CONFIG_BLOB_KEY` from `"app_config"` to `"trusted_server_config"`, so `ts config push` (default key = store id) and the boot read agree with **zero** env/`--key`. **Do NOT repoint request-signing at the app-config store.** Request signing uses its **own** stores: reads use the hard-coded runtime names `JWKS_CONFIG_STORE_NAME = "jwks_store"` (config) and `SIGNING_SECRET_STORE_NAME = "signing_keys"` (secret); writes use `request_signing.config_store_id`/`secret_store_id`, which are the **management-API identifiers of those same jwks/signing stores** (Fastly separates runtime name from management id). Today `trusted-server.example.toml` sets `config_store_id = "app_config"` / `secret_store_id = "secrets"`, which points **writes at the wrong store** (writes JWKS state into app-config while reads look in `jwks_store`). Phase 1 fixes this: set `request_signing.config_store_id` to the **jwks store**'s id and `secret_store_id` to the **signing_keys** store's id, so reads and writes target the same store. Declare `jwks_store` (config) and `signing_keys` (secret) as their own logical ids in `edgezero.toml`. From c6fdf62562ce1105bba6325858e56a348f785d87 Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Fri, 3 Jul 2026 08:46:58 -0700 Subject: [PATCH 15/30] Fix remaining spec key app_config -> trusted_server_config; S3 s3-auth in R9 --- .../specs/2026-07-02-edgezero-full-migration-design.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md b/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md index a9bd6324..edc05233 100644 --- a/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md +++ b/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md @@ -85,7 +85,7 @@ Phase 1 (stores) ──> Phase 2 (config) ──> Phase 3 (secrets) Phase 4 (e **D4 — One typed `Settings` as the AppConfig root.** Replace `TrustedServerAppConfig` (wrapper with empty `SECRET_FIELDS`) by deriving `AppConfig` directly on `Settings`, with `#[secret]` on the real secret fields (Phase 3). Removes a transitional indirection. -**D5 — Reconcile ALL runtime store ids with `edgezero.toml`, kind by kind.** Not just the app-config blob, and **not just config/secrets**: EdgeZero's registry lookup is **strict** (unknown id → `None`), so every logical store id any config field or call site names at runtime must appear in `edgezero.toml` under the **correct kind** — `[stores.kv]`, `[stores.config]`, or `[stores.secrets]`. Reconciliation is kind-aware (`Settings::referenced_store_ids_by_kind()`): **KV** ids = `ec.ec_store` (`ec_identity_store`), `consent.consent_store`, `auction.creative_store` (deprecated); **config** ids = the app-config store `trusted_server_config` (key `app_config`), `jwks_store`, `datadome-ip-bypass`; **secret** ids = `signing_keys`, DataDome `ts_secrets`, the S3 secret store `s3-auth` (`default_s3_secret_store()`). (`counter_store`/`opid_store` are Fastly-adapter constants, not `Settings` ids.) Every declared id must map to a real platform store, opened **by logical id** (D7 — no `EDGEZERO__STORES__*__NAME`). The app-config-store rename and the request-signing store-id fix are detailed just below; the full inventory is the Phase 1 plan's Task 1. +**D5 — Reconcile ALL runtime store ids with `edgezero.toml`, kind by kind.** Not just the app-config blob, and **not just config/secrets**: EdgeZero's registry lookup is **strict** (unknown id → `None`), so every logical store id any config field or call site names at runtime must appear in `edgezero.toml` under the **correct kind** — `[stores.kv]`, `[stores.config]`, or `[stores.secrets]`. Reconciliation is kind-aware (`Settings::referenced_store_ids_by_kind()`): **KV** ids = `ec.ec_store` (`ec_identity_store`), `consent.consent_store`, `auction.creative_store` (deprecated); **config** ids = the app-config store `trusted_server_config` (blob key == id: `trusted_server_config`), `jwks_store`, `datadome-ip-bypass`; **secret** ids = `signing_keys`, DataDome `ts_secrets`, the S3 secret store `s3-auth` (`default_s3_secret_store()`). (`counter_store`/`opid_store` are Fastly-adapter constants, not `Settings` ids.) Every declared id must map to a real platform store, opened **by logical id** (D7 — no `EDGEZERO__STORES__*__NAME`). The app-config-store rename and the request-signing store-id fix are detailed just below; the full inventory is the Phase 1 plan's Task 1. **D6 — Runtime write path for request-signing key rotation.** EdgeZero `ConfigStore`/`SecretStore` are **read-only** at runtime; writes go through provisioning (author/ops time). But `KeyRotationManager` writes+deletes config (JWKS) and secrets (private keys) **at request time** via `/_ts/admin/keys/{rotate,deactivate,delete}`, backed by `management_api.rs`. Three resolutions: - **(a) Keep a write-capable admin abstraction** — retain the trusted-server `put`/`create`/`delete` traits + `management_api.rs` for the admin write path only; EdgeZero read-only stores serve the read path. Least disruptive; leaves a non-EdgeZero write path (so "completely on EdgeZero" is not literally met for admin writes). @@ -265,7 +265,7 @@ Two consequences: (1) edgezero #305 **must** ship `ArrayEach` + `Option` | R1 | Do any `Settings` secrets live inside **arrays**? | **Resolved: yes** (`ec.partners[].api_token`, `handlers[].password`) + optional (`ts_pull_token`). edgezero #305 must ship `ArrayEach` + `Option` (see §5 Phase 3 inventory + Phase 0 note). | | R7 | P0-C: upstream a header-preserving Fastly dispatch, or keep a permanent Fastly dispatch shim? | Decide with edgezero maintainer (§4a); gates the Fastly end-state and Phase 5. | | R8 | P-BOOT: boot-time store handle (a) vs lazy cached first-request load (b), per adapter? | Phase 2 plan (§4a). | -| R9 | D5: reconcile **all** runtime store ids **by kind** with `edgezero.toml` (strict lookup fails otherwise) — KV: `ec_identity_store`, `consent_store`, `creative_store`; config: `trusted_server_config` (app-config blob, key `app_config`), `jwks_store`, `datadome-ip-bypass`; secrets: `signing_keys`, `ts_secrets`, S3. Request signing uses `jwks_store`/`signing_keys` (not the app-config store). | Phase 1 plan task 1/2. | +| R9 | D5: reconcile **all** runtime store ids **by kind** with `edgezero.toml` (strict lookup fails otherwise) — KV: `ec_identity_store`, `consent_store`, `creative_store`; config: `trusted_server_config` (app-config blob, key == id), `jwks_store`, `datadome-ip-bypass`; secrets: `signing_keys`, `ts_secrets`, `s3-auth`. Request signing uses `jwks_store`/`signing_keys` (not the app-config store). | Phase 1 plan task 1/2. | | R12 | Fastly `EnvConfig` reader is private / `fastly::ConfigStore` has no `iter()`. | **Resolved by D7** — runtime opens stores by logical id; no store-name env/dictionary read; no local `EnvConfig` reader needed. | | R10 | D6: runtime write path for key rotation — keep write-capable admin abstraction (a), move to ops/CLI (b), or upstream an EdgeZero write API (c)? | **Blocks Phase 1 deletions.** Phase 1 plan locks to **(a)**; (b)/(c) → separate plan. | | R11 | Should EdgeZero expose a **public** Fastly registry-builder helper? | Lower priority under D7 — local builders open by logical id and need only public store constructors. Decide with edgezero maintainer if convenient. | From dc91e5d834b424d5f55065eb01d3d7a39e15d18c Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Fri, 3 Jul 2026 09:20:07 -0700 Subject: [PATCH 16/30] Amend plan per review 10 (integration surfaces, flat secrets, blob-key file) - Add config_payload.rs (CONFIG_BLOB_KEY) to Task 2 files + git add (was missing) - Task 2 Step 6b: rename app_config in generate-viceroy-config.rs (+ its test), tests/common/config.rs, tests/environments/axum.rs env var (they run in adapter suites and break under the store/key rename) - Task 2 Step 6: Cloudflare/Spin secrets are FLAT (ignore store_name) -> provision the concrete secret KEYS the code reads (signing KIDs, DataDome key name, S3 access_key_id/secret_access_key/session token), not the store id - Task 6: add app.rs to files; test passes only after Step 4b (not 3-4) - Task 1 output table: drop EDGEZERO__STORES__*__NAME column (D7 -> platform resource per adapter, no env mapping) --- ...07-02-edgezero-store-registry-migration.md | 21 +++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-) diff --git a/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md b/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md index 3be20092..1f5fd830 100644 --- a/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md +++ b/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md @@ -65,7 +65,7 @@ Expected: only `KeyRotationManager` in `crates/trusted-server-core/src/request_s - [ ] **Step 3: Record the kind-partitioned D5 map** -Append a table to "Task 1 Output": for each `{kv|config|secrets}` id → resolution (declare in `edgezero.toml`, or collapse onto the kind's default) → `EDGEZERO__STORES______NAME`. Spec default: app-config blob → config id `trusted_server_config` key `trusted_server_config` (key == id); JWKS → its own config id; `ec_identity_store` → kv id; collapse `secrets`→`trusted_server_secrets` where identical; declare DataDome/S3/signing as distinct secret ids. +Append a table to "Task 1 Output": for each `{kv|config|secrets}` id → resolution (declare in `edgezero.toml`, or collapse onto the kind's default) → the concrete platform resource per adapter. **Under D7 there is no `EDGEZERO__STORES__*__NAME` mapping** — the logical id opens the same-named platform store, so the table records the *platform resource per adapter* (Fastly local/prod store, CF KV namespace / flat secret keys, Spin KV label / variable), **not** an env var. Spec default: app-config blob → config id `trusted_server_config` key `trusted_server_config` (key == id); JWKS → its own config id; `ec_identity_store` → kv id; collapse `secrets`→`trusted_server_secrets` where identical; declare DataDome/S3/signing as distinct secret ids. - [ ] **Step 4: Confirm D6-a (or STOP)** @@ -85,8 +85,10 @@ git commit -m "Record Phase 1 kind-aware store-id map and confirm D6-a" **Files (exact):** - Modify: `edgezero.toml` (`[stores.kv]`/`[stores.config]`/`[stores.secrets]` `ids`) - Modify: `crates/trusted-server-core/src/settings_data.rs` (`DEFAULT_CONFIG_STORE_ID`) +- Modify: `crates/trusted-server-core/src/config_payload.rs` (`CONFIG_BLOB_KEY` → `"trusted_server_config"`) - Modify: `trusted-server.example.toml`, `crates/trusted-server-integration-tests/fixtures/configs/trusted-server.integration.toml` - Create: `crates/trusted-server-core/src/testdata/all-store-refs.toml` +- **Modify (integration test surfaces that hard-code `app_config` — break under the rename):** `crates/trusted-server-integration-tests/src/bin/generate-viceroy-config.rs` (`[local_server.config_stores.app_config]` + `app_config = '''…'''` + the generator test asserting them → `trusted_server_config`), `crates/trusted-server-integration-tests/tests/common/config.rs` (`{"app_config": envelope}` → `{"trusted_server_config": …}`), `crates/trusted-server-integration-tests/tests/environments/axum.rs` (env `TRUSTED_SERVER_CONFIG_APP_CONFIG_APP_CONFIG` → the trusted_server_config-keyed name) - Modify (platform manifests): `fastly.toml`, `crates/trusted-server-adapter-cloudflare/wrangler.toml`, `crates/trusted-server-adapter-spin/spin.toml` (Axum uses `.edgezero/local-config-*.json` local dev files — document but do not commit machine-local state) - Create: `crates/trusted-server-core/src/stores.rs` (shared `stores_metadata()`); Modify: each `crates/trusted-server-adapter-{fastly,axum,cloudflare,spin}/src/app.rs` (`impl Hooks` → add `fn stores()`) - Test: `crates/trusted-server-core/src/settings.rs` (`#[cfg(test)]`) @@ -143,8 +145,10 @@ Apply the **D5 renames**: set `settings_data.rs::DEFAULT_CONFIG_STORE_ID = "trus - [ ] **Step 6: Declare the stores in every PLATFORM manifest (Blocker — local resources missing), per each adapter's real mapping.** D7 requires each logical id to be openable as a real platform store. The adapters map kinds to concrete resources differently — declare exactly: - **Fastly** (`fastly.toml`): KV ids → `[[local_server.kv_stores.]]`; config ids → `[local_server.config_stores.]`; secret ids → `[local_server.secret_stores.]`. Also add the production-service store bindings for each id. - - **Cloudflare** (`wrangler.toml`): EdgeZero backs **config stores by a KV namespace binding** (`config_store.rs`) — so each **config** id (`trusted_server_config`, `jwks_store`, `datadome-ip-bypass`) gets a `[[kv_namespaces]]` binding (as does each KV id). **Secrets are flat Worker secrets**, provisioned via `wrangler secret put ` / the dashboard — **not** `wrangler.toml` bindings; document the `wrangler secret put` commands for `signing_keys`/`ts_secrets`/`s3-auth` in the operator runbook rather than in `wrangler.toml`. - - **Spin** (`spin.toml`): config **and** KV ids open **KV-store labels** (`request.rs:282`) — declare each under the component's `key_value_stores = [...]`; secrets map to Spin variables/config — declare each secret id accordingly. + - **Cloudflare** (`wrangler.toml`): EdgeZero backs **config stores by a KV namespace binding** (`config_store.rs`) — so each **config** id (`trusted_server_config`, `jwks_store`, `datadome-ip-bypass`) gets a `[[kv_namespaces]]` binding (as does each KV id). **Secrets use a FLAT namespace — `CloudflareSecretStore::get_bytes` ignores `store_name` and reads `env.secret(key)`.** So do **not** `wrangler secret put signing_keys` (that provisions the wrong name). Provision the concrete secret **keys the code reads**: the signing KIDs written by `KeyRotationManager`, the DataDome `server_side_key_secret_name`, and the S3 `access_key_id` / `secret_access_key` / optional session-token keys. Document the exact `wrangler secret put ` commands in the operator runbook; `store_name`/store-id is irrelevant on Cloudflare. + - **Spin** (`spin.toml`): config **and** KV ids open **KV-store labels** (`request.rs:282`) — declare each under the component's `key_value_stores = [...]`. Secrets are likewise a **flat** namespace (`SpinSecretStore` ignores `store_name`) mapped to Spin variables — provision the concrete secret **keys** (as for Cloudflare), lowercased per Spin's variable rules, not the store id. + +- [ ] **Step 6b: Update integration-test surfaces that hard-code the old `app_config` store/key** (they run in the adapter suites, so the rename breaks them): in `generate-viceroy-config.rs` rename the generated `[local_server.config_stores.app_config]` block + `app_config = '''…'''` entry to `trusted_server_config` (store and key) and update the generator's own assertion test; in `tests/common/config.rs` change `{"app_config": envelope}` to `{"trusted_server_config": envelope}`; in `tests/environments/axum.rs` rename the `TRUSTED_SERVER_CONFIG_APP_CONFIG_APP_CONFIG` env var to the `trusted_server_config`-keyed equivalent the Axum config store expects. - **Axum**: dev-only local files `.edgezero/local-config-.json` (config) and the redb KV default; document how to seed them, do not commit machine-local state. Cross-check each existing manifest — some ids (`jwks_store`, `signing_keys`) are already partially declared (`request_signing/mod.rs` doc references `fastly.toml`); add only the missing ones. @@ -161,8 +165,12 @@ git add edgezero.toml fastly.toml trusted-server.example.toml \ crates/trusted-server-adapter-cloudflare/wrangler.toml \ crates/trusted-server-adapter-spin/spin.toml \ crates/trusted-server-integration-tests/fixtures/configs/trusted-server.integration.toml \ + crates/trusted-server-integration-tests/src/bin/generate-viceroy-config.rs \ + crates/trusted-server-integration-tests/tests/common/config.rs \ + crates/trusted-server-integration-tests/tests/environments/axum.rs \ crates/trusted-server-core/src/settings.rs \ crates/trusted-server-core/src/settings_data.rs \ + crates/trusted-server-core/src/config_payload.rs \ crates/trusted-server-core/src/stores.rs \ crates/trusted-server-core/src/lib.rs \ crates/trusted-server-core/src/testdata/all-store-refs.toml \ @@ -170,7 +178,7 @@ git add edgezero.toml fastly.toml trusted-server.example.toml \ crates/trusted-server-adapter-axum/src/app.rs \ crates/trusted-server-adapter-cloudflare/src/app.rs \ crates/trusted-server-adapter-spin/src/app.rs -git commit -m "Declare kv/config/secret store ids in edgezero.toml, platform manifests, and Hooks::stores()" +git commit -m "Declare store ids in edgezero.toml, manifests, Hooks::stores(); rename app-config store/key to trusted_server_config" ``` --- @@ -366,7 +374,8 @@ EdgeZero's Fastly `dispatch_with_registries` and its registry builders are `pub( **Files:** - Create: `crates/trusted-server-adapter-fastly/src/registries.rs` (`build_config_registry`, `build_secret_registry`, `build_kv_registry`) - Modify: `crates/trusted-server-adapter-fastly/src/main.rs:477` (the `oneshot` dispatch block) -- Test: `crates/trusted-server-adapter-fastly/src/registries.rs` (`#[cfg(test)]`) + a route test +- Modify: `crates/trusted-server-adapter-fastly/src/app.rs:238` (`build_per_request_services` → build from composite, Step 4b) +- Test: `crates/trusted-server-adapter-fastly/src/registries.rs` (`#[cfg(test)]`) + a route test in `route_tests.rs` **Interfaces:** - Consumes: `StoresMetadata` (from `Hooks::stores()`), EdgeZero `FastlyConfigStore`/`FastlyKvStore`/`FastlySecretStore` open primitives, `StoreRegistry::from_parts` (which returns **`Option`** — `None` when the default id is absent from `by_id`). @@ -389,7 +398,7 @@ Run: `cargo test-fastly build_config_registry_resolves_declared_ids` → Expecte - [ ] **Step 5: Write a failing Fastly route test** — `GET /.well-known/trusted-server.json` via the EdgeZero `oneshot` path returns the JWKS doc read through the injected `ConfigRegistry` (built with default + `jwks_store` ids). Name: `oneshot_discovery_reads_jwks_via_registry` (mirror the `StubJwksConfigStore`/`JWKS_CONFIG_STORE_NAME` pattern in `route_tests.rs`, but drive the EdgeZero path, not `route_request`). -Run: `cargo test-fastly oneshot_discovery_reads_jwks_via_registry` → Expected: FAIL then PASS after Steps 3–4. +Run: `cargo test-fastly oneshot_discovery_reads_jwks_via_registry` → Expected: FAIL, then PASS only after Steps 3, 4, **and 4b** (the test reads through `RuntimeServices`, which is composite-backed only after 4b — without 4b the injected registries are unused and the read still hits the old direct store). - [ ] **Step 6: Fastly suite + parity + commit** From 99c67ae58eaa1e77d28bc5519c358148f1ad707b Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Fri, 3 Jul 2026 15:38:10 -0700 Subject: [PATCH 17/30] Review 10b: pin note + Cloudflare side-channel/Viceroy merge fixes The prior review's Critical 'APIs do not exist' findings were against a stale cargo-cache checkout (ce6bcf7, edgezero #253 'Add store support for Spin') that predates the registry refactor. Our actual pin is 6ebc29a5 (PR #306), which HAS StoreRegistry/StoresMetadata/Hooks::stores()/dispatch_with_registries/ with_*_registry/[stores.*] ids+default/CF-KV-namespace/Axum-local-file. Added a 'Pinned dependency (verified)' note recording this to prevent re-litigation. Valid fixes: - Cloudflare app.rs settings_from_cloudflare_config_json reads literal value.get("app_config") from the side-channel; change to CONFIG_BLOB_KEY so the key rename doesn't break Cloudflare boot (Phase 2 does the store migration) - Viceroy generator already emits a trusted_server_config store for rollout flags; the app_config->trusted_server_config rename must MERGE the blob into that table, not emit a duplicate table --- .../2026-07-02-edgezero-store-registry-migration.md | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md b/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md index 1f5fd830..5760d14c 100644 --- a/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md +++ b/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md @@ -10,6 +10,10 @@ **Spec:** `docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md` §5 Phase 1, D5, D6, §4a. +## Pinned dependency (verified) + +This plan targets the EdgeZero commit pinned in `Cargo.lock`: **`branch = worktree-state-nested-secrets-spec-review` @ `6ebc29a5`** (PR [stackpop/edgezero#306](https://github.com/stackpop/edgezero/pull/306)). That commit **has** every API this plan uses — verified by inspecting the pinned checkout: `store_registry.rs` (`StoreRegistry`/`ConfigRegistry`/`SecretRegistry`/`from_parts`), `StoresMetadata` + `Hooks::stores()` (`app.rs`), `dispatch_with_registries` + `build_*_registry` (adapter-fastly), `AxumDevServer::{with_config,with_kv,with_secret}_registry`, the `[stores.*]` `ids`/`default` manifest schema (`manifest.rs`, `StoreDeclaration`, `deny_unknown_fields`), CloudflareConfigStore backed by **KV namespaces**, and AxumConfigStore backed by **`.edgezero/local-config-.json`**. Older cargo-cache checkouts (e.g. `ce6bcf7`, "Add store support for Spin adapter #253") **predate** the registry refactor and lack these — do not review against them; confirm any API check against `6ebc29a5`. + ## Global Constraints - Rust **2024 edition**, toolchain **1.95.0**; WASM target `wasm32-wasip1`. @@ -90,6 +94,7 @@ git commit -m "Record Phase 1 kind-aware store-id map and confirm D6-a" - Create: `crates/trusted-server-core/src/testdata/all-store-refs.toml` - **Modify (integration test surfaces that hard-code `app_config` — break under the rename):** `crates/trusted-server-integration-tests/src/bin/generate-viceroy-config.rs` (`[local_server.config_stores.app_config]` + `app_config = '''…'''` + the generator test asserting them → `trusted_server_config`), `crates/trusted-server-integration-tests/tests/common/config.rs` (`{"app_config": envelope}` → `{"trusted_server_config": …}`), `crates/trusted-server-integration-tests/tests/environments/axum.rs` (env `TRUSTED_SERVER_CONFIG_APP_CONFIG_APP_CONFIG` → the trusted_server_config-keyed name) - Modify (platform manifests): `fastly.toml`, `crates/trusted-server-adapter-cloudflare/wrangler.toml`, `crates/trusted-server-adapter-spin/spin.toml` (Axum uses `.edgezero/local-config-*.json` local dev files — document but do not commit machine-local state) +- Modify: `crates/trusted-server-adapter-cloudflare/src/app.rs` (`settings_from_cloudflare_config_json` side-channel key `app_config` → `CONFIG_BLOB_KEY`) - Create: `crates/trusted-server-core/src/stores.rs` (shared `stores_metadata()`); Modify: each `crates/trusted-server-adapter-{fastly,axum,cloudflare,spin}/src/app.rs` (`impl Hooks` → add `fn stores()`) - Test: `crates/trusted-server-core/src/settings.rs` (`#[cfg(test)]`) @@ -148,7 +153,11 @@ Apply the **D5 renames**: set `settings_data.rs::DEFAULT_CONFIG_STORE_ID = "trus - **Cloudflare** (`wrangler.toml`): EdgeZero backs **config stores by a KV namespace binding** (`config_store.rs`) — so each **config** id (`trusted_server_config`, `jwks_store`, `datadome-ip-bypass`) gets a `[[kv_namespaces]]` binding (as does each KV id). **Secrets use a FLAT namespace — `CloudflareSecretStore::get_bytes` ignores `store_name` and reads `env.secret(key)`.** So do **not** `wrangler secret put signing_keys` (that provisions the wrong name). Provision the concrete secret **keys the code reads**: the signing KIDs written by `KeyRotationManager`, the DataDome `server_side_key_secret_name`, and the S3 `access_key_id` / `secret_access_key` / optional session-token keys. Document the exact `wrangler secret put ` commands in the operator runbook; `store_name`/store-id is irrelevant on Cloudflare. - **Spin** (`spin.toml`): config **and** KV ids open **KV-store labels** (`request.rs:282`) — declare each under the component's `key_value_stores = [...]`. Secrets are likewise a **flat** namespace (`SpinSecretStore` ignores `store_name`) mapped to Spin variables — provision the concrete secret **keys** (as for Cloudflare), lowercased per Spin's variable rules, not the store id. -- [ ] **Step 6b: Update integration-test surfaces that hard-code the old `app_config` store/key** (they run in the adapter suites, so the rename breaks them): in `generate-viceroy-config.rs` rename the generated `[local_server.config_stores.app_config]` block + `app_config = '''…'''` entry to `trusted_server_config` (store and key) and update the generator's own assertion test; in `tests/common/config.rs` change `{"app_config": envelope}` to `{"trusted_server_config": envelope}`; in `tests/environments/axum.rs` rename the `TRUSTED_SERVER_CONFIG_APP_CONFIG_APP_CONFIG` env var to the `trusted_server_config`-keyed equivalent the Axum config store expects. +- [ ] **Step 6b: Update every surface that hard-codes the old `app_config` store/key** (they run in the adapter suites / boot paths, so the rename breaks them): + - **`generate-viceroy-config.rs` — MERGE, don't duplicate.** The generator already emits `[local_server.config_stores.trusted_server_config]` holding the **rollout flags** (`edgezero_enabled`, `edgezero_rollout_pct`), separate from the `app_config` store holding the envelope blob. After the rename both live in ONE store `trusted_server_config`, so **merge the envelope entry into the existing `trusted_server_config.contents` table under key `trusted_server_config`** (alongside the flag keys) — do **not** emit a second `[local_server.config_stores.trusted_server_config]` block (duplicate table). Update the generator's assertion test accordingly. + - **`tests/common/config.rs`:** `{"app_config": envelope}` → `{"trusted_server_config": envelope}`. + - **`tests/environments/axum.rs`:** rename the `TRUSTED_SERVER_CONFIG_APP_CONFIG_APP_CONFIG` env var to the `trusted_server_config`-keyed name the Axum config store expects. + - **`crates/trusted-server-adapter-cloudflare/src/app.rs` (`settings_from_cloudflare_config_json`):** it reads the literal `value.get("app_config")` from the `TRUSTED_SERVER_CONFIG` side-channel (Cloudflare stays on the side-channel until Phase 2). Change this literal to `CONFIG_BLOB_KEY` (now `"trusted_server_config"`) so Cloudflare boot doesn't break under the rename. (This is a key-string update only; the Phase 2 store migration is separate.) - **Axum**: dev-only local files `.edgezero/local-config-.json` (config) and the redb KV default; document how to seed them, do not commit machine-local state. Cross-check each existing manifest — some ids (`jwks_store`, `signing_keys`) are already partially declared (`request_signing/mod.rs` doc references `fastly.toml`); add only the missing ones. @@ -168,6 +177,7 @@ git add edgezero.toml fastly.toml trusted-server.example.toml \ crates/trusted-server-integration-tests/src/bin/generate-viceroy-config.rs \ crates/trusted-server-integration-tests/tests/common/config.rs \ crates/trusted-server-integration-tests/tests/environments/axum.rs \ + crates/trusted-server-adapter-cloudflare/src/app.rs \ crates/trusted-server-core/src/settings.rs \ crates/trusted-server-core/src/settings_data.rs \ crates/trusted-server-core/src/config_payload.rs \ From 7efe64867477b4d778ab127eec8132ea8d14725d Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Fri, 3 Jul 2026 15:52:49 -0700 Subject: [PATCH 18/30] Adopt full-convergence direction: app! macro + run_app + State User chose full convergence on the canonical app-demo wiring. Reshape the umbrella spec: - End-state: every adapter is one-line run_app::; App from app! macro; handlers #[action] + State>; no TS-local registry/dispatch wiring - Two NEW required upstream edgezero prerequisites (verified gaps in pinned 6ebc29a): - P0-C: header-preserving Fastly run_app dispatch + pre-dispatch extension hook (Set-Cookie/JA4/logger) - permanent-exception fallback removed - P0-D: macro/run_app app-state injection (with_state is builder-only; app! router + run_app never call it) - a Hooks state hook run_app inserts per request - D1 retained (load-once Settings) but injected via P0-D state hook, not with_state - Phase 4 rewritten as full convergence (app! macro, run_app, extractors, catch-all fallback for integration/publisher dispatch) - Sequencing note (R14): Phase 1 adapter-registry builders (Tasks 5-6) are throwaway under run_app; recommend landing P0-C/P0-D early to skip them - Risks R7 (P0-C required), R13 (P0-D), R14 (sequencing) --- ...26-07-02-edgezero-full-migration-design.md | 54 ++++++++++++------- 1 file changed, 35 insertions(+), 19 deletions(-) diff --git a/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md b/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md index edc05233..ca5cfa1c 100644 --- a/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md +++ b/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md @@ -10,9 +10,12 @@ ## 1. End-state -trusted-server is a fully EdgeZero-native app: adapter binaries are thin entry points (`run_app::` where the platform allows it, or a **documented adapter-level dispatch shim** where it does not — see Fastly below); core is platform-neutral; config, KV, and secrets flow exclusively through EdgeZero's `StoreRegistry`; app config is a signed blob published by `ts config push` and read back typed with secrets resolved from the secret store; handlers are `#[action]` functions taking `FromRequest` extractors; and no *pre-EdgeZero* shim remains in core or the adapters. +trusted-server is a fully EdgeZero-native app **converged on the canonical `app-demo` wiring** (decision: full convergence): every adapter binary is the one-line `run_app::` entry; `App` is generated by the `edgezero_core::app!("edgezero.toml")` macro (routes from `[[triggers.http]]`, `stores()` from `[stores.*]`); core is platform-neutral; config, KV, and secrets flow exclusively through EdgeZero's `StoreRegistry`; app config is a signed blob published by `ts config push`; handlers are `#[action]` functions taking `FromRequest` extractors, with shared app state reaching them via `State>`; and no *pre-EdgeZero* shim **and no trusted-server-local registry/dispatch wiring** remains. -**Fastly is not `run_app::` today and may not be at the end** (Blocker, verified `adapter-fastly/src/main.rs`): Fastly deliberately calls `app.router().oneshot()` directly instead of the standard dispatch helpers, because (a) the helpers convert through `fastly::Response` via `set_header`, which **drops duplicate `Set-Cookie` values** from publisher/origin responses, and (b) `run_app_*` triggers a **logger reinit** Fastly must avoid. Fastly also injects `client_info` + `device_signals` (TLS JA4 / H2 fingerprint) into request extensions from the *original* `FastlyRequest` before conversion — signals a reconstructed EdgeZero request cannot expose. This is an **EdgeZero-adapter capability gap**, not trusted-server cruft. Resolution is a **prerequisite (P0-C)**, see §4a. +**Reaching this requires two upstream EdgeZero capabilities** (Fastly today calls `app.router().oneshot()` directly — verified `adapter-fastly/src/main.rs` — because the standard helpers drop duplicate `Set-Cookie` values via `set_header`, trigger a logger reinit, and can't capture JA4/H2 from the raw `FastlyRequest`; and macro/`run_app` apps have no way to inject `State` since `with_state` is builder-only): +- **P0-C** — a header-preserving Fastly `run_app` dispatch + a pre-dispatch extension hook (§4a). +- **P0-D** — app-state injection for macro/`run_app` apps: a `Hooks` state hook `run_app` inserts per request (§4a). +Both are **required prerequisites** under full convergence — Fastly is `run_app::`, not a permanent exception. Concretely, at the end of this migration: @@ -70,14 +73,18 @@ Phase 1 (stores) ──> Phase 2 (config) ──> Phase 3 (secrets) Phase 4 (e - **Phase 1** depends on nothing upstream (EdgeZero store APIs already exist). - **Phase 2** depends on Phase 1 (config store must be EdgeZero-native first). - **Phase 3** depends on Phase 2 **and** Phase 0's nested `#[secret]`. -- **Phase 4** depends on Phase 0's `State`; it can run in parallel with 1–3 once Phase 0 lands, but is cleaner after Phase 1 (so handlers pull EdgeZero stores, not `RuntimeServices`). +- **Phase 4** (full convergence) depends on Phase 0's `State` **plus P0-C (Fastly `run_app` dispatch) and P0-D (macro app-state injection)** — both new required edgezero prerequisites (§4a). It is cleaner after Phases 1–3. - **Phase 5** is last and **gated on the edgezero rollout reaching 100%** (issue #495). +**Full-convergence sequencing note.** Because Phase 4 moves all adapters to `run_app` (which builds registries internally from `stores()`), the Phase 1 **adapter-registry wiring** (Fastly local builders + custom-`oneshot` injection, Task 6; Axum `with_*_registry`, Task 5 Step 0) is **interim scaffolding thrown away in Phase 4**. Two ways to sequence: +- **(Recommended) Land P0-C + P0-D early** (alongside Phase 0 A/B), so Phase 1's adapter wiring can target `run_app` from the start and skip the throwaway local builders — Phase 1 then only does the *core* store work (composite, store-id reconciliation, boot read), and the adapters get registries free from `run_app`. +- **(Fallback) Proceed with the interim builders** if P0-C/P0-D slip; accept that Task 5 Step 0 / Task 6 are deleted in Phase 4. The core Phase 1 work (composite, D5/D6, chunk-resolver deletion) is unaffected either way. + --- ## 4. Cross-cutting design decisions -**D1 — Config caching, not per-request extraction.** Keep the load-once model: at startup each adapter reads the blob from the config store, verifies the envelope, resolves secrets, validates, and stores `Arc` in `AppState`. Handlers read it via `State` (Phase 4). Rationale: `AppConfig`'s per-request re-parse/verify/secret-walk is prohibitive for a struct this large. Trade-off: config changes require a new deploy/boot to take effect (already true today). *This diverges deliberately from the stock `AppConfig` extractor; documented as such.* +**D1 — Config caching, not per-request extraction (retained under full convergence).** The user's full-convergence choice explicitly "reconsiders D1", but the rationale stands: `AppConfig`'s per-request re-parse/verify/secret-walk is prohibitive for a struct this large, so keep the **load-once** model. Under full convergence the *mechanism* changes: instead of a hand-built router with `with_state`, `Settings` is loaded once (lazily on first use, cached in a `OnceCell` inside `AppState`) and reaches `#[action]` handlers via `State>` injected through the **P0-D** state hook. Handlers that genuinely want the stock per-request typed extractor can still use `AppConfig` for small sub-configs, but the hot `Settings` path stays cached. Trade-off unchanged: config changes need a new deploy/boot (already true today). **D2 — Integration proxy router stays put (for now).** Phase 4 migrates the **named/core routes** to extractors. The integration registry's nested `matchit` dispatch and `IntegrationProxy::handle` signature are internal, working, and orthogonal; migrating them is a **follow-up** (Phase 4b, optional), not silently dropped. Called out so the extractor migration isn't mistaken for "all handlers." @@ -106,10 +113,16 @@ Phase 1 (stores) ──> Phase 2 (config) ──> Phase 3 (secrets) Phase 4 (e These are not trusted-server refactors; they are EdgeZero-adapter capability gaps or up-front decisions that gate the phases. -**P0-C — EdgeZero adapter dispatch that preserves multi-value headers and skips logger reinit (Fastly).** For Fastly to reach a thin entry point, EdgeZero's Fastly adapter dispatch must: (1) preserve duplicate response headers (esp. `Set-Cookie`) instead of collapsing via `set_header`; (2) allow the app to opt out of the per-call logger reinit; and (3) provide a hook to inject request-scoped extensions (`client_info`, `device_signals`) derived from the raw `FastlyRequest` before conversion. **Two resolutions:** -- **(Recommended) Upstream to EdgeZero** as a header-preserving `run_app`/dispatch variant + a pre-dispatch extension hook. Add to the edgezero prerequisite set alongside Phase 0 (A/B). Then Fastly's `main.rs` collapses to that variant. -- **(Fallback) Permanent documented exception** — Fastly keeps a small adapter-level dispatch shim calling `app.router().oneshot()`. The end-state (§1) already allows this. This is *not* pre-EdgeZero cruft and would survive Phase 5. -Decision needed with the edgezero maintainer; feeds the same PR track as #305. +**P0-C — Header-preserving Fastly `run_app` dispatch + pre-dispatch extension hook (REQUIRED under full convergence).** For Fastly to be the one-line `run_app::`, EdgeZero's Fastly `run_app`/dispatch must: (1) preserve duplicate response headers (esp. `Set-Cookie`) instead of collapsing via `set_header`; (2) allow opting out of the per-call logger reinit; and (3) expose a **pre-dispatch hook** to inject request-scoped extensions (`client_info`, `device_signals` from JA4/H2) derived from the raw `FastlyRequest` before conversion. **This must be upstreamed** — the "permanent Fastly exception" fallback is off the table now that the decision is full convergence. New edgezero PR (same track as #305/#306). Until it lands, Fastly keeps its interim custom `oneshot` + local registry builders (Phase 1 Task 6) as **throwaway scaffolding**, replaced by `run_app` once P0-C merges. + +**P0-D — App-state injection for macro/`run_app` apps (REQUIRED, new).** `State` (from #306) is read from request extensions, and `RouterBuilder::with_state` inserts it — but the `app!` macro generates the router and `run_app` dispatches it, so a macro app has **no builder to call `with_state` on**. EdgeZero must add a way for a `Hooks` app to register app-level state that `run_app` inserts into every request — e.g. a `Hooks::app_state() -> Extensions` (or `configure`-time state layer / `app!` `[app] state` support) that the per-adapter `run_app` copies into each request's extensions alongside the store registries. Without this, `State>` cannot reach `#[action]` handlers in a macro app. New edgezero PR. + +**P-BOOT — Boot-time store access for startup config + secret load.** Define, per adapter, how `build_state()` obtains a config-store (and secret-store) handle at boot, before request context. Options: +- **(a) Boot-time handle from the adapter environment** — Cloudflare builds a config-store handle from the `env` binding passed to `run_app`; Spin from the host component config; Fastly/Axum open the store eagerly (already do). Requires EdgeZero to expose a boot-time store constructor (or trusted-server constructs it from the adapter's env directly, mirroring today's `TRUSTED_SERVER_CONFIG` side-channel but reading the store instead). +- **(b) Lazy first-request load + cache** — defer the config load to the first request (where the registry exists), cache `Arc` in a `OnceCell`. Keeps D1's load-once semantics but moves the load off the boot path. Trade-off: first request pays the cost and must handle a config-load error as a request error. +Recommendation: **(a)** where the adapter env is available at boot (Cloudflare/Spin both pass it to `run_app`), falling back to **(b)** only if an adapter genuinely cannot construct a boot-time handle. Settle in the Phase 2 plan. + +*(P-BOOT's `build_state()` gets subsumed under full convergence: once `run_app` owns startup, boot config load is `run_app` reading the config store via the registry it builds; the boot reader collapses into option (a)/(b) inside `run_app`, and D1's load-once `Settings` is populated from the P0-D state hook at first use.)* **P-BOOT — Boot-time store access for startup config + secret load.** Define, per adapter, how `build_state()` obtains a config-store (and secret-store) handle at boot, before request context. Options: - **(a) Boot-time handle from the adapter environment** — Cloudflare builds a config-store handle from the `env` binding passed to `run_app`; Spin from the host component config; Fastly/Axum open the store eagerly (already do). Requires EdgeZero to expose a boot-time store constructor (or trusted-server constructs it from the adapter's env directly, mirroring today's `TRUSTED_SERVER_CONFIG` side-channel but reading the store instead). @@ -203,21 +216,22 @@ Two consequences: (1) edgezero #305 **must** ship `ArrayEach` + `Option` --- -### Phase 4 — Handlers → extractors +### Phase 4 — Full convergence: `app!` macro + `run_app` + `#[action]` extractors -**Goal:** core route handlers become `#[action]` functions taking `FromRequest` extractors; per-adapter handler shims deleted. +**Goal:** trusted-server becomes the canonical `app-demo` shape — every adapter binary is `run_app::`, `App` is `app!`-macro-generated from `edgezero.toml`, handlers are `#[action]` fns with `FromRequest` extractors, and **all** trusted-server-local registry/dispatch wiring (the Phase 1 interim Fastly/Axum builders, the custom `oneshot`, the per-adapter handler shims) is deleted. -**Depends on:** Phase 0 (A) `State`; cleaner after Phase 1. +**Depends on:** Phase 0 (A) `State`, **P0-C** (Fastly `run_app` dispatch), **P0-D** (macro app-state injection). Cleaner after Phases 1–3. **Changes:** -- Introduce `State>` (or narrower `State>` / `State>` / `State>`) wired via `RouterBuilder::with_state` in each adapter's `Hooks::routes()`. *Granularity (one `Arc` vs per-component states) is a Phase 4 plan decision.* -- Rewrite core `handle_*` (`proxy.rs`, `publisher.rs`, `auction/endpoints.rs`, `request_signing/endpoints.rs`, `ec/*.rs`) from `(&Settings, &RuntimeServices, Request)` to `#[action]` signatures using `State<…>`, `Json`/`Query`/`Path`/`Headers`/`Host`, and the store extractors (`Kv`, `Secrets`, `Config`). -- Delete the per-adapter shims (`execute_handler`/`execute_named`/`named_route_handler` + `NamedRouteHandler` enums) and shrink/retire `RuntimeServices` (its store fields already gone in Phase 1; remaining bundle folds into `State` + extractors). -- **EC lifecycle & pre-route filters** (`build_ec_request_state`, `run_pre_route_filters`, `attach_dispatch_extensions`, `FinalizeResponseMiddleware`) are cross-cutting — keep them as **middleware**, not per-arg extractors. -- **Phase 4b (optional follow-up, D2):** migrate the integration proxy nested router / `IntegrationProxy::handle` onto `RouterService` + extractors. Deferred by default. +- **Entry points → `run_app::`** on all four adapters. Fastly uses the P0-C header-preserving dispatch (deleting the custom `oneshot` + local registry builders from Phase 1 Task 6). Axum bridges `PORT`/`axum.toml` → `EDGEZERO__ADAPTER__PORT/HOST` and calls `dev_server::run_app` (deleting the `AxumDevServer::with_config` + `with_*_registry` wiring from Phase 1 Task 5). Cloudflare/Spin already `run_app`. +- **Routing → `app!` macro.** Declare trusted-server's routes in `edgezero.toml` `[[triggers.http]]` (named routes + a `/{*rest}` catch-all trigger for the publisher/integration fallback), and let the macro generate `Hooks::routes()`/`stores()`. This replaces the hand-written `routes_for_state`/`NAMED_ROUTES` tables. +- **App state via P0-D.** `Arc` (Settings/orchestrator/integration registry) is registered through the P0-D `Hooks` state hook and reaches `#[action]` handlers as `State>`. *Granularity (one `Arc` vs per-component states) is a Phase 4 plan decision.* +- **Handlers → `#[action]`.** Rewrite core `handle_*` (`proxy.rs`, `publisher.rs`, `auction/endpoints.rs`, `request_signing/endpoints.rs`, `ec/*.rs`) from `(&Settings, &RuntimeServices, Request)` to `#[action]` signatures using `State<…>`, `Json`/`Query`/`Path`/`Headers`/`Host`, and the store extractors. Delete the per-adapter handler shims + `NamedRouteHandler` enums; retire `RuntimeServices`. +- **EC lifecycle & pre-route filters** stay as **middleware** (declared via `[app].middleware` in `edgezero.toml`), plus the P0-C pre-dispatch hook for Fastly JA4/client_info. +- **Integration proxy router** (`IntegrationProxy::handle`) is dispatched from the catch-all fallback `#[action]` handler — it keeps its internal `matchit` routing but is reached as an `#[action]`, so no separate dispatch system survives. -**Deletions:** per-adapter handler shims, `NamedRouteHandler` enums, `RuntimeServices` (final form). -**Acceptance:** every named route **that a given adapter supports** is served via an `#[action]` handler on that adapter (route sets are *not* uniform — Fastly exposes EC identity routes `/_ts/api/v1/{identify,batch-sync}`; Spin and Axum deliberately omit them to match non-Fastly adapters); middleware carries EC lifecycle, and **Fastly-only EC after-send / finalize ordering** is preserved; parity test green. +**Deletions:** custom Fastly `oneshot` + Phase 1 Fastly/Axum local registry builders (throwaway scaffolding), `AxumDevServer::with_config` wiring, per-adapter handler shims, `NamedRouteHandler` enums, hand-written `routes_for_state`/`NAMED_ROUTES`, `RuntimeServices` (final form). +**Acceptance:** all four binaries are one-line `run_app::`; routes come from `edgezero.toml`; every named route a given adapter supports is served via an `#[action]` handler (route sets are not uniform — Fastly-only EC routes preserved); Fastly EC after-send / finalize ordering preserved; parity test green. --- @@ -263,7 +277,9 @@ Two consequences: (1) edgezero #305 **must** ship `ArrayEach` + `Option` | ID | Question | Owner / resolution | |----|----------|--------------------| | R1 | Do any `Settings` secrets live inside **arrays**? | **Resolved: yes** (`ec.partners[].api_token`, `handlers[].password`) + optional (`ts_pull_token`). edgezero #305 must ship `ArrayEach` + `Option` (see §5 Phase 3 inventory + Phase 0 note). | -| R7 | P0-C: upstream a header-preserving Fastly dispatch, or keep a permanent Fastly dispatch shim? | Decide with edgezero maintainer (§4a); gates the Fastly end-state and Phase 5. | +| R7 | P0-C: header-preserving Fastly `run_app` dispatch + pre-dispatch hook — **REQUIRED** under full convergence (no permanent-exception fallback). | New edgezero PR; gates Phase 4 Fastly convergence. | +| R13 | P0-D: macro/`run_app` app-state injection (`with_state` is builder-only). | New edgezero PR; gates `State` in `#[action]` handlers. | +| R14 | Sequencing: land P0-C/P0-D early (skip Phase 1 throwaway adapter builders) vs proceed with interim builders? | Decide now (§3 sequencing note) — affects Phase 1 Tasks 5–6 scope. | | R8 | P-BOOT: boot-time store handle (a) vs lazy cached first-request load (b), per adapter? | Phase 2 plan (§4a). | | R9 | D5: reconcile **all** runtime store ids **by kind** with `edgezero.toml` (strict lookup fails otherwise) — KV: `ec_identity_store`, `consent_store`, `creative_store`; config: `trusted_server_config` (app-config blob, key == id), `jwks_store`, `datadome-ip-bypass`; secrets: `signing_keys`, `ts_secrets`, `s3-auth`. Request signing uses `jwks_store`/`signing_keys` (not the app-config store). | Phase 1 plan task 1/2. | | R12 | Fastly `EnvConfig` reader is private / `fastly::ConfigStore` has no `iter()`. | **Resolved by D7** — runtime opens stores by logical id; no store-name env/dictionary read; no local `EnvConfig` reader needed. | From 571ac6867e62f97214a912ed6fb65ded58bc2ebc Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Fri, 3 Jul 2026 16:02:48 -0700 Subject: [PATCH 19/30] Add P0-C + P0-D edgezero spec (Fastly run_app fidelity + macro app-state) Implementation-ready spec to hand to the edgezero dev, grounded in pinned 6ebc29a: - P0-C: C1 append_header for multi-value Set-Cookie (response.rs:29 set_header bug), C2 Hooks::owns_logging() opt-out, C3 run_app_with_request_extensions pre-dispatch hook for JA4/H2/client_info from the raw fastly::Request - P0-D: Hooks::app_state() injected per request (symmetric with registries) + app! macro state= argument; note P0-D is avoidable via hand-built routes()+with_state --- ...0cd-fastly-dispatch-and-appstate-design.md | 171 ++++++++++++++++++ 1 file changed, 171 insertions(+) create mode 100644 docs/superpowers/specs/2026-07-03-edgezero-p0cd-fastly-dispatch-and-appstate-design.md diff --git a/docs/superpowers/specs/2026-07-03-edgezero-p0cd-fastly-dispatch-and-appstate-design.md b/docs/superpowers/specs/2026-07-03-edgezero-p0cd-fastly-dispatch-and-appstate-design.md new file mode 100644 index 00000000..24fe9ae8 --- /dev/null +++ b/docs/superpowers/specs/2026-07-03-edgezero-p0cd-fastly-dispatch-and-appstate-design.md @@ -0,0 +1,171 @@ +# EdgeZero P0-C + P0-D — Fastly `run_app` dispatch fidelity + app-state injection + +- **Status:** Draft for edgezero maintainer +- **Date:** 2026-07-03 +- **Target repo:** `github.com/stackpop/edgezero` (`edgezero-adapter-fastly`, `edgezero-core`, `edgezero-macros`) +- **Consumed by:** trusted-server "full convergence" migration — the decision that every adapter binary becomes the one-line `run_app::` with `#[action]` handlers. These two capabilities are the remaining gaps that block Fastly (P0-C) and macro-based app state (P0-D). Independent of the earlier Phase 0 spec (State + nested `#[secret]`, PR #306). +- **Verified against:** pinned commit `6ebc29a5` (branch `worktree-state-nested-secrets-spec-review`). + +--- + +## Why + +trusted-server is converging on the canonical `app-demo` wiring: `run_app::` on every adapter, `#[action]` handlers, `State>`. Two things stop that today: + +1. **Fastly `run_app` loses fidelity** that trusted-server's hand-written custom dispatch preserves: multi-value `Set-Cookie` headers, an opt-out from the per-call logger reinit, and a pre-dispatch hook to capture Fastly-only request signals (TLS JA4 / H2 fingerprint, client IP) from the raw `fastly::Request` before it is converted to the neutral core request. → **P0-C.** +2. **Macro/`run_app` apps can't inject app-owned state.** `State` + `RouterBuilder::with_state` exist (PR #306) and the router injects registered state at dispatch — but the `app!` macro generates the router and never calls `with_state`, and `run_app` doesn't inject app state. So `State>` can't reach handlers in a macro app. → **P0-D.** + +**P0-D is optional** (see §4): if a downstream keeps a hand-written `Hooks::routes()` that calls `RouterBuilder::with_state`, the existing dispatch-time injection already delivers `State` under `run_app` — no edgezero change needed. P0-D is required only to support app-owned state **through the `app!` macro**. It is specified here so the maintainer can choose to support the fully-macro path. + +--- + +## P0-C — Fastly `run_app` dispatch fidelity + +Three independent sub-changes in `edgezero-adapter-fastly`. Each is small and separately testable. + +### C1 — Preserve multi-value response headers (`Set-Cookie`) + +**Current (bug):** `crates/edgezero-adapter-fastly/src/response.rs` builds the `fastly::Response` by looping over the core response's `HeaderMap` and calling `set_header`, which **replaces** — so N `Set-Cookie` values collapse to the last one: + +```rust +// response.rs (~line 28) +for (name, value) in &parts.headers { + fastly_response.set_header(name.as_str(), value.as_bytes()); +} +``` + +`http::HeaderMap`'s iterator yields **one entry per value** (duplicates included), and the `fastly::Response` starts empty (`FastlyResponse::from_status(...)`). So the fix is to **append** instead of set: + +```rust +for (name, value) in &parts.headers { + fastly_response.append_header(name.as_str(), value.as_bytes()); +} +``` + +`append_header` adds without clobbering, so all `Set-Cookie` (and any other multi-value header) survive. This is unconditionally correct given a fresh response; no per-header special-casing needed. + +**Same defect on the outbound proxy path:** `crates/edgezero-adapter-fastly/src/proxy.rs:53` uses `set_header` when building the upstream `fastly::Request` — audit whether request-side multi-value headers (rare, but `Cookie` folding differs) need the same treatment; at minimum document why request-side `set_header` is acceptable. + +**Test:** a handler returns a `Response` with two `Set-Cookie` values; assert the converted `fastly::Response` (via `get_header_all("set-cookie")`) contains both. + +### C2 — Let the app opt out of the `run_app` logger init + +**Current:** `run_app` (`lib.rs:113`) initializes the Fastly logger unconditionally when `use_fastly_logger`: + +```rust +let logging = logging_from_env(&env); +if logging.use_fastly_logger { + init_logger(endpoint, logging.level, logging.echo_stdout)?; +} +``` + +An app that already owns `log`/`log-fastly` initialization (trusted-server does) cannot use `run_app` without a double-init conflict. Provide an opt-out. **Preferred:** a `Hooks` flag consulted by every adapter's `run_app`, so it is platform-neutral: + +```rust +// edgezero-core/src/app.rs — Hooks +/// When `true`, the adapter's `run_app` skips its own logger +/// initialization; the app is responsible for installing a `log` backend. +/// Default `false` (adapter initializes logging as today). +fn owns_logging() -> bool { false } +``` + +`run_app` becomes `if logging.use_fastly_logger && !A::owns_logging() { init_logger(...)?; }`. (Alternative: a `run_app_without_logger::` variant — but the `Hooks` flag composes with the `app!` macro and applies uniformly across adapters, so prefer it.) + +**Test:** an app with `owns_logging() == true` runs `run_app` twice / after the app initialized its own logger without the init error. + +### C3 — Pre-dispatch hook for raw-request signals (JA4 / H2 / client IP) + +**Current:** `run_app` → `dispatch_with_registries` → `dispatch_with_handles` converts the `fastly::Request` into the neutral core request and inserts the store registries into its extensions. There is **no hook** to read the *original* `fastly::Request` (whose `get_tls_ja4()`, `get_client_h2_fingerprint()`, `client_ip` are only available pre-conversion) and stash derived values into the core request's extensions. trusted-server's custom path does exactly this before dispatch. + +**Proposed:** a Fastly-adapter `run_app` variant that accepts a pre-dispatch closure which populates extensions from the raw request: + +```rust +// edgezero-adapter-fastly/src/lib.rs +pub fn run_app_with_request_extensions( + req: fastly::Request, + extend: F, +) -> Result +where + A: Hooks, + F: FnOnce(&fastly::Request, &mut http::Extensions), +{ /* same as run_app, but call `extend(&req, core_req.extensions_mut())` + inside dispatch, after conversion and after registry insertion, + before the router runs */ } +``` + +The closure runs once per request, receives the raw `fastly::Request` and the core request's `Extensions`, and inserts whatever typed values the app needs (trusted-server inserts its `ClientInfo` + `DeviceSignals`). `run_app` stays as the no-hook convenience wrapper (`run_app_with_request_extensions::(req, |_, _| {})`). + +This requires threading the closure from `run_app_with_request_extensions` → `dispatch_with_registries` → `dispatch_with_handles` (add a generic `extend: F` parameter, or an `Option<&mut dyn FnMut(&FastlyRequest, &mut Extensions)>`). Keep the existing `dispatch_with_registries` signature working (the no-op closure). + +**Test:** a handler reads a value from extensions that only the pre-dispatch closure could have set (e.g. a synthetic `Ja4` newtype); assert it is present. + +### P0-C acceptance + +- Multi-value `Set-Cookie` round-trips through `run_app` (C1). +- An app that owns logging runs under `run_app` without a logger-init error (C2). +- A pre-dispatch closure can populate core-request extensions from the raw `fastly::Request` (C3). +- `app-demo` still builds/serves; existing Fastly tests green; `run_app` (no-hook) behavior unchanged for apps that don't opt in. + +--- + +## P0-D — App-state injection for macro / `run_app` apps + +### The gap + +`State` (`extractor.rs:550`) reads from request extensions; `RouterBuilder::with_state` (`router.rs`) registers a value that the router's `state_inserters` clone into each request at dispatch (`router.rs` ~line 256). That works when the app **hand-builds** its router. But the `app!` macro generates `Hooks::routes()` and never calls `with_state`, and `run_app` doesn't inject app state — so a macro app has no way to provide `State>`. + +### Design — symmetric with registry injection + +Registries are injected **per request** by each adapter's `run_app` (in `dispatch_with_handles`), not baked into the router. Mirror that for app state: + +1. **`edgezero-core/src/app.rs` — new `Hooks` method** (default: no state): +```rust +/// App-owned state inserted into every request's extensions before dispatch, +/// making it available to the `State` extractor in macro-based apps. +/// Returns type-erased inserters (same shape as RouterBuilder's state layer). +/// Default: none. +fn app_state() -> AppState { AppState::default() } +``` +where `AppState` is a small type-erased carrier (reuse the `StateInserter = Arc` shape already in `router.rs`, exposed as a public builder, e.g. `AppState::default().with(value)`). + +2. **Each adapter's `run_app` applies `A::app_state()` to every request's extensions** — in the same spot it inserts the store registries (Fastly `dispatch_with_handles`; Axum `EdgeZeroAxumService`; Cloudflare/Spin equivalents). Precedence note: app-state inserts should not overwrite the store registries (distinct `TypeId`s; document last-writer-wins if an app registers a colliding type). + +3. **`edgezero-macros` — `app!` gains an optional `state` argument** so macro apps can wire it: +```rust +edgezero_core::app!("edgezero.toml", state = crate::app_state); +// expands to `fn app_state() -> AppState { crate::app_state() }` in the generated Hooks impl +``` +Without the argument the generated `app_state()` uses the default (no state), preserving current behavior. + +### Alternative that needs NO P0-D (document in the guide) + +A downstream that keeps a **hand-written `Hooks::routes()`** can call `RouterBuilder::with_state(app_state)` there; the existing dispatch-time `state_inserters` then inject it under `run_app` with zero further change. The trade-off is routes are built in Rust rather than declared in `edgezero.toml`. trusted-server may take this path to avoid P0-D — but P0-D is what makes app state work for the **fully macro-driven** shape `app-demo` models. + +### P0-D acceptance + +- A macro app declaring `app!("...", state = f)` can extract `State` (where `T` is what `f` returns) in an `#[action]` handler on all four adapters. +- An app that provides no state is unaffected (`State` for an unregistered `T` returns the existing "no state registered" 500). +- `app-demo` gains a small example using `app!(..., state = ...)` + a `State` handler. + +--- + +## Sequencing & interaction with trusted-server Phase 1 + +- **P0-C is required** for trusted-server Phase 4 (Fastly `run_app`). Until it lands, trusted-server's Phase 1 keeps interim Fastly local registry builders + custom `oneshot`; those are deleted in Phase 4 once P0-C exists. Landing P0-C early lets Phase 1 skip that throwaway scaffolding. +- **P0-D is required only for the `app!`-macro path.** If trusted-server keeps hand-built `routes()` + `with_state`, P0-C alone suffices for full `run_app` convergence. Decide this before Phase 4. +- Both are independent of the nested-`#[secret]` work already in #306. + +## Files to touch (edgezero) + +**P0-C** +- `crates/edgezero-adapter-fastly/src/response.rs` — `set_header` → `append_header` (C1) +- `crates/edgezero-adapter-fastly/src/proxy.rs` — audit request-side `set_header` (C1) +- `crates/edgezero-core/src/app.rs` — `Hooks::owns_logging()` (C2) +- `crates/edgezero-adapter-fastly/src/lib.rs` — consult `owns_logging()`; add `run_app_with_request_extensions` (C2, C3) +- `crates/edgezero-adapter-fastly/src/request.rs` — thread the pre-dispatch closure through `dispatch_with_registries`/`dispatch_with_handles` (C3) + +**P0-D** +- `crates/edgezero-core/src/app.rs` — `Hooks::app_state()` + `AppState` carrier +- `crates/edgezero-core/src/router.rs` — expose the `StateInserter`/state-carrier type publicly (reuse existing) +- `crates/edgezero-adapter-{fastly,axum,cloudflare,spin}/src/…` — apply `A::app_state()` per request alongside registries +- `crates/edgezero-macros/src/app.rs` — optional `state = ` argument From 669a4282a07e13e705cde057cfba7c9340dae71a Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Fri, 3 Jul 2026 16:03:41 -0700 Subject: [PATCH 20/30] Move P0-C/P0-D spec to the edgezero worktree (edgezero deliverable) --- ...0cd-fastly-dispatch-and-appstate-design.md | 171 ------------------ 1 file changed, 171 deletions(-) delete mode 100644 docs/superpowers/specs/2026-07-03-edgezero-p0cd-fastly-dispatch-and-appstate-design.md diff --git a/docs/superpowers/specs/2026-07-03-edgezero-p0cd-fastly-dispatch-and-appstate-design.md b/docs/superpowers/specs/2026-07-03-edgezero-p0cd-fastly-dispatch-and-appstate-design.md deleted file mode 100644 index 24fe9ae8..00000000 --- a/docs/superpowers/specs/2026-07-03-edgezero-p0cd-fastly-dispatch-and-appstate-design.md +++ /dev/null @@ -1,171 +0,0 @@ -# EdgeZero P0-C + P0-D — Fastly `run_app` dispatch fidelity + app-state injection - -- **Status:** Draft for edgezero maintainer -- **Date:** 2026-07-03 -- **Target repo:** `github.com/stackpop/edgezero` (`edgezero-adapter-fastly`, `edgezero-core`, `edgezero-macros`) -- **Consumed by:** trusted-server "full convergence" migration — the decision that every adapter binary becomes the one-line `run_app::` with `#[action]` handlers. These two capabilities are the remaining gaps that block Fastly (P0-C) and macro-based app state (P0-D). Independent of the earlier Phase 0 spec (State + nested `#[secret]`, PR #306). -- **Verified against:** pinned commit `6ebc29a5` (branch `worktree-state-nested-secrets-spec-review`). - ---- - -## Why - -trusted-server is converging on the canonical `app-demo` wiring: `run_app::` on every adapter, `#[action]` handlers, `State>`. Two things stop that today: - -1. **Fastly `run_app` loses fidelity** that trusted-server's hand-written custom dispatch preserves: multi-value `Set-Cookie` headers, an opt-out from the per-call logger reinit, and a pre-dispatch hook to capture Fastly-only request signals (TLS JA4 / H2 fingerprint, client IP) from the raw `fastly::Request` before it is converted to the neutral core request. → **P0-C.** -2. **Macro/`run_app` apps can't inject app-owned state.** `State` + `RouterBuilder::with_state` exist (PR #306) and the router injects registered state at dispatch — but the `app!` macro generates the router and never calls `with_state`, and `run_app` doesn't inject app state. So `State>` can't reach handlers in a macro app. → **P0-D.** - -**P0-D is optional** (see §4): if a downstream keeps a hand-written `Hooks::routes()` that calls `RouterBuilder::with_state`, the existing dispatch-time injection already delivers `State` under `run_app` — no edgezero change needed. P0-D is required only to support app-owned state **through the `app!` macro**. It is specified here so the maintainer can choose to support the fully-macro path. - ---- - -## P0-C — Fastly `run_app` dispatch fidelity - -Three independent sub-changes in `edgezero-adapter-fastly`. Each is small and separately testable. - -### C1 — Preserve multi-value response headers (`Set-Cookie`) - -**Current (bug):** `crates/edgezero-adapter-fastly/src/response.rs` builds the `fastly::Response` by looping over the core response's `HeaderMap` and calling `set_header`, which **replaces** — so N `Set-Cookie` values collapse to the last one: - -```rust -// response.rs (~line 28) -for (name, value) in &parts.headers { - fastly_response.set_header(name.as_str(), value.as_bytes()); -} -``` - -`http::HeaderMap`'s iterator yields **one entry per value** (duplicates included), and the `fastly::Response` starts empty (`FastlyResponse::from_status(...)`). So the fix is to **append** instead of set: - -```rust -for (name, value) in &parts.headers { - fastly_response.append_header(name.as_str(), value.as_bytes()); -} -``` - -`append_header` adds without clobbering, so all `Set-Cookie` (and any other multi-value header) survive. This is unconditionally correct given a fresh response; no per-header special-casing needed. - -**Same defect on the outbound proxy path:** `crates/edgezero-adapter-fastly/src/proxy.rs:53` uses `set_header` when building the upstream `fastly::Request` — audit whether request-side multi-value headers (rare, but `Cookie` folding differs) need the same treatment; at minimum document why request-side `set_header` is acceptable. - -**Test:** a handler returns a `Response` with two `Set-Cookie` values; assert the converted `fastly::Response` (via `get_header_all("set-cookie")`) contains both. - -### C2 — Let the app opt out of the `run_app` logger init - -**Current:** `run_app` (`lib.rs:113`) initializes the Fastly logger unconditionally when `use_fastly_logger`: - -```rust -let logging = logging_from_env(&env); -if logging.use_fastly_logger { - init_logger(endpoint, logging.level, logging.echo_stdout)?; -} -``` - -An app that already owns `log`/`log-fastly` initialization (trusted-server does) cannot use `run_app` without a double-init conflict. Provide an opt-out. **Preferred:** a `Hooks` flag consulted by every adapter's `run_app`, so it is platform-neutral: - -```rust -// edgezero-core/src/app.rs — Hooks -/// When `true`, the adapter's `run_app` skips its own logger -/// initialization; the app is responsible for installing a `log` backend. -/// Default `false` (adapter initializes logging as today). -fn owns_logging() -> bool { false } -``` - -`run_app` becomes `if logging.use_fastly_logger && !A::owns_logging() { init_logger(...)?; }`. (Alternative: a `run_app_without_logger::` variant — but the `Hooks` flag composes with the `app!` macro and applies uniformly across adapters, so prefer it.) - -**Test:** an app with `owns_logging() == true` runs `run_app` twice / after the app initialized its own logger without the init error. - -### C3 — Pre-dispatch hook for raw-request signals (JA4 / H2 / client IP) - -**Current:** `run_app` → `dispatch_with_registries` → `dispatch_with_handles` converts the `fastly::Request` into the neutral core request and inserts the store registries into its extensions. There is **no hook** to read the *original* `fastly::Request` (whose `get_tls_ja4()`, `get_client_h2_fingerprint()`, `client_ip` are only available pre-conversion) and stash derived values into the core request's extensions. trusted-server's custom path does exactly this before dispatch. - -**Proposed:** a Fastly-adapter `run_app` variant that accepts a pre-dispatch closure which populates extensions from the raw request: - -```rust -// edgezero-adapter-fastly/src/lib.rs -pub fn run_app_with_request_extensions( - req: fastly::Request, - extend: F, -) -> Result -where - A: Hooks, - F: FnOnce(&fastly::Request, &mut http::Extensions), -{ /* same as run_app, but call `extend(&req, core_req.extensions_mut())` - inside dispatch, after conversion and after registry insertion, - before the router runs */ } -``` - -The closure runs once per request, receives the raw `fastly::Request` and the core request's `Extensions`, and inserts whatever typed values the app needs (trusted-server inserts its `ClientInfo` + `DeviceSignals`). `run_app` stays as the no-hook convenience wrapper (`run_app_with_request_extensions::(req, |_, _| {})`). - -This requires threading the closure from `run_app_with_request_extensions` → `dispatch_with_registries` → `dispatch_with_handles` (add a generic `extend: F` parameter, or an `Option<&mut dyn FnMut(&FastlyRequest, &mut Extensions)>`). Keep the existing `dispatch_with_registries` signature working (the no-op closure). - -**Test:** a handler reads a value from extensions that only the pre-dispatch closure could have set (e.g. a synthetic `Ja4` newtype); assert it is present. - -### P0-C acceptance - -- Multi-value `Set-Cookie` round-trips through `run_app` (C1). -- An app that owns logging runs under `run_app` without a logger-init error (C2). -- A pre-dispatch closure can populate core-request extensions from the raw `fastly::Request` (C3). -- `app-demo` still builds/serves; existing Fastly tests green; `run_app` (no-hook) behavior unchanged for apps that don't opt in. - ---- - -## P0-D — App-state injection for macro / `run_app` apps - -### The gap - -`State` (`extractor.rs:550`) reads from request extensions; `RouterBuilder::with_state` (`router.rs`) registers a value that the router's `state_inserters` clone into each request at dispatch (`router.rs` ~line 256). That works when the app **hand-builds** its router. But the `app!` macro generates `Hooks::routes()` and never calls `with_state`, and `run_app` doesn't inject app state — so a macro app has no way to provide `State>`. - -### Design — symmetric with registry injection - -Registries are injected **per request** by each adapter's `run_app` (in `dispatch_with_handles`), not baked into the router. Mirror that for app state: - -1. **`edgezero-core/src/app.rs` — new `Hooks` method** (default: no state): -```rust -/// App-owned state inserted into every request's extensions before dispatch, -/// making it available to the `State` extractor in macro-based apps. -/// Returns type-erased inserters (same shape as RouterBuilder's state layer). -/// Default: none. -fn app_state() -> AppState { AppState::default() } -``` -where `AppState` is a small type-erased carrier (reuse the `StateInserter = Arc` shape already in `router.rs`, exposed as a public builder, e.g. `AppState::default().with(value)`). - -2. **Each adapter's `run_app` applies `A::app_state()` to every request's extensions** — in the same spot it inserts the store registries (Fastly `dispatch_with_handles`; Axum `EdgeZeroAxumService`; Cloudflare/Spin equivalents). Precedence note: app-state inserts should not overwrite the store registries (distinct `TypeId`s; document last-writer-wins if an app registers a colliding type). - -3. **`edgezero-macros` — `app!` gains an optional `state` argument** so macro apps can wire it: -```rust -edgezero_core::app!("edgezero.toml", state = crate::app_state); -// expands to `fn app_state() -> AppState { crate::app_state() }` in the generated Hooks impl -``` -Without the argument the generated `app_state()` uses the default (no state), preserving current behavior. - -### Alternative that needs NO P0-D (document in the guide) - -A downstream that keeps a **hand-written `Hooks::routes()`** can call `RouterBuilder::with_state(app_state)` there; the existing dispatch-time `state_inserters` then inject it under `run_app` with zero further change. The trade-off is routes are built in Rust rather than declared in `edgezero.toml`. trusted-server may take this path to avoid P0-D — but P0-D is what makes app state work for the **fully macro-driven** shape `app-demo` models. - -### P0-D acceptance - -- A macro app declaring `app!("...", state = f)` can extract `State` (where `T` is what `f` returns) in an `#[action]` handler on all four adapters. -- An app that provides no state is unaffected (`State` for an unregistered `T` returns the existing "no state registered" 500). -- `app-demo` gains a small example using `app!(..., state = ...)` + a `State` handler. - ---- - -## Sequencing & interaction with trusted-server Phase 1 - -- **P0-C is required** for trusted-server Phase 4 (Fastly `run_app`). Until it lands, trusted-server's Phase 1 keeps interim Fastly local registry builders + custom `oneshot`; those are deleted in Phase 4 once P0-C exists. Landing P0-C early lets Phase 1 skip that throwaway scaffolding. -- **P0-D is required only for the `app!`-macro path.** If trusted-server keeps hand-built `routes()` + `with_state`, P0-C alone suffices for full `run_app` convergence. Decide this before Phase 4. -- Both are independent of the nested-`#[secret]` work already in #306. - -## Files to touch (edgezero) - -**P0-C** -- `crates/edgezero-adapter-fastly/src/response.rs` — `set_header` → `append_header` (C1) -- `crates/edgezero-adapter-fastly/src/proxy.rs` — audit request-side `set_header` (C1) -- `crates/edgezero-core/src/app.rs` — `Hooks::owns_logging()` (C2) -- `crates/edgezero-adapter-fastly/src/lib.rs` — consult `owns_logging()`; add `run_app_with_request_extensions` (C2, C3) -- `crates/edgezero-adapter-fastly/src/request.rs` — thread the pre-dispatch closure through `dispatch_with_registries`/`dispatch_with_handles` (C3) - -**P0-D** -- `crates/edgezero-core/src/app.rs` — `Hooks::app_state()` + `AppState` carrier -- `crates/edgezero-core/src/router.rs` — expose the `StateInserter`/state-carrier type publicly (reuse existing) -- `crates/edgezero-adapter-{fastly,axum,cloudflare,spin}/src/…` — apply `A::app_state()` per request alongside registries -- `crates/edgezero-macros/src/app.rs` — optional `state = ` argument From b0d27cc93fbfebbcaa59f8b64f8e4434d6b2e6d2 Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Sat, 4 Jul 2026 12:51:13 -0700 Subject: [PATCH 21/30] Link Phase 0 to edgezero PR #306 (impl PR) instead of #305 (spec PR) --- ...026-07-02-edgezero-full-migration-design.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md b/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md index ca5cfa1c..417067ce 100644 --- a/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md +++ b/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md @@ -4,7 +4,7 @@ - **Date:** 2026-07-02 - **Scope:** Move trusted-server **completely** onto EdgeZero primitives: config push, KV, secret store, config injection without an embedded `trusted-server.toml`, extractor-based handlers, and deletion of every pre-EdgeZero workaround. - **Shape:** Umbrella roadmap. Defines the end-state, the current-state gap, and an ordered set of phases with dependencies. **Each phase gets its own implementation plan** (`writing-plans`) before code is written. -- **Companion spec:** Phase 0 (`State` extractor + nested `#[secret]`) is an **edgezero-repo** change, specified separately (`…-state-and-nested-secrets-design.md`) and tracked via edgezero PR [stackpop/edgezero#305](https://github.com/stackpop/edgezero/pull/305). This umbrella depends on it but does not re-specify it. +- **Companion spec:** Phase 0 (`State` extractor + nested `#[secret]`) is an **edgezero-repo** change, specified separately (`…-state-and-nested-secrets-design.md`) and tracked via edgezero PR [stackpop/edgezero#306](https://github.com/stackpop/edgezero/pull/306). This umbrella depends on it but does not re-specify it. --- @@ -113,7 +113,7 @@ Phase 1 (stores) ──> Phase 2 (config) ──> Phase 3 (secrets) Phase 4 (e These are not trusted-server refactors; they are EdgeZero-adapter capability gaps or up-front decisions that gate the phases. -**P0-C — Header-preserving Fastly `run_app` dispatch + pre-dispatch extension hook (REQUIRED under full convergence).** For Fastly to be the one-line `run_app::`, EdgeZero's Fastly `run_app`/dispatch must: (1) preserve duplicate response headers (esp. `Set-Cookie`) instead of collapsing via `set_header`; (2) allow opting out of the per-call logger reinit; and (3) expose a **pre-dispatch hook** to inject request-scoped extensions (`client_info`, `device_signals` from JA4/H2) derived from the raw `FastlyRequest` before conversion. **This must be upstreamed** — the "permanent Fastly exception" fallback is off the table now that the decision is full convergence. New edgezero PR (same track as #305/#306). Until it lands, Fastly keeps its interim custom `oneshot` + local registry builders (Phase 1 Task 6) as **throwaway scaffolding**, replaced by `run_app` once P0-C merges. +**P0-C — Header-preserving Fastly `run_app` dispatch + pre-dispatch extension hook (REQUIRED under full convergence).** For Fastly to be the one-line `run_app::`, EdgeZero's Fastly `run_app`/dispatch must: (1) preserve duplicate response headers (esp. `Set-Cookie`) instead of collapsing via `set_header`; (2) allow opting out of the per-call logger reinit; and (3) expose a **pre-dispatch hook** to inject request-scoped extensions (`client_info`, `device_signals` from JA4/H2) derived from the raw `FastlyRequest` before conversion. **This must be upstreamed** — the "permanent Fastly exception" fallback is off the table now that the decision is full convergence. New edgezero PR (same track as #306). Until it lands, Fastly keeps its interim custom `oneshot` + local registry builders (Phase 1 Task 6) as **throwaway scaffolding**, replaced by `run_app` once P0-C merges. **P0-D — App-state injection for macro/`run_app` apps (REQUIRED, new).** `State` (from #306) is read from request extensions, and `RouterBuilder::with_state` inserts it — but the `app!` macro generates the router and `run_app` dispatches it, so a macro app has **no builder to call `with_state` on**. EdgeZero must add a way for a `Hooks` app to register app-level state that `run_app` inserts into every request — e.g. a `Hooks::app_state() -> Extensions` (or `configure`-time state layer / `app!` `[app] state` support) that the per-adapter `run_app` copies into each request's extensions alongside the store registries. Without this, `State>` cannot reach `#[action]` handlers in a macro app. New edgezero PR. @@ -135,10 +135,10 @@ Recommendation: **(a)** where the adapter env is available at boot (Cloudflare/S ### Phase 0 — EdgeZero prerequisites (external, edgezero repo) -**Owner:** edgezero. **Tracked by:** its own spec + PR [stackpop/edgezero#305](https://github.com/stackpop/edgezero/pull/305) — "add State + nested #[secret] design spec". +**Owner:** edgezero. **Tracked by:** its own spec + PR [stackpop/edgezero#306](https://github.com/stackpop/edgezero/pull/306) — "State extractor + nested/array #[secret] support". **Delivers:** (A) `State` extractor + `RouterBuilder::with_state`; (B) nested/array `#[secret]` in `#[derive(AppConfig)]` + path-aware `secret_walk`; **(C, if resolved upstream) P0-C** header-preserving Fastly dispatch + pre-dispatch extension hook (§4a). **Blocks:** Phase 3 (B), Phase 4 (A), Phase 5/Fastly end-state (C). **This umbrella consumes it as a versioned dependency** — bump the pinned `edgezero` rev once merged. -**Note for #305:** the trusted-server secret audit (Phase 3 / §5) confirms **array secrets exist** (`ec.partners[].api_token`, `handlers[].password`) and **optional-string secrets exist** (`ts_pull_token`). So edgezero #305's `ArrayEach` and `Option` support are **required**, not deferrable — this settles that PR's open question B-1. +**Note for #306:** the trusted-server secret audit (Phase 3 / §5) confirms **array secrets exist** (`ec.partners[].api_token`, `handlers[].password`) and **optional-string secrets exist** (`ts_pull_token`). So edgezero #306's `ArrayEach` and `Option` support are **required**, not deferrable — this settles that PR's open question B-1. --- @@ -196,20 +196,20 @@ Recommendation: **(a)** where the adapter env is available at boot (Cloudflare/S - Operator migration: `ts` provisions secrets into the secret store (via EdgeZero provision), and a migration guide moves existing inline secrets out of `trusted-server.toml`. `reject_placeholder_secrets` becomes a check on the resolved values at boot. - Startup load resolves `#[secret]` fields against the secret store (D1/D3), then validates. -**Secret inventory (spec artifact — verify + extend during the Phase 3 plan).** Preliminary audit of `Settings`; shapes drive the edgezero #305 requirements: +**Secret inventory (spec artifact — verify + extend during the Phase 3 plan).** Preliminary audit of `Settings`; shapes drive the edgezero #306 requirements: | Secret | Path | Shape | Notes | |---|---|---|---| -| Partner API tokens | `ec.partners[].api_token` | **array element** | needs `ArrayEach` (edgezero #305) | +| Partner API tokens | `ec.partners[].api_token` | **array element** | needs `ArrayEach` (edgezero #306) | | Handler passwords | `handlers[].password` | **array element** | needs `ArrayEach` | | EC passphrase | `ec.passphrase` | scalar `String` | nested | -| Pull token | `ts_pull_token` | **`Option`** | needs optional-secret support (edgezero #305) | +| Pull token | `ts_pull_token` | **`Option`** | needs optional-secret support (edgezero #306) | | Publisher proxy secret | `publisher.proxy_secret` | scalar `String` | nested | | DataDome server-side key | `integrations.datadome.*` (store-ref name+key) | store-ref | already resolves via secret-store name+key | | S3 / proxy secret access key | `proxy.secret_access_key` (+ `proxy.secret_store`) | store-ref | already store-backed | | Request-signing keys | `request_signing.*` (`secret_store_id`) | store-ref | already store-backed | -Two consequences: (1) edgezero #305 **must** ship `ArrayEach` + `Option` (see Phase 0 note); (2) the already-store-backed secrets (DataDome, S3, request-signing) need only re-expression as `#[secret(store_ref)]`, not relocation. +Two consequences: (1) edgezero #306 **must** ship `ArrayEach` + `Option` (see Phase 0 note); (2) the already-store-backed secrets (DataDome, S3, request-signing) need only re-expression as `#[secret(store_ref)]`, not relocation. **Deletions:** inline secrets in the blob, `Redacted`, `SECRET_FIELDS = &[]` wrapper. **Acceptance:** pushed blob contains only secret **key names**; boot resolves them; a config with nested **and array** secrets validates and serves; operator migration guide published; tests green. @@ -276,7 +276,7 @@ Two consequences: (1) edgezero #305 **must** ship `ArrayEach` + `Option` | ID | Question | Owner / resolution | |----|----------|--------------------| -| R1 | Do any `Settings` secrets live inside **arrays**? | **Resolved: yes** (`ec.partners[].api_token`, `handlers[].password`) + optional (`ts_pull_token`). edgezero #305 must ship `ArrayEach` + `Option` (see §5 Phase 3 inventory + Phase 0 note). | +| R1 | Do any `Settings` secrets live inside **arrays**? | **Resolved: yes** (`ec.partners[].api_token`, `handlers[].password`) + optional (`ts_pull_token`). edgezero #306 must ship `ArrayEach` + `Option` (see §5 Phase 3 inventory + Phase 0 note). | | R7 | P0-C: header-preserving Fastly `run_app` dispatch + pre-dispatch hook — **REQUIRED** under full convergence (no permanent-exception fallback). | New edgezero PR; gates Phase 4 Fastly convergence. | | R13 | P0-D: macro/`run_app` app-state injection (`with_state` is builder-only). | New edgezero PR; gates `State` in `#[action]` handlers. | | R14 | Sequencing: land P0-C/P0-D early (skip Phase 1 throwaway adapter builders) vs proceed with interim builders? | Decide now (§3 sequencing note) — affects Phase 1 Tasks 5–6 scope. | From dcd7038bf7ac1e736f9445714e3194b41aef1331 Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Sat, 4 Jul 2026 14:13:10 -0700 Subject: [PATCH 22/30] Amend spec/plan per review 11 (Phase 1/5 boundary + drift/KV/manifest gaps) - BLOCKER: Task 8 made Fastly read impls write-only, but legacy_main (live until Phase 5) reads through FastlyPlatformConfigStore via build_runtime_services. Fix: only Axum/CF/Spin go write-only in Phase 1; Fastly read impls stay until Phase 5 deletes legacy_main (spec Phase 1/5 boundary + ledger updated) - stores() drift: add Step 5b test asserting stores_metadata() and each adapter's Hooks::stores() equal edgezero.toml (registries build from stores(), not the toml that Step 1 validates) - KV multi-id: CF/Spin build RuntimeServices from kv_store_default() only, so non-default KV ids (consent_store/ec_store) don't resolve; Step 2c decides (registry-backed KV selector vs documented defer + narrowed acceptance) - cloudflare.toml still uses legacy [stores.*].name schema the pinned manifest rejects; Task 2 reconciles/deletes it - Add Cloudflare/Spin non-default config/secret tests (Step 2b) --- ...07-02-edgezero-store-registry-migration.md | 29 ++++++++++++++----- ...26-07-02-edgezero-full-migration-design.md | 5 ++-- 2 files changed, 25 insertions(+), 9 deletions(-) diff --git a/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md b/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md index 5760d14c..4f13e131 100644 --- a/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md +++ b/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md @@ -146,10 +146,13 @@ Apply the **D5 renames**: set `settings_data.rs::DEFAULT_CONFIG_STORE_ID = "trus - [ ] **Step 4: Declare every id in `edgezero.toml`** — `[stores.kv]` = `trusted_server_kv`, `ec_identity_store`, `consent_store`, `creative_store`; `[stores.config]` = `trusted_server_config`, `jwks_store`, `datadome-ip-bypass`; `[stores.secrets]` = `trusted_server_secrets`, `signing_keys`, `ts_secrets`, `s3-auth`. (Names double as the platform store names under D7.) Also set `config_payload.rs::CONFIG_BLOB_KEY = "trusted_server_config"` (blob key == store id, per the D5 rule) so `ts config push`'s default key and the boot read agree with no env/`--key`. -- [ ] **Step 5: Wire `Hooks::stores()` on all four adapters (Blocker — metadata is not wired today).** Each `impl Hooks for TrustedServerApp` currently overrides only `routes()`; the default `stores()` returns **empty** `StoresMetadata`, so no registries can be built from it. Add `fn stores() -> StoresMetadata` returning the `[stores.*]` metadata, generated once from `edgezero.toml`. Prefer a single shared `const`/fn in `trusted-server-core` (e.g. `pub fn stores_metadata() -> StoresMetadata`) that all four adapters return, so the ids live in one place. Verify against `edgezero_core::app::StoresMetadata`/`StoreMetadata` shape. +- [ ] **Step 5: Wire `Hooks::stores()` on all four adapters (Blocker — metadata is not wired today).** Each `impl Hooks for TrustedServerApp` currently overrides only `routes()`; the default `stores()` returns **empty** `StoresMetadata`, so no registries can be built from it. Add `fn stores() -> StoresMetadata` returning the `[stores.*]` metadata, generated once from `edgezero.toml`. Prefer a single shared fn in `trusted-server-core` (`pub fn stores_metadata() -> StoresMetadata`) that all four adapters return, so the ids live in one place. Verify against `edgezero_core::app::StoresMetadata`/`StoreMetadata` shape. + +- [ ] **Step 5b: Anti-drift test — `stores_metadata()` and every adapter's `Hooks::stores()` must equal `edgezero.toml`.** Registries are built from `TrustedServerApp::stores()`, **not** from the `edgezero.toml` that Step 1's test validates — so a stale/incomplete `stores_metadata()` would pass Step 1 while runtime registries silently miss ids. Add a test that parses `edgezero.toml`'s `[stores.*]` ids/default and asserts they equal `trusted_server_core::stores_metadata()` **and** each `::TrustedServerApp::stores()` (per kind, ids as sets + default). Put the core half in `trusted-server-core` and one assertion in each adapter's test module (so a future adapter that forgets to return `stores_metadata()` fails). - [ ] **Step 6: Declare the stores in every PLATFORM manifest (Blocker — local resources missing), per each adapter's real mapping.** D7 requires each logical id to be openable as a real platform store. The adapters map kinds to concrete resources differently — declare exactly: - **Fastly** (`fastly.toml`): KV ids → `[[local_server.kv_stores.]]`; config ids → `[local_server.config_stores.]`; secret ids → `[local_server.secret_stores.]`. Also add the production-service store bindings for each id. + - **Cloudflare manifest `cloudflare.toml` — reconcile the STALE schema (Medium blocker).** `crates/trusted-server-adapter-cloudflare/cloudflare.toml` still uses the pre-rewrite manifest schema (`[stores.kv].name = …`, `[stores.kv.adapters.cloudflare].name = …`), which the pinned EdgeZero manifest parser (`manifest.rs`, `deny_unknown_fields`) **rejects** in favor of `[stores.*]` `ids`/`default`. Either migrate it to the `ids`/`default` schema (matching `edgezero.toml`) or **delete it if `edgezero.toml` is the single source** and nothing loads `cloudflare.toml`. Do not leave a stale-schema manifest that a tool/test could load. - **Cloudflare** (`wrangler.toml`): EdgeZero backs **config stores by a KV namespace binding** (`config_store.rs`) — so each **config** id (`trusted_server_config`, `jwks_store`, `datadome-ip-bypass`) gets a `[[kv_namespaces]]` binding (as does each KV id). **Secrets use a FLAT namespace — `CloudflareSecretStore::get_bytes` ignores `store_name` and reads `env.secret(key)`.** So do **not** `wrangler secret put signing_keys` (that provisions the wrong name). Provision the concrete secret **keys the code reads**: the signing KIDs written by `KeyRotationManager`, the DataDome `server_side_key_secret_name`, and the S3 `access_key_id` / `secret_access_key` / optional session-token keys. Document the exact `wrangler secret put ` commands in the operator runbook; `store_name`/store-id is irrelevant on Cloudflare. - **Spin** (`spin.toml`): config **and** KV ids open **KV-store labels** (`request.rs:282`) — declare each under the component's `key_value_stores = [...]`. Secrets are likewise a **flat** namespace (`SpinSecretStore` ignores `store_name`) mapped to Spin variables — provision the concrete secret **keys** (as for Cloudflare), lowercased per Spin's variable rules, not the store id. @@ -172,6 +175,7 @@ Expected: PASS. ```bash git add edgezero.toml fastly.toml trusted-server.example.toml \ crates/trusted-server-adapter-cloudflare/wrangler.toml \ + crates/trusted-server-adapter-cloudflare/cloudflare.toml \ crates/trusted-server-adapter-spin/spin.toml \ crates/trusted-server-integration-tests/fixtures/configs/trusted-server.integration.toml \ crates/trusted-server-integration-tests/src/bin/generate-viceroy-config.rs \ @@ -363,10 +367,16 @@ Expected: FAIL (all three). - [ ] **Step 2: Build `RuntimeServices` via the composite** in each adapter's `build_runtime_services(ctx: &RequestContext)`. **Extract the whole registry from request extensions** — `ctx.request().extensions().get::().cloned()` / `get::()` — the same way EdgeZero's `Config`/`Secrets` extractors do. Do **not** use `ctx.config_store_default()`/`config_store(id)` (those return a single bound handle and would wire only the default store). Pass the cloned registry as the composite reader (Task 3) and the per-adapter **write-only** impl (`PlatformConfigWriter`/`PlatformSecretWriter`) as the writer. If a registry is absent from extensions, that is a wiring bug (Step 0 / EdgeZero dispatch) — surface it, don't silently fall back. -- [ ] **Step 3: Run to verify pass** (all three) +- [ ] **Step 2b: Non-default coverage on Cloudflare AND Spin (not just Axum).** Cloudflare/Spin platform mappings differ (config = KV-namespace/label backed; secrets = flat namespace), so default-only assertions are insufficient. Add, in each of the Cloudflare and Spin test modules, tests proving **non-default** resolution: a `jwks_store` **config** read and a `ts_secrets` / S3 **secret**-key read resolve through the composite (route tests if cheap, else small `build_runtime_services` + composite-read tests seeding a 2-id registry). + +- [ ] **Step 2c: Decide KV multi-id handling (Cloudflare/Spin gap).** Cloudflare (`platform.rs:568`) and Spin (`platform.rs:725`) build `RuntimeServices` from `ctx.kv_store_default()` **only** — a non-default KV id (`consent.consent_store`, `ec.ec_store`, `creative_store`) will **not** resolve there, though Fastly handles it via special store reopening (`app.rs:205`). Config/secret tests would pass while KV stays default-only. Choose one and record it: + - **(Recommended) Resolve named KV via the registry:** have `build_runtime_services` on Cloudflare/Spin (and Fastly's per-request services) obtain the `KvRegistry` from extensions and expose `kv_store(id)` through it (a KV analogue of the config/secret composite), so `consent_store`/`ec_store` resolve everywhere. Add a non-default-KV test per adapter. + - **(Defer)** Explicitly scope Phase 1 to config/secret multi-id + KV-**default** only; document that non-default KV on Cloudflare/Spin is unresolved (Fastly-only today) and **narrow Phase 1 acceptance accordingly**, tracking KV multi-id as a Phase 1 follow-up. Do not leave it as a silent gap. + +- [ ] **Step 3: Run to verify pass** (all three, incl. the non-default tests from 2b) Run: `cargo test-axum && cargo test-cloudflare && cargo test-spin` -Expected: PASS. (Cloudflare/Spin reuse the same composite; their route tests assert the default-id read at minimum.) +Expected: PASS. - [ ] **Step 4: Commit** @@ -443,15 +453,20 @@ git commit -m "Delete duplicated Fastly config-chunk resolver; rely on EdgeZero --- -## Task 8: Retire per-adapter config/secret READ impls; keep the write path (D6-a) +## Task 8: Retire per-adapter config/secret READ impls — EXCEPT Fastly's, which the live legacy path still needs (D6-a) + +Now that all reads (boot + request) flow through EdgeZero **on the edgezero path**, convert the per-adapter management impls to **write-only** (`PlatformConfigWriter`/`PlatformSecretWriter` from Task 3 Step 0) + `management_api.rs` (D6-a). -Now that all reads (boot + request, all adapters) flow through EdgeZero, delete the config/secret **read** implementations. The per-adapter management impls become **write-only** (`PlatformConfigWriter`/`PlatformSecretWriter` from Task 3 Step 0) + `management_api.rs` (D6-a). Update the legacy `route_tests.rs` stubs that construct `RuntimeServices` from bespoke read stores. +**⚠️ Phase 1 / Phase 5 boundary (Blocker fix):** Fastly's `legacy_main` (`adapter-fastly/src/main.rs:726`) is **still live** until Phase 5 (gated on 100% rollout, issue #495). It builds `RuntimeServices` via `build_runtime_services` (`adapter-fastly/src/platform.rs:578`), which wires `FastlyPlatformConfigStore` / `FastlyPlatformSecretStore` **for reads**. So **Fastly's read impls must NOT become write-only in Phase 1** — doing so breaks (or fails to compile) the legacy path before it is deleted. Therefore: +- **Axum / Cloudflare / Spin** read impls → **write-only** now (they have no legacy path). +- **Fastly** `FastlyPlatformConfigStore`/`FastlyPlatformSecretStore` stay **read+write** (full `PlatformConfigStore`/`PlatformSecretStore`) until Phase 5. The edgezero path on Fastly reads via the composite (Task 3–6); `legacy_main` reads via the direct impl. Both coexist. Fastly's read impls are deleted / narrowed to write-only in **Phase 5**, together with `legacy_main`. **Files:** -- Modify: `crates/trusted-server-adapter-{fastly,axum,cloudflare,spin}/src/platform.rs` +- Modify: `crates/trusted-server-adapter-{axum,cloudflare,spin}/src/platform.rs` (→ write-only) +- Leave: `crates/trusted-server-adapter-fastly/src/platform.rs` config/secret **read** impls in place (write-only conversion deferred to Phase 5) - Modify: `crates/trusted-server-adapter-fastly/src/route_tests.rs` (update stubs to the composite/registry shape) -- [ ] **Step 1: Convert per-adapter config/secret impls to write-only.** The old impls implemented the read+write `PlatformConfigStore`/`PlatformSecretStore` (`FastlyPlatformConfigStore` with `get`+`put`+`delete`, `AxumPlatformConfigStore`, `NoopConfigStore`, Cloudflare/Spin equivalents, secret impls). Now that the composite serves reads, **re-implement them as `PlatformConfigWriter`/`PlatformSecretWriter`** (drop `get`/`get_bytes`; keep `put`/`create`/`delete`). This compiles only because Task 3 Step 0 split the traits — otherwise deleting `get` from a `PlatformConfigStore` impl is a trait-incompleteness error. Keep `management_api.rs`. +- [ ] **Step 1: Convert the NON-Fastly config/secret impls to write-only.** For **Axum / Cloudflare / Spin**, re-implement the old read+write impls (`AxumPlatformConfigStore`, `NoopConfigStore`, Cloudflare/Spin equivalents, secret impls) as `PlatformConfigWriter`/`PlatformSecretWriter` (drop `get`/`get_bytes`; keep `put`/`create`/`delete`). This compiles only because Task 3 Step 0 split the traits. **Leave `FastlyPlatformConfigStore`/`FastlyPlatformSecretStore` as full read+write** — `legacy_main` still reads through them until Phase 5 (see the boundary note above). Keep `management_api.rs`. - [ ] **Step 2: Update `route_tests.rs`** — the stub stores (`StubJwksConfigStore`, etc.) and `RuntimeServices` construction move to the composite/registry shape: build the composite reader from a real `ConfigRegistry`/`SecretRegistry` with **at least two ids** (default + a non-default such as `jwks_store`/`ts_secrets`), and assert an **unknown store id resolves strictly to an error** (not a silent fallback to default). Writer = a recording stub; keep coverage of the write path (`put`/`create`/`delete`) so key-rotation delegation stays tested. diff --git a/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md b/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md index 417067ce..5b0e3a78 100644 --- a/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md +++ b/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md @@ -164,7 +164,7 @@ Recommendation: **(a)** where the adapter env is available at boot (Cloudflare/S - Delete the 4× per-adapter `platform.rs` config/secret **read** impls; adapters build registries from `[stores.*]` metadata (via `dispatch_with_registries` on Axum/Cloudflare/Spin, via the Fastly-specific injection above). - Delete `settings_data.rs`'s `FastlyChunkPointer` resolver — EdgeZero's `FastlyConfigStore` resolves chunks transparently. `get_settings_from_config_store` collapses to `ConfigStore::get` + `settings_from_config_blob`. -**Deletions (after D6/D5 resolved):** `settings_data.rs` chunk resolver, `platform/traits.rs` config/secret **read** traits, 4× `platform.rs` config/secret read impls. **`management_api.rs` deletion is conditional on D6** (may move to CLI/ops instead, or stay as a runtime write path). +**Deletions (after D6/D5 resolved):** `settings_data.rs` chunk resolver, `platform/traits.rs` config/secret **read** traits, and the **Axum/Cloudflare/Spin** `platform.rs` config/secret read impls (→ write-only). **Fastly's read impls stay** until Phase 5 — `legacy_main` reads through `FastlyPlatformConfigStore`/`FastlyPlatformSecretStore` via `build_runtime_services` and is live until the 100%-rollout cutover, so converting them to write-only in Phase 1 would break the legacy path (Phase 1/Phase 5 boundary). **`management_api.rs` deletion is conditional on D6.** **Keeps:** `RuntimeServices` as a shrinking bundle (removed in Phase 4); the runtime write path until D6 resolves it; `StoreName`/`StoreId` where writes/provisioning need the management-id split. **Acceptance:** all adapters build; `cargo test-fastly/-axum/-cloudflare/-spin` + parity green; secret/config **reads** go through EdgeZero registries; **key rotation/delete still works** (per the D6 resolution); every declared store id resolves (no strict-lookup `None`). @@ -245,6 +245,7 @@ Two consequences: (1) edgezero #306 **must** ship `ArrayEach` + `Option` - Delete `legacy_main` / `route_request` (`adapter-fastly/src/main.rs`), `compat.rs` (fastly↔http shim), and the flag machinery (`edgezero_enabled`, `edgezero_rollout_pct`, `select_edgezero_entrypoint`, `should_route_to_edgezero`, IP-bucket hashing). - `main()` calls the EdgeZero path unconditionally — the P0-C dispatch variant, or the documented Fastly dispatch shim (§4a), depending on how P0-C resolves. - Retire the `trusted_server_config` rollout-flag reads (the flags, not the config store — after D5 the store may still hold app config). +- **Now convert Fastly's `FastlyPlatformConfigStore`/`FastlyPlatformSecretStore` read impls to write-only** (the conversion deferred from Phase 1 Task 8) — safe here because `legacy_main`/`build_runtime_services`'s direct reads are deleted with the legacy path. - **Ancillary cleanup (easy to miss):** Fastly route tests importing legacy stores + `route_request` (`adapter-fastly/src/route_tests.rs`); generated Viceroy config rollout flags (`integration-tests/src/bin/generate-viceroy-config.rs`); `fastly.toml` local `edgezero_enabled`/`edgezero_rollout_pct` config; and the rollout runbook `docs/internal/EDGEZERO_MIGRATION.md`. **Deletions:** `legacy_main`, `route_request`, `compat.rs`, rollout flags, `route_tests.rs` legacy imports, viceroy-config flags, `fastly.toml` flag config, `EDGEZERO_MIGRATION.md` runbook. @@ -258,7 +259,7 @@ Two consequences: (1) edgezero #306 **must** ship `ArrayEach` + `Option` |---|---|---|---| | Fastly chunk-pointer resolver | `core/src/settings_data.rs` | 1 | EdgeZero `FastlyConfigStore` + `chunked_config.rs` | | Bespoke config/secret store traits | `core/src/platform/traits.rs` (config+secret trait defs); `mod.rs`/`types.rs` edited, not deleted (KV re-export + shrinking `RuntimeServices` stay) | 1 | EdgeZero `ConfigStore`/`SecretStore`/`StoreRegistry` | -| 4× per-adapter store impls | `adapter-*/src/platform.rs` | 1 | per-adapter EdgeZero store impls | +| Per-adapter config/secret read impls → write-only | `adapter-{axum,cloudflare,spin}/src/platform.rs` **Phase 1**; `adapter-fastly/src/platform.rs` **Phase 5** (Fastly reads kept for `legacy_main` until cutover) | 1 / 5 | EdgeZero registries + composite for reads | | Fastly management REST client (**runtime writes**) | `adapter-fastly/src/management_api.rs` | **conditional (D6)** — 1 only if key rotation moves to ops/CLI; otherwise retained as the admin write path | EdgeZero provisioning (if writes leave runtime) — else no replacement | | Adapter/runtime app-config baking | `adapter-{cloudflare,spin}/src/app.rs` (`include_str!` + Cloudflare `TRUSTED_SERVER_CONFIG` side-channel) | 2 | boot-time store-loaded config (P-BOOT). *`ts config init` template embed is out of scope.* | | Legacy env overlay + `config` dep | `core/src/settings.rs` (`from_toml_and_env`, `ENVIRONMENT_VARIABLE_*`) | 2 | `EDGEZERO__*` / AppConfig env layers | From 52487b525017a5146d44e37e55ea32a6abac4220 Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Sat, 4 Jul 2026 15:25:14 -0700 Subject: [PATCH 23/30] Amend spec/plan per review 12 (final cleanup) - KV multi-id DECIDED: registry-backed KV now (not defer). All four adapters resolve named KV via KvRegistry from extensions; remove Fastly consent-store special-casing; per-adapter non-default-KV test. Spec acceptance updated. - Spec: fix stale Phase 1 'delete 4x read impls' bullet to match the corrected Fastly-stays-until-Phase-5 deletion note (no self-contradiction) - Spec: remove duplicate P-BOOT paragraph (kept the full-convergence note) - app_config rename: add tests/environments/cloudflare.rs literals + wrangler.toml placeholder JSON/comment to Task 2 Step 6b + git add --- .../2026-07-02-edgezero-store-registry-migration.md | 9 +++++---- .../specs/2026-07-02-edgezero-full-migration-design.md | 9 ++------- 2 files changed, 7 insertions(+), 11 deletions(-) diff --git a/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md b/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md index 4f13e131..f745a652 100644 --- a/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md +++ b/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md @@ -160,7 +160,9 @@ Apply the **D5 renames**: set `settings_data.rs::DEFAULT_CONFIG_STORE_ID = "trus - **`generate-viceroy-config.rs` — MERGE, don't duplicate.** The generator already emits `[local_server.config_stores.trusted_server_config]` holding the **rollout flags** (`edgezero_enabled`, `edgezero_rollout_pct`), separate from the `app_config` store holding the envelope blob. After the rename both live in ONE store `trusted_server_config`, so **merge the envelope entry into the existing `trusted_server_config.contents` table under key `trusted_server_config`** (alongside the flag keys) — do **not** emit a second `[local_server.config_stores.trusted_server_config]` block (duplicate table). Update the generator's assertion test accordingly. - **`tests/common/config.rs`:** `{"app_config": envelope}` → `{"trusted_server_config": envelope}`. - **`tests/environments/axum.rs`:** rename the `TRUSTED_SERVER_CONFIG_APP_CONFIG_APP_CONFIG` env var to the `trusted_server_config`-keyed name the Axum config store expects. - - **`crates/trusted-server-adapter-cloudflare/src/app.rs` (`settings_from_cloudflare_config_json`):** it reads the literal `value.get("app_config")` from the `TRUSTED_SERVER_CONFIG` side-channel (Cloudflare stays on the side-channel until Phase 2). Change this literal to `CONFIG_BLOB_KEY` (now `"trusted_server_config"`) so Cloudflare boot doesn't break under the rename. (This is a key-string update only; the Phase 2 store migration is separate.) + - **`crates/trusted-server-adapter-cloudflare/src/app.rs` (`settings_from_cloudflare_config_json`):** it reads the literal `value.get("app_config")` from the `TRUSTED_SERVER_CONFIG` side-channel (Cloudflare stays on the side-channel until Phase 2). Change this literal to `CONFIG_BLOB_KEY` (now `"trusted_server_config"`) so Cloudflare boot doesn't break under the rename. (Key-string update only; the Phase 2 store migration is separate.) + - **`crates/trusted-server-integration-tests/tests/environments/cloudflare.rs`:** the `inject_cloudflare_config` tests assert against `{"app_config":"blob"}` / `TRUSTED_SERVER_CONFIG = '''{"app_config":…}'''` literals (lines ~206/210/221/232) — update to the `trusted_server_config` key. + - **`crates/trusted-server-adapter-cloudflare/wrangler.toml`:** the placeholder `TRUSTED_SERVER_CONFIG = '{"app_config":""}'` (line ~23) + its `app_config` comment (line ~21) — update to `trusted_server_config`. - **Axum**: dev-only local files `.edgezero/local-config-.json` (config) and the redb KV default; document how to seed them, do not commit machine-local state. Cross-check each existing manifest — some ids (`jwks_store`, `signing_keys`) are already partially declared (`request_signing/mod.rs` doc references `fastly.toml`); add only the missing ones. @@ -181,6 +183,7 @@ git add edgezero.toml fastly.toml trusted-server.example.toml \ crates/trusted-server-integration-tests/src/bin/generate-viceroy-config.rs \ crates/trusted-server-integration-tests/tests/common/config.rs \ crates/trusted-server-integration-tests/tests/environments/axum.rs \ + crates/trusted-server-integration-tests/tests/environments/cloudflare.rs \ crates/trusted-server-adapter-cloudflare/src/app.rs \ crates/trusted-server-core/src/settings.rs \ crates/trusted-server-core/src/settings_data.rs \ @@ -369,9 +372,7 @@ Expected: FAIL (all three). - [ ] **Step 2b: Non-default coverage on Cloudflare AND Spin (not just Axum).** Cloudflare/Spin platform mappings differ (config = KV-namespace/label backed; secrets = flat namespace), so default-only assertions are insufficient. Add, in each of the Cloudflare and Spin test modules, tests proving **non-default** resolution: a `jwks_store` **config** read and a `ts_secrets` / S3 **secret**-key read resolve through the composite (route tests if cheap, else small `build_runtime_services` + composite-read tests seeding a 2-id registry). -- [ ] **Step 2c: Decide KV multi-id handling (Cloudflare/Spin gap).** Cloudflare (`platform.rs:568`) and Spin (`platform.rs:725`) build `RuntimeServices` from `ctx.kv_store_default()` **only** — a non-default KV id (`consent.consent_store`, `ec.ec_store`, `creative_store`) will **not** resolve there, though Fastly handles it via special store reopening (`app.rs:205`). Config/secret tests would pass while KV stays default-only. Choose one and record it: - - **(Recommended) Resolve named KV via the registry:** have `build_runtime_services` on Cloudflare/Spin (and Fastly's per-request services) obtain the `KvRegistry` from extensions and expose `kv_store(id)` through it (a KV analogue of the config/secret composite), so `consent_store`/`ec_store` resolve everywhere. Add a non-default-KV test per adapter. - - **(Defer)** Explicitly scope Phase 1 to config/secret multi-id + KV-**default** only; document that non-default KV on Cloudflare/Spin is unresolved (Fastly-only today) and **narrow Phase 1 acceptance accordingly**, tracking KV multi-id as a Phase 1 follow-up. Do not leave it as a silent gap. +- [ ] **Step 2c: Resolve named KV via the registry (DECIDED — registry-backed KV now).** Cloudflare (`platform.rs:568`) and Spin (`platform.rs:725`) build `RuntimeServices` from `ctx.kv_store_default()` **only** — a non-default KV id (`consent.consent_store`, `ec.ec_store`, `creative_store`) will **not** resolve there, though Fastly handles it via special store reopening (`app.rs:205`). Config/secret tests would otherwise pass while KV stays default-only. Per full convergence + D5 ("every declared id resolves"), Phase 1 **resolves named KV through the registry**, not default-only: have `build_runtime_services` on **all four** adapters obtain the `KvRegistry` from request extensions (`ctx.request().extensions().get::()`) and expose `kv_store(id)` through it — a KV analogue of the config/secret composite (KV needs no writer split; `KvRegistry::named(id)` returns a `KvHandle` directly). Migrate the Fastly special-case consent-store reopening (`app.rs:205`) onto this registry path too. Add a **non-default KV** resolution test per adapter (e.g. `consent_store` resolves and is distinct from the default). *(This makes Fastly's `runtime_services_for_consent_route` special-casing redundant — remove it.)* - [ ] **Step 3: Run to verify pass** (all three, incl. the non-default tests from 2b) diff --git a/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md b/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md index 5b0e3a78..8458810e 100644 --- a/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md +++ b/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md @@ -124,11 +124,6 @@ Recommendation: **(a)** where the adapter env is available at boot (Cloudflare/S *(P-BOOT's `build_state()` gets subsumed under full convergence: once `run_app` owns startup, boot config load is `run_app` reading the config store via the registry it builds; the boot reader collapses into option (a)/(b) inside `run_app`, and D1's load-once `Settings` is populated from the P0-D state hook at first use.)* -**P-BOOT — Boot-time store access for startup config + secret load.** Define, per adapter, how `build_state()` obtains a config-store (and secret-store) handle at boot, before request context. Options: -- **(a) Boot-time handle from the adapter environment** — Cloudflare builds a config-store handle from the `env` binding passed to `run_app`; Spin from the host component config; Fastly/Axum open the store eagerly (already do). Requires EdgeZero to expose a boot-time store constructor (or trusted-server constructs it from the adapter's env directly, mirroring today's `TRUSTED_SERVER_CONFIG` side-channel but reading the store instead). -- **(b) Lazy first-request load + cache** — defer the config load to the first request (where the registry exists), cache `Arc` in a `OnceCell`. Keeps D1's load-once semantics but moves the load off the boot path. Trade-off: first request pays the cost and must handle a config-load error as a request error. -Recommendation: **(a)** where the adapter env is available at boot (Cloudflare/Spin both pass it to `run_app`), falling back to **(b)** only if an adapter genuinely cannot construct a boot-time handle. Settle in the Phase 2 plan; this is the load-bearing detail that makes "no baked TOML on Cloudflare/Spin" actually implementable. - --- ## 5. Phases @@ -161,12 +156,12 @@ Recommendation: **(a)** where the adapter env is available at boot (Cloudflare/S **`StoreName` semantics (D7):** `platform/types.rs::StoreName` is documented as an "edge-visible platform name". Under D7 runtime reads resolve through the registry by **logical id** (`registry.named(id)`), so for reads `StoreName` now carries the **logical store id** (== platform name by default). Phase 1 updates the `StoreName` doc and audits read call sites to pass logical ids, not physical platform names. - **Fastly registry injection (ties to P0-C):** Fastly's custom `oneshot` path (§1) currently inserts only a `ConfigStoreHandle`, not registries via `dispatch_with_registries`. EdgeZero's `dispatch_with_registries` and its registry builders are **`pub(crate)`** (verified in the pinned checkout), so trusted-server must build the registries **locally** (from `StoresMetadata` + `EnvConfig` + the EdgeZero Fastly store open primitives) and insert them into extensions before `oneshot` — or an EdgeZero public builder must be added upstream (**R11**). -- Delete the 4× per-adapter `platform.rs` config/secret **read** impls; adapters build registries from `[stores.*]` metadata (via `dispatch_with_registries` on Axum/Cloudflare/Spin, via the Fastly-specific injection above). +- Convert the **Axum/Cloudflare/Spin** `platform.rs` config/secret **read** impls to write-only (Fastly's stay read+write until Phase 5 — see the deletions note below); adapters build registries from `[stores.*]` metadata (via `dispatch_with_registries` on Axum/Cloudflare/Spin, via the Fastly-specific injection above). - Delete `settings_data.rs`'s `FastlyChunkPointer` resolver — EdgeZero's `FastlyConfigStore` resolves chunks transparently. `get_settings_from_config_store` collapses to `ConfigStore::get` + `settings_from_config_blob`. **Deletions (after D6/D5 resolved):** `settings_data.rs` chunk resolver, `platform/traits.rs` config/secret **read** traits, and the **Axum/Cloudflare/Spin** `platform.rs` config/secret read impls (→ write-only). **Fastly's read impls stay** until Phase 5 — `legacy_main` reads through `FastlyPlatformConfigStore`/`FastlyPlatformSecretStore` via `build_runtime_services` and is live until the 100%-rollout cutover, so converting them to write-only in Phase 1 would break the legacy path (Phase 1/Phase 5 boundary). **`management_api.rs` deletion is conditional on D6.** **Keeps:** `RuntimeServices` as a shrinking bundle (removed in Phase 4); the runtime write path until D6 resolves it; `StoreName`/`StoreId` where writes/provisioning need the management-id split. -**Acceptance:** all adapters build; `cargo test-fastly/-axum/-cloudflare/-spin` + parity green; secret/config **reads** go through EdgeZero registries; **key rotation/delete still works** (per the D6 resolution); every declared store id resolves (no strict-lookup `None`). +**Acceptance:** all adapters build; `cargo test-fastly/-axum/-cloudflare/-spin` + parity green; secret/config **reads** go through EdgeZero registries; **named KV ids resolve on all four adapters** via the `KvRegistry` (not default-only — `consent_store`/`ec_store` resolve on Cloudflare/Spin, and Fastly's consent-store special-casing is removed); **key rotation/delete still works** (per the D6 resolution); every declared store id (kv/config/secret) resolves (no strict-lookup `None`); Fastly `legacy_main` still reads (its read impls untouched until Phase 5). --- From d8d099a66da8d588ceb1d5565682a465c12086de Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Sat, 4 Jul 2026 15:47:40 -0700 Subject: [PATCH 24/30] Amend spec/plan per review 13 (KV-registry cross-cutting task) - KV named resolution is a CORE change, not adapter-only: RuntimeServices only has kv_store() single handle and consent (publisher.rs:626) drops the store id via .map(|_| services.kv_store()). Step 2c now adds RuntimeServices::kv_store_named + kv_registry field, updates consent call sites to pass consent_store, removes Fastly special-casing, and lists core files (types.rs, publisher.rs) - Step 2d: test-support helper to build registry-populated RequestContexts + migrate existing direct-context tests (strict registries break them otherwise) - ec_store scope narrowed: Phase 1 only validates it declares/resolves to a KvHandle; NOT EC identity-graph wiring on non-Fastly (separate larger effort) --- ...26-07-02-edgezero-store-registry-migration.md | 16 +++++++++++++--- .../2026-07-02-edgezero-full-migration-design.md | 4 +++- 2 files changed, 16 insertions(+), 4 deletions(-) diff --git a/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md b/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md index f745a652..6c512db3 100644 --- a/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md +++ b/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md @@ -335,8 +335,11 @@ git commit -m "Load boot config via EdgeZero config store on Fastly and Axum" **Files:** - Modify: `crates/trusted-server-adapter-axum/src/main.rs` (keep `AxumDevServer::with_config`, chain registry setters) - Create: `crates/trusted-server-adapter-axum/src/registries.rs` (`build_{config,kv,secret}_registry_axum(&StoresMetadata)`) -- Modify: `crates/trusted-server-adapter-{axum,cloudflare,spin}/src/platform.rs` (`build_runtime_services` → composite) -- Test: `crates/trusted-server-adapter-axum/src/app.rs` route tests (+ cloudflare/spin equivalents) +- **Modify (core KV surface, Step 2c):** `crates/trusted-server-core/src/platform/types.rs` (`RuntimeServices::kv_store_named` + `kv_registry` field/builder), `crates/trusted-server-core/src/publisher.rs` (consent call site passes `consent_store`), + any other id-dropping KV consumers +- Modify: `crates/trusted-server-adapter-{axum,cloudflare,spin}/src/platform.rs` (`build_runtime_services` → composite + `kv_registry` from extensions) +- Modify: `crates/trusted-server-adapter-fastly/src/{platform.rs,app.rs}` (populate `kv_registry`; remove consent-store special-casing) +- Test-support (Step 2d): a shared `test_context_with_registries(...)` helper; migrate existing direct-context tests (`adapter-{axum,cloudflare,spin,fastly}` test modules) +- Test: `crates/trusted-server-adapter-axum/src/app.rs` route tests (+ cloudflare/spin equivalents; non-default KV test per adapter) - [ ] **Step 0: Wire registries into Axum while keeping the custom PORT behavior.** Do **not** call `dev_server::run_app` — it reads bind config only from `EDGEZERO__ADAPTER__HOST/PORT` and would drop trusted-server's `PORT`/`axum.toml` handling (`main.rs:11`, `port_from_env`). Instead keep the current `AxumDevServer::with_config(router, config)` and chain the builder's registry setters (verified present: `AxumDevServer::{with_config_registry, with_kv_registry, with_secret_registry}`): ```rust @@ -372,7 +375,14 @@ Expected: FAIL (all three). - [ ] **Step 2b: Non-default coverage on Cloudflare AND Spin (not just Axum).** Cloudflare/Spin platform mappings differ (config = KV-namespace/label backed; secrets = flat namespace), so default-only assertions are insufficient. Add, in each of the Cloudflare and Spin test modules, tests proving **non-default** resolution: a `jwks_store` **config** read and a `ts_secrets` / S3 **secret**-key read resolve through the composite (route tests if cheap, else small `build_runtime_services` + composite-read tests seeding a 2-id registry). -- [ ] **Step 2c: Resolve named KV via the registry (DECIDED — registry-backed KV now).** Cloudflare (`platform.rs:568`) and Spin (`platform.rs:725`) build `RuntimeServices` from `ctx.kv_store_default()` **only** — a non-default KV id (`consent.consent_store`, `ec.ec_store`, `creative_store`) will **not** resolve there, though Fastly handles it via special store reopening (`app.rs:205`). Config/secret tests would otherwise pass while KV stays default-only. Per full convergence + D5 ("every declared id resolves"), Phase 1 **resolves named KV through the registry**, not default-only: have `build_runtime_services` on **all four** adapters obtain the `KvRegistry` from request extensions (`ctx.request().extensions().get::()`) and expose `kv_store(id)` through it — a KV analogue of the config/secret composite (KV needs no writer split; `KvRegistry::named(id)` returns a `KvHandle` directly). Migrate the Fastly special-case consent-store reopening (`app.rs:205`) onto this registry path too. Add a **non-default KV** resolution test per adapter (e.g. `consent_store` resolves and is distinct from the default). *(This makes Fastly's `runtime_services_for_consent_route` special-casing redundant — remove it.)* +- [ ] **Step 2c: Named-KV resolution — a CORE surface change, not adapter-only (DECIDED — registry-backed KV now).** The core `RuntimeServices` exposes only `kv_store(&self) -> &dyn PlatformKvStore` — a **single** handle — and consumers that have a store id today **drop it**: `publisher.rs:626` does `settings.consent.consent_store.as_deref().map(|_| services.kv_store())`. So adapter-only changes cannot make `consent_store` resolve. This step is cross-cutting: + - **Core (`platform/types.rs`):** add a registry-backed named accessor, e.g. `RuntimeServices::kv_store_named(&self, id: &str) -> Option` resolving from a `KvRegistry` carried on `RuntimeServices` (add a `kv_registry` field + builder setter, populated by adapters from `ctx.request().extensions().get::()`). Keep `kv_store()`/`kv_handle()` as the default for existing consumers. + - **Consent call sites (`publisher.rs`):** replace `.map(|_| services.kv_store())` with `settings.consent.consent_store.as_deref().and_then(|id| services.kv_store_named(id))` — so the configured id actually selects the store. Audit other KV consumers (`storage/kv_store.rs`, `ec/*`) for the same "id dropped" pattern. + - **Adapters (all four `platform.rs`):** populate `RuntimeServices.kv_registry` from extensions in `build_runtime_services`. Remove Fastly's special consent-store reopening (`app.rs:205`, `runtime_services_for_consent_route`) — now redundant. + - **Files:** `crates/trusted-server-core/src/platform/types.rs`, `crates/trusted-server-core/src/publisher.rs` (+ any other id-dropping call sites), `crates/trusted-server-adapter-{fastly,axum,cloudflare,spin}/src/platform.rs`, `crates/trusted-server-adapter-fastly/src/app.rs` (remove consent special-case), adapter test helpers (Step 2d). + - **Test:** per adapter, a non-default KV id (`consent_store`) resolves via `kv_store_named` and is **distinct** from the default handle; an unknown id → `None`. + +- [ ] **Step 2d: Test-support — registry-populated `RequestContext` helper + migrate existing direct-context tests.** Strict registries make a missing registry a wiring bug, but existing adapter tests call `build_runtime_services(&ctx)` / `build_per_request_services(&ctx)` on **hand-built** `RequestContext`s with no registries inserted (e.g. `adapter-axum/src/app.rs:130`, `adapter-cloudflare/src/app.rs:151,314`, `adapter-spin/src/app.rs:440`, `adapter-cloudflare/src/platform.rs:729`). Those will now fail (composite → `registry.named()` → `None`). Add a shared test helper (e.g. `test_context_with_registries(config: &[…], kv: &[…], secrets: &[…]) -> RequestContext`) that inserts `ConfigRegistry`/`KvRegistry`/`SecretRegistry` into the context, and **migrate every existing direct-context test** to use it. Enumerate them during the run (`rg 'build_(runtime|per_request)_services'` in adapter test modules). - [ ] **Step 3: Run to verify pass** (all three, incl. the non-default tests from 2b) diff --git a/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md b/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md index 8458810e..120045ba 100644 --- a/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md +++ b/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md @@ -161,7 +161,9 @@ Recommendation: **(a)** where the adapter env is available at boot (Cloudflare/S **Deletions (after D6/D5 resolved):** `settings_data.rs` chunk resolver, `platform/traits.rs` config/secret **read** traits, and the **Axum/Cloudflare/Spin** `platform.rs` config/secret read impls (→ write-only). **Fastly's read impls stay** until Phase 5 — `legacy_main` reads through `FastlyPlatformConfigStore`/`FastlyPlatformSecretStore` via `build_runtime_services` and is live until the 100%-rollout cutover, so converting them to write-only in Phase 1 would break the legacy path (Phase 1/Phase 5 boundary). **`management_api.rs` deletion is conditional on D6.** **Keeps:** `RuntimeServices` as a shrinking bundle (removed in Phase 4); the runtime write path until D6 resolves it; `StoreName`/`StoreId` where writes/provisioning need the management-id split. -**Acceptance:** all adapters build; `cargo test-fastly/-axum/-cloudflare/-spin` + parity green; secret/config **reads** go through EdgeZero registries; **named KV ids resolve on all four adapters** via the `KvRegistry` (not default-only — `consent_store`/`ec_store` resolve on Cloudflare/Spin, and Fastly's consent-store special-casing is removed); **key rotation/delete still works** (per the D6 resolution); every declared store id (kv/config/secret) resolves (no strict-lookup `None`); Fastly `legacy_main` still reads (its read impls untouched until Phase 5). +**Acceptance:** all adapters build; `cargo test-fastly/-axum/-cloudflare/-spin` + parity green; secret/config **reads** go through EdgeZero registries; **named KV ids resolve as registry handles on all four adapters** via `RuntimeServices::kv_store_named(id)` (not default-only — e.g. `consent_store` selects its store on Cloudflare/Spin, and Fastly's consent-store special-casing is removed); **key rotation/delete still works** (per the D6 resolution); every declared store id (kv/config/secret) is **declared and openable** (a `KvHandle` resolves by id — no strict-lookup `None`); Fastly `legacy_main` still reads (its read impls untouched until Phase 5). + +> **Scope clarification (ec_store):** Phase 1 only validates that `ec.ec_store` (`ec_identity_store`) is **declared and resolves to a `KvHandle`** through the registry. It does **not** wire the EC identity graph (`KvIdentityGraph`/`EcKvStore`) on non-Fastly adapters — those deliberately omit the EC API routes today (`adapter-spin/src/app.rs:115`), and portable EC support is a separate, larger effort out of Phase 1 scope. Registry-level handle resolution ≠ EC graph wiring. --- From 87d408793637a02f5cee6fd14665b936709b42bd Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Sat, 4 Jul 2026 16:12:05 -0700 Subject: [PATCH 25/30] Amend plan per review 14 (KvHandle type decision + Task 5 scope) - KvHandle/consent type path DECIDED: KvRegistry::named yields KvHandle (a wrapper, not a PlatformKvStore impl), and ConsentPipelineInput.kv_store is Option<&dyn PlatformKvStore>. Migrate the consent KV surface to KvHandle: RuntimeServices::kv_handle_named(id) -> Option (mirrors existing kv_handle()); ConsentPipelineInput.kv_store -> Option; consent persistence fns take &KvHandle. Files: consent/mod.rs, storage/kv_store.rs added. - Task 5 Step 3/4 now run all four adapters + wasm checks and commit core + Fastly changes (task touches types.rs/publisher.rs/Fastly), per the all-four-green rule --- ...07-02-edgezero-store-registry-migration.md | 27 ++++++++++++------- ...26-07-02-edgezero-full-migration-design.md | 2 +- 2 files changed, 19 insertions(+), 10 deletions(-) diff --git a/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md b/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md index 6c512db3..8866f17d 100644 --- a/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md +++ b/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md @@ -376,24 +376,33 @@ Expected: FAIL (all three). - [ ] **Step 2b: Non-default coverage on Cloudflare AND Spin (not just Axum).** Cloudflare/Spin platform mappings differ (config = KV-namespace/label backed; secrets = flat namespace), so default-only assertions are insufficient. Add, in each of the Cloudflare and Spin test modules, tests proving **non-default** resolution: a `jwks_store` **config** read and a `ts_secrets` / S3 **secret**-key read resolve through the composite (route tests if cheap, else small `build_runtime_services` + composite-read tests seeding a 2-id registry). - [ ] **Step 2c: Named-KV resolution — a CORE surface change, not adapter-only (DECIDED — registry-backed KV now).** The core `RuntimeServices` exposes only `kv_store(&self) -> &dyn PlatformKvStore` — a **single** handle — and consumers that have a store id today **drop it**: `publisher.rs:626` does `settings.consent.consent_store.as_deref().map(|_| services.kv_store())`. So adapter-only changes cannot make `consent_store` resolve. This step is cross-cutting: - - **Core (`platform/types.rs`):** add a registry-backed named accessor, e.g. `RuntimeServices::kv_store_named(&self, id: &str) -> Option` resolving from a `KvRegistry` carried on `RuntimeServices` (add a `kv_registry` field + builder setter, populated by adapters from `ctx.request().extensions().get::()`). Keep `kv_store()`/`kv_handle()` as the default for existing consumers. - - **Consent call sites (`publisher.rs`):** replace `.map(|_| services.kv_store())` with `settings.consent.consent_store.as_deref().and_then(|id| services.kv_store_named(id))` — so the configured id actually selects the store. Audit other KV consumers (`storage/kv_store.rs`, `ec/*`) for the same "id dropped" pattern. + - **Type decision — resolve named KV as `KvHandle`, and migrate consent onto `KvHandle`.** `KvRegistry::named(id)` yields a `KvHandle` (a wrapper `{ store: Arc }`), **not** a `&dyn PlatformKvStore` — and `ConsentPipelineInput.kv_store` is currently `Option<&dyn PlatformKvStore>` (`consent/mod.rs:89`), so a `KvHandle` does not fit directly. Do **not** wrap; **migrate the consent KV surface to `KvHandle`** (the idiomatic edgezero handle — `RuntimeServices` already exposes `kv_handle()` for the default). Specifically: + - **Core (`platform/types.rs`):** add `RuntimeServices::kv_handle_named(&self, id: &str) -> Option` (mirroring the existing `kv_handle()`), resolving from a `KvRegistry` carried on `RuntimeServices` (add a `kv_registry` field + builder setter, populated by adapters from `ctx.request().extensions().get::()`). + - **Consent (`consent/mod.rs`, `storage/kv_store.rs`, `publisher.rs`):** change `ConsentPipelineInput.kv_store` from `Option<&dyn PlatformKvStore>` to `Option`; update the consent persistence fns (`load_consent_from_kv`/`save_consent_to_kv`/`delete_consent_from_kv`) to take a `&KvHandle` and use its async methods (they already `block_on`). At the call site (`publisher.rs:626`) pass `settings.consent.consent_store.as_deref().and_then(|id| services.kv_handle_named(id))`. + - Audit other KV consumers (`ec/*`) for the same "id dropped" pattern; they already use `kv_handle()` so are lower-risk. - **Adapters (all four `platform.rs`):** populate `RuntimeServices.kv_registry` from extensions in `build_runtime_services`. Remove Fastly's special consent-store reopening (`app.rs:205`, `runtime_services_for_consent_route`) — now redundant. - - **Files:** `crates/trusted-server-core/src/platform/types.rs`, `crates/trusted-server-core/src/publisher.rs` (+ any other id-dropping call sites), `crates/trusted-server-adapter-{fastly,axum,cloudflare,spin}/src/platform.rs`, `crates/trusted-server-adapter-fastly/src/app.rs` (remove consent special-case), adapter test helpers (Step 2d). - - **Test:** per adapter, a non-default KV id (`consent_store`) resolves via `kv_store_named` and is **distinct** from the default handle; an unknown id → `None`. + - **Files:** `crates/trusted-server-core/src/platform/types.rs`, `crates/trusted-server-core/src/consent/mod.rs` (`ConsentPipelineInput.kv_store` type), `crates/trusted-server-core/src/storage/kv_store.rs` (consent persistence fns → `&KvHandle`), `crates/trusted-server-core/src/publisher.rs` (call site), `crates/trusted-server-adapter-{fastly,axum,cloudflare,spin}/src/platform.rs`, `crates/trusted-server-adapter-fastly/src/app.rs` (remove consent special-case), adapter test helpers (Step 2d). + - **Test:** per adapter, a non-default KV id (`consent_store`) resolves via `kv_handle_named` and is **distinct** from the default handle; an unknown id → `None`. - [ ] **Step 2d: Test-support — registry-populated `RequestContext` helper + migrate existing direct-context tests.** Strict registries make a missing registry a wiring bug, but existing adapter tests call `build_runtime_services(&ctx)` / `build_per_request_services(&ctx)` on **hand-built** `RequestContext`s with no registries inserted (e.g. `adapter-axum/src/app.rs:130`, `adapter-cloudflare/src/app.rs:151,314`, `adapter-spin/src/app.rs:440`, `adapter-cloudflare/src/platform.rs:729`). Those will now fail (composite → `registry.named()` → `None`). Add a shared test helper (e.g. `test_context_with_registries(config: &[…], kv: &[…], secrets: &[…]) -> RequestContext`) that inserts `ConfigRegistry`/`KvRegistry`/`SecretRegistry` into the context, and **migrate every existing direct-context test** to use it. Enumerate them during the run (`rg 'build_(runtime|per_request)_services'` in adapter test modules). -- [ ] **Step 3: Run to verify pass** (all three, incl. the non-default tests from 2b) +- [ ] **Step 3: Run to verify pass** — this task touches **core** (`platform/types.rs`, consent, `publisher.rs`) and **Fastly** (`platform.rs`/`app.rs`) as well as Axum/CF/Spin, so run **all four** adapters + wasm checks (per the global "all four green" rule), incl. the non-default config/secret/KV tests from 2b/2c. -Run: `cargo test-axum && cargo test-cloudflare && cargo test-spin` +Run: `cargo test-fastly && cargo test-axum && cargo test-cloudflare && cargo test-spin && cargo check-cloudflare && cargo check-spin` Expected: PASS. -- [ ] **Step 4: Commit** +- [ ] **Step 4: Commit** (include the core + Fastly changes, not just the three adapters) ```bash -git add crates/trusted-server-adapter-axum crates/trusted-server-adapter-cloudflare crates/trusted-server-adapter-spin -git commit -m "Build RuntimeServices via composite store in Axum, Cloudflare, and Spin" +git add crates/trusted-server-core/src/platform/types.rs \ + crates/trusted-server-core/src/consent/mod.rs \ + crates/trusted-server-core/src/storage/kv_store.rs \ + crates/trusted-server-core/src/publisher.rs \ + crates/trusted-server-adapter-fastly \ + crates/trusted-server-adapter-axum \ + crates/trusted-server-adapter-cloudflare \ + crates/trusted-server-adapter-spin +git commit -m "Wire RuntimeServices via composite + registry-backed named KV across all adapters" ``` --- diff --git a/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md b/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md index 120045ba..3c66b6ce 100644 --- a/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md +++ b/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md @@ -161,7 +161,7 @@ Recommendation: **(a)** where the adapter env is available at boot (Cloudflare/S **Deletions (after D6/D5 resolved):** `settings_data.rs` chunk resolver, `platform/traits.rs` config/secret **read** traits, and the **Axum/Cloudflare/Spin** `platform.rs` config/secret read impls (→ write-only). **Fastly's read impls stay** until Phase 5 — `legacy_main` reads through `FastlyPlatformConfigStore`/`FastlyPlatformSecretStore` via `build_runtime_services` and is live until the 100%-rollout cutover, so converting them to write-only in Phase 1 would break the legacy path (Phase 1/Phase 5 boundary). **`management_api.rs` deletion is conditional on D6.** **Keeps:** `RuntimeServices` as a shrinking bundle (removed in Phase 4); the runtime write path until D6 resolves it; `StoreName`/`StoreId` where writes/provisioning need the management-id split. -**Acceptance:** all adapters build; `cargo test-fastly/-axum/-cloudflare/-spin` + parity green; secret/config **reads** go through EdgeZero registries; **named KV ids resolve as registry handles on all four adapters** via `RuntimeServices::kv_store_named(id)` (not default-only — e.g. `consent_store` selects its store on Cloudflare/Spin, and Fastly's consent-store special-casing is removed); **key rotation/delete still works** (per the D6 resolution); every declared store id (kv/config/secret) is **declared and openable** (a `KvHandle` resolves by id — no strict-lookup `None`); Fastly `legacy_main` still reads (its read impls untouched until Phase 5). +**Acceptance:** all adapters build; `cargo test-fastly/-axum/-cloudflare/-spin` + parity green; secret/config **reads** go through EdgeZero registries; **named KV ids resolve as registry handles on all four adapters** via `RuntimeServices::kv_handle_named(id) -> Option` (not default-only — e.g. `consent_store` selects its store on Cloudflare/Spin; the consent KV surface migrates from `&dyn PlatformKvStore` to `KvHandle`, and Fastly's consent-store special-casing is removed); **key rotation/delete still works** (per the D6 resolution); every declared store id (kv/config/secret) is **declared and openable** (a `KvHandle` resolves by id — no strict-lookup `None`); Fastly `legacy_main` still reads (its read impls untouched until Phase 5). > **Scope clarification (ec_store):** Phase 1 only validates that `ec.ec_store` (`ec_identity_store`) is **declared and resolves to a `KvHandle`** through the registry. It does **not** wire the EC identity graph (`KvIdentityGraph`/`EcKvStore`) on non-Fastly adapters — those deliberately omit the EC API routes today (`adapter-spin/src/app.rs:115`), and portable EC support is a separate, larger effort out of Phase 1 scope. Registry-level handle resolution ≠ EC graph wiring. From affc79f5cbe1fc0c5061900deed4ed5e668aef26 Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Sat, 4 Jul 2026 16:12:31 -0700 Subject: [PATCH 26/30] Fix remaining kv_store_named -> kv_handle_named in Task 5 files list --- .../plans/2026-07-02-edgezero-store-registry-migration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md b/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md index 8866f17d..4cf1592c 100644 --- a/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md +++ b/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md @@ -335,7 +335,7 @@ git commit -m "Load boot config via EdgeZero config store on Fastly and Axum" **Files:** - Modify: `crates/trusted-server-adapter-axum/src/main.rs` (keep `AxumDevServer::with_config`, chain registry setters) - Create: `crates/trusted-server-adapter-axum/src/registries.rs` (`build_{config,kv,secret}_registry_axum(&StoresMetadata)`) -- **Modify (core KV surface, Step 2c):** `crates/trusted-server-core/src/platform/types.rs` (`RuntimeServices::kv_store_named` + `kv_registry` field/builder), `crates/trusted-server-core/src/publisher.rs` (consent call site passes `consent_store`), + any other id-dropping KV consumers +- **Modify (core KV surface, Step 2c):** `crates/trusted-server-core/src/platform/types.rs` (`RuntimeServices::kv_handle_named` + `kv_registry` field/builder), `crates/trusted-server-core/src/consent/mod.rs` + `storage/kv_store.rs` (consent KV surface → `KvHandle`), `crates/trusted-server-core/src/publisher.rs` (call site passes `consent_store`) - Modify: `crates/trusted-server-adapter-{axum,cloudflare,spin}/src/platform.rs` (`build_runtime_services` → composite + `kv_registry` from extensions) - Modify: `crates/trusted-server-adapter-fastly/src/{platform.rs,app.rs}` (populate `kv_registry`; remove consent-store special-casing) - Test-support (Step 2d): a shared `test_context_with_registries(...)` helper; migrate existing direct-context tests (`adapter-{axum,cloudflare,spin,fastly}` test modules) From eb787f821ceff5f5ad7ad5b5b556c7864bcee5d9 Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Sat, 4 Jul 2026 20:32:33 -0700 Subject: [PATCH 27/30] Amend spec/plan per review 16 (Fastly KV sequencing + consent behavioral test) - Fastly named-KV sequencing: Task 5 wired kv_registry + removed consent special-casing on Fastly, but Fastly injects registries only in Task 6. Move Fastly named-KV + build_per_request_services composite + runtime_services_for_consent_route removal into Task 6 Step 4b; Task 5 does core + Axum/CF/Spin and leaves Fastly compiling (kv_handle_named returns None) - Behavioral consent test: prove consent_store loads/saves/deletes through the NON-default store and leaves the default untouched (not just handle distinctness) - Convert existing core consent tests (consent/mod.rs kv_store: Some(&store) at ~1450/1481/1492) from &dyn PlatformKvStore to KvHandle - Fix understated KV summaries: spec gap table + plan goal now say default KV is EdgeZero but named/consent-store KV selection is not --- ...26-07-02-edgezero-store-registry-migration.md | 16 +++++++++++----- .../2026-07-02-edgezero-full-migration-design.md | 2 +- 2 files changed, 12 insertions(+), 6 deletions(-) diff --git a/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md b/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md index 4cf1592c..9a032d88 100644 --- a/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md +++ b/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md @@ -2,7 +2,7 @@ > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. -**Goal:** Route trusted-server's runtime **and boot-time** config/secret **reads** through EdgeZero stores/registries (as KV already is), reconcile every logical store id (kv/config/secrets) with `edgezero.toml`, and delete the duplicated Fastly chunk resolver — while **keeping** the runtime **write** path (key rotation) intact via a composite store (decision **D6-a**). +**Goal:** Route trusted-server's runtime **and boot-time** config/secret **reads** through EdgeZero stores/registries, add **named/non-default KV** selection (default KV is already EdgeZero; named KV / `consent_store` selection is **not** — see Step 2c), reconcile every logical store id (kv/config/secrets) with `edgezero.toml`, and delete the duplicated Fastly chunk resolver — while **keeping** the runtime **write** path (key rotation) intact via a composite store (decision **D6-a**). **Architecture:** trusted-server core reads/writes stores through the bespoke `PlatformConfigStore`/`PlatformSecretStore` traits (each mixes read `get`/`get_string` + write `put`/`create`/`delete`), surfaced via `RuntimeServices` (one trait object per kind). EdgeZero's `ConfigStore`/`SecretStore` are **read-only**; per-request `ConfigRegistry`/`SecretRegistry` live in request extensions. This phase introduces a **composite store** whose *reads* resolve from EdgeZero and whose *writes* delegate to the existing management-API-backed impl, migrates the Fastly/Axum **boot** config read to EdgeZero, and adds **local** registry builders for Fastly's custom `oneshot` dispatch (EdgeZero's builders are `pub(crate)`). @@ -337,7 +337,7 @@ git commit -m "Load boot config via EdgeZero config store on Fastly and Axum" - Create: `crates/trusted-server-adapter-axum/src/registries.rs` (`build_{config,kv,secret}_registry_axum(&StoresMetadata)`) - **Modify (core KV surface, Step 2c):** `crates/trusted-server-core/src/platform/types.rs` (`RuntimeServices::kv_handle_named` + `kv_registry` field/builder), `crates/trusted-server-core/src/consent/mod.rs` + `storage/kv_store.rs` (consent KV surface → `KvHandle`), `crates/trusted-server-core/src/publisher.rs` (call site passes `consent_store`) - Modify: `crates/trusted-server-adapter-{axum,cloudflare,spin}/src/platform.rs` (`build_runtime_services` → composite + `kv_registry` from extensions) -- Modify: `crates/trusted-server-adapter-fastly/src/{platform.rs,app.rs}` (populate `kv_registry`; remove consent-store special-casing) +- **Fastly named-KV + composite + `runtime_services_for_consent_route` removal → Task 6** (Fastly injects registries only in Task 6). Task 5 leaves Fastly compiling: the core `kv_handle_named` is additive and returns `None` on Fastly until Task 6 wires `kv_registry` (consent falls back safely). - Test-support (Step 2d): a shared `test_context_with_registries(...)` helper; migrate existing direct-context tests (`adapter-{axum,cloudflare,spin,fastly}` test modules) - Test: `crates/trusted-server-adapter-axum/src/app.rs` route tests (+ cloudflare/spin equivalents; non-default KV test per adapter) @@ -380,12 +380,14 @@ Expected: FAIL (all three). - **Core (`platform/types.rs`):** add `RuntimeServices::kv_handle_named(&self, id: &str) -> Option` (mirroring the existing `kv_handle()`), resolving from a `KvRegistry` carried on `RuntimeServices` (add a `kv_registry` field + builder setter, populated by adapters from `ctx.request().extensions().get::()`). - **Consent (`consent/mod.rs`, `storage/kv_store.rs`, `publisher.rs`):** change `ConsentPipelineInput.kv_store` from `Option<&dyn PlatformKvStore>` to `Option`; update the consent persistence fns (`load_consent_from_kv`/`save_consent_to_kv`/`delete_consent_from_kv`) to take a `&KvHandle` and use its async methods (they already `block_on`). At the call site (`publisher.rs:626`) pass `settings.consent.consent_store.as_deref().and_then(|id| services.kv_handle_named(id))`. - Audit other KV consumers (`ec/*`) for the same "id dropped" pattern; they already use `kv_handle()` so are lower-risk. - - **Adapters (all four `platform.rs`):** populate `RuntimeServices.kv_registry` from extensions in `build_runtime_services`. Remove Fastly's special consent-store reopening (`app.rs:205`, `runtime_services_for_consent_route`) — now redundant. + - **Adapters — Axum/Cloudflare/Spin here; Fastly in Task 6 (sequencing).** Populate `RuntimeServices.kv_registry` from extensions in `build_runtime_services` for **Axum/Cloudflare/Spin** (they inject registries via EdgeZero `run_app`/`dispatch_with_registries`). **Fastly's** named-KV wiring belongs in **Task 6**, because Fastly only injects registries into extensions in Task 6 (its custom `oneshot`); its active per-request services are built in `app.rs:238` (`build_per_request_services`), not `platform.rs`, and its consent special-casing (`app.rs:205`, `runtime_services_for_consent_route`, used at `app.rs:588/735`) is removed **there** once the `kv_registry` is present. Doing Fastly named-KV in Task 5 would populate from a registry not yet in extensions. - **Files:** `crates/trusted-server-core/src/platform/types.rs`, `crates/trusted-server-core/src/consent/mod.rs` (`ConsentPipelineInput.kv_store` type), `crates/trusted-server-core/src/storage/kv_store.rs` (consent persistence fns → `&KvHandle`), `crates/trusted-server-core/src/publisher.rs` (call site), `crates/trusted-server-adapter-{fastly,axum,cloudflare,spin}/src/platform.rs`, `crates/trusted-server-adapter-fastly/src/app.rs` (remove consent special-case), adapter test helpers (Step 2d). - - **Test:** per adapter, a non-default KV id (`consent_store`) resolves via `kv_handle_named` and is **distinct** from the default handle; an unknown id → `None`. + - **Test (behavioral, not just resolution) — write it FAILING first.** The migration exists because consent **drops** the id and hits the default store. So the core test that proves the fix is behavioral: with `settings.consent.consent_store = "consent_store"` and a registry holding a **default** KV + a distinct `consent_store` KV, a consent persistence round-trip (load/save/delete) must read/write the **`consent_store`** handle and leave the **default** store **untouched**. Assert against both stores (the target has the entry; the default does not). Add this in `trusted-server-core` consent tests. Also keep the cheaper per-adapter check: `kv_handle_named("consent_store")` resolves and is distinct from the default; unknown id → `None`. - [ ] **Step 2d: Test-support — registry-populated `RequestContext` helper + migrate existing direct-context tests.** Strict registries make a missing registry a wiring bug, but existing adapter tests call `build_runtime_services(&ctx)` / `build_per_request_services(&ctx)` on **hand-built** `RequestContext`s with no registries inserted (e.g. `adapter-axum/src/app.rs:130`, `adapter-cloudflare/src/app.rs:151,314`, `adapter-spin/src/app.rs:440`, `adapter-cloudflare/src/platform.rs:729`). Those will now fail (composite → `registry.named()` → `None`). Add a shared test helper (e.g. `test_context_with_registries(config: &[…], kv: &[…], secrets: &[…]) -> RequestContext`) that inserts `ConfigRegistry`/`KvRegistry`/`SecretRegistry` into the context, and **migrate every existing direct-context test** to use it. Enumerate them during the run (`rg 'build_(runtime|per_request)_services'` in adapter test modules). +Also convert the **existing core consent tests** that construct `ConsentPipelineInput` with a `&dyn PlatformKvStore` store — `crates/trusted-server-core/src/consent/mod.rs` has `kv_store: Some(&store)` call sites at ~lines 1450 / 1481 / 1492 — to the new `Option` shape (build a `KvHandle` over an in-memory `KvStore` test double). These fail to compile the moment `ConsentPipelineInput.kv_store` changes type, so migrate them in the same step (`rg 'kv_store: Some\(' crates/trusted-server-core/src/consent` to find them all). + - [ ] **Step 3: Run to verify pass** — this task touches **core** (`platform/types.rs`, consent, `publisher.rs`) and **Fastly** (`platform.rs`/`app.rs`) as well as Axum/CF/Spin, so run **all four** adapters + wasm checks (per the global "all four green" rule), incl. the non-default config/secret/KV tests from 2b/2c. Run: `cargo test-fastly && cargo test-axum && cargo test-cloudflare && cargo test-spin && cargo check-cloudflare && cargo check-spin` @@ -434,7 +436,11 @@ Run: `cargo test-fastly build_config_registry_resolves_declared_ids` → Expecte - [ ] **Step 4: Insert registries in the oneshot block** — replace the lone `core_req.extensions_mut().insert(config_store)` at `main.rs:477`: build the three registries via Step 3 (propagate `build_kv_registry`'s `FastlyError` into the dispatch's `Result`), and `if let Some(reg) = ...` insert each into `core_req.extensions_mut()`, preserving the existing `client_info`/`device_signals` inserts. -- [ ] **Step 4b: Build Fastly `RuntimeServices` from the composite (else the injected registries are unused).** Fastly's `build_per_request_services` (`adapter-fastly/src/app.rs:238`) currently does `RuntimeServices::builder().config_store(Arc::new(FastlyPlatformConfigStore))…` — reading directly, ignoring the registries. Change it to extract the registries from extensions (`ctx.request().extensions().get::().cloned()` / `SecretRegistry`) and build `CompositeConfigStore`/`CompositeSecretStore` (reader = registry; writer = the Fastly write-only impl), exactly as Task 5 does for the other adapters. Without this, Steps 1–4 wire registries nothing reads. +- [ ] **Step 4b: Build Fastly `RuntimeServices` from the composite + wire named KV + remove consent special-casing (Fastly half of Step 2c, sequenced here).** Fastly's `build_per_request_services` (`adapter-fastly/src/app.rs:238`) currently does `RuntimeServices::builder().config_store(Arc::new(FastlyPlatformConfigStore))…` — reading directly, ignoring the registries. Now that Step 4 injects `Config`/`Secret`/`Kv` registries into extensions, change it to: + - extract the registries from extensions (`ctx.request().extensions().get::().cloned()` / `SecretRegistry` / `KvRegistry`) and build `CompositeConfigStore`/`CompositeSecretStore` (reader = registry; writer = the Fastly **read+write** impl kept for legacy, per Task 8) — as Task 5 does for the other adapters; + - populate `RuntimeServices.kv_registry` from the `KvRegistry` (Step 2c core surface), so `kv_handle_named("consent_store")` works on Fastly; + - **remove `runtime_services_for_consent_route` (`app.rs:205`) and its call sites (`app.rs:588/735`)** — consent now selects its store via `kv_handle_named`, so the special reopening is redundant. + Without this, Step 4 wires registries nothing reads, and named consent KV stays Fastly-special. - [ ] **Step 5: Write a failing Fastly route test** — `GET /.well-known/trusted-server.json` via the EdgeZero `oneshot` path returns the JWKS doc read through the injected `ConfigRegistry` (built with default + `jwks_store` ids). Name: `oneshot_discovery_reads_jwks_via_registry` (mirror the `StubJwksConfigStore`/`JWKS_CONFIG_STORE_NAME` pattern in `route_tests.rs`, but drive the EdgeZero path, not `route_request`). diff --git a/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md b/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md index 3c66b6ce..c0d8d52a 100644 --- a/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md +++ b/docs/superpowers/specs/2026-07-02-edgezero-full-migration-design.md @@ -34,7 +34,7 @@ Verified across `trusted-server-core`, the four adapters, `trusted-server-cli`, | Concern | Today | Gap to close | |---|---|---| -| **KV** | ✅ 100% on EdgeZero (`KvStore`/`KvHandle`, re-exported as `PlatformKvStore`) | None (baseline for the pattern) | +| **KV** | ⚠️ **Default** KV is on EdgeZero (`KvStore`/`KvHandle`, re-exported as `PlatformKvStore`), but **named/non-default KV is not selected**: `RuntimeServices` exposes only a single default handle and consent drops the configured `consent_store` id (`publisher.rs:626`) | Phase 1 (named-KV surface + consent migration — see D5/Step 2c) | | **Routing** | ✅ All 4 adapters route through EdgeZero `RouterService` + `Hooks` | None structurally; handler authoring changes in Phase 4 | | **Core off `fastly::` types** | ✅ Enforced by `migration_guards.rs` | Keep the guard; extend coverage as adapters shrink | | **Config load** | ⚠️ Fastly + Axum load the blob from the config store; **Cloudflare** reads a `TRUSTED_SERVER_CONFIG` env side-channel (native fallback `include_str!`); **Spin `include_str!` `trusted-server.example.toml`** — none of these is a boot-time config-store read | Phase 2 (P-BOOT) | From 59b504be086bfa943af55ec34d6183725fcd8a20 Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Sat, 4 Jul 2026 21:51:46 -0700 Subject: [PATCH 28/30] Amend plan per review 17 (consent cutover sequencing + secret/consent tests) - Interim Fastly regression: Task 5 flipped consent to kv_handle_named but Fastly has no KvRegistry until Task 6 -> consent would silently skip on Fastly. Fix: Task 5 migrates the consent TYPE but keeps the call site on kv_handle() (default, = Fastly's swapped consent store); the named-lookup FLIP + behavioral test are atomic in Task 6 Step 5b once all four inject a KvRegistry - Absent-registry policy: builders stay infallible (return RuntimeServices); build a composite over an EMPTY registry so reads error, not silently fall back - Task 6 Step 5c: Fastly named-KV/consent route test (guards the special-case removal) - Task 3 Step 1b: explicit CompositeSecretStore test (named id, unknown-id error, create/delete StoreId-preserving delegation) - flat CF/Spin secret namespaces mean route tests don't prove store-id binding - Task 5 hygiene: no longer edits/commits Fastly (moved to Task 6) --- ...07-02-edgezero-store-registry-migration.md | 58 ++++++++++++++----- 1 file changed, 45 insertions(+), 13 deletions(-) diff --git a/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md b/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md index 9a032d88..0d94fb47 100644 --- a/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md +++ b/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md @@ -251,9 +251,38 @@ fn composite_config_reads_named_store_and_writes_delegate() { } ``` -- [ ] **Step 2: Run to verify it fails** +- [ ] **Step 1b: Write the failing SECRET-composite test too** (not just config — Cloudflare/Spin secrets are a **flat** namespace, so route tests won't prove store-id binding; this unit test does). Mirror the config test for `CompositeSecretStore`: +```rust +#[test] +fn composite_secret_reads_named_store_and_writes_delegate() { + // SecretRegistry with default + a non-default `ts_secrets` id. + let reader = secret_registry(&[ + ("trusted_server_secrets", "API_KEY", b"default-key"), + ("ts_secrets", "server-side-key", b"dd-secret"), + ], "trusted_server_secrets"); + let writer = Arc::new(RecordingSecretWriter::default()); + let composite = CompositeSecretStore::new(reader, writer.clone()); + // Non-default store resolves. + let v = composite.get_bytes(&StoreName::from("ts_secrets"), "server-side-key").expect("read"); + assert_eq!(v, b"dd-secret"); + // Unknown store id is a strict error. + assert!(matches!( + composite.get_bytes(&StoreName::from("nope"), "x").expect_err("should error on unknown secret store").current_context(), + PlatformError::SecretStore, + )); + // create/delete delegate with the target StoreId preserved. + composite.create(&StoreId::from("ts_secrets"), "new", "val").expect("create delegates"); + assert_eq!( + writer.creates.lock().expect("should acquire writer lock").as_slice(), + &[("ts_secrets".to_owned(), "new".to_owned(), "val".to_owned())], + "create must delegate with the SAME StoreId", + ); +} +``` -Run: `cargo test-fastly composite_config_reads_named_store_and_writes_delegate` +- [ ] **Step 2: Run to verify both fail** + +Run: `cargo test-fastly composite_config_reads_named_store_and_writes_delegate composite_secret_reads_named_store_and_writes_delegate` Expected: FAIL (module does not exist). - [ ] **Step 3: Implement `composite.rs`** @@ -371,40 +400,39 @@ cargo test-axum first_party_proxy_reads_s3_secret ``` Expected: FAIL (all three). -- [ ] **Step 2: Build `RuntimeServices` via the composite** in each adapter's `build_runtime_services(ctx: &RequestContext)`. **Extract the whole registry from request extensions** — `ctx.request().extensions().get::().cloned()` / `get::()` — the same way EdgeZero's `Config`/`Secrets` extractors do. Do **not** use `ctx.config_store_default()`/`config_store(id)` (those return a single bound handle and would wire only the default store). Pass the cloned registry as the composite reader (Task 3) and the per-adapter **write-only** impl (`PlatformConfigWriter`/`PlatformSecretWriter`) as the writer. If a registry is absent from extensions, that is a wiring bug (Step 0 / EdgeZero dispatch) — surface it, don't silently fall back. +- [ ] **Step 2: Build `RuntimeServices` via the composite** in each adapter's `build_runtime_services(ctx: &RequestContext)`. **Extract the whole registry from request extensions** — `ctx.request().extensions().get::().cloned()` / `get::()` — the same way EdgeZero's `Config`/`Secrets` extractors do. Do **not** use `ctx.config_store_default()`/`config_store(id)` (those return a single bound handle and would wire only the default store). Pass the cloned registry as the composite reader (Task 3) and the per-adapter **write-only** impl (`PlatformConfigWriter`/`PlatformSecretWriter`) as the writer. **Absent-registry policy (concrete):** `build_runtime_services` returns `RuntimeServices` (not `Result`) on all adapters, so don't add a fallible signature. Instead, when a registry is absent from extensions, build a composite over an **empty** registry (`StoreRegistry` with no ids) — its strict `named()`/`default()` return `None`, so the composite's `get`/`get_bytes` **error** (`PlatformError`) on first read rather than silently reading a default store. A missing registry thus surfaces as a read error at the call site, not a silent fallback, with no builder signature change. - [ ] **Step 2b: Non-default coverage on Cloudflare AND Spin (not just Axum).** Cloudflare/Spin platform mappings differ (config = KV-namespace/label backed; secrets = flat namespace), so default-only assertions are insufficient. Add, in each of the Cloudflare and Spin test modules, tests proving **non-default** resolution: a `jwks_store` **config** read and a `ts_secrets` / S3 **secret**-key read resolve through the composite (route tests if cheap, else small `build_runtime_services` + composite-read tests seeding a 2-id registry). - [ ] **Step 2c: Named-KV resolution — a CORE surface change, not adapter-only (DECIDED — registry-backed KV now).** The core `RuntimeServices` exposes only `kv_store(&self) -> &dyn PlatformKvStore` — a **single** handle — and consumers that have a store id today **drop it**: `publisher.rs:626` does `settings.consent.consent_store.as_deref().map(|_| services.kv_store())`. So adapter-only changes cannot make `consent_store` resolve. This step is cross-cutting: - **Type decision — resolve named KV as `KvHandle`, and migrate consent onto `KvHandle`.** `KvRegistry::named(id)` yields a `KvHandle` (a wrapper `{ store: Arc }`), **not** a `&dyn PlatformKvStore` — and `ConsentPipelineInput.kv_store` is currently `Option<&dyn PlatformKvStore>` (`consent/mod.rs:89`), so a `KvHandle` does not fit directly. Do **not** wrap; **migrate the consent KV surface to `KvHandle`** (the idiomatic edgezero handle — `RuntimeServices` already exposes `kv_handle()` for the default). Specifically: - **Core (`platform/types.rs`):** add `RuntimeServices::kv_handle_named(&self, id: &str) -> Option` (mirroring the existing `kv_handle()`), resolving from a `KvRegistry` carried on `RuntimeServices` (add a `kv_registry` field + builder setter, populated by adapters from `ctx.request().extensions().get::()`). - - **Consent (`consent/mod.rs`, `storage/kv_store.rs`, `publisher.rs`):** change `ConsentPipelineInput.kv_store` from `Option<&dyn PlatformKvStore>` to `Option`; update the consent persistence fns (`load_consent_from_kv`/`save_consent_to_kv`/`delete_consent_from_kv`) to take a `&KvHandle` and use its async methods (they already `block_on`). At the call site (`publisher.rs:626`) pass `settings.consent.consent_store.as_deref().and_then(|id| services.kv_handle_named(id))`. + - **Consent type migration (Task 5) vs behavioral flip (Task 6) — avoid an interim Fastly regression.** Change `ConsentPipelineInput.kv_store` from `Option<&dyn PlatformKvStore>` to `Option` and update the persistence fns (`load_consent_from_kv`/`save_consent_to_kv`/`delete_consent_from_kv`) to take `&KvHandle` (they already `block_on`) — **in Task 5**. But **do NOT flip the `publisher.rs:626` call site to `kv_handle_named` in Task 5**: Fastly has no `KvRegistry` until Task 6, so `kv_handle_named("consent_store")` would return `None` there and consent persistence would **silently skip on Fastly** between Task 5 and Task 6. Today Fastly consent works because `runtime_services_for_consent_route` (`app.rs:205`) reopens the consent store and **swaps the default** kv via `with_kv_store`. So in **Task 5**, the call site keeps today's behavior: pass `services.kv_handle()` (the default handle — which on Fastly is the swapped consent store; on the others matches current behavior). The **named-lookup flip** — `settings.consent.consent_store.as_deref().and_then(|id| services.kv_handle_named(id))` — and removal of the Fastly swap happen **atomically in Task 6**, once all four adapters (Fastly included) inject a `KvRegistry`. The behavioral test (below) therefore lands in **Task 6**. - Audit other KV consumers (`ec/*`) for the same "id dropped" pattern; they already use `kv_handle()` so are lower-risk. - **Adapters — Axum/Cloudflare/Spin here; Fastly in Task 6 (sequencing).** Populate `RuntimeServices.kv_registry` from extensions in `build_runtime_services` for **Axum/Cloudflare/Spin** (they inject registries via EdgeZero `run_app`/`dispatch_with_registries`). **Fastly's** named-KV wiring belongs in **Task 6**, because Fastly only injects registries into extensions in Task 6 (its custom `oneshot`); its active per-request services are built in `app.rs:238` (`build_per_request_services`), not `platform.rs`, and its consent special-casing (`app.rs:205`, `runtime_services_for_consent_route`, used at `app.rs:588/735`) is removed **there** once the `kv_registry` is present. Doing Fastly named-KV in Task 5 would populate from a registry not yet in extensions. - **Files:** `crates/trusted-server-core/src/platform/types.rs`, `crates/trusted-server-core/src/consent/mod.rs` (`ConsentPipelineInput.kv_store` type), `crates/trusted-server-core/src/storage/kv_store.rs` (consent persistence fns → `&KvHandle`), `crates/trusted-server-core/src/publisher.rs` (call site), `crates/trusted-server-adapter-{fastly,axum,cloudflare,spin}/src/platform.rs`, `crates/trusted-server-adapter-fastly/src/app.rs` (remove consent special-case), adapter test helpers (Step 2d). - - **Test (behavioral, not just resolution) — write it FAILING first.** The migration exists because consent **drops** the id and hits the default store. So the core test that proves the fix is behavioral: with `settings.consent.consent_store = "consent_store"` and a registry holding a **default** KV + a distinct `consent_store` KV, a consent persistence round-trip (load/save/delete) must read/write the **`consent_store`** handle and leave the **default** store **untouched**. Assert against both stores (the target has the entry; the default does not). Add this in `trusted-server-core` consent tests. Also keep the cheaper per-adapter check: `kv_handle_named("consent_store")` resolves and is distinct from the default; unknown id → `None`. + - **Test (Task 5):** the `kv_handle_named` surface resolves — per adapter, `kv_handle_named("consent_store")` returns a handle distinct from the default; unknown id → `None`. (The **behavioral** consent test — that `consent_store` is actually selected and the default is left untouched — lands in **Task 6** with the call-site flip; see Task 6.) - [ ] **Step 2d: Test-support — registry-populated `RequestContext` helper + migrate existing direct-context tests.** Strict registries make a missing registry a wiring bug, but existing adapter tests call `build_runtime_services(&ctx)` / `build_per_request_services(&ctx)` on **hand-built** `RequestContext`s with no registries inserted (e.g. `adapter-axum/src/app.rs:130`, `adapter-cloudflare/src/app.rs:151,314`, `adapter-spin/src/app.rs:440`, `adapter-cloudflare/src/platform.rs:729`). Those will now fail (composite → `registry.named()` → `None`). Add a shared test helper (e.g. `test_context_with_registries(config: &[…], kv: &[…], secrets: &[…]) -> RequestContext`) that inserts `ConfigRegistry`/`KvRegistry`/`SecretRegistry` into the context, and **migrate every existing direct-context test** to use it. Enumerate them during the run (`rg 'build_(runtime|per_request)_services'` in adapter test modules). Also convert the **existing core consent tests** that construct `ConsentPipelineInput` with a `&dyn PlatformKvStore` store — `crates/trusted-server-core/src/consent/mod.rs` has `kv_store: Some(&store)` call sites at ~lines 1450 / 1481 / 1492 — to the new `Option` shape (build a `KvHandle` over an in-memory `KvStore` test double). These fail to compile the moment `ConsentPipelineInput.kv_store` changes type, so migrate them in the same step (`rg 'kv_store: Some\(' crates/trusted-server-core/src/consent` to find them all). -- [ ] **Step 3: Run to verify pass** — this task touches **core** (`platform/types.rs`, consent, `publisher.rs`) and **Fastly** (`platform.rs`/`app.rs`) as well as Axum/CF/Spin, so run **all four** adapters + wasm checks (per the global "all four green" rule), incl. the non-default config/secret/KV tests from 2b/2c. +- [ ] **Step 3: Run to verify pass** — this task changes **core** (`platform/types.rs`, consent type, `publisher.rs` interim call site) and **Axum/CF/Spin** platform wiring. **Fastly is NOT modified here** (its named-KV wiring + consent flip are Task 6) — but run **all four** anyway to confirm Fastly still compiles/passes with the core changes (the `kv_handle()` interim call site preserves Fastly's swap behavior), per the "all four green" rule. Run: `cargo test-fastly && cargo test-axum && cargo test-cloudflare && cargo test-spin && cargo check-cloudflare && cargo check-spin` Expected: PASS. -- [ ] **Step 4: Commit** (include the core + Fastly changes, not just the three adapters) +- [ ] **Step 4: Commit** (core + Axum/CF/Spin — **not** Fastly; Fastly is committed in Task 6) ```bash git add crates/trusted-server-core/src/platform/types.rs \ crates/trusted-server-core/src/consent/mod.rs \ crates/trusted-server-core/src/storage/kv_store.rs \ crates/trusted-server-core/src/publisher.rs \ - crates/trusted-server-adapter-fastly \ crates/trusted-server-adapter-axum \ crates/trusted-server-adapter-cloudflare \ crates/trusted-server-adapter-spin -git commit -m "Wire RuntimeServices via composite + registry-backed named KV across all adapters" +git commit -m "Add named-KV surface + composite reads on Axum/Cloudflare/Spin; migrate consent to KvHandle" ``` --- @@ -446,12 +474,16 @@ Run: `cargo test-fastly build_config_registry_resolves_declared_ids` → Expecte Run: `cargo test-fastly oneshot_discovery_reads_jwks_via_registry` → Expected: FAIL, then PASS only after Steps 3, 4, **and 4b** (the test reads through `RuntimeServices`, which is composite-backed only after 4b — without 4b the injected registries are unused and the read still hits the old direct store). -- [ ] **Step 6: Fastly suite + parity + commit** +- [ ] **Step 5b: Flip the consent call site to named KV + behavioral test (the atomic cutover — all four adapters now inject a `KvRegistry`).** Now that Fastly (Step 4b) and Axum/CF/Spin (Task 5) all inject a `KvRegistry`, change `publisher.rs:626` from `services.kv_handle()` to `settings.consent.consent_store.as_deref().and_then(|id| services.kv_handle_named(id))`. This is the point where the configured id actually selects the store on **every** adapter (and `runtime_services_for_consent_route`'s swap, removed in 4b, is no longer needed). Add the **behavioral** core test (write it failing first): with `consent_store = "consent_store"` and a registry holding a **default** KV + a distinct `consent_store` KV, a consent round-trip (load/save/delete) reads/writes the **`consent_store`** handle and leaves the **default** store **untouched** (assert against both). Files: `crates/trusted-server-core/src/publisher.rs` (+ core consent test). + +- [ ] **Step 5c: Fastly named-KV / consent route test.** With `runtime_services_for_consent_route` removed (4b), add a Fastly test proving `consent_store` resolves via the **injected `KvRegistry`** — a consent-persisting route (or a `build_per_request_services`-level test) writes/reads through the `consent_store` handle, not the default. This guards the special-case removal. + +- [ ] **Step 6: Fastly suite + parity + commit** (core consent flip is committed here with the Fastly work, since the flip is only safe once Fastly injects registries) -Run: `cargo test-fastly && cargo test --manifest-path crates/trusted-server-integration-tests/Cargo.toml --test parity` +Run: `cargo test-fastly && cargo test-axum && cargo test-cloudflare && cargo test-spin && cargo test --manifest-path crates/trusted-server-integration-tests/Cargo.toml --test parity` ```bash -git add crates/trusted-server-adapter-fastly -git commit -m "Add local Fastly registry builders and inject them into the oneshot dispatch" +git add crates/trusted-server-adapter-fastly crates/trusted-server-core/src/publisher.rs +git commit -m "Inject Fastly registries; flip consent to named KV; remove consent special-casing" ``` --- From 32e35ec035aa2c5d6f81da94a5e854f7ef25bde2 Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Sat, 4 Jul 2026 22:34:34 -0700 Subject: [PATCH 29/30] Amend plan per review 18 (empty-registry impossible + secret builder + commit/filter) - BLOCKER: absent-registry policy said 'composite over empty StoreRegistry', but StoreRegistry fields are private and from_parts returns None on empty by_id. Fix: Composite{Config,Secret}Store hold Option (reader = the Option from extensions.get().cloned()); error on None. No empty-registry construction. - Fastly secret builder: does NOT open per id. One SecretHandle over FastlySecretStore (stateless; opens per get_bytes) + BoundSecretStore per id. Tightened Task 6 builders per kind (KV opens/Result, config handle, secret bound-no-open) - Task 6 commit: add consent/mod.rs (behavioral test file) - Task 3 test command: single shared-prefix filter 'composite_' (not two filters) --- ...-07-02-edgezero-store-registry-migration.md | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md b/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md index 0d94fb47..63954525 100644 --- a/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md +++ b/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md @@ -214,8 +214,8 @@ Concrete D6-a mechanism. The bespoke traits read **by `StoreName`** and callers - Consumes: `edgezero_core::store_registry::{ConfigRegistry, SecretRegistry, ConfigStoreBinding, BoundSecretStore}`, write-only `Arc`/`Arc`. - Produces: - New **write-only** traits `PlatformConfigWriter { put; delete }` and `PlatformSecretWriter { create; delete }` (extracted from the read+write `PlatformConfigStore`/`PlatformSecretStore`). This is what lets Task 8 delete the per-adapter **read** impls while keeping the writer object — the writer no longer needs `get`/`get_bytes`. - - `CompositeConfigStore::new(reader: ConfigRegistry, writer: Arc) -> Self` implementing the full read+write `PlatformConfigStore`. **`ConfigRegistry::named(id)` returns `Option`, not a handle** — so `get(store_name, key)` = resolve `binding = reader.named(store_name.as_str()).ok_or(PlatformError::ConfigStore)?`, then `block_on(binding.handle.get(key))`. EdgeZero `ConfigStore::get` returns `Result, ConfigStoreError>`; the bespoke `get` returns `Result`, so map `Ok(None)`/`Err(ConfigStoreError::*)` → `PlatformError::ConfigStore`. `put`/`delete` → `writer`. - - `CompositeSecretStore::new(reader: SecretRegistry, writer: Arc) -> Self` implementing `PlatformSecretStore`: `get_bytes(store_name, key)` = `reader.named(store_name.as_str()).ok_or(PlatformError::SecretStore)?` → `block_on(bound.get_bytes(key))`; map `Ok(None)`/`Err` → `PlatformError::SecretStore`. `create`/`delete` → `writer`. A store_name not in the registry is a hard error (strict), not a silent fallback. + - `CompositeConfigStore::new(reader: Option, writer: Arc) -> Self` implementing the full read+write `PlatformConfigStore`. The reader is `Option` because **an empty `StoreRegistry` cannot be constructed** (fields are private; `from_parts` returns `None` on empty `by_id`) — so the "absent registry" case is `None`, not an empty registry. `get(store_name, key)` = `let reg = self.reader.as_ref().ok_or(PlatformError::ConfigStore)?;` then (since **`ConfigRegistry::named(id)` returns `Option`, not a handle**) `let binding = reg.named(store_name.as_str()).ok_or(PlatformError::ConfigStore)?;` then `block_on(binding.handle.get(key))`. EdgeZero `ConfigStore::get` returns `Result, ConfigStoreError>`; the bespoke `get` returns `Result`, so map `Ok(None)`/`Err(ConfigStoreError::*)` → `PlatformError::ConfigStore`. `put`/`delete` → `writer`. + - `CompositeSecretStore::new(reader: Option, writer: Arc) -> Self` implementing `PlatformSecretStore`: `get_bytes(store_name, key)` = `self.reader.as_ref().ok_or(PlatformError::SecretStore)?.named(store_name.as_str()).ok_or(PlatformError::SecretStore)?` → `block_on(bound.get_bytes(key))`; map `Ok(None)`/`Err` → `PlatformError::SecretStore`. `create`/`delete` → `writer`. Both an **absent registry** (`None`) and a store_name not in the registry are hard errors (strict), never a silent fallback. - [ ] **Step 0: Split write-only traits.** In `traits.rs`, define `pub trait PlatformConfigWriter: Send + Sync { put; delete }` and `pub trait PlatformSecretWriter: Send + Sync { create; delete }` (matching the parent traits' `Send + Sync` bounds, since they're held as `Arc`). Keep `PlatformConfigStore`/`PlatformSecretStore` as the read+write surface `RuntimeServices` exposes. This split is the prerequisite that makes Task 8's "delete reads, keep writes" compile. Run `cargo check-axum` to confirm the split compiles before proceeding. @@ -282,7 +282,7 @@ fn composite_secret_reads_named_store_and_writes_delegate() { - [ ] **Step 2: Run to verify both fail** -Run: `cargo test-fastly composite_config_reads_named_store_and_writes_delegate composite_secret_reads_named_store_and_writes_delegate` +Run: `cargo test-fastly composite_` (one shared-prefix filter runs both `composite_config_…` and `composite_secret_…`; `cargo test` takes a single filter). Expected: FAIL (module does not exist). - [ ] **Step 3: Implement `composite.rs`** @@ -400,7 +400,7 @@ cargo test-axum first_party_proxy_reads_s3_secret ``` Expected: FAIL (all three). -- [ ] **Step 2: Build `RuntimeServices` via the composite** in each adapter's `build_runtime_services(ctx: &RequestContext)`. **Extract the whole registry from request extensions** — `ctx.request().extensions().get::().cloned()` / `get::()` — the same way EdgeZero's `Config`/`Secrets` extractors do. Do **not** use `ctx.config_store_default()`/`config_store(id)` (those return a single bound handle and would wire only the default store). Pass the cloned registry as the composite reader (Task 3) and the per-adapter **write-only** impl (`PlatformConfigWriter`/`PlatformSecretWriter`) as the writer. **Absent-registry policy (concrete):** `build_runtime_services` returns `RuntimeServices` (not `Result`) on all adapters, so don't add a fallible signature. Instead, when a registry is absent from extensions, build a composite over an **empty** registry (`StoreRegistry` with no ids) — its strict `named()`/`default()` return `None`, so the composite's `get`/`get_bytes` **error** (`PlatformError`) on first read rather than silently reading a default store. A missing registry thus surfaces as a read error at the call site, not a silent fallback, with no builder signature change. +- [ ] **Step 2: Build `RuntimeServices` via the composite** in each adapter's `build_runtime_services(ctx: &RequestContext)`. **Extract the whole registry from request extensions** — `ctx.request().extensions().get::().cloned()` / `get::()` — the same way EdgeZero's `Config`/`Secrets` extractors do. Do **not** use `ctx.config_store_default()`/`config_store(id)` (those return a single bound handle and would wire only the default store). Pass the cloned registry **`Option`** as the composite reader (Task 3) and the per-adapter **write-only** impl (`PlatformConfigWriter`/`PlatformSecretWriter`) as the writer. **Absent-registry policy (concrete):** `build_runtime_services` returns `RuntimeServices` (not `Result`) on all adapters, so don't add a fallible signature. The composite reader is `Option` / `Option` (an empty `StoreRegistry` is unconstructable — private fields; `from_parts` → `None` on empty). So pass `ctx.request().extensions().get::().cloned()` (already an `Option`) straight into `CompositeConfigStore::new(...)`; when it's `None`, the composite's `get`/`get_bytes` **error** (`PlatformError`) on first read rather than silently reading a default store. A missing registry surfaces as a read error at the call site — no builder signature change, no empty-registry construction. - [ ] **Step 2b: Non-default coverage on Cloudflare AND Spin (not just Axum).** Cloudflare/Spin platform mappings differ (config = KV-namespace/label backed; secrets = flat namespace), so default-only assertions are insufficient. Add, in each of the Cloudflare and Spin test modules, tests proving **non-default** resolution: a `jwks_store` **config** read and a `ts_secrets` / S3 **secret**-key read resolve through the composite (route tests if cheap, else small `build_runtime_services` + composite-read tests seeding a 2-id registry). @@ -460,7 +460,11 @@ EdgeZero's Fastly `dispatch_with_registries` and its registry builders are `pub( Run: `cargo test-fastly build_config_registry_resolves_declared_ids` → Expected: FAIL. -- [ ] **Step 3: Implement the three builders** in `registries.rs` with the signatures above: iterate `StoreMetadata.ids`, open the EdgeZero Fastly store **by the logical id** (`FastlyConfigStore`/`FastlyKvStore`/`FastlySecretStore` open primitive), collect into a `BTreeMap`, and `StoreRegistry::from_parts(by_id, default_id.to_owned())` (propagating the KV `open` error in `build_kv_registry`). No `EnvConfig`, no runtime dictionary. +- [ ] **Step 3: Implement the three builders** in `registries.rs` with the signatures above — the three kinds construct **differently** (mirror EdgeZero's own private Fastly builders, `request.rs`): + - **KV** (`build_kv_registry -> Result, FastlyError>`): for each id, `FastlyKvStore::open(id)` (this **can fail** → propagate `FastlyError`) → `KvHandle`; collect into `BTreeMap`; `StoreRegistry::from_parts(by_id, default_id)`. + - **Config** (`-> Option`): for each id, build a `ConfigStoreHandle` over `FastlyConfigStore` for that id (+ `ConfigStoreBinding { handle, default_key }`); `from_parts`. + - **Secret** (`-> Option`): **do NOT open per id.** Create **one** `SecretHandle::new(Arc::new(FastlySecretStore))` (the provider is stateless — `FastlySecretStore::get_bytes(store_name, key)` opens the named store per call), then bind each id via `BoundSecretStore::new(handle.clone(), store_name)` where `store_name` = the logical id (D7); `from_parts`. + - All open **by logical id** (D7 — no `EnvConfig`/runtime dictionary). `from_parts` yields `None` if a kind is undeclared or the default id is absent. - [ ] **Step 4: Insert registries in the oneshot block** — replace the lone `core_req.extensions_mut().insert(config_store)` at `main.rs:477`: build the three registries via Step 3 (propagate `build_kv_registry`'s `FastlyError` into the dispatch's `Result`), and `if let Some(reg) = ...` insert each into `core_req.extensions_mut()`, preserving the existing `client_info`/`device_signals` inserts. @@ -482,7 +486,9 @@ Run: `cargo test-fastly oneshot_discovery_reads_jwks_via_registry` → Expected: Run: `cargo test-fastly && cargo test-axum && cargo test-cloudflare && cargo test-spin && cargo test --manifest-path crates/trusted-server-integration-tests/Cargo.toml --test parity` ```bash -git add crates/trusted-server-adapter-fastly crates/trusted-server-core/src/publisher.rs +git add crates/trusted-server-adapter-fastly \ + crates/trusted-server-core/src/publisher.rs \ + crates/trusted-server-core/src/consent/mod.rs git commit -m "Inject Fastly registries; flip consent to named KV; remove consent special-casing" ``` From 05aab520a187dfef1b173523ff6185aae77cffb3 Mon Sep 17 00:00:00 2001 From: Aram Grigoryan <132480+aram356@users.noreply.github.com> Date: Sun, 5 Jul 2026 00:43:02 -0700 Subject: [PATCH 30/30] Amend plan per review 19 (Task 3 Option snippets + KV open type + Task 5 scope) - Task 3 test snippets: pass Some(reader) to Composite{Config,Secret}Store::new (constructor takes Option, snippets passed bare registries) - Fastly KV builder: FastlyKvStore::open(id) returns Result, not KvHandle; wrap KvHandle::new(Arc::new(store)) - Task 5 stale scope: publisher.rs call site is interim kv_handle() (flip is Task 6); remove Fastly platform.rs/app.rs from Task 5 files (Task 6 owns them) - Task 3 Step 4 pass command: cargo test-fastly composite_ (runs both tests) --- .../2026-07-02-edgezero-store-registry-migration.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md b/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md index 63954525..50d5fbc5 100644 --- a/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md +++ b/docs/superpowers/plans/2026-07-02-edgezero-store-registry-migration.md @@ -230,7 +230,7 @@ fn composite_config_reads_named_store_and_writes_delegate() { ("jwks_store", "kid-1", "{\"kty\":\"OKP\"}"), ], "trusted_server_config"); let writer = Arc::new(RecordingConfigWriter::default()); - let composite = CompositeConfigStore::new(reader, writer.clone()); + let composite = CompositeConfigStore::new(Some(reader), writer.clone()); // Act + Assert: non-default store resolves. let jwk = composite .get(&StoreName::from("jwks_store"), "kid-1") @@ -261,7 +261,7 @@ fn composite_secret_reads_named_store_and_writes_delegate() { ("ts_secrets", "server-side-key", b"dd-secret"), ], "trusted_server_secrets"); let writer = Arc::new(RecordingSecretWriter::default()); - let composite = CompositeSecretStore::new(reader, writer.clone()); + let composite = CompositeSecretStore::new(Some(reader), writer.clone()); // Non-default store resolves. let v = composite.get_bytes(&StoreName::from("ts_secrets"), "server-side-key").expect("read"); assert_eq!(v, b"dd-secret"); @@ -291,7 +291,7 @@ Expected: FAIL (module does not exist). - [ ] **Step 4: Run to verify it passes** -Run: `cargo test-fastly composite_config_reads_named_store_and_writes_delegate` +Run: `cargo test-fastly composite_` (runs both config + secret composite tests) Expected: PASS. - [ ] **Step 5: Reconcile `StoreName` semantics (D7).** `platform/types.rs::StoreName` is documented as an "edge-visible **platform** name". The composite now resolves `registry.named(store_name.as_str())` by **logical id**, so `StoreName` for reads must carry the **logical store id**. Update the `StoreName` doc comment to say "logical runtime store id" for reads, and audit read call sites (`request_signing/{signing,rotation}.rs`, `proxy.rs`, `integrations/datadome/{protection,protection_scope}.rs`) to confirm they pass **logical ids** (`trusted_server_config`, `jwks_store`, `ts_secrets`, `datadome-ip-bypass`, …), not physical platform names. No functional change if ids already equal names (D7 convention), but the doc + audit prevent implementers from passing physical names into logical registries. @@ -364,7 +364,7 @@ git commit -m "Load boot config via EdgeZero config store on Fastly and Axum" **Files:** - Modify: `crates/trusted-server-adapter-axum/src/main.rs` (keep `AxumDevServer::with_config`, chain registry setters) - Create: `crates/trusted-server-adapter-axum/src/registries.rs` (`build_{config,kv,secret}_registry_axum(&StoresMetadata)`) -- **Modify (core KV surface, Step 2c):** `crates/trusted-server-core/src/platform/types.rs` (`RuntimeServices::kv_handle_named` + `kv_registry` field/builder), `crates/trusted-server-core/src/consent/mod.rs` + `storage/kv_store.rs` (consent KV surface → `KvHandle`), `crates/trusted-server-core/src/publisher.rs` (call site passes `consent_store`) +- **Modify (core KV surface, Step 2c):** `crates/trusted-server-core/src/platform/types.rs` (`RuntimeServices::kv_handle_named` + `kv_registry` field/builder), `crates/trusted-server-core/src/consent/mod.rs` + `storage/kv_store.rs` (consent KV surface → `KvHandle`), `crates/trusted-server-core/src/publisher.rs` (interim call site keeps `kv_handle()`; the named `consent_store` flip is **Task 6**) - Modify: `crates/trusted-server-adapter-{axum,cloudflare,spin}/src/platform.rs` (`build_runtime_services` → composite + `kv_registry` from extensions) - **Fastly named-KV + composite + `runtime_services_for_consent_route` removal → Task 6** (Fastly injects registries only in Task 6). Task 5 leaves Fastly compiling: the core `kv_handle_named` is additive and returns `None` on Fastly until Task 6 wires `kv_registry` (consent falls back safely). - Test-support (Step 2d): a shared `test_context_with_registries(...)` helper; migrate existing direct-context tests (`adapter-{axum,cloudflare,spin,fastly}` test modules) @@ -410,7 +410,7 @@ Expected: FAIL (all three). - **Consent type migration (Task 5) vs behavioral flip (Task 6) — avoid an interim Fastly regression.** Change `ConsentPipelineInput.kv_store` from `Option<&dyn PlatformKvStore>` to `Option` and update the persistence fns (`load_consent_from_kv`/`save_consent_to_kv`/`delete_consent_from_kv`) to take `&KvHandle` (they already `block_on`) — **in Task 5**. But **do NOT flip the `publisher.rs:626` call site to `kv_handle_named` in Task 5**: Fastly has no `KvRegistry` until Task 6, so `kv_handle_named("consent_store")` would return `None` there and consent persistence would **silently skip on Fastly** between Task 5 and Task 6. Today Fastly consent works because `runtime_services_for_consent_route` (`app.rs:205`) reopens the consent store and **swaps the default** kv via `with_kv_store`. So in **Task 5**, the call site keeps today's behavior: pass `services.kv_handle()` (the default handle — which on Fastly is the swapped consent store; on the others matches current behavior). The **named-lookup flip** — `settings.consent.consent_store.as_deref().and_then(|id| services.kv_handle_named(id))` — and removal of the Fastly swap happen **atomically in Task 6**, once all four adapters (Fastly included) inject a `KvRegistry`. The behavioral test (below) therefore lands in **Task 6**. - Audit other KV consumers (`ec/*`) for the same "id dropped" pattern; they already use `kv_handle()` so are lower-risk. - **Adapters — Axum/Cloudflare/Spin here; Fastly in Task 6 (sequencing).** Populate `RuntimeServices.kv_registry` from extensions in `build_runtime_services` for **Axum/Cloudflare/Spin** (they inject registries via EdgeZero `run_app`/`dispatch_with_registries`). **Fastly's** named-KV wiring belongs in **Task 6**, because Fastly only injects registries into extensions in Task 6 (its custom `oneshot`); its active per-request services are built in `app.rs:238` (`build_per_request_services`), not `platform.rs`, and its consent special-casing (`app.rs:205`, `runtime_services_for_consent_route`, used at `app.rs:588/735`) is removed **there** once the `kv_registry` is present. Doing Fastly named-KV in Task 5 would populate from a registry not yet in extensions. - - **Files:** `crates/trusted-server-core/src/platform/types.rs`, `crates/trusted-server-core/src/consent/mod.rs` (`ConsentPipelineInput.kv_store` type), `crates/trusted-server-core/src/storage/kv_store.rs` (consent persistence fns → `&KvHandle`), `crates/trusted-server-core/src/publisher.rs` (call site), `crates/trusted-server-adapter-{fastly,axum,cloudflare,spin}/src/platform.rs`, `crates/trusted-server-adapter-fastly/src/app.rs` (remove consent special-case), adapter test helpers (Step 2d). + - **Files (Task 5):** `crates/trusted-server-core/src/platform/types.rs`, `crates/trusted-server-core/src/consent/mod.rs` (`ConsentPipelineInput.kv_store` type), `crates/trusted-server-core/src/storage/kv_store.rs` (consent persistence fns → `&KvHandle`), `crates/trusted-server-core/src/publisher.rs` (interim call site, kept on `kv_handle()`), `crates/trusted-server-adapter-{axum,cloudflare,spin}/src/platform.rs`, adapter test helpers (Step 2d). **Fastly `platform.rs`/`app.rs` (populate `kv_registry`, remove consent special-case) + the `publisher.rs` named-lookup flip are Task 6**, not here. - **Test (Task 5):** the `kv_handle_named` surface resolves — per adapter, `kv_handle_named("consent_store")` returns a handle distinct from the default; unknown id → `None`. (The **behavioral** consent test — that `consent_store` is actually selected and the default is left untouched — lands in **Task 6** with the call-site flip; see Task 6.) - [ ] **Step 2d: Test-support — registry-populated `RequestContext` helper + migrate existing direct-context tests.** Strict registries make a missing registry a wiring bug, but existing adapter tests call `build_runtime_services(&ctx)` / `build_per_request_services(&ctx)` on **hand-built** `RequestContext`s with no registries inserted (e.g. `adapter-axum/src/app.rs:130`, `adapter-cloudflare/src/app.rs:151,314`, `adapter-spin/src/app.rs:440`, `adapter-cloudflare/src/platform.rs:729`). Those will now fail (composite → `registry.named()` → `None`). Add a shared test helper (e.g. `test_context_with_registries(config: &[…], kv: &[…], secrets: &[…]) -> RequestContext`) that inserts `ConfigRegistry`/`KvRegistry`/`SecretRegistry` into the context, and **migrate every existing direct-context test** to use it. Enumerate them during the run (`rg 'build_(runtime|per_request)_services'` in adapter test modules). @@ -461,7 +461,7 @@ EdgeZero's Fastly `dispatch_with_registries` and its registry builders are `pub( Run: `cargo test-fastly build_config_registry_resolves_declared_ids` → Expected: FAIL. - [ ] **Step 3: Implement the three builders** in `registries.rs` with the signatures above — the three kinds construct **differently** (mirror EdgeZero's own private Fastly builders, `request.rs`): - - **KV** (`build_kv_registry -> Result, FastlyError>`): for each id, `FastlyKvStore::open(id)` (this **can fail** → propagate `FastlyError`) → `KvHandle`; collect into `BTreeMap`; `StoreRegistry::from_parts(by_id, default_id)`. + - **KV** (`build_kv_registry -> Result, FastlyError>`): `FastlyKvStore::open(id)` returns `Result` (not a `KvHandle`), so for each id `let store = FastlyKvStore::open(id)?;` then wrap `KvHandle::new(Arc::new(store))` (map `KvError` → `FastlyError`); collect into `BTreeMap`; `StoreRegistry::from_parts(by_id, default_id)`. - **Config** (`-> Option`): for each id, build a `ConfigStoreHandle` over `FastlyConfigStore` for that id (+ `ConfigStoreBinding { handle, default_key }`); `from_parts`. - **Secret** (`-> Option`): **do NOT open per id.** Create **one** `SecretHandle::new(Arc::new(FastlySecretStore))` (the provider is stateless — `FastlySecretStore::get_bytes(store_name, key)` opens the named store per call), then bind each id via `BoundSecretStore::new(handle.clone(), store_name)` where `store_name` = the logical id (D7); `from_parts`. - All open **by logical id** (D7 — no `EnvConfig`/runtime dictionary). `from_parts` yields `None` if a kind is undeclared or the default id is absent.