fix(agent-core): only send prompt_cache_key to official OpenAI endpoints#2203
fix(agent-core): only send prompt_cache_key to official OpenAI endpoints#2203B143KC47 wants to merge 3 commits into
Conversation
…dpoint Since 0.29.0 every OpenAI-compatible provider received the session prompt_cache_key in the request body. Strictly-validating endpoints reject the unknown parameter with a 400, breaking custom providers. Gate the field on the effective base URL targeting api.openai.com (or being unset, which the client defaults there), in both the v1 provider config resolution and the v2 OpenAI chat-completions/responses bases. Vendors that support the field (e.g. Kimi) keep encoding it through their own branch or cacheKey trait hook. Fixes MoonshotAI#2166
…pt cache affinity Data-residency endpoints (eu.api.openai.com, us.api.openai.com) are official OpenAI hosts and accept prompt_cache_key; match any *.api.openai.com hostname instead of the apex only.
🦋 Changeset detectedLatest commit: 0f89f11 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a6909f91e6
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| /** | ||
| * `prompt_cache_key` is an official-OpenAI request field: strictly-validating | ||
| * OpenAI-compatible endpoints reject unknown parameters with a 400, so the | ||
| * bases only fall back to it when the effective base URL targets | ||
| * `api.openai.com` or a regional variant like `eu.api.openai.com` (or is |
There was a problem hiding this comment.
This function-level JSDoc violates the package convention that comments may appear only in the top-of-file block. Move the helper’s externally relevant responsibility into the existing module header and remove this local narration.
AGENTS.md reference: packages/agent-core-v2/AGENTS.md:L11-L13
Useful? React with 👍 / 👎.
| // The bare `prompt_cache_key` fallback is official-OpenAI only: | ||
| // strictly-validating compatible endpoints 400 on unknown fields | ||
| // (#2166); vendors that support it encode it via their cacheKey hook. |
There was a problem hiding this comment.
These inline comments narrate the branch beside a statement, contrary to this package’s header-only comment convention. Keep any externally relevant responsibility in the file header and let the conditional express the implementation.
AGENTS.md reference: packages/agent-core-v2/AGENTS.md:L11-L13
Useful? React with 👍 / 👎.
| // Per-turn intent overlays in the fixed contract order. The | ||
| // `prompt_cache_key` overlay is official-OpenAI only: strictly-validating | ||
| // compatible endpoints reject unknown fields with a 400 (#2166). |
There was a problem hiding this comment.
This statement-adjacent explanation violates the scoped rule that comments live solely in the top-of-file block. Incorporate the externally relevant endpoint responsibility into that header and remove the inline narration.
AGENTS.md reference: packages/agent-core-v2/AGENTS.md:L11-L13
Useful? React with 👍 / 👎.
| // Strictly-validating OpenAI-compatible servers reject the unknown | ||
| // `prompt_cache_key` field with a 400, so the bare fallback stays | ||
| // official-endpoint only (#2166). |
There was a problem hiding this comment.
The new explanatory comment is inside the test body, but the package convention permits comments only in the top-of-file scenario header. Move any necessary scenario responsibility to that existing header and keep the test’s setup and assertions self-explanatory.
AGENTS.md reference: packages/agent-core-v2/AGENTS.md:L11-L13
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Pull request overview
Fixes a regression where prompt_cache_key was being sent to all OpenAI-compatible endpoints, causing strict third-party providers (e.g. NVIDIA NIM) to reject requests with 400 Unsupported parameter(s): prompt_cache_key. The change scopes prompt_cache_key emission to official OpenAI base URLs (or unset base URL), restoring compatibility for custom endpoints while keeping session cache affinity for the official API.
Changes:
- Gate
prompt_cache_keybehind anisOfficialOpenAIBaseUrlcheck for v1openaiandopenai_responsesprovider config generation. - Apply the same official-endpoint-only fallback behavior in v2 OpenAI Chat Completions and Responses bases.
- Add tests covering both “custom endpoint omits” and “official endpoint keeps” behavior for v1 and v2.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| packages/agent-core/test/harness/runtime-provider.test.ts | Adds v1 regression tests ensuring prompt_cache_key is omitted for custom base URLs and kept for official OpenAI URLs. |
| packages/agent-core/src/session/provider-manager.ts | Gates prompt_cache_key in OpenAI provider configs using a new isOfficialOpenAIBaseUrl helper. |
| packages/agent-core-v2/test/kosong/provider/composition.test.ts | Adds v2 behavior probes verifying omission on custom endpoints and retention on official OpenAI endpoints. |
| packages/agent-core-v2/src/kosong/provider/bases/openai/openai-responses.ts | Gates the per-turn prompt_cache_key overlay to official OpenAI base URLs. |
| packages/agent-core-v2/src/kosong/provider/bases/openai/openai-legacy.ts | Makes the base fallback prompt_cache_key overlay official-endpoint-only while preserving vendor cacheKey hook behavior. |
| packages/agent-core-v2/src/kosong/provider/bases/openai/openai-common.ts | Introduces shared isOfficialOpenAIBaseUrl helper for v2 OpenAI bases. |
| .changeset/prompt-cache-key-official-openai-only.md | Adds a patch changeset for the published CLI package describing the user-visible fix. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
The package convention allows comments only in the top-of-file block; move the prompt_cache_key gating notes there.
|
Codex Review: Didn't find any major issues. Nice work! Reviewed commit: ℹ️ About Codex in GitHubCodex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback". |
Related Issue
Resolve #2166
Problem
Since v0.29.0 (#1970), every OpenAI-compatible provider receives the session
prompt_cache_keyin the request body. Strictly-validating endpoints (e.g. NVIDIA NIM) reject the unknown parameter with400 Unsupported parameter(s): prompt_cache_key, which makes customopenai/openai_responsesproviders unusable.What changed
The field is now only sent when the effective base URL targets the official OpenAI API (
api.openai.comor a regional variant likeeu.api.openai.com, or no base URL, which the client defaults there):toKosongProviderConfiggatesprompt_cache_keyin both theopenaiandopenai_responsesbranches on a newisOfficialOpenAIBaseUrlhelper. Thekimibranch is intentionally untouched — Kimi endpoints support the field.kimi web/ experimental print mode): the same gate on the bare fallback in the OpenAI chat-completions and Responses bases. Vendors that support the field keep encoding it through theircacheKeytrait hook (e.g. Kimi), which is unaffected.This restores pre-0.29.0 behavior for custom endpoints while preserving #1970's cache-affinity intent for the official API. Trade-off, stated explicitly: OpenAI-compatible gateways on custom base URLs that do accept
prompt_cache_keylose cache affinity again — exactly the pre-0.29.0 behavior; a per-provider opt-in flag could cover them as a follow-up if wanted.Tests:
runtime-provider.test.tsgains custom-endpoint (omitted) and official-endpoint (kept) cases for both provider types;composition.test.tsgains the same probes on the v2 wire. All four fail without the fix.Checklist
gen-changesetsskill, or this PR needs no changeset.gen-docsskill, or this PR needs no doc update.