fix(responses): refuse input beyond the advertised context window (#1128) - #1167
Closed
HoshimiRox1 wants to merge 2 commits into
Closed
fix(responses): refuse input beyond the advertised context window (#1128)#1167HoshimiRox1 wants to merge 2 commits into
HoshimiRox1 wants to merge 2 commits into
Conversation
A chained-turn replay can balloon a request far past the model's context window (observed: a 4x expansion pushed a ~400k-token conversation to 1.6M input tokens). The proxy forwarded it verbatim; processing it on Windows ballooned bun RSS and native-crashed the whole service (upstream Bun memory bug, lidge-jun#314), taking every active thread down until restart. Reject the request with a clean 413 before any upstream I/O when the parsed input exceeds the model's configured modelContextWindows value. The client compacts well before the window, so the guard only fires on abnormal duplication.
The lidge-jun#875 transport policy forces a bounded JSON upstream for providers like deepseek (modelResponsesUpstreamStreaming: false), even when the client asked for SSE. The passthrough branch already reframes that JSON back to the canonical terminal SSE sequence, but routed-compaction turns skip the passthrough branch: they returned application/json, so Codex SSE parser hit EOF at the first byte and the remote compact task failed with "stream closed before response.completed", retrying 6x and leaving the thread unusable. Mirror the passthrough reframe in both routed branches (runTurn and parseStream): when the client requested stream and the upstream policy forced JSON, emit response.created, one output_item.done per output item (including the synthetic compaction item), response.completed, then [DONE].
Contributor
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
⏳ DRAFT
What to do
Review readiness checklist
1/4 boxes ticked. This PR stays in draft until every box above is ticked. |
Contributor
|
✅ Deterministic PR hygiene checks passed. |
Author
|
Closing for now — will re-open after reviewing the process. Thanks! |
5 tasks
HoshimiRox1
changed the title
7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Verification
Checklist
Review readiness checklist
This PR stays in draft until every box below is ticked. Tick all four boxes once the requirements are met:
All CI tests are green on my local testing.
I pushed my PR to the latest dev commit.
I resolved all correct Codex and CodeRabbit findings.
My PR is ready for review.