fix(gateway): 修复跨账号 compaction 重写引起的499错误 与 zstd 大请求引发的413错误 - #397
Open
MDX-Tom wants to merge 2 commits into
Open
fix(gateway): 修复跨账号 compaction 重写引起的499错误 与 zstd 大请求引发的413错误#397MDX-Tom wants to merge 2 commits into
MDX-Tom wants to merge 2 commits into
Conversation
MDX-Tom
force-pushed
the
codex/fix-zstd-compaction-recovery
branch
from
July 28, 2026 05:41
dcb2c6c to
ee62ccf
Compare
qxcnm
requested changes
Jul 28, 2026
Owner
There was a problem hiding this comment.
compaction 修复方向没有发现阻塞问题,但 zstd 解压改动存在可被压缩炸弹触发的内存放大风险,暂时不能合并。
默认 CODEXMANAGER_FRONT_PROXY_MAX_BODY_BYTES=0 时,当前实现会对 zstd 请求直接 read_to_end,使解压后大小完全不受限。该解压发生在后端鉴权之前;即便限制并发任务数量,高压缩比请求仍可能持续放大内存并导致 OOM。
可以取消旧的 64 MiB 限制以兼容官方大请求,但请保留独立、可配置且有安全默认值的解压后大小上限(或可靠的扩张比限制),不要让 0 等价于无限解压,并增加压缩炸弹/超高扩张比回归测试。compaction 部分可以保留,必要时也可以拆成单独 PR。
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
背景
在启用官方 zstd 请求压缩,并使用
OpenAIprovider name 后,长会话/图片场景可能出现两类失败:input[N].encrypted_content缺失;根因
1. compaction 被留下无效对象壳
跨账号恢复会移除账号绑定的
encrypted_content。旧逻辑会删除compaction.encrypted_content字段,但只把reasoning/encrypted_content类型的数组项整体移除,导致compaction本体继续发往上游。Responses API 仍要求该类型携带encrypted_content,因此报missing_required_parameter。2. zstd 路径存在未配置的 64 MiB 隐式上限
项目配置中
CODEXMANAGER_FRONT_PROXY_MAX_BODY_BYTES=0表示不限制,普通请求也遵循该语义;但 zstd 解压路径额外把 0 映射成固定 64 MiB。官方 Codex 会将当前任务的完整 Responses JSON 编码后再用 zstd level 3 压缩,图片密集型长任务的解压尺寸可以超过 64 MiB,官方请求代码没有对应的 64 MiB 客户端上限。修改
reasoning、encrypted_content、compaction、compaction_summary、context_compaction作为账号绑定历史项整体移除,避免留下缺字段对象壳。FRONT_PROXY_MAX_BODY_BYTES语义:0:不添加请求体大小拒绝;>0:继续按解压后大小限制,并保留 413。spawn_blocking,未改变请求路由、缓存或图片加载逻辑。.cargo/config.toml。高概率复现
499
/v1/responses输入:前 24 项为普通 message,第 25 项为带encrypted_content的type=compaction,末尾再追加一条普通 message。strip_session_affinity=true。encrypted_content的壳并发送,稳定触发input[24].encrypted_content缺失;修复后 compaction 项被整体移除。413
data:image/png;base64,...的合法 Responses JSON,使解压后长度为67,108,865bytes。Content-Encoding: zstd,保持默认 body limit 语义(0)。413 ... after zstd decompression: >67108864;修复后成功解压并进入后端转发。max_body_bytes=32和 64-byte 解压内容验证显式限制仍返回 413。测试
通过:
cargo test -p codexmanager-service strip_encrypted_content -- --nocapture(2/2)cargo test -p codexmanager-service stripped_candidate_removes_account_scoped_items_without_leaving_invalid_shells -- --nocapture(1/1)cargo test -p codexmanager-service default_front_proxy_limit_decodes_official_image_request_above_64_mib -- --nocapture(1/1)cargo test -p codexmanager-service decompressed_zstd_request_body_respects_front_proxy_limit -- --nocapture(1/1)cargo test -p codexmanager-service zstd -- --nocapture(6/6)cargo test --workspace --exclude codexmanager-service(通过)cargo fmt --all -- --checkgit diff --check HEAD~2..HEAD完整
cargo test --workspace还会命中一个基线已存在的 macOS 临时目录规范化失败:directory_import_detects_same_size_file_replacement_and_fifo_entries;该用例在未应用本 PR 的基线 commit 上同样稳定失败。本 PR 涉及的回归、gateway 集成和其余 workspace 均已验证。Commit 划分
b619e0fcfix(gateway): drop account-scoped compaction on failoverdcb2c6cafix(gateway): honor unbounded zstd request body settingFixes #224