feat(runtime): add Codex server-side history compaction

[M4n5ter]

English

Summary

Add provider-native server-side history compaction for OpenAI Codex subscription models and enable it by default for those models.

The implementation keeps the existing text-summary checkpoint for portable compaction and adds a distinct opaque provider-state checkpoint for Codex. The runtime owns the common checkpoint lifecycle—coverage, persistence, replay, rolling replacement, overflow recovery, and model-switch compatibility—while the Codex-specific compactor owns only the provider protocol and request options.

Key behavior:

• configure the Codex remote compactor by default in Runtime Host model composition;
• set providerOptions.openai.compactionTrigger only on the dedicated compactor request, leaving ordinary model requests unchanged;
• persist and replay native Codex compaction state without fake summaries or synthetic transcript markers;
• preserve the selected provider checkpoint through empty tails and text-only replay fallbacks;
• lower settled provider-executed tool history into paired Responses function-call items so store: false never produces dangling outputs;
• redact only the opaque compaction payload in provider-request telemetry while preserving unrelated URLs, metadata, and business JSON;
• retain existing text summarization behavior for non-Codex providers and safely recompress full source history after incompatible model/provider switches.

The abstraction is intentionally narrow: one discriminated checkpoint representation and one compactor contract, without introducing a provider registry or a generic transport framework before another integration requires one.

Upstream context

This work exposed a missing typed capability in the Vercel AI SDK. M4n5ter reported vercel/ai#18894, which directly prompted vercel/ai#18895. The upstream change has since been merged and released; Maka now uses the released providerOptions.openai.compactionTrigger support directly, with no compatibility transport layer.

Verification

• clean npm ci, including dependency patch application;
• npm run format:check;
• npm run build;
• npm run typecheck;
• npm test — 6,671 tests: 6,645 passed, 0 failed, 26 skipped;
• git diff --check;
• real Codex OAuth verification with gpt-5.6-luna: initial compaction, rolling compaction from the previous opaque state, and ordinary provider-state replay all succeeded. The two compaction requests each carried exactly one terminal trigger; the replay request carried no trigger and recovered both facts from the compacted history.

Checklist

• Tests cover the change and its regression cases.
• Format, typecheck, build, and the full test suite pass locally.
• The provider-native path was verified against a real Codex OAuth subscription.

AI disclosure

This change was completed with assistance from Codex. I have reviewed it and accept responsibility for the result.

中文

概要

为 OpenAI Codex 订阅模型接入 provider 原生的服务端历史压缩能力，并对这类模型默认启用。

实现保留现有可移植的文本摘要 checkpoint，同时为 Codex 增加独立的 opaque provider-state checkpoint。Runtime 负责公共生命周期，包括覆盖范围、持久化、回放、滚动替换、超窗恢复和模型切换兼容性；Codex 专用 compactor 仅负责 provider 协议和请求选项。

主要行为：

• 在 Runtime Host 的模型组合阶段为 Codex 默认配置远程 compactor；
• 仅在专用 compactor 请求上设置 providerOptions.openai.compactionTrigger，普通模型请求不受影响；
• 持久化并回放原生 Codex 压缩状态，不使用虚假摘要或合成 transcript marker；
• 在空 tail 和 text-only 回放降级路径中继续保留已选中的 provider checkpoint；
• 将已结算的 provider-executed tool 历史转换成严格配对的 Responses function-call items，避免 store: false 产生 dangling output；
• provider-request telemetry 只脱敏 opaque 压缩载荷，不破坏无关 URL、metadata 和业务 JSON；
• 非 Codex provider 继续使用现有文本摘要；发生不兼容的模型/provider 切换后，会从完整原始历史安全地重新压缩。

抽象有意保持克制：当前只有一个可辨识联合 checkpoint 表示和一个 compactor contract；在出现第二个明确集成需求前，不引入 provider registry 或通用 transport framework。

上游背景

本次工作暴露了 Vercel AI SDK 缺少类型化显式压缩触发能力。M4n5ter 提交了 vercel/ai#18894，并直接促成了 vercel/ai#18895。该上游变更现已合并并发布；Maka 直接使用正式发布的 providerOptions.openai.compactionTrigger，不再保留兼容 transport 层。

验证

• 全新执行 npm ci，dependency patch 正常应用；
• npm run format:check；
• npm run build；
• npm run typecheck；
• npm test：共 6,671 项，6,645 passed、0 failed、26 skipped；
• git diff --check；
• 使用 gpt-5.6-luna 和真实 Codex OAuth 订阅验证：首次压缩、基于旧 opaque state 的滚动压缩，以及普通 provider-state 回放均成功。两次压缩请求都只携带一个位于末尾的 trigger；回放请求不携带 trigger，并准确恢复了压缩历史中的两个事实。

检查清单

• 测试覆盖本次行为与相关回归路径。
• format、typecheck、build 和全量测试均在本地通过。
• provider-native 路径已使用真实 Codex OAuth 订阅验证。

AI 披露

本变更由 Codex 辅助完成。我已完成审核，并对最终结果负责。

[likun666661]

整体方向我认可：把 provider-native compaction 建模成带 coverage 的 checkpoint 投影，而不是伪造文本 summary；RuntimeEvents 继续作为事实源，公共生命周期继续负责 coverage、digest、持久化、rolling、replay 和模型切换。这一层抽象比较克制，V2 text / V3 provider-state 的 discriminated union 也比引入 provider registry 或通用 transport framework 更合适。

不过我建议合并前处理或明确以下几点：

1. 不要仅凭 providerType === 'openai-codex' 就无条件替换文本 summarizer。 当前实现对所有 Codex subscription model 默认选择 remote compactor，但 compaction_trigger 仍不是公开 OpenAI compaction 文档中的标准入口，AI SDK 的 typed support 也还在 feat: support explicit OpenAI Responses compaction triggers vercel/ai#18895。若某个模型、账号或服务端 rollout 不支持 trigger，当前路径只会得到 provider_error / invalid_provider_state，不会退回原有文本 summarizer。建议至少增加 per-model/capability gate，或在首次 native compaction 失败时 fallback 到 text summary；更稳妥的是先 feature flag 灰度，再根据成功率默认开启。

2. 补测真正的 buildOpenAiCodexHistoryCompactor() 和 Host composition。 现有测试很好地覆盖了 transport、V3 checkpoint、replay、tool pairing 和 telemetry redaction，但没有直接覆盖：从 fullStream 提取唯一 compaction item、0/多个/畸形 item、previous V3 + newly folded tail、abort/error taxonomy，以及 Host 对 Codex/native 与其他 provider/text 的实际选择。这个函数现在是默认生产路径，只靠手工验证风险偏高；建议给 streamText/wrapLanguageModel 增加可注入 seam。

3. 需要说明与 Codex 参考请求语义的差异。 Maka 当前 compactor 只传重建后的 messages；Codex Remote V2 参考实现还会传 base_instructions、model-visible tool specs，并在请求前 trim function-call history。Maka 只压缩历史、不把当前 system/tools 固化进 checkpoint 可能是有意设计，但最好用 tool-heavy/oversized-result 的测试证明：compaction request 本身能 fit，且缺少 instructions/tools 不影响 continuation state 的质量。

从奥卡姆剃刀看，我认为工程问题定义正确，产品问题定义还需要收窄：

• 正确的问题是：Runtime 需要支持一种非文本、provider-bound、可持久化和 replay 的 history projection。这个 PR 的 V3 checkpoint 正好解决它。
• 尚未被证明的问题是：provider-native compaction 应默认替代现有文本 compaction。现有 text summary 已经解决容量/overflow；native 路径新增的价值主要是 continuation fidelity，但 PR 目前证明的是“协议可以工作”，还没有 native vs text 的质量、成功率、成本和延迟对照。

因此我建议把初始承诺定义为：Codex 可在能力确认且 native 调用成功时使用 provider-native checkpoint，否则复用现有文本 checkpoint；通过数据再决定是否默认开启。

本地验证：受影响的 387 个测试全部通过，Runtime Host typecheck 通过。整体实现质量不错，主要阻塞点是默认 rollout 和 fallback，而不是 V3 checkpoint 方向本身。

[M4n5ter]

@likun666661 Thanks for the detailed review. I checked each point against the production path and the Codex reference implementation, and pushed the confirmed fix in 8eea9a90b.

English

1. Default enablement and fallback

I am keeping provider-native compaction enabled by default for openai-codex, without adding a feature flag, per-model allowlist, or native-to-text fallback.

In Maka, openai-codex is a dedicated Codex subscription connection and therefore the capability boundary, rather than an arbitrary OpenAI-compatible model. The Codex reference implementation likewise selects Remote Compaction V2 through provider capability rather than a model-name list.

The production path has been manually verified with a real Codex OAuth subscription, covering initial compaction, rolling compaction, and tool-bearing history. A native failure follows the existing source-derived fail-open path and cannot persist invalid provider state. Silently switching to a text checkpoint would introduce two continuation semantics behind the same default capability without evidence that such a fallback is needed.

vercel/ai#18895 provides typed request support. Once released, it will allow the narrow transport compatibility layer to be removed, but it does not change this capability decision.

2. Test coverage

The missing direct coverage for compaction output extraction was valid. The new tests cover:

• exactly one complete openai.compaction item;
• zero, multiple, and incomplete compaction items;
• oversized historical Tool Result projection;
• local input_too_large failure before provider dispatch.

I did not add an injectable streamText / wrapLanguageModel seam. Such a seam would exist only to test a mocked provider response and would not validate the Codex server contract. The pure extraction and request-shaping boundaries are tested directly, while the complete production path has been exercised through the real OAuth subscription.

Existing tests continue to cover V3 checkpoint persistence, rolling coverage, native replay, model/provider switches, text-only replay fallback, overflow recovery, telemetry redaction, and settled hosted-tool pairing.

3. Request semantics and input capacity

The compaction request being able to exceed its own context window through a large Tool Result was a real issue and is fixed.

The common compactor input now carries the active inputBudget. The Codex adapter:

• estimates the complete provider-native message projection before dispatch;
• replaces older Tool Result payloads with a deterministic omission marker when needed;
• preserves every Tool Call / Tool Result pair and the later grounded conversation;
• throws the typed input_too_large failure without dispatch when the remaining projection still cannot fit.

The system prompt and tool catalog difference is intentional and is now documented in both architecture documents. Maka’s checkpoint contract covers history only: current instructions and tools are not frozen into checkpoint state, and the subsequent inference request always supplies their current versions.

The existing text summarizer does not yet consume the new input budget. That behavior predates this PR and is not required for the Codex integration, so it is tracked separately in #3013 instead of expanding this PR.

Verification

• Full Runtime suite: 2,864 tests; 2,852 passed, 0 failed, 12 skipped.
• Runtime Host CI suite: 933 passed, 0 failed.
• Runtime and Runtime Host typechecks passed.
• Biome lint and git diff --check passed.
• Real Codex OAuth verification covered initial, rolling, and tool-bearing native compaction.

The remaining red CI check is the unrelated node-pty/SQLite fd-reuse failure already tracked in #2978; this PR does not modify the affected PTY, shell-run, or storage code.

中文

感谢详细 review。我逐项对照了生产路径和 Codex 参考实现，并在 8eea9a90b 中推送了确认需要处理的修复。

1. 默认启用与 fallback

我会继续对 openai-codex 默认启用 provider-native compaction，不增加 feature flag、per-model allowlist 或 native-to-text fallback。

在 Maka 中，openai-codex 是独立的 Codex 订阅 connection，本身就是 capability boundary，并不是任意 OpenAI-compatible model。Codex 参考实现同样通过 provider capability 选择 Remote Compaction V2，而不是维护模型名称列表。

生产路径已经使用真实 Codex OAuth 订阅完成手动验证，覆盖首次压缩、rolling compaction 和包含工具历史的压缩。native 调用失败时会进入现有的 source-derived fail-open 路径，无法持久化无效 provider state。在没有实际不支持证据的情况下静默切换到 text checkpoint，会让同一个默认能力背后出现两套 continuation 语义。

vercel/ai#18895 提供的是类型化请求支持。其发布后可以删除当前范围很窄的 transport 兼容层，但不会改变这里的 capability decision。

2. 测试覆盖

缺少 compaction output 提取的直接覆盖是有效意见。新增测试覆盖了：

• 唯一且完整的 openai.compaction item；
• 0 个、多个及字段不完整的 compaction item；
• 超大历史 Tool Result 的输入投影；
• provider dispatch 前的本地 input_too_large 失败。

我没有增加 streamText / wrapLanguageModel 注入层，因为它只会用于模拟 provider response，不能验证 Codex 服务端契约。纯 output extraction 和 request shaping 边界已经直接测试，完整生产路径则通过真实 OAuth 订阅验证。

现有测试继续覆盖 V3 checkpoint 持久化、rolling coverage、native replay、模型/provider 切换、text-only replay fallback、overflow recovery、telemetry redaction，以及已结算 hosted-tool 的严格配对。

3. 请求语义与输入容量

超大 Tool Result 可能让 compaction request 自身再次超出 context window，这是一个真实问题，现已修复。

公共 compactor input 现在会携带当前 inputBudget。Codex adapter 会：

• 在 dispatch 前估算完整 provider-native message projection；
• 必要时将较旧的 Tool Result payload 替换为确定性的 omission marker；
• 保留每一组 Tool Call / Tool Result 配对以及之后的 grounded conversation；
• 如果剩余 projection 仍无法容纳，则不发送请求，直接抛出类型化的 input_too_large。

system prompt 和 tool catalog 的差异是有意设计，并已补充到中英文架构文档。Maka 的 checkpoint contract 只覆盖历史；当前 instructions 和 tools 不会被冻结进 checkpoint，后续 inference request 始终使用它们当时最新的版本。

现有 text summarizer 尚未消费新增的 input budget。这是本 PR 之前就存在的行为，也不是 Codex 集成的必要范围，因此已单独记录为 #3013，不在本 PR 中扩大修改范围。

验证

• Runtime 完整测试：2,864 项，2,852 passed、0 failed、12 skipped。
• Runtime Host CI：933 passed、0 failed。
• Runtime 与 Runtime Host typecheck：通过。
• Biome lint 与 git diff --check：通过。
• 使用真实 Codex OAuth 验证了首次、rolling 和包含工具历史的 native compaction。

目前剩余的 CI 红灯是 #2978 已记录的 node-pty/SQLite fd-reuse 问题；本 PR 没有修改受影响的 PTY、shell-run 或 storage 代码。

[M4n5ter]

I have manually reviewed the final implementation and its validation results. I am satisfied that this change is ready, take responsibility for the decision and its outcome, and have decided to merge this pull request.

This work was completed with assistance from OpenAI Codex. I reviewed and approved the resulting changes; final responsibility remains mine.