fix(runtime): gate Responses replay on encrypted reasoning - #2518
Conversation
041fb5b to
17be48c
Compare
|
Upstream update: vercel/ai#18515 has merged, and the request-side plaintext reasoning fix is available in @ai-sdk/open-responses 2.0.27. Maka currently uses @ai-sdk/openai for this Responses path, so upgrading the existing dependency alone would not consume that fix. The full solution needs a dialect-specific provider adapter and a plaintext replay contract for DeepSeek, while this PR remains the encrypted-only containment boundary. @Astro-Han, if that direction looks right, I’ll open a separate Draft PR from current main for the complete Open Responses integration. Once the replacement proves the real request body and preserves encrypted OpenAI replay, we can decide whether to merge this containment first or mark it superseded. |
|
The complete dialect-specific implementation is now up as Draft #2972. It includes the fail-closed encrypted-content boundary from this PR, while also enabling the full plaintext reasoning round trip through |
Astro-Han
left a comment
There was a problem hiding this comment.
Yes, this direction is right, and I'd like to merge it. Answering your question first since you're waiting on it.
Don't build a dialect-specific adapter — take @ai-sdk/open-responses. For us that is a net deletion: it reads the standard content[].reasoning_text shape natively, so openai-responses-plaintext-reasoning-transport.ts retires with it, and 2.0.27 covers the request side you'd otherwise be writing. Anything missing there is worth filing upstream — the AI SDK turns those around quickly, and we'd rather carry an upstream version than a parallel implementation.
The target shape is already documented, so no reverse-engineering needed. DeepSeek's Responses API accepts reasoning as an input item type, carrying "a list of reasoning_text content parts" — the mirror of what it emits, plaintext, no encryption in the picture:
{ "type": "reasoning", "content": [{ "type": "reasoning_text", "text": "..." }] }Worth recording in #2513: the thinking-mode guide says that when tool calls are present the reasoning must be passed back, and omitting it returns a 400. We don't see those, because DeepSeek recovers it server-side — we've tested that. So nothing is degrading today. But we're relying on undocumented server behavior for a field the contract requires, which is a real gap rather than a nice-to-have, and it fails on tool loops specifically if that ever tightens.
Things I verified so you don't have to. The wire body is unchanged: store: false makes the SDK filter reasoning items with no encrypted content anyway, so dropping them earlier produces the same request, with item order unaffected. Narrowing null is safe — the SDK writes encrypted_content ?? null, so null has always meant "nothing to replay". Genuine OpenAI encrypted replay still works: encrypted content arrives at reasoning-end rather than on the deltas, and the itemId merge in ai-sdk-backend.ts:2257-2282 keeps it. The rename is complete, no residue.
One thing before merge. #2513 sets the acceptance check "a fake-fetch assertion checks the actual request body, not only Maka's intermediate model messages" — the new test asserts the intermediate messages. Detail inline; responses-wire-contract.test.ts:86-100 already has the pattern.
Two smaller notes. reasoningReplayContract still returns the same kind for every openai-responses wire, so the contract isn't dialect-aware yet — the data guard is what does the work. And the narrowing reaches more than DeepSeek: volcengine-agent-plan, openai-responses-compatible, gpt-5* and grok-4.5 all ride this wire. Conservative for all of them, but worth a line in the description.
Review assistance: Claude Code (Opus) ran parallel passes over architecture and test quality and drafted this. I verified the wire-body equivalence, the null semantics, the DeepSeek docs, and the test's assertion target myself.
中文
方向是对的,我倾向合入。先回答你等着的那个问题。
不要自建 dialect-specific adapter,直接上 @ai-sdk/open-responses。 对我们来说这是净删代码:它原生读标准的 content[].reasoning_text,所以 openai-responses-plaintext-reasoning-transport.ts 可以跟着退役,而 2.0.27 已经覆盖了你原本要自己写的请求侧。那边缺什么值得提给上游——AI SDK 迭代很快,我们宁可跟一个上游版本,也不愿维护一套并行实现。
目标形状文档里就有,不用反推。DeepSeek 的 Responses API 接受 reasoning 作为 input item 类型,内容是「a list of reasoning_text content parts」——就是它吐出来那个形状的镜像,明文,完全不涉及加密:
{ "type": "reasoning", "content": [{ "type": "reasoning_text", "text": "..." }] }有一点值得记进 #2513:thinking mode 文档写明,存在工具调用时 reasoning 必须回传,不回传返回 400。我们没有遇到,是因为 DeepSeek 在服务端替我们找回了——这个我们实测过。所以今天没有任何东西在掉分。但我们是在用一个未文档化的服务端行为,去顶一个契约要求的字段,这是实打实的缺口而不是锦上添花,而且一旦对方收紧,只会在工具回合上炸。
我核过、你不用再验的部分。 线上请求体不变:store: false 本来就会让 SDK 过滤掉没有加密内容的 reasoning item,提前丢弃产生的是同一个请求,item 顺序也不受影响。收窄 null 是安全的——SDK 写的是 encrypted_content ?? null,null 一直就是「没有可重放的东西」。真正的 OpenAI 加密重放仍然正常:加密内容是在 reasoning-end 而不是 delta 上到达的,ai-sdk-backend.ts:2257-2282 的 itemId 归并会保住它。重命名彻底,没有残留。
合并前有一件事。 #2513 里你自己定的验收标准是「a fake-fetch assertion checks the actual request body, not only Maka's intermediate model messages」,而新测试断言的正是中间态 model messages。细节在行内;responses-wire-contract.test.ts:86-100 已经有现成的模式。
两条小的。reasoningReplayContract 对所有 openai-responses wire 仍然返回同一个 kind,所以 contract 还不是 dialect-aware——真正做判别的是那道数据守卫。另外这次收窄覆盖的不止 DeepSeek:volcengine-agent-plan、openai-responses-compatible、gpt-5* 和 grok-4.5 都走这条 wire,对它们都是保守的,但值得在描述里写一句。
评审协助说明:Claude Code (Opus) 并行跑了架构与测试质量两轮审查并起草了本条评论。请求体等价性、null 语义、DeepSeek 文档,以及这个测试到底断言了什么,都是我自己核实的。
|
That's the test I was after. Real Merging. The wire body is unchanged either way, so this is safe to land now. 中文这正是我想要的那个测试。真 合入。两种情况下线上请求体都不变,现在落地是安全的。 |
Summary
Make the current Responses replay boundary explicit: Maka projects a reasoning item into the native OpenAI Responses shape only when it carries non-empty provider-issued encrypted content. Plaintext reasoning remains stored and visible, but is no longer misrepresented as an OpenAI encrypted item that the SDK warns about and drops.
Genuine encrypted OpenAI Responses reasoning and its tool ordering remain unchanged.
Part of #2513. The complete plaintext reasoning round trip remains tracked by Draft #2972.
Boundary and scope
The plaintext reasoning serializer fix landed upstream through vercel/ai#18515 in
@ai-sdk/open-responses. This PR does not makereasoningReplayContractdialect-aware: every model on the currentopenai-responseswire still has the same encrypted replay contract, and the data guard inai-sdk-backend.tsdecides whether a stored item is safe to project.That guard conservatively reaches every model currently using this wire, including
volcengine-agent-plan,openai-responses-compatible,gpt-5*, andgrok-4.5. It only removes reasoning items that the native OpenAI serializer cannot replay withstore: false; non-empty encrypted OpenAI reasoning continues unchanged.The full plaintext path belongs in an
open-responses-plaintextsibling backed by@ai-sdk/open-responses, as proposed in Draft #2972, rather than in a parallel serializer here.Verification
null, and emptyreasoningEncryptedContentcases omit the reasoning item while preservingmessage → function_call → function_call_outputstructure and values.npm --workspace @maka/runtime test— 2,788 passed, 6 skipped, 0 failednpm run build:testnpm run format:checknpm run lintChecklist
Does this PR entail a change in behavior?