deepseek-v4-pro: no working reasoning-budget control (only all-or-nothing thinking)

[justrach]

Summary

On the OpenAI-compatible gateway (POST /v1/chat/completions) with deepseek-v4-pro, end-to-end latency is dominated by reasoning tokens, and there's no working way to get a bounded amount of reasoning. The only effective control is turning reasoning fully on or fully off — no middle setting.

What I measured

All via POST /v1/chat/completions, model: deepseek-v4-pro, standard OpenAI-compatible client.

• Reasoning is the latency driver, not throughput. Streaming rate is ~90 tok/s (healthy). But the model emits hidden reasoning tokens before the answer:
  • simple prompt → ~149 reasoning tokens → ~4s
  • rich prompt → ~1000–1700 reasoning tokens (+ large output) → ~40s
• Concurrency is fine (not the problem): 6 concurrent requests run in parallel — ~7s wall for light prompts, ~55s for heavy (serial would be ~240s). Per-key throughput is good; this is purely per-request reasoning cost.

The gap: no working reasoning-effort / budget knob

param tried	result
thinking: { type: "disabled" }	✅ works — reasoning_tokens=0, ~2s replies
reasoning_effort: "minimal"	❌ HTTP 400 invalid_request_error — "Failed to deserialize the JSON body"
reasoning_effort: "low"	⚠️ accepted but ignored — reasoning tokens not reduced (sometimes higher)
reasoning: { effort: "low" }	⚠️ partial / inconsistent (~400 reasoning tokens, not reliably controllable)

So the only real choice is ~2s with zero reasoning (which hurts quality on logic / multi-constraint tasks) or ~40s with full reasoning. There's no "give me a small reasoning budget and cap latency" option in between.

Request

A documented, working reasoning-budget control for deepseek-v4-pro, e.g. one of:

• reasoning: { max_tokens: N } — a hard cap on reasoning tokens, and/or
• a real reasoning_effort: "minimal" | "low" | "medium" | "high" enum that actually scales reasoning (and doesn't 400 on "minimal").

This lets latency-sensitive apps trade a little reasoning for a lot of speed instead of the current all-or-nothing.

Environment

• Endpoint: https://gateway.codegraff.com/v1/chat/completions
• Model: deepseek-v4-pro
• Auth: Bearer cg_sk_… (redacted)

[justrach]

Confirmed — reproduced today against deepseek-v4-pro on the gateway, and the findings match this report:

• reasoning_effort:"minimal" → 400 (unknown variant minimal; the accepted set is high/low/medium/max/xhigh)
• reasoning_effort:"low" is accepted but doesn't reduce reasoning — the low/medium/high scale isn't even monotonic (low produced more reasoning than high)
• reasoning.max_tokens / budget_tokens (and the thinking.* equivalents) are ignored

So today it really is all-or-nothing: thinking:{type:"disabled"} (0 reasoning, ~4s) vs. full reasoning, nothing in between.

This is a gateway-side fix (request parsing + how effort/budget maps to the upstream model), not the codegraff client — so I'm tracking it internally on the gateway and closing this here. Two fixes are queued: (1) accept minimal and make low < medium < high actually monotonic, and (2) honor a real reasoning budget (reasoning.max_tokens). I'll follow up here when it ships.