Conversation
…ompat buzz-agent's openai-compat path sets reasoning control only through the OpenAI `reasoning_effort` body field. That field is honoured by llama.cpp's HTTP server (which maps it onto Qwen3's /think switch) but is *ignored* by vLLM's OpenAI-compat endpoint, which reads thinking control exclusively from the Qwen3 chat-template variable `chat_template_kwargs.enable_thinking`. As a result, agents pointed at a vLLM-served Qwen3 could not be switched off: BUZZ_AGENT_THINKING_EFFORT=none sent reasoning_effort="none", the server dropped it, and the model ran at its default (thinking on) regardless. Also emit chat_template_kwargs.enable_thinking (false for none/minimal, true for any on-level) alongside reasoning_effort. Each backend reads only the field it understands, so sending both keeps the on/off intent correct across backends and is a no-op where a field is ignored. When no effort is set, neither field is emitted and the provider's default is preserved. Signed-off-by: knowhao <hao.a.le@gmail.com>
🔐 Codex Security Review
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The OpenAI-compat agent path set reasoning control only through the OpenAI reasoning_effort body field. llama.cpp's HTTP server honours that field, but vLLM's OpenAI-compat endpoint ignores it and reads thinking control exclusively from the Qwen3 chat-template variable chat_template_kwargs.enable_thinking. Agents pointed at a vLLM-served Qwen3 therefore couldn't be switched off: BUZZ_AGENT_THINKING_EFFORT=none sent reasoning_effort:"none", the server dropped it, and the model ran at its default (thinking on) regardless.
This makes openai_body also emit chat_template_kwargs.enable_thinking — false for none/minimal, true for any on-level — alongside the existing reasoning_effort. Each backend reads only the field it understands, so sending both keeps the on/off intent correct across llama.cpp and vLLM, and is a no-op where a field is ignored. When no effort is set, neither field is emitted and the provider's default is preserved.
Related issue
Fixes #7831
Testing
No UI changes (backend request-body change). Verified two ways:
Unit tests in buzz-agent: 545 pass, including new tests asserting enable_thinking=false for none/minimal, enable_thinking=true for every on-level, and that no chat_template_kwargs is emitted when effort is unset. just fmt-check and just clippy pass; DCO signed.
Live end-to-end against real servers: a vLLM 0.27 Qwen3 build — with the fix, none drops completion tokens 73→5 (thinking off) and on-levels restore it; a llama.cpp server — none stays off and medium/high still vary depth as before, confirming the added field is a no-op there.