Skip to content

buzz-agent ignores reasoning_effort on vLLM OpenAI-compat endpoints — thinking can't be turned off #7831

Description

@knowhao

Motivation
What problem does this solve? Who runs into it and when?
Agents configured with an OpenAI-compatible provider (e.g. Qwen3 served by vLLM) can't have thinking disabled. BUZZ_AGENT_THINKING_EFFORT=none has no effect — the model still thinks.

Root Cause
The OpenAI-compat path in buzz-agent sets reasoning control only via the OpenAI reasoning_effort body field. llama.cpp's HTTP server honours reasoning_effort (it maps it onto Qwen3's /think switch), but vLLM's OpenAI-compat endpoint ignores reasoning_effort and reads thinking control exclusively from the Qwen3 chat-template variable chat_template_kwargs.enable_thinking. So none → reasoning_effort:"none" is dropped by vLLM and the model runs at its default (thinking on).

Proposed solution
Also emit chat_template_kwargs.enable_thinking alongside reasoning_effort in the OpenAI-compat body: false for none/minimal, true for any on-level. Each backend reads only the field it understands, so sending both is correct on every backend and a no-op where a field is ignored. When no effort is set, neither field is emitted and the provider default is preserved.

Alternatives considered
What other approaches did you consider?
Relying on reasoning_effort only (current) — breaks on vLLM.
A per-backend compat flag instead of always sending both — more config surface; sending both is simpler and safe since each backend reads only its own field.

Additional context
Anything else that helps — links, screenshots, prior art.
Verified against a live vLLM 0.27 Qwen3 build (off: 73→5 completion tokens; enable_thinking:false honoured, reasoning_effort/top-level enable_thinking ignored) and a llama.cpp server (both reasoning_effort and chat_template_kwargs honoured). Prior art: vllm-project/vllm#15524, vllm-project/vllm#288, zed-industries/zed#58289.


Before opening: please search open issues and PRs for duplicates — link the closest one, or say "none found".

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions