Conversation
…emplates without it) Clients such as pi send the system prompt with role "developer" by default (OpenAI's current spelling of "system"). Most chat templates do not know the role and raise "Unexpected message role", which the server returns as a 400. Map developer to system before rendering unless the template spells the role itself (gpt-oss does). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0173pf9k9fSVtwbm3f898HDt
Author
|
Re-tested against current main ( Method. This PR's head merged onto main, then the full Result: 1206 passed, 350 skipped, no new failures. The +1 over main are this PR's own tests, and they ran (not skipped). 🤖 Generated with Claude Code |
Author
|
Closing this. It was written with heavy AI assistance, and the maintainers have indicated that do not want such contributions. The description and the diff stay here for anyone who wants to pick the idea up. |
This was referenced Sep 15, 2026
MT-z
added a commit
to MT-z/FreeToken
that referenced
this pull request
Sep 16, 2026
…easured on this box) gdevenyi's FlashML-org#391, open since 2026-09-05. OpenAI's `developer` role is the current spelling of `system`; a template that does not spell it raises, and the request dies at `could not encode request: Unexpected message role.` The PR maps it to `system` unless the template handles the role itself. Reproduced before taking it, on this branch, Ornith-1.5-35B-A3B-NVFP4: role=developer 400 could not encode request: Unexpected message role. role=system 200 and after the merge both answer 200 with the same text ('Paris is the capital of France.'). `/v1/responses` already had its own mapping for codex (`responses_api.py:258`); this is the chat-completions path, which did not. Conflicted twice in `tokenizer/tokenize.py`, both "two things added at the same spot" -- the vision bundle on this side, `_map_developer_role` on the PR's -- so both are kept. The Anthropic route still answers 200. Full suite: 11 failed, 1861 passed, 60 skipped -- the same 11, plus the test the PR adds. Not applicable on this box, checked and set aside in the same pass: FlashML-org#275 the GDN in_proj per-tensor FP8 shape it fixes IS Ornith's (config group_0 lists linear_attn.in_proj_qkv / in_proj_z as FP8), but the PR is the qwen3_5 *dense* loader and the MoE path already handles it -- Ornith loads and serves today FlashML-org#294 needs experts NVFP4 with the shared expert per-tensor FP8; Ornith's group_1 puts the shared expert in NVFP4 alongside the routed experts, so the shape does not match FlashML-org#296 compressed-tensors (llm-compressor); Ornith is quant_method=modelopt FlashML-org#300 the KV ladder is `--max-running-requests 1` only, and a Claude Code request's input + max_output_tokens lands on a high rung on the first call. What it automates IS real here though -- see FINDINGS: KV 98,304 buys +17% decode at 87k of context, because slots/layer crosses experts_for_90pct Assisted-by: Claude Opus 5
gdevenyi
added a commit
to gdevenyi/FreeToken
that referenced
this pull request
Sep 18, 2026
gdevenyi
added a commit
to gdevenyi/FreeToken
that referenced
this pull request
Sep 19, 2026
gdevenyi
added a commit
to gdevenyi/FreeToken
that referenced
this pull request
Sep 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this fixes
A request whose system prompt uses the OpenAI
developerrole (the current spelling ofsystem; pi sends it by default foropenai-completionsproviders,compat.supportsDeveloperRole: true) fails with400 could not encode request: Unexpected message role.on every model whose chat template does not spell the role, which is most of them (Qwen3.8-Flash-Next, Qwen3.x, GLM, ...). The template raises, the server relays it.TokenizeManager._rendernow mapsdevelopertosystembeforeapply_chat_template, unless the template mentionsdeveloperitself (gpt-oss renders the two roles differently and keeps its input). The mapping copies the message; the request is not mutated.Testing
tests/tokenizer/test_tokenize.py::test_developer_role_maps_to_system_unless_the_template_knows_it.{"role": "developer", ...}went from the 400 above to the sameprompt_tokensas thesystemspelling.🤖 Generated with Claude Code
https://claude.ai/code/session_0173pf9k9fSVtwbm3f898HDt