serve/glmtools.py: GLM's own tool protocol, beside kimitools

[hendrikras]

A container served from chat.json could declare no tools unless its tokenizer carried Kimi K2's five control tokens, so GLM-5.3-Flash — whose tokenizer instead carries <tool_call>, <arg_key> and <arg_value> as single specials — answered a tools request with a 400 naming the field. That refusal was correct about the declarative format and wrong about the container: the grammar was there, in specials.json and in the release's own chat_template.jinja, and four prefix/suffix strings were never going to be where it lived.

So the protocol gets the same thing K2's got: a module of its own, beside xtml.py and kimitools.py, and an oracle that diffs its rendering against the template that defines it.

• glmtools.detect() requires all nine markers to be single tokens, the same all-or-none gate kimitools keeps: half of this XML renders as ordinary text, so a partial set is a different protocol, not a smaller one.
• The grammars differ where it matters. A Kimi call is an id and a JSON block inside a section; a GLM call is flat XML, the name after the opening tag and one <arg_key>/<arg_value> pair per argument, no id — so calls come back named by position. A Kimi result is a system turn named for the tool; a GLM result is an <|observation|> turn wrapping <tool_response> blocks, and consecutive results share one opener, which chatfmt now emits by looking back a message.
• ChatFormat carries which protocol resolved and branches on it; the container with neither is still refused by name, now naming both.
• PlainParser takes the tool reader from the caller, and server.new_parser hands it glmtools.ToolParser on a GLM container. api.py maps GlmToolError to a 400 the way it maps KimiToolError.

The security boundary survives unchanged: the tags go out as markup segments, the signatures and argument values as plain text, so a caller's JSON cannot forge a control token the message content is not.

tests/serve/test_glm_upstream.py is the oracle, GLM_DIR naming the release, the way K2_DIR does for test_chatfmt_upstream; run.sh grew the matching check. The template needs jinja2's loopcontrols extension — it uses {% break %}.

Docs updated where they said GLM is refused: README, CLAUDE.md, docs/SERVE.md, docs/GLM.md.

[mfethe1]

Independent third-party validation at the current PR head (e354221517159f5ffb30ec5d26723fed54da812b):

• PR: 263 repository unittests passed, 5 skipped
• Base c66c7b3febf261590abea540e74602d8e7759d62: 244 passed, 4 skipped
• Exact test-ID diff: 19 added, 0 removed
• Five targeted mutations were detected by the tests
• A three-case oracle using the fetched upstream GLM tool template plus this PR's parser/renderer passed

Scope: this is ad-hoc Tier B software validation, not canonical CI green. The oracle executes the real PR parser/renderer but uses a synthetic tokenizer/engine; no licensed real GLM model or container was run. So this supports the narrower claim that the PR addresses the reported tool-protocol behavior under tests, not end-to-end model compatibility or merge readiness.

Disclosure: the validation was executed and analyzed by an AI agent under my direction; I reviewed the evidence and am sharing the scoped result.

[mfethe1]

One concrete CI follow-up to the validation above, checked against the unchanged head e354221517159f5ffb30ec5d26723fed54da812b: the upstream-template oracle can run without model weights. tests/serve/test_glm_upstream.py needs the release's chat_template.jinja and Jinja2; it uses FakeEngine for the tokenizer/engine boundary. Both the test and tests/run.sh currently allow the check to skip when its prerequisites are absent.

Would you prefer a small follow-up that makes this a required, CPU-only CI check? Suggested scope:

• Fetch only the template from an immutable upstream revision and verify its SHA-256 (or vendor it with appropriate license/provenance, if preferred).
• Supply GLM_DIR and a pinned Jinja2 dependency, then run tests.serve.test_glm_upstream.
• Require all three oracle tests to execute with zero skips; missing dependencies/template or unresolved markers must fail this designated CI job rather than produce a green skipped run.

That would preserve the existing optional local workflow while making the independent grammar comparison repeatable in CI. This is a proposal, not a claim that CI has been wired or run. The earlier result remains ad-hoc verification passed; this is not canonical suite green, and this template/FakeEngine check does not establish real-model compatibility.

Happy to keep it as a narrow follow-up rather than expand this PR. Is a pinned download or a licensed vendored fixture preferable?

Disclosure: AI-agent-assisted source review and drafting, posted with human authorization.

[hendrikras]

Vendored template vs pinned download

• Chose vendoring the chat_template.jinja file under tests/serve/glm_upstream/ instead of fetching it in CI.

Rationale:

• Makes the CI job fully self‑contained and reproducible without outbound network calls.
• Avoids potential flakiness or policy issues around fetching from Hugging Face / GitHub in CI.
• Keeps the test runnable from a fresh clone with no extra setup.

The vendored file is taken from zai-org/GLM-5.3-Flash @ eb9eb208eb0d988989d07a6a12d0fdeb5f52574a (Hugging Face). The directory’s README.md includes:

• Exact source URL and commit hash.
• Licensing note:
• The template lives in the HF model repo, which carries an MIT notice (reproduced in the README).
• The GitHub code repo zai-org/GLM-5 is Apache‑2.0 but does not ship this file directly.

A one‑liner that this is used only with FakeEngine in CI; no real GLM model or weights are involved.

This gives clear provenance and respects the license as stated where the file actually comes from (the model repo).

Pinning Jinja2

• Added requirements-test.txt with Jinja2==3.1.6.
• The new glm_upstream_oracle CI job installs this pinned dependency.

Rationale:

• Ensures stable, repeatable rendering behavior for the oracle.
• Avoids subtle differences across Jinja2 versions affecting the tool‑protocol checks.

Strict mode: no silent skips in CI

• Introduced CI_GLM_ORACLE_STRICT=1 for the CI job.

In this mode:

• Missing Jinja2, missing template, or unresolved markers fail the job instead of skipping.
• The job asserts that all three oracle checks ran; it cannot be “green because it did nothing”.

Local development behavior is intentionally softer:

• tests/run.sh defaults to the vendored template and runs out of a fresh clone.
• If Jinja2/uv is absent, it still skips, but with a clear warning.
• GLM_DIR can still override with a real release directory for local experiments.

This split keeps CI strict while preserving a convenient local workflow.

Scope

This is narrow infrastructure to support the tool‑protocol validation from #61.

It does not:

• Change production parser/renderer logic.
• Test end‑to‑end compatibility with real GLM models or weights.

It supports the narrower claim that the PR addresses the reported tool‑protocol behavior under tests, as per the original ad‑hoc validation.

Disclaimer: GLM-5.3-flash was used to implement, perplexity.ai was used to draft this message.

[mfethe1]

Thanks @hendrikras — I independently exercised the revised strict oracle at f9a46b5a6c82945e9dac604874bfeaa0c1c445c0 on macOS arm64 / Python 3.11.15 with Jinja2 3.1.6.

Positive control: CI_GLM_ORACLE_STRICT=1 GLM_DIR=tests/serve/glm_upstream python -m unittest tests.serve.test_glm_upstream -v ran 3 tests, zero skips, OK (exit 0). A fresh repeat after the negative controls also ran all 3 successfully.

Negative controls, each in a separate subprocess:

• GLM_DIR pointing to an empty directory: exit 1, strict missing-template assertion.
• A separate isolated virtualenv without Jinja2: exit 1, strict missing-dependency assertion.
• Remove token 21 (<tool_call>) from the imported oracle's synthetic GLM_MARKERS mapping before loading the suite: exit 1, strict unresolved-markers assertion.
• Remove test_tool_result_matches_the_template from the imported test class before suite discovery: the remaining 2 tests pass, but teardown fails with strict oracle ran 2 of 3 checks; exit 1.

The last two are deliberate in-memory fault injections, not modifications to production code. These checks confirm the designated oracle fails closed on the prerequisite/registration cases we discussed.

Provenance: I fetched chat_template.jinja directly from the documented upstream revision eb9eb208eb0d988989d07a6a12d0fdeb5f52574a. Its 10,950 bytes match the vendored fixture exactly; SHA-256 is 0c4099f3382d6c92700dfb99725025360966fd73032f0ecf32377c0d9e6309c5.

One small license-notice correction: the LICENSE fetched from that same revision begins Copyright (c) 2026 Z.AI Co., Ltd, while the vendored README substitutes Copyright (c) Z.ai / zai-org. Could you preserve the original copyright notice verbatim (ideally vendor the exact upstream LICENSE alongside the template)? The MIT text explicitly asks that the above copyright notice be included. This does not change the oracle result.

Scope: independent CPU-only renderer/template validation through FakeEngine. I ran the targeted oracle, not the full canonical suite or the hosted Python 3.12 CI job; this is not real-model compatibility, GPU validation, or a merge-readiness claim. No weights were loaded.

Disclosure: validation and this reply were performed by an AI agent under human authorization.

[hendrikras]

added LICENCE file and edited README.md file

[marcobambini]

I cannot merge this as-is because the GLM tool parser loses argument types. I reproduced this by rendering a call with call_section() and feeding its segments into ToolParser: {"count": 3, "enabled": true, "items": [1, 2]} comes back as {"count": "3", "enabled": "true", "items": "[1, 2]"}. _finish_arg() always stores the raw text as a string, so valid calls can fail tool schema validation.

Please preserve argument types according to the tool schema, including declared string values, and add regression coverage before merging. The existing GLM rendering oracle passes but does not cover this parser issue.

[mfethe1]

Confirming Marco's report on 3da511f — reproduced independently on macOS arm64 / Python 3.11, and I think the root cause is narrower than it looks.

Reproduction. Rendering with call_section() and feeding the segments back through ToolParser on the untouched head:

sent {"n": 5, "fl": 1.5, "b": true, "a": [1, 2], "o": {"k": 1}, "s": "7", "nul": null}
got  {"n": "5", "fl": "1.5", "b": "true", "a": "[1, 2]", "o": "{\"k\": 1}", "s": "7", "nul": "null"}

Every non-string arrives as its own source text.

Why. This is not a lossy parse — it is a protocol fact. GLM's grammar states an argument as <arg_value>text</arg_value> with no type attached, unlike the XTML path, where _typed_value() already reconstructs values because the markup carries a type. _finish_arg() therefore has nothing to reconstruct from. The caller's tools schema is the only place the type still exists, and it was never reaching the parser.

What I'd suggest, in the shape the repo already uses:

• arg_types(tools) in glmtools.py → {tool: {arg: declared type}}.
• Rename regions._typed_value → typed_value and reuse it; no second converter.
• ToolParser(types), applied in _finish_arg().
• Carry tools on Prompt and pass it to new_parser() — this is the only wiring change.

One judgement call worth flagging: I convert only when a type was declared. An undeclared argument keeps the text it was, because guessing types here is how a zip code "02134" becomes 2134. That also keeps test_a_non_string_value_is_read_back_as_json meaningful rather than silently redefining it.

Verified on the full matrix you listed — number, boolean, array, object, null all round-trip, and a declared string "02134" stays a string. tests/serve/test_glmtools.py 18 passed (2 added: declared-type round-trip, undeclared-stays-text); test_chatfmt.py 44 passed, test_regions.py 31 passed, test_glm_upstream.py 3 passed.

test_server.py shows 4 errors — but those are libwaste not found on the pristine head too, identical set, so they are environmental here, not from this change.

Happy to open this as a PR against your branch, @hendrikras, or leave it for you — your call, I don't want to duplicate work you've already started.