fix(server-stateless): requiredCapabilities is a ClientCapabilities object, not an array - #376
Merged
pcarleton merged 1 commit intoJul 1, 2026
Conversation
…bject, not an array
The sep-2575-server-rejects-undeclared-capability check required
error.data.requiredCapabilities to be an array containing 'sampling'. The
schema's MissingRequiredClientCapabilityError defines it as a
ClientCapabilities object ({ "sampling": {} }), as do this repo's own
spec-types and the tasks/required-task-error scenario, so a spec-correct
server could not pass the check. It shipped because the positive -32021
branch had no self-test.
- stateless.ts: accept the object shape; each capability value must itself
be an object ({ "sampling": null } fails). State the shape in the
scenario's implementer-facing description.
- examples/servers/typescript/everything-server.ts: emit the object.
- stateless.test.ts: self-tests for object-passes / array-fails /
null-value-fails, sharing one capability-probe mock helper.
commit: |
10 tasks
pcarleton
approved these changes
Jul 1, 2026
panyam
added a commit
to panyam/mcpkit
that referenced
this pull request
Jul 2, 2026
Bump conf-upstream-main to modelcontextprotocol/conformance@794dcab (alpha.7 -> alpha.9) and merge the same upstream into the fork feature branches (pending, chore/sep-2640-yaml). Stateless goes 25/26 -> 30/30: upstream `modelcontextprotocol/conformance#376` fixed the former array-vs-object requiredCapabilities test, which mcpkit was always correct on. The testconf-stateless target now requires zero failures instead of tolerating that one known fail. Docs: refresh CONFORMANCE.md + UPSTREAM_AUDIT.md against the new pin, update the stateless note in local-suites.yaml, and fix stale conf-main -> conf-skills skills-path references (the skills suite defaults to ../conf-skills on branch chore/sep-2640-yaml). Claude-Session: https://claude.ai/code/session_018j9S7ia7xxKnrTRD9G4y7y
8 tasks
pcarleton
pushed a commit
that referenced
this pull request
Jul 27, 2026
Every JSON-RPC message the harness sends or receives is now validated (ajv) against the vendored schema.json for the run's spec version, at the wire choke points: sendStatelessRequest (outbound request, JSON response, SSE events), connectStateful (request/result/notifications), and both mock servers (inbound client requests, outbound responses/handler results). Beyond the JSONRPCMessage envelope, messages are validated against the most specific definition the schema has: typed requests/notifications by method const, typed error responses by error.code const (e.g. -32021 -> MissingRequiredClientCapabilityError, the #376 hallucination), and typed results via the XxxRequest -> XxxResult pair, with SEP-2322 InputRequiredResult discriminated on resultType. Violations are recorded per scenario and surfaced by the runners as a synthetic wire-schema-valid check (FAILURE when the implementation under test sent an invalid message) plus a wire-schema-harness-error check when the invalid message was harness-authored. A vitest-wide afterEach guard fails any self-test that recorded violations, so hallucinated fixtures can't ship (#376, #271). Opt-outs are explicit and greppable: per-call skipValidation for deliberately malformed traffic, takeWireViolations() drains in tests that exercise deliberately broken fixtures. Offenders the new validation caught, now fixed: - everything-server stateless path omitted resultType (and the caching hints on server/discover) required by the draft schema - sep-2164-empty-contents fixture omitted resultType/ttlMs/cacheScope (its intended violation, empty contents, is schema-valid prose-only) - connection/mock-server unit-test fixtures used schema-invalid stubs (tools/call without name, ListToolsResult without required members, Tool without inputSchema, ProgressNotification without progressToken, ElicitRequest without message/requestedSchema) - http-standard-headers test mocks returned bare { tools: [] } results - input-required-result-validate-input deliberately sends malformed inputResponses; now marked skipValidation Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JHNrpU3Vr6Q1rPZiji7zLY
pcarleton
added a commit
that referenced
this pull request
Jul 27, 2026
…399) * Vendor schema/{version}/schema.json alongside schema.ts sync-schema now copies the spec repo's JSON Schema for each version into src/spec-types/{version}.schema.json (same pinned SHA recorded in SOURCE: f817239). These are the inputs for runtime wire-message validation. Vendored verbatim, so they join schema.ts in .prettierignore. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JHNrpU3Vr6Q1rPZiji7zLY * Validate all wire messages against the per-version spec JSON schema Every JSON-RPC message the harness sends or receives is now validated (ajv) against the vendored schema.json for the run's spec version, at the wire choke points: sendStatelessRequest (outbound request, JSON response, SSE events), connectStateful (request/result/notifications), and both mock servers (inbound client requests, outbound responses/handler results). Beyond the JSONRPCMessage envelope, messages are validated against the most specific definition the schema has: typed requests/notifications by method const, typed error responses by error.code const (e.g. -32021 -> MissingRequiredClientCapabilityError, the #376 hallucination), and typed results via the XxxRequest -> XxxResult pair, with SEP-2322 InputRequiredResult discriminated on resultType. Violations are recorded per scenario and surfaced by the runners as a synthetic wire-schema-valid check (FAILURE when the implementation under test sent an invalid message) plus a wire-schema-harness-error check when the invalid message was harness-authored. A vitest-wide afterEach guard fails any self-test that recorded violations, so hallucinated fixtures can't ship (#376, #271). Opt-outs are explicit and greppable: per-call skipValidation for deliberately malformed traffic, takeWireViolations() drains in tests that exercise deliberately broken fixtures. Offenders the new validation caught, now fixed: - everything-server stateless path omitted resultType (and the caching hints on server/discover) required by the draft schema - sep-2164-empty-contents fixture omitted resultType/ttlMs/cacheScope (its intended violation, empty contents, is schema-valid prose-only) - connection/mock-server unit-test fixtures used schema-invalid stubs (tools/call without name, ListToolsResult without required members, Tool without inputSchema, ProgressNotification without progressToken, ElicitRequest without message/requestedSchema) - http-standard-headers test mocks returned bare { tools: [] } results - input-required-result-validate-input deliberately sends malformed inputResponses; now marked skipValidation Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JHNrpU3Vr6Q1rPZiji7zLY * Validate the stateful wire at the SDK transport, close review-panel gaps Address the review-panel findings on the wire-schema validation PR: - Stateful wire: hook the SDK transport inside connectToServer, wrapping send/onmessage so every raw wire message in both directions is validated for the scenario's spec version -- including the initialize handshake, server->client requests (elicitation/sampling), and the harness's responses to them. Request ids are correlated per direction so responses validate against their typed result definition (e.g. an outbound ElicitResult). The reconstructed stand-ins in connectStateful (fake id: 0 envelopes, Zod-reparsed results) are removed; the hook sees the real bytes. The four scenarios that call connectToServer directly (elicitation-defaults, elicitation-enums, tools, lifecycle) now pass ctx.specVersion and get validation for free; the runner surfaces their violations through the existing recorder. Known remaining gap (documented in the module doc): the client-auth scenarios' bespoke express mock is not instrumented. - skipValidation now skips only the request direction; the implementation's response to a deliberately malformed request is still validated, with a narrow carve-out tolerating the JSON-RPC 2.0 id: null an error response must carry when the request could not be processed (the MCP schema's RequestId forbids null). Covered by new unit tests pinning both the still-validated response and the narrowness of the carve-out. - negative-mrtr.test.ts: the file-wide drain now asserts every recorded violation is implementation-origin (the deliberately broken fixture) and throws on any harness-origin violation. - everything-server: hoist the cacheable-methods set to module scope and replace the per-request res.json monkey-patch with an explicit sendStatelessJson(res, method, payload) helper applied at each dispatch site. - Pin the extracted dispatch maps (methodDefs/errorDefs, e.g. -32021 -> MissingRequiredClientCapabilityError under the draft) per spec version in wire-schema.test.ts, so a schema sync that breaks the const extraction fails loudly instead of degrading to envelope-only validation. Also note the batch branch's requestMethod forwarding limitation. Offender the transport hook flushed out: the everything-server's test_elicitation_sep1034_defaults tool sent a number-typed elicitation field with default: 95.5, which the 2025-11-25 schema forbids (NumberSchema.default is typed integer there; widened to number only in the draft). The fixture and the scenario's expected default are now the integer-valued 95, with comments explaining why. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JHNrpU3Vr6Q1rPZiji7zLY * Condense comments to three lines max Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JHNrpU3Vr6Q1rPZiji7zLY * Re-vendor spec schemas at pinned SHA 71e30695 Regenerate src/spec-types/*.schema.json via npm run sync-schema at the SHA already recorded in SOURCE. The previously vendored draft schema predated spec PR #3002, so it still required DiscoverResult.serverInfo at the top level; the mock server now correctly reports serverInfo via result _meta, which the stale schema rejected. * Scope the wire-violation recorder per scenario for parallel suite runs Client --suite runs execute scenarios concurrently over what was a module-global violation recorder, so scenarios could steal or wipe each other's recorded violations. Give each suite scenario its own recorder via AsyncLocalStorage; code outside a scope (single-scenario runs, the sequential server suite, vitest hooks) falls back to the global recorder. Also condense a four-line comment in connection/stateful.ts to three lines. * Patch the 2025-11-25 NumberSchema generator bug as a load-time erratum The released 2025-11-25 schema.json types NumberSchema minimum/maximum/default as integer, contradicting its own schema.ts (default?: number) — a typescript-json-schema artifact, fixed for draft in modelcontextprotocol#2710 and proposed for 2025-11-25 in modelcontextprotocol#3139. Patch the schema at load time so SDK fixtures with fractional number defaults (all tier-1 SDKs send 95.5) are not flagged; restore the sep-1034 scenario and example-server fixture to 95.5. Delete the erratum when #3139 lands and the schemas are re-vendored. Also scope the sequential server-suite loop with withWireRecorder so a connection leaked by a failed scenario cannot attribute late traffic to the next scenario. --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Paul Carleton <paulc@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
sep-2575-server-rejects-undeclared-capabilityrequireserror.data.requiredCapabilitiesto be an array containing'sampling'. The schema defines it as aClientCapabilitiesobject, so a spec-correct server cannot pass the check.Motivation and Context
Spec —
schema/draft/schema.ts#L482(rendered):and the spec's own example,
missing-elicitation-capability.json:Both have carried the object shape since the commit that introduced SEP-2575 (
8e192a22, May 11) — it was never an array at any point. The array assertion arrived a week later in the initial scenario scaffold (#271, May 19) without a spec citation, plausibly from the prose "whosedata.requiredCapabilitieslists the missing capabilities" (basic/index), which reads array-ish in isolation; the schema and example are unambiguous.This repo already uses the object everywhere else —
src/spec-types/draft.ts#L499types it asClientCapabilities, andtasks/required-task-error.ts#L192assertsdata.requiredCapabilities.extensions[...]as an object. The array assertion went unnoticed because it is only reachable once a server actually returns-32021, and that positive branch had no self-test.Before / after, against a real server (typescript-sdk's conformance everythingServer) answering the probe with the spec shape:
before (
0.2.0-alpha.8):after (this branch, same server):
How Has This Been Tested?
npx vitest run src/scenarios/server/stateless.test.ts— 18/18, including three new self-tests: object passes, array fails,{ "sampling": null }fails (capability values must themselves be objects).server-statelessagainst typescript-sdk's everythingServer (with itstest_missing_capabilityfixture armed): 30/30 — the same server the shipped alpha.8 fails, as captured above.Breaking Changes
For servers, no — the object shape is what the schema has always required. For fixture servers written against the old assertion: python-sdk's reference everything-server emits the array (and pins it in two tests), and
server-statelessis in neither of its baselines, so its conformance CI will flag this scenario on its next referee bump. The fix there is the same one-line shape change — happy to send it.Types of changes
Checklist
Additional context