Test our OIDC login against the official OpenID conformance suite#21
Merged
Conversation
Implements the conformance item spiked in notes/oidc-conformance-spike.md: - SCRATCHWORK_LOCAL_OAUTH_ISSUER: expected-issuer override riding the loopback-gated local endpoint overrides, replacing (never widening) the Google issuer check — the conformance suite issues under its own URL. - Token-endpoint client auth moves to client_secret_basic (RFC 6749 §2.3.1), which the suite's basic RP plan enforces at FAILURE level; the hermetic e2e provider now requires the Basic header and rejects body-borne secrets. - e2e/conformance/: pinned suite compose (release-v5.2.0, devmode) and a runner (`bun run conformance`) that drives the full oidcc-client-basic-certification-test-plan through a loopback proxy, one fresh backend + browser per module, and reports per-module verdicts. Verified against the real suite: 8/14 modules green — every ID-token tampering/rejection lane passes (invalid iss/aud/sub/iat/nonce, bad signature, kid-absent-multiple, alg=none refused). The six that require completing a login stall on a known product gap (no userinfo support; email required in the ID token, which the suite's OP never provides; single-key kid-less JWKS rejected) — recorded with the decision needed in the spike note. The lane joins bun run ci once that's resolved and the plan is green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Decision (Pete, 2026-07-20, recorded in notes/oidc-conformance-spike.md): keep the RP as is — no userinfo call, the ID token stays the only credential consumed — and require exactly the seven modules that judge the RP by rejecting a tampered ID token. The plan's login-completion modules are N/A by design and skipped with a log line, narrowing "conformance coverage" from certification-green to security-modules-green deliberately. The scoped lane runs green: 7/7 PASSED in ~2m20s wall including suite boot. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
We use Google sign-in via OIDC, but until now nothing independently verified that our client rejects bad ID tokens. This PR adds a test setup that runs the official OpenID Foundation conformance suite against our server and makes sure we fail the way we're supposed to when a token is tampered with.
What's in here
e2e/conformance/, run withbun run conformance): spins up the OpenID conformance suite in Docker (pinned to release-v5.2.0), points it at a fresh instance of our server, and drives each test module through a real browser. Reports pass/fail per module.SCRATCHWORK_LOCAL_OAUTH_ISSUER): the conformance suite acts as the identity provider, so we need to tell the server to expect its issuer URL instead of Google's. This only works alongside the existing loopback-only local endpoint overrides — it can't loosen anything in production.What we test (and what we skip)
The suite's full certification plan has 14 modules. Seven of them check that a client rejects tampered ID tokens — wrong issuer, wrong audience, bad signature,
alg=none, stale timestamps, wrong nonce, ambiguous signing key. Those are the security-critical ones, and all seven pass.The other seven check full login completion, which requires features we deliberately don't have (a userinfo call — we treat the ID token as the only credential). Those are skipped with a log line explaining why. Details and the decision are written up in
notes/oidc-conformance-spike.md.Testing
Ran the suite for real: 7/7 required modules pass in about 2.5 minutes including boot.
bun run cicovers the unit tests.🤖 Generated with Claude Code