Make RFC 8693 delegate clients reachable and usable

[jhrozek]

Summary

RFC 8693 token-exchange is implemented in the embedded authorization server but unreachable in production: no supported path can provision a client capable of holding the grant (DCR and CIMD are both public-only), and discovery never advertises the grant or secret-based client auth even though the handler is always registered. This blocks the agentic-delegation use case #5194 exists for: an agent acting on behalf of a user, with tokens carrying sub=user and act.sub=agent instead of collapsing into one identity or losing the user entirely.

This PR:

• Adds RunConfig.delegate_clients / the operator CRD's delegateClients — pre-provisioned confidential clients, secret always by reference (file/env var/Kubernetes Secret), restricted to exactly the token-exchange grant, with required scope/audience narrowing.
• Resolves secrets and registers these clients at server startup, before any HTTP traffic can be served.
• Fixes discovery metadata so the token-exchange grant and secret-based client auth are actually advertised.
• Fixes a consent-model gap found by running the new e2e test against a real cluster: a delegate client could never actually complete a first exchange, because its grant types are locked to token-exchange-only (it can never run authorization_code itself), so it could never satisfy the old "subject token must be issued to the presenting client" check. Relaxed that check specifically for operator-declared delegate clients on the self-issued path, with an explicit guard so the existing external-issuer consent protections (Consent model for external OIDC subject tokens in multi-issuer token exchange #5989) are untouched. Verified against a live Kind cluster end-to-end, and against RFC 8693 §2.1 and comparable real-world STS implementations (Keycloak, Okta/Auth0, Curity) via oauth-expert review — see "Special notes" below.
• Wires the CRD through both MCPExternalAuthConfig/MCPServer/MCPRemoteProxy and VirtualMCPServer (the shared EmbeddedAuthServerConfig type), injecting the Secret via valueFrom.secretKeyRef — never copied into a ConfigMap, controller memory, or status.
• Adds a real Kind-cluster e2e test proving the whole path: secret stays out of the ConfigMap, the Deployment env uses SecretKeyRef, and a real HTTP token exchange succeeds with the correct act claim.
• A cleanup pass: removed the grant_types field from the wire format (it could only ever hold one legal value, and the CRD side had already made that call), replaced a hand-rolled contains() with stdlib slices.Contains, added a minimum-length check on delegate-client secrets (they're operator-supplied, unlike DCR-minted secrets, so nothing else checks their entropy), and added a startup warning that a configured delegate client can exchange any user's self-issued token.

Closes #6082

Type of change

• New feature
• Bug fix
• Refactoring (no behavior change)
• Dependency update
• Documentation
• Other (describe):

Test plan

• Unit tests (task test)
• E2E tests (task test-e2e)
• Linting (task lint-fix)
• Manual testing (describe below)

Manually verified end-to-end against a real Kind cluster: deployed the operator with this branch's images, created a VirtualMCPServer with authServerConfig.delegateClients and an embedded Dex upstream, and confirmed the e2e test virtualmcp_delegate_clients_test.go passes — including the real RFC 8693 exchange over the network, the act.sub claim assertion, and a wrong-secret request correctly getting 401.

API Compatibility

• This PR does not break the v1beta1 API, OR the api-break-allowed label is applied and the migration guidance is described above.

Purely additive: a new optional delegateClients field on the shared EmbeddedAuthServerConfig type, consumed by both MCPExternalAuthConfig and VirtualMCPServer. Existing resources without it are unaffected.

Changes

Area	Files	What
Runtime config & validation	pkg/authserver/config.go	DelegateClientRunConfig/DelegateClient types, validation (unique ID, secret reference required, scope/audience subset, minimum secret length)
Secret resolution & registration	pkg/authserver/runner/embeddedauthserver.go, pkg/authserver/server_impl.go, pkg/authserver/server/registration/client.go	Resolves file/env secrets before storage init; registers clients before the provider/router is built; startup warning log
Discovery & consent	pkg/authserver/server/handlers/discovery.go, pkg/authserver/server/provider.go, pkg/authserver/server/tokenexchange/{handler,factory}.go	Advertises the grant and secret-based auth methods; relaxes self-issued consent for configured delegate clients only
Operator CRD API	cmd/thv-operator/api/v1beta1/mcpexternalauthconfig_types.go	DelegateClientConfig (no inline secret, no redirect URI, no grant selection), CEL admission rules
Operator wiring	cmd/thv-operator/pkg/controllerutil/authserver.go, cmd/thv-operator/controllers/{mcpserver,mcpremoteproxy,virtualmcpserver}*.go, cmd/thv-operator/pkg/vmcpconfig/converter.go	CRD → RunConfig conversion, reserved pod env var names (indexed by position, not client ID), Secret validation, generated manifests/docs
E2E	test/e2e/thv-operator/virtualmcp/virtualmcp_delegate_clients_test.go	Full-stack proof against a real cluster
Architecture docs	docs/arch/17-token-exchange-delegation.md	Trust model, consent signals, a worked example, and the accepted blanket-trust tradeoff

Does this introduce a user-facing change?

Yes: operators can now declare pre-provisioned confidential OAuth clients (delegateClients) on MCPExternalAuthConfig/VirtualMCPServer's embedded auth server config, enabling RFC 8693 token-exchange delegation for agentic use cases. No action required for existing deployments — the field is optional and additive.

Implementation plan

Approved implementation plan (original 6-step feature plan; a follow-up plan for the consent-model fix, reviewed by an oauth-expert/code-reviewer/security-advisor panel, is summarized in "Special notes for reviewers" below)

Implementation plan: delegate clients and operator CRD support

Execution workflow

Treat the existing working-tree patch as untrusted. Salvage code only after verifying it against the plan; do not reset or overwrite unrelated work. HANDOVER.md remains uncommitted
unless explicitly requested.

For every implementation step:

1Implement only that step.
2Run its focused tests and applicable repository checks using task targets.
3Create one scoped commit.
4Run the step’s MoE review with the listed read-only specialist agents in parallel.
5Consolidate findings by severity.
6Fix all blocking findings, rerun verification, amend the same commit, and repeat the MoE review until approved.
7Continue to the next step only after the review gate passes.

Commit subjects will use imperative mood, no conventional prefix, no trailing period, and a maximum of 50 characters. Nothing will be pushed and no PR will be opened unless separately
requested.

─────────────────────

Step 1: Establish the runtime configuration contract

Work

Update pkg/authserver/config.go :

Add serializable RunConfig.DelegateClients .
Define DelegateClientRunConfig with:
client_id
client_secret_file
client_secret_env_var
grant_types
scopes
audiences
Define a resolved runtime client type containing the resolved secret.
Keep runtime grant support restricted to exactly RFC 8693 token exchange.
Require explicit, narrowed scopes and audiences.
Validate static confidential clients against existing HTTP transport protections without coupling them to confidential DCR.
Remove the broken patch’s optional allowed_delegate_clients cross-reference and exported wildcard; that compatibility change is outside this scope.
Correct stale comments claiming DCR can issue only public clients.

Acceptance criteria

Empty and duplicate client IDs are rejected.
Neither secret reference is rejected.
Missing, unknown, or additional grants are rejected.
Empty or unsupported scopes are rejected.
Empty or unsupported audiences are rejected.
A token-exchange-only client does not require redirect URIs.
Static clients work with confidential DCR disabled and do not enable it.
Static clients are rejected over unsafe HTTP according to the confidential-client transport policy.
The serialized configuration exposes no inline-secret field.
Runtime validation also protects direct authserver.Config callers.

Tests

Add table-driven coverage in pkg/authserver/config_test.go for every acceptance criterion.

Deliverables

Stable runtime and wire-format delegate-client types.
Authoritative runtime validation.
Focused validation tests.

Commit

Define delegate client configuration

MoE review

oauth-expert : OAuth grant and redirect-flow correctness.
security-advisor : secret surface, transport safety, and privilege narrowing.
code-reviewer : Go API shape, validation quality, and repository conventions.

─────────────────────

Step 2: Resolve secrets and register clients at startup

Work

Update:

pkg/authserver/runner/embeddedauthserver.go
pkg/authserver/server/registration/client.go
pkg/authserver/server_impl.go

Implementation requirements:

Resolve delegate secrets immediately after serializable validation and before storage initialization or upstream DCR/network side effects.
Reuse resolveSecret(file, envVar) with file-over-environment precedence.
Reject unreadable, missing, or empty resolved secrets without including secret material in errors or logs.
Clone grant, scope, and audience slices when crossing into runtime configuration.
Construct a confidential fosite.DefaultClient with a hashed secret.
Apply only configured grants, scopes, and audiences.
Do not apply DCR defaults or registration.DCRIssued .
Register configured clients after storage setup but before provider/router construction.
Preserve static-client precedence when its ID collides with a persisted DCR client.
Ensure the replacement has no DCR TTL in memory or Redis storage.

Acceptance criteria

Invalid secret configuration fails before any external registration side effect.
File secrets and environment secrets both work.
File contents are trimmed and take precedence over environment values.
Authorization-critical slices cannot be mutated through retained caller storage.
The resulting client is confidential and accepts no redirect flow.
It is available before the first HTTP request can be served.
A static client replaces a same-ID DCR client.
The replacement is not marked DCRIssued and receives no DCR expiration.
Repeated startup with the same configuration is idempotent.
Errors and logs never contain client secrets.

Tests

Secret resolution and startup-order tests in pkg/authserver/runner/embeddedauthserver_test.go .
Constructor tests in pkg/authserver/server/registration/client_test.go .
Memory and Redis registration/collision tests.
Authentication tests for both client_secret_basic and client_secret_post .

Deliverables

Early secret-reference resolution.
Static confidential-client constructor.
Pre-serving registration.
Collision and persistence guarantees.
Unit/storage coverage.

Commit

Register configured delegate clients

MoE review

oauth-expert : Fosite client shape and authentication-method behavior.
security-advisor : secret lifecycle, hashing, error redaction, and collision safety.
code-reviewer : startup ordering, resource handling, and storage behavior.

─────────────────────

Step 3: Complete discovery and runtime end-to-end proof

Work

Update:

pkg/authserver/server/provider.go
pkg/authserver/server/handlers/discovery.go
pkg/authserver/integration_test.go

Discovery decision:

Always advertise RFC 8693 token exchange because the handler is always registered.
Advertise client_secret_basic and client_secret_post when confidential DCR is enabled or at least one static delegate client exists.
Keep public-only token-endpoint auth methods exactly ["none"] .
Document that token-exchange advertisement is the deliberate exception to the handover’s otherwise byte-identical public-only metadata requirement.
Keep DCR’s allowed grants unchanged; never permit token exchange through unauthenticated DCR.

Replace the new feature’s direct-storage test setup with the real configuration/startup path.

Acceptance criteria

Both OAuth Authorization Server and OIDC discovery advertise token exchange.
A static client causes secret-based methods to be advertised without enabling confidential DCR.
Public-only configurations retain ["none"] as their auth-method list.
DCR continues to reject the token-exchange grant.
A configured client completes RFC 8693 exchange through the HTTP endpoint.
Both Basic and form-post client authentication work.
Wrong or missing credentials fail.
Public clients and confidential clients lacking the exchange grant fail.
The issued token has the expected subject, audience, scopes, and act.sub .
Out-of-policy scopes and audiences fail.

End-to-end integration test

Extend pkg/authserver/integration_test.go to:

1Configure the client through the actual runtime configuration path.
2Start the embedded server.
3Produce a valid subject token.
4POST a real RFC 8693 request to /oauth/token .
5Exercise both supported secret-authentication forms.
6Decode the issued token and verify act.sub == client_id .
7Exercise invalid-secret, scope, and audience failures.

Each parallel subtest must own its mockoidc instance and register cleanup with t.Cleanup .

Deliverables

Correct discovery metadata.
DCR security regression coverage.
HTTP-level RFC 8693 end-to-end proof using configured startup registration.

Commit

Expose configured token exchange

MoE review

oauth-expert : RFC 8693 request/response and discovery correctness.
unit-test-writer : coverage quality, isolation, and failure cases.
security-advisor : DCR boundary and authorization narrowing.
code-reviewer : integration-test reliability and implementation quality.

─────────────────────

Step 4: Add the shared operator API

Work

Extend the existing EmbeddedAuthServerConfig ; do not create a standalone client CRD.

Add a Kubernetes-facing delegate-client type containing:

clientId
clientSecretRef
scopes
audiences

Do not expose:

an inline secret;
redirect URIs;
configurable grant types.

The operator will synthesize the single supported token-exchange grant.

Apply structural schema validation:

required, non-empty client ID;
required SecretKeyRef.name and .key ;
required non-empty scopes and audiences;
bounded atomic lists;
transport validation rejecting delegate clients over unsafe HTTP.

Because the shared configuration is exposed by both MCPExternalAuthConfig and VirtualMCPServer , neither consumer may silently ignore the field.

Acceptance criteria

Both existing CRD surfaces accept the new field.
Inline secret material cannot be represented.
Invalid structural entries are rejected at admission.
Static clients remain independent of confidential DCR.
Unsafe HTTP combinations are rejected at admission.
The CRD shape exposes no unsupported grant or redirect configuration.
Existing resources without delegate clients remain schema-compatible.

Tests

API type and CEL/schema validation tests.
Serialization tests for the Kubernetes-facing shape.
Compatibility test for an existing configuration without delegate clients.

Deliverables

Shared EmbeddedAuthServerConfig.delegateClients API.
Kubernetes-native SecretKeyRef shape.
Admission validation and API tests.

Commit

Add delegate clients to operator API

MoE review

kubernetes-expert : CRD design, kubebuilder markers, CEL, and shared-type behavior.
oauth-expert : confirmation that the reduced CRD surface correctly represents the runtime contract.
security-advisor : Secret reference and plaintext-transport safeguards.
code-reviewer : API compatibility and Go conventions.

─────────────────────

Step 5: Wire both operator deployment paths

Work

Update the shared operator conversion and environment helpers, including:

BuildAuthServerRunConfig
GenerateAuthServerEnvVars

existing Secret-reference helpers
MCPExternalAuthConfig-backed workload generation
VirtualMCPServer auth-server ConfigMap and Deployment generation

For each configured client:

1Generate a deterministic reserved environment-variable name based on list position, not arbitrary client-ID text.
2Add a pod environment variable using valueFrom.secretKeyRef .
3Put only that environment-variable name in client_secret_env_var .
4Synthesize exactly the RFC 8693 grant.
5Copy narrowed scopes and audiences.
6Validate against the effective scopes and audiences available in the conversion context.

Never read Secret values into the controller or place them in ConfigMaps, status, logs, or errors.

Regenerate all affected API artifacts with verified Taskfile targets.

Acceptance criteria

MCPExternalAuthConfig-backed workloads receive the mapped clients.
VirtualMCPServer workloads receive the mapped clients.
Multiple clients receive deterministic, distinct bindings.
Generated ConfigMaps contain references only, never credentials.
Deployments use EnvVar.ValueFrom.SecretKeyRef ; EnvVar.Value remains empty.
Generated runtime config contains the fixed token-exchange grant.
Invalid effective scopes/audiences produce an actionable terminal validation result rather than a retry loop.
Neither shared API consumer silently drops the field.
Existing workloads without delegate clients remain unchanged.

Tests

Table-driven conversion tests in operator controller utilities.
Environment-generation tests verifying exact Secret selectors.
Tests that search generated ConfigMaps/status for absence of secret values.
MCPExternalAuthConfig envtest reconciliation coverage.
VirtualMCPServer conversion/reconciliation coverage.
Idempotent reconciliation assertions where applicable.

Generated deliverables

Regenerate and commit applicable artifacts through Taskfile targets:

deepcopy code;
CRD manifests;
Helm CRD files/templates;
CRD API reference documentation.

Expected commands, after confirming exact targets in the Taskfiles:

task operator-generate
task operator-manifests
task crdref-gen

Deliverables

Shared CRD-to- RunConfig adapter.
SecretKeyRef-to-environment wiring.
Complete MCPExternalAuthConfig and VirtualMCPServer support.
Operator integration tests.
Generated CRDs, deepcopy code, Helm artifacts, and API reference.

Commit

Wire operator delegate clients

MoE review

kubernetes-expert : reconciliation, Deployment/ConfigMap construction, API conventions, and generated artifacts.
toolhive-expert : coverage of every embedded-auth-server deployment path.
security-advisor : proof that Secret values never enter shared state.
code-reviewer : idempotency, validation errors, and maintainability.

─────────────────────

Step 6: Add Kubernetes E2E and documentation

Work

Add a real operator E2E using the repository’s existing Kind/Ginkgo infrastructure:

1Create a namespace-local Secret containing a delegate client credential.
2Create the embedded-auth-server configuration with delegateClients .
3Create the consuming workload.
4Wait for the managed endpoint to become available.
5Verify discovery metadata.
6Obtain or produce the test subject token through the existing fixture.
7Perform a real RFC 8693 exchange using the Secret-backed credential.
8Decode the result and verify act.sub , subject, scope, and audience.
9Verify an invalid credential is rejected.

Readiness, ConfigMap contents, or Deployment state alone do not satisfy this E2E requirement.

Update docs/arch/17-token-exchange-delegation.md and relevant operator API narrative documentation:

configuration examples;
secret-reference-only handling;
static/DCR independence;
scope and audience narrowing;
discovery behavior;
collision precedence;
supported operator consumers;
Secret rotation behavior as actually implemented, without claiming automatic rollout if none exists.

Acceptance criteria

The E2E deploys through the operator rather than manually constructing runtime configuration.
The running pod receives the credential from a Kubernetes Secret.
A real RFC 8693 exchange succeeds.
act.sub identifies the declared client.
Invalid credentials fail.
No test inspects or copies plaintext secret values from generated shared state.
Documentation matches the shipped behavior and no longer says token exchange is unreachable.

Verification

Run the applicable repository tasks, including:

focused unit and integration tasks during development;
task test ;
task lint-fix ;
task license-check ;
operator generation consistency checks;
the relevant Kind/operator E2E task from the repository root.

Do not use bare go test , go build , go vet , or golangci-lint .

Deliverables

Kind-based operator RFC 8693 E2E.
Updated architecture and operator documentation.
Final full-suite verification record.

Commit

Test operator token exchange end to end

MoE review

kubernetes-expert : E2E deployment fidelity and operator behavior.
oauth-expert : protocol assertions and delegated-token claims.
documentation-writer : documentation accuracy and completeness.
code-reviewer : test reliability and final change quality.

─────────────────────

Final aggregate review gate

After all six commits pass their individual gates, run one final parallel MoE review across the complete branch:

tech-lead-orchestrator : scope, commit boundaries, and architectural coherence.
oauth-expert : complete OAuth/DCR/token-exchange model.
kubernetes-expert : CRD, reconciliation, generated artifacts, and E2E.
security-advisor : secrets, transport, authorization narrowing, and persistence.
code-reviewer : repository conventions, correctness, and regression risk.

Resolve all blocking findings by amending the commit that introduced them, rerun that step’s tests, rerun the affected step review, then rerun the aggregate review.

Before any PR is proposed, inspect production-code/file counts against repository limits. If the combined runtime and operator work exceeds the PR scope limits, preserve the six
reviewed commits but split delivery into two PRs:

1Runtime contract and RFC 8693 integration: steps 1–3.
2Operator API, wiring, E2E, and docs: steps 4–6.
↑69.7K ↓4.5K · 71.9s

▌ you
│ trim the reviewers per step to 2, leave all you chose for the aggregate gate only otherwise this will be too costly

● mecatl

Implementation plan: delegate clients and operator CRD support

Workflow

Treat the existing patch as untrusted and salvage only verified pieces. Do not commit HANDOVER.md .

For each step:

1Implement only that step.
2Run focused tests and applicable task checks.
3Create one scoped commit.
4Run the two listed read-only reviewers in parallel.
5Fix blocking findings, amend the same commit, rerun checks, and repeat review until approved.
6Continue only after the review gate passes.

Do not push or open a PR unless separately requested. Commit subjects use imperative mood, no conventional prefix or trailing period, and at most 50 characters.

Step 1: Runtime configuration contract

Work

Add serializable RunConfig.DelegateClients and a resolved runtime type in pkg/authserver/config.go .
Wire fields: client ID, file/environment secret reference, grants, scopes, and audiences; never an inline secret.
Restrict grants to exactly RFC 8693 token exchange.
Require explicitly narrowed scopes and audiences.
Apply confidential-client HTTP transport protection without coupling static clients to confidential DCR.
Remove the broken patch’s optional allowed_delegate_clients cross-check/exported wildcard and correct stale DCR comments.

Acceptance criteria

Reject missing/duplicate IDs, missing secret references, invalid grants, empty/out-of-policy scopes, and empty/out-of-policy audiences.
Accept token-exchange-only clients without redirect URIs.
Static clients neither require nor enable confidential DCR.
Reject unsafe HTTP configurations.
Protect both RunConfig and direct runtime Config callers.
Expose no inline-secret field.

Tests: table-driven config validation tests.

Deliverables: stable runtime/wire types, authoritative validation, unit coverage.

Commit: Define delegate client configuration

Reviewers: oauth-expert , security-advisor

Step 2: Secret resolution and startup registration

Work

Resolve secrets immediately after serializable validation, before storage/upstream side effects.
Reuse resolveSecret with file precedence and clone permission slices.
Build a confidential Fosite client with a hashed secret, exact permissions, no redirects, and no DCR marker/defaults.
Register clients before provider/router construction.
Ensure a static client replaces a same-ID DCR row without inheriting its TTL.

Acceptance criteria

Missing/unreadable/empty secrets fail before external side effects and never appear in logs/errors.
File and environment secrets work; file content is trimmed and wins over environment.
Caller mutation cannot change validated permissions.
Clients exist before serving and repeated startup is idempotent.
Collision replacement is permanent and non-DCR in memory and Redis.
Both Basic and form-post client authentication work.

Tests: runner resolution/order tests, registration constructor tests, memory/Redis collision tests.

Deliverables: safe resolution, static-client construction, pre-serving registration, persistence guarantees.

Commit: Register configured delegate clients

Reviewers: oauth-expert , security-advisor

Step 3: Discovery and runtime end-to-end proof

Work

Always advertise token exchange because its handler is always registered.
Advertise secret auth methods when confidential DCR or static clients are configured.
Preserve ["none"] for public-only token auth methods.
Keep token exchange forbidden through DCR.
Drive the existing integration harness through configured startup registration rather than direct storage insertion.

Acceptance criteria

Both discovery endpoints advertise token exchange.
Static clients advertise secret methods without enabling confidential DCR.
Public-only auth methods remain ["none"] .
DCR rejects token exchange.
Real HTTP exchanges work with Basic and form-post authentication.
Wrong/missing credentials, public clients, and clients lacking the grant fail.
Issued tokens contain expected subject, audience, scopes, and act.sub .
Out-of-policy scopes/audiences fail.

E2E integration test: configure client through the real runtime path, start the server, issue a subject token, call /oauth/token , decode the result, and verify act.sub . Parallel
subtests use isolated mockoidc instances and t.Cleanup .

Deliverables: discovery support, DCR regression protection, HTTP RFC 8693 proof.

Commit: Expose configured token exchange

Reviewers: oauth-expert , unit-test-writer

Step 4: Shared operator API

Work

Add delegateClients to existing EmbeddedAuthServerConfig ; do not create a new CRD.
CRD-facing entries contain clientId , clientSecretRef , scopes, and audiences.
Do not expose inline secrets, redirect URIs, or grant selection; the adapter synthesizes token exchange.
Add required/bounded atomic-list schema validation and unsafe-HTTP validation.
Because the type is shared, require support for both MCPExternalAuthConfig and VirtualMCPServer.

Acceptance criteria

Both CRD surfaces accept the field.
Invalid structure and unsafe HTTP combinations fail admission.
Inline secrets and unsupported grant/redirect configuration cannot be represented.
Static clients remain independent of DCR.
Existing resources remain schema-compatible.
Neither consumer may silently ignore the field.

Tests: API/CEL validation, serialization, and backward-compatibility tests.

Deliverables: shared CRD field, Kubernetes-native Secret reference, admission coverage.

Commit: Add delegate clients to operator API

Reviewers: kubernetes-expert , security-advisor

Step 5: Wire both operator deployment paths

Work

Extend shared RunConfig conversion and environment generation.
For each client, generate a deterministic reserved environment name, inject valueFrom.secretKeyRef , reference only that name from RunConfig, synthesize the token-exchange grant, and
copy scopes/audiences.
Validate against effective scopes/audiences.
Wire MCPExternalAuthConfig-backed workloads and VirtualMCPServer ConfigMap/Deployment paths.
Never read or persist Secret values.
Regenerate deepcopy, CRDs, Helm artifacts, and CRD API docs with confirmed Taskfile targets.

Acceptance criteria

Both deployment paths receive all configured clients.
Multiple clients have stable, distinct bindings.
ConfigMaps contain references only; pod environment entries use SecretKeyRef with no literal value.
Invalid effective permissions produce terminal actionable validation.
Existing workloads without clients remain unchanged.
Reconciliation remains idempotent.

Tests: conversion/env generation tests, secret non-leak assertions, MCPExternalAuthConfig envtest, VirtualMCPServer reconciliation tests.

Deliverables: CRD adapter, Secret environment wiring, both consumer paths, generated API artifacts, operator integration coverage.

Commit: Wire operator delegate clients

Reviewers: kubernetes-expert , toolhive-expert

Step 6: Kubernetes E2E and documentation

Work

Add a Kind-based operator E2E that creates the Secret, auth configuration, and workload; waits for the endpoint; performs a real RFC 8693 exchange; validates claims; and verifies
invalid credentials fail.
Update delegation architecture and relevant operator documentation with configuration, narrowing, discovery, collision, consumer support, and actual Secret-rotation behavior.

Acceptance criteria

The test deploys through the operator and obtains the credential from a Kubernetes Secret.
A real exchange succeeds and returns expected act.sub , subject, scope, and audience.
Invalid credentials fail.
Readiness-only assertions are not treated as E2E proof.
Documentation matches the shipped behavior and no longer says exchange is unreachable.

Verification: focused tasks, task test , task lint-fix , task license-check , generator consistency, and the relevant operator Kind E2E task. Never use bare Go test/build/vet/lint
commands.

Deliverables: operator RFC 8693 E2E, architecture/operator docs, final verification record.

Commit: Test operator token exchange end to end

Reviewers: kubernetes-expert , oauth-expert

Aggregate review gate

After all six commits pass their step gates, run one parallel review across the complete branch:

tech-lead-orchestrator
oauth-expert
kubernetes-expert
security-advisor
code-reviewer

Fix findings by amending the commit that introduced them, rerun affected checks and the relevant step review, then repeat the aggregate gate.

Special notes for reviewers

• The consent-model fix (commit "Let delegate clients present unrelated self-issued tokens") is the one part of this PR not in the original plan — it was discovered by actually running the new e2e test against a live cluster rather than by design review. It went through its own plan → 3-agent MoE review (oauth-expert, code-reviewer, security-advisor) → per-step sonnet-implement/opus-review cycle before merging into this branch. Two follow-up reviews are worth reviewer attention:
  • An oauth-expert comparison against Keycloak/Okta/Auth0/Curity found this design (any operator-declared delegate client may exchange any user's self-issued token, no per-subject binding) is more permissive than all of those — Keycloak in particular binds exchange to the requesting client via the subject token's own aud. The verdict was still "acceptable to ship" given this is an operator-controlled, non-multi-tenant deployment model (not a public self-service IdP), and the alternative (per-subject allowlisting) was deliberately rejected in favor of this simpler blanket-trust model, matching how the unmerged spiffee-authserver branch already handles the equivalent SPIFFE-based case. Flagging so reviewers can independently agree or disagree with that risk acceptance.
  • The blast radius: a compromised delegate-client secret is a path to impersonating any user whose self-issued token the attacker can also obtain, not just "impersonate this one client." There's no per-jti single-use enforcement anywhere in this codebase (pre-existing, not introduced here). Both are now documented explicitly in docs/arch/17-token-exchange-delegation.md.
• Known, unresolved test failures in this PR's own new test suite (not introduced by the consent fix or the cleanup pass — present since the original implementation, root cause not yet diagnosed):
  • TestConfiguredDelegateClientTokenExchange/missing_credentials (pkg/authserver/delegate_client_runner_integration_test.go) expects 401, gets 400.
  • TestNewStaticDelegateClient (pkg/authserver/server/registration/client_test.go) fails a fosite.Arguments vs []string type-strictness comparison in assert.Equal, not an actual behavior mismatch.
    Neither blocks the feature (confirmed via the real e2e run and manual testing above), but both should get a follow-up fix rather than ship silently broken.
• A deliberate, intentional CEL/Go asymmetry exists around loopback-HTTP delegate clients: the CRD categorically rejects any plaintext-HTTP issuer when delegateClients is set (CEL has no URL parser to express a loopback exception safely), while the Go-level RunConfig path still permits it under the existing InsecureAllowConfidentialOverLoopbackHTTP opt-in — reviewed and confirmed intentional (not a gap), now documented in both types' doc comments so a future reader doesn't "fix" it into a regression.
• Also included, unrelated to Token-exchange grant is unreachable: no way to provision a confidential client #6082 but discovered while getting this branch's tests to run clean: removal of an orphaned test (TestMakeTrustedIssuerRunConfigs) referencing a TrustedIssuerConfig CRD type and converter that were never actually implemented anywhere in the codebase — confirmed via git log -S that the type was never committed, only its generated docs were (a leftover from an earlier, superseded draft this branch's implementation plan explicitly salvaged from).

🤖 Generated with Claude Code

[codecov]

Codecov Report

❌ Patch coverage is 81.94444% with 78 lines in your changes missing coverage. Please review.
✅ Project coverage is 72.96%. Comparing base (d057929) to head (6b56b26).

Files with missing lines	Patch %	Lines
...perator/controllers/virtualmcpserver_controller.go	58.13%	13 Missing and 5 partials ⚠️
cmd/thv-operator/pkg/controllerutil/authserver.go	80.28%	14 Missing ⚠️
...-operator/controllers/mcpremoteproxy_controller.go	79.36%	10 Missing and 3 partials ⚠️
pkg/authserver/runner/embeddedauthserver.go	73.52%	5 Missing and 4 partials ⚠️
pkg/authserver/config.go	91.78%	2 Missing and 4 partials ⚠️
pkg/authserver/server_impl.go	76.00%	3 Missing and 3 partials ⚠️
...perator/controllers/virtualmcpserver_deployment.go	0.00%	4 Missing and 1 partial ⚠️
...d/thv-operator/controllers/mcpserver_controller.go	93.18%	2 Missing and 1 partial ⚠️
cmd/thv-operator/pkg/vmcpconfig/converter.go	66.66%	2 Missing ⚠️
pkg/authserver/server/registration/client.go	90.90%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #6320      +/-   ##
==========================================
+ Coverage   72.84%   72.96%   +0.11%     
==========================================
  Files         742      742              
  Lines       77807    78167     +360     
==========================================
+ Hits        56681    57035     +354     
+ Misses      17155    17137      -18     
- Partials     3971     3995      +24     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.