Summary
The TLS-bridge leaf minter caches forged leaf certs on a TTL compared with time.Now(), which carries Go's monotonic clock. Across a host suspend / VM pause the monotonic clock freezes while wall time — and the leaf's x509 NotAfter — keeps advancing. The cache then serves a leaf the client rejects as certificate has expired.
Impact (silent fail-open)
The client's rejection looks like a forged-cert failure, so the bridge auto-skips the host (SkipSet) and falls back to undecrypted passthrough. Interception — and all inference / MCP / A2A parsing — for that host silently stops until the proxy is restarted. The agent keeps working via the blind tunnel, so nothing looks broken; the only signal is missing events in abctl.
How it was hit
Dev cluster on a laptop that slept overnight: litellm's cached forged leaf expired, every inference call fell through to passthrough, and abctl showed no inference events despite inference-parser being active. Restarting authbridge-proxy (clearing the in-memory leaf cache + SkipSet) restored interception.
Root cause
authlib/tlsbridge/minter.go gates cache freshness on a monotonic-carrying deadline; authlib/tlsbridge/decision.go SkipSet has the same pattern. Monotonic ≠ wall across suspend, so the cache keeps a wall-clock-expired leaf "fresh."
Fix
Gate cache freshness on wall-clock deadlines + the cert's real NotAfter:
- populate
tls.Certificate.Leaf; gate the cache hit on a monotonic-stripped (.Round(0)) wall-clock expires plus a NotAfter backstop that never serves a leaf past its real validity;
- give
SkipSet expiry the same .Round(0) treatment.
Scope
Primarily a dev-environment robustness issue (laptop sleep / VM pause) — production nodes don't suspend — but the silent fail-open-to-passthrough behavior warrants the guard, and wall-clock gating is the correct invariant regardless.
Part of rossoctl/rossoctl#2071.
Summary
The TLS-bridge leaf minter caches forged leaf certs on a TTL compared with
time.Now(), which carries Go's monotonic clock. Across a host suspend / VM pause the monotonic clock freezes while wall time — and the leaf's x509NotAfter— keeps advancing. The cache then serves a leaf the client rejects ascertificate has expired.Impact (silent fail-open)
The client's rejection looks like a forged-cert failure, so the bridge auto-skips the host (
SkipSet) and falls back to undecrypted passthrough. Interception — and all inference / MCP / A2A parsing — for that host silently stops until the proxy is restarted. The agent keeps working via the blind tunnel, so nothing looks broken; the only signal is missing events in abctl.How it was hit
Dev cluster on a laptop that slept overnight: litellm's cached forged leaf expired, every inference call fell through to passthrough, and abctl showed no inference events despite
inference-parserbeing active. Restartingauthbridge-proxy(clearing the in-memory leaf cache +SkipSet) restored interception.Root cause
authlib/tlsbridge/minter.gogates cache freshness on a monotonic-carrying deadline;authlib/tlsbridge/decision.goSkipSethas the same pattern. Monotonic ≠ wall across suspend, so the cache keeps a wall-clock-expired leaf "fresh."Fix
Gate cache freshness on wall-clock deadlines + the cert's real
NotAfter:tls.Certificate.Leaf; gate the cache hit on a monotonic-stripped (.Round(0)) wall-clockexpiresplus aNotAfterbackstop that never serves a leaf past its real validity;SkipSetexpiry the same.Round(0)treatment.Scope
Primarily a dev-environment robustness issue (laptop sleep / VM pause) — production nodes don't suspend — but the silent fail-open-to-passthrough behavior warrants the guard, and wall-clock gating is the correct invariant regardless.
Part of rossoctl/rossoctl#2071.