fix(media): explain clock skew behind an opaque upload 401 - #6929
Open
gabmichels wants to merge 1 commit into
Open
gabmichels wants to merge 1 commit into
gabmichels wants to merge 1 commit into
Conversation
Media uploads fail with a bare `401 authentication failed` when the client's system clock has drifted. Every media auth failure collapses into that one message to defeat enumeration, so a user-fixable, non-Buzz problem is indistinguishable from a signing, scope, or authorization bug. After an upload is refused with a 401, the client now makes one short `HEAD` of the upload route and reads the relay's `Date` header. The local clock is sampled either side of that round trip, so the offset is bracketed into the interval the observation proves rather than estimated — latency can only widen the interval, never manufacture a verdict. When the interval lies wholly outside what media upload auth allows, the drift is reported alongside the relay's own message. The probe is deliberately a separate request. The relay verifies Blossom auth before it reads the body, so the `Date` on a large upload's rejection was stamped at the start of a transfer that may have run for minutes; reading the clock off it would blame a healthy clock for someone else's auth failure. The message reports a measurement, not a diagnosis. The relay compares whole seconds across an unknown transit delay, so a drift near the tolerance may be accepted or rejected depending on fractional alignment — claiming causation would be wrong in exactly the band this serves. The relay's message is appended to, never replaced, and the error keeps its variant, status, exit code and JSON category. Both directions are covered. The backward bound uses the `expiration` the client stamped on its own token, which it knows exactly, unlike `max_age_secs` — the relay selects the 600s or 3600s window by sniffing the body bytes rather than trusting `Content-Type`. Signed-off-by: Gab Michels <gab.michels@hotmail.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #4075.
Media uploads fail with
401 authentication failedwhen the client's system clock has drifted. The 401 is deliberately opaque — all thirteen media auth failures collapse into it (buzz-media/src/error.rs) — so a user-fixable, non-Buzz problem is presented as an authentication failure, and people go looking for auth bugs. On #4075 the drift was ~6.75s on a Windows box whosew32timeservice had never started; uploads had worked days earlier and then stopped, with no client change in between.This is option (2) from that issue: compare the local clock against the relay's
Dateheader and say so, instead of leaving the user with a bare 401. No relay change —crates/buzz-mediais untouched.What it does
When an upload is refused with a 401, the client makes one short
HEADrequest to the upload route, reads theDateheader, and appends an explanation if — and only if — the observation proves the clock is at fault:The relay's own message is kept. Fifteen distinct auth failures collapse into that 401 and only one of them is the clock, so replacing the body would destroy the evidence every time the guess is wrong. The error also stays a
CliError::Relay { status: 401 }, so exit codes and the JSON error category are unchanged.What the message claims, and what it does not
It reports a measurement — this clock is provably outside the window — and offers it as the first thing to rule out. It does not claim to have diagnosed the 401 in hand, because a client cannot: the relay compares whole seconds (
created > now + 5), both sides are truncated, and the transit between them is unknown, so a drift of ~5.6s is accepted or rejected depending on where the two clocks fall inside their respective seconds. Asserting causation would be wrong in exactly the band this feature exists to serve. Appending rather than replacing means a wrong guess costs the user nothing.Why the diagnosis runs after the failure, not before it
The obvious design is a pre-flight check before minting the token. I built that first and it was wrong twice over.
Reading the
Dateoff the rejection does not work.crates/buzz-relay/src/api/media.rs:139verifies Blossom auth inFromRequestParts, before the body is read — the code comments call this out as the "pre-body auth-rejection guarantee". So on a 200 MB upload the 401'sDatewas stamped at t≈0 while the client spent the next minute pushing bytes. Treating it as "the relay's clock now" would pin every slow-upload 401 on a perfectly healthy clock. Hence a dedicated round trip: it is the only one short enough to bound.A pre-flight that can refuse is a regression risk. Any check that runs before the upload and returns an error can turn a diagnostic into a new failure mode. Running only after a refusal makes that structurally impossible: there is no path where this code is the reason an upload does not happen.
The measurement is bracketed, not estimated
The local clock is read on both sides of the probe. Writing
Δfor the true offset,Dfor the parsed header, andSfor the server's real clock at stamp time,D ≤ S < D + 1sandbefore − Δ ≤ S ≤ after − Δ, which givesBoth bounds hold whatever the latency was, so no fudge factor is needed and none is applied. Each direction uses only its proven side: the "ahead" verdict reads the lower bound, sampled before the request, so latency cannot inflate it. A test pins this — 600 seconds of simulated latency on a synced clock yields no verdict. The cost of that rigour is silence when the interval straddles a bound, which is the right way to be wrong.
The local readings are in milliseconds; only the server's stamp is truncated. That matters for the reported case: with whole-second readings a 6.75s drift straddles the 5s tolerance and stays silent, while millisecond readings prove ≥5.75s.
A probe response carrying an
Ageheader is discarded rather than measured — only a cache sendsAge, and a cached reply'sDateis as stale as the cache entry.Both directions, and the backward bound is the token's own lifetime
Skew goes both ways, and the verifier's branches do not share a bound:
The check uses the
expirationthe client stamped on its own token. That is the tightest of the backward bounds — a clock behind by more than the token's whole lifetime mints one already expired on arrival — and, unlikemax_age_secs, the client knows it exactly. It cannot knowmax_age_secs: the relay selects the image (600s) or video (3600s) pipeline by sniffing the body bytes, not fromContent-Type(upload_blob: "Content-Typeis advisory only; a bounded body prefix selects the streaming video path from actual bytes"), so any client-side guess keyed on MIME can disagree with what the relay actually applied.Not done deliberately
The client could read the relay's clock and backdate
created_atso skewed uploads succeed. That is a worse fix: a drifted clock also corrupts event timestamps, so masking it at the media layer hides a problem that is broken elsewhere too. The only correct repair is to fix the clock, and this change says so.TimestampOutOfWindowis still collapsed into the blind 401. That is option (1) from the issue and belongs in its own PR.Testing
buzz-core— 11 unit tests:Dateparsing (IMF-fixdate, RFC 2822 at zero and non-zero offsets, junk), the bracketing arithmetic, out-of-order readings, the reported ~6.75s drift, the backward bound tracking the token lifetime, and three that exist to fail if the design regresses — latency alone never produces a verdict; a drift inside the tolerance stays silent at every fractional alignment of the server's stamp; the message never asserts causation.buzz-cli— 6 integration tests against an axum stand-in relay that dates the probe and the rejection independently, so a test can prove which one the diagnosis trusts. Includes the regression test for the pre-body rejection (a rejection dated 600s stale with a clean probe must not be read as drift), an unreadable-Datecase, and a non-401 failure that must not be probed at all.desktop— the auth event's tag shape and that itsexpirationderives from the lifetime the diagnosis measures against, theserver-tag omission path, and the probe guard for non-401 and cancelled uploads.cargo clippy --workspace --all-targetsand the Tauri clippy pass clean;cargo fmt --all --checkclean; the desktop file-size ratchet passes (the new desktop code lives in its own module, andcommands/media.rsis smaller than before).200, and a non-401 failure (422 invalid image data) still surfaces exactly as it did.No UI screenshot: the desktop change alters the text of an existing upload-failure error string rather than any layout or interaction. Happy to add one if you would rather see it in situ.