feat(gkapi): meter Tor exits through one shared invite ceiling - #82
Conversation
## Problem The per-IP invite limit (4/24h) is structurally defeated by Tor circuit rotation: every exit node is a fresh IP and therefore a fresh bucket. Measured 2026-07-25 from the live rate-limit store, one actor pulled 246 invites in ~20 minutes across 202 distinct IPs -- 200 of them via Tor exits -- without any single IP approaching the limit. Tightening the per-IP number cannot fix this; rotation routes around it at any value. Downstream effect: every invite mints a fresh keypair, so the Official River room got a stream of new identities that moderation could not converge on (banning is per-identity). ## Approach The Tor exit set is enumerable, so stop treating exits as independent identities and meter the whole set through ONE shared hourly bucket. Rotation then buys the attacker nothing. Sizing from the same data: organic Tor traffic peaked at 11 invites/hour (mean 6.8) across non-burst hours; the burst hit 158/hour. A ceiling of 25/hour leaves 2.3x headroom over the worst organic hour while capping a rotation burst hard. Deliberately NOT a Tor block. Tor is ~4.8% of non-burst invite traffic (69 requests, 56 distinct exits, nearly all 1-2 requests each -- ordinary usage). Freenet is a privacy project and quickstart is the main onboarding path; blocking that traffic costs real users for no extra protection a bucket does not already give. Validated against the live exit list: 159/202 burst IPs are listed, covering 200/246 burst invites. The ceiling would have admitted 25 and refused 175 -- a ~71% cut to that burst. The rest came from non-Tor IPs and is unaffected, which the ceiling does not claim to address. ## Failure policy Fails OPEN throughout. If the list cannot be fetched and no cache exists, nothing is treated as Tor and gkapi behaves exactly as before. A garbage 200 never wipes a good list; a fetch error never clears one; the cache is written via write-then-rename so a crash cannot truncate it. Ordering in the handler is deliberate: the Tor capacity check runs BEFORE the per-IP check so a refused burst does not burn the requester's per-IP allowance, and the matching record() runs only AFTER the per-IP check passes so a per-IP rejection never consumes shared Tor capacity. ## Testing 9 new tests: parsing (v4/v6/junk/whitespace), fail-open on empty, corrupt, and missing cache, atomic cache write, bucket limit/expiry, shared-not-per-identity, and a sizing regression pinning the ceiling above the observed organic peak and below the observed burst. Plus an #[ignore]d live test against the real Tor Project list (run before deploying changes to the fetch path); it fetched 1386 exits and round-tripped the cache. Refs #81 [AI-assisted - Claude]
[AI-assisted - Claude]
External review (Codex,
|
…ceiling Five reviewers (4 Claude perspectives + Codex) found 20+ issues. The critical one would have taken gkapi down on deploy. ## CRITICAL: HTTPS startup panic (would have caused an outage) `reqwest`'s `rustls-tls` enables `rustls/ring`; axum-server's `tls-rustls` enables `rustls/aws-lc-rs`. With BOTH provider features on, rustls 0.23's `CryptoProvider::from_crate_features()` returns None by design and TLS construction panics: no process-level CryptoProvider available Confirmed empirically: the binary exits 101 before binding :443, taking down donations and cert-signing too, not just invites. No unit test could catch it (none construct a RustlsConfig) and `cargo run` without --tls-cert takes the plain-HTTP branch. Fix: install the aws-lc-rs provider explicitly in main(), preserving the provider axum-server used before reqwest existed. Pinned by a new tests/https_startup.rs that spawns the real binary with real TLS material and makes a real HTTPS request -- verified to FAIL without the fix and pass with it. ## Sizing was wrong: 25/hour would have refused real users The original ceiling came from a smaller sample filtered by MAGNITUDE (hours above a threshold excluded as "burst", then the remaining max called the organic peak) -- circular, and it understated the peak 3x. Re-derived from one dataset excluding the burst window BY TIME: organic Tor peaks at 33/hour (mean 11.9), not 11. Default raised to 60 and now overridable at runtime. ## Honest reframing The claim "ordinary Tor users are unaffected" was false and is removed. Sharing a budget across an anonymity set means the loudest member takes it: an attacker sustaining the ceiling denies invites to every Tor user, and since refused requests cost nothing, a poller beats a human to each freed slot. Inherent to any aggregate cap over an unlinkable pool; closing it needs proof-of-work (#81). This is rate-shaping that buys time, not a fix. ## Other fixes - AggregateBucket: `try_acquire()` is now the atomic admission authority (check+record under one lock), so the ceiling holds under any concurrency and stays correct if an `.await` is later introduced. Pinned by a 16-thread contention test. - Monotonic `Instant` instead of wall clock: an NTP step backwards would have stalled pruning and pinned the bucket full, denying all Tor users. - Runtime off-switch: `--tor-invites-per-hour` / `TOR_INVITES_PER_HOUR`, `0` disables. Previously required a rebuild to disable or resize. - Tor slot released when invite generation fails, so a generation-failure burst no longer locks out Tor users on top of the outage. - IPv4-mapped IPv6 canonicalised: if the bind ever changes to `::`, every exit would otherwise silently escape metering. - Fetch streams with a running cap; `response.text()` buffered the whole body BEFORE the size check, so a chunked response could OOM a small VM. - List validation: "non-empty" was insufficient -- a truncated 200 or an HTML error page containing one IP could replace a good ~1400-entry list with a handful and silently disable metering. Now requires a plausible minimum and rejects implausible shrinkage. - Refresh backoff (30s/1m/5m/15m) while the list is empty; previously one transient failure at startup disabled the ceiling for a full hour. - Refresher panic is now logged at error!; previously it would freeze the exit list forever, silently. - Exhaustion logged at warn! not info! -- it means real users are being refused. - 429 copy no longer advises privacy users to stop using Tor, and no longer states the exact ceiling (which told an attacker what budget to drain). - Per-IP 429 message interpolates the constant instead of hardcoding a stale 4. - river-invite-button.html rendered every sub-hour retry as "approximately 1 hour(s)". The Tor window is 60 min, so this PR made that routinely wrong; now renders seconds/minutes/hours, and treats 0 as a value rather than absent. - Cargo.toml comment corrected: it claimed to avoid openssl, which was false (integration_test pulls native-tls in via feature unification). ## Testing 35 unit tests + 1 HTTPS startup integration test. New handler tests close the gap every reviewer flagged: nothing previously exercised the wiring, so dropping try_acquire or reordering it past the per-IP check left the suite green and the limiter defeated. Mutation-verified -- both regressions are caught by 4 tests each. New CI job (rust-api-tests.yml): CI never built rust/api at all, so none of these tests ran and a compile error would merge green. Refs #81 [AI-assisted - Claude]
Consolidated review: 5 reviewers, 20+ findings, all addressedFull-tier review per the multi-model rule: four Claude perspectives (code-first, The one that mattered: HTTPS startup panicFound independently by code-first and skeptical, from feature analysis
Nothing would have caught it: no unit test constructs a Fixed by installing the aws-lc-rs provider explicitly in The sizing constant was wrong, and would have blocked real usersbig-picture and code-first both flagged that my burst numbers disagreed Re-derived from one dataset, excluding the burst window by time:
A 25/hour ceiling would have refused real users. Default is now 60 (above Honest caveat now in the code: hours 04Z (28 invites / 25 exits) and 05Z (33/33) The central claim was false and has been removedEvery reviewer independently reached the same conclusion, and they are right: Sustaining the ceiling costs an attacker ~150 exits/day out of ~1400 available, That is inherent to any aggregate cap over an unlinkable pool, not an Also fixed
Testing gap closedtesting and skeptical both found that the wiring this PR calls Six handler tests added, and mutation-verified: both regressions are caught testing also found that CI never built Now: 35 unit tests + 1 HTTPS startup integration test. Not fixed, deliberately
Status: NOT deployed, and I don't think it should be without your callIan, you authorised deploy — but that was on the strength of two claims I have The immediate spam is already handled out of band (the room's ban FIFO was [AI-assisted - Claude] |
Problem
gkapi's per-IP invite limit (4/24h) is structurally defeated by Tor circuit
rotation: every exit node is a fresh IP and therefore a fresh bucket.
Measured from
invite_rate_limits.json(1728 invites,2026-07-24T06:23Z–2026-07-25T05:54Z), classified against the official bulk exit
list: the 02:35–03:05Z burst was 246 invites from 202 distinct IPs, 200 of
them via 159 Tor exits — with no single IP near the per-IP limit. Tightening
the per-IP number cannot fix this; rotation routes around it at any value.
Approach
The Tor exit set is enumerable, so meter the whole set through one shared
hourly bucket instead of treating exits as independent identities.
Ceiling: 60/hour, overridable at runtime (
TOR_INVITES_PER_HOUR,0disables).
An earlier revision used 25, derived by excluding high hours as "burst" by
magnitude and taking the max of the rest — circular, and it understated the
peak 3x. 25 would have refused real users. 60 clears the measured organic
max with ~1.8x headroom and still cuts the burst hour ~71%.
This is a judgement call on ambiguous data, not a derived constant: hours 04Z
(28 invites / 25 exits) and 05Z (33/33) run ~1.0 invites per exit, which looks
like many real users — but a one-request-per-exit attacker is indistinguishable
from that by volume alone.
What this costs, stated plainly
Not a Tor block: Tor is ~7% of non-burst invite traffic (106 requests, 90 exits),
nearly all 1–2 requests each. A block denies those users in all states.
But the bucket is not free, and an earlier version of this PR was wrong to
say ordinary Tor users are unaffected. Sharing one budget across an anonymity set
means the loudest member can take all of it. Sustaining the ceiling costs an
attacker ~150 exits/day out of ~1400 available, refused requests cost them
nothing, and an automated poller beats a human to every freed slot. Before this
change, denying Tor users invites was not possible; after it, it is cheap.
That is inherent to any aggregate cap over an unlinkable pool. Closing it needs a
per-request cost signal — proof-of-work — which is #81 and remains the real fix.
Treat this as rate-shaping that buys time.
Failure policy: fails OPEN
No list and no cache ⇒ nothing is treated as Tor ⇒ pre-PR behaviour. A garbage or
truncated 200 never replaces a good list (plausible-minimum + shrinkage checks); a
fetch error never clears one; the cache is written write-then-rename; poisoned
locks resolve to "not Tor" / "has capacity".
0disables the ceiling outright.Handler ordering is load-bearing
The cheap capacity pre-check runs before the per-IP check, so a ceiling
refusal doesn't burn the requester's per-IP allowance. The atomic
try_acquire()— the actual admission authority — runs only after the per-IP check passes,
so a per-IP rejection never consumes shared Tor capacity. Both directions are now
pinned by mutation-verified tests.
Testing
35 unit tests + 1 HTTPS startup integration test.
try_acquireor reordering it left the suite green and the limiter defeated).Mutation-verified — each regression is caught by 4 tests.
try_acquireatomicity.tests/https_startup.rsspawns the real binary with real TLS material;verified to fail without the crypto-provider fix.
#[ignore]d live Tor-list fetch (1386 exits, cache round-trip):cargo test --bins -- --ignoredNew CI job (
rust-api-tests.yml): CI previously never builtrust/apiatall, so none of these tests ran and a compile error would have merged green.
Operational surface (new)
check.torproject.org/var/lib/gkapi/tor_exit_list.txt--tor-exit-cache/TOR_EXIT_CACHE,--tor-invites-per-hour/TOR_INVITES_PER_HOURreqwest,rustls(the latter required to fix the provider clash)cargo fmtreflow inerrors.rs/invite.rsRefs #81
[AI-assisted - Claude]