Our per-token rate limit can be bypassed.
In server/verifyTokenAndRateLimit.ts (around line 48) we call rateLimiter.consume(token), where token is the argon2 hash the client sends. The client generates that hash fresh each time, with a new random salt (client/modules/searchTokenHash.ts), so every request can carry a different valid token. Different key each time => the 10-req/10s limit never really bites.
The same value also goes into verifiedTokens.add(token) (server/verifiedTokens.ts), which is a plain Set with no bound or TTL. Over time that set just keeps growing (slow memory growth).
What we can do:
- Key the rate limiter on the client IP instead of the token (careful with
X-Forwarded-For behind the HF Spaces proxy, so we read the real client IP and not the proxy's).
- Bound
verifiedTokens (a TTL or LRU), or rethink whether we need to keep every verified token in memory at all.
I want to get the IP handling right for the Hugging Face deployment specifically, since that's the main public instance. Anyone knows offhand which header HF Spaces sets for the client IP? (Otherwise I'll dig into it during implementation.)
Note: this changes runtime behavior a bit (users behind the same IP would share a bucket), so worth calling out in the PR.
Our per-token rate limit can be bypassed.
In
server/verifyTokenAndRateLimit.ts(around line 48) we callrateLimiter.consume(token), wheretokenis the argon2 hash the client sends. The client generates that hash fresh each time, with a new random salt (client/modules/searchTokenHash.ts), so every request can carry a different valid token. Different key each time => the 10-req/10s limit never really bites.The same value also goes into
verifiedTokens.add(token)(server/verifiedTokens.ts), which is a plainSetwith no bound or TTL. Over time that set just keeps growing (slow memory growth).What we can do:
X-Forwarded-Forbehind the HF Spaces proxy, so we read the real client IP and not the proxy's).verifiedTokens(a TTL or LRU), or rethink whether we need to keep every verified token in memory at all.I want to get the IP handling right for the Hugging Face deployment specifically, since that's the main public instance. Anyone knows offhand which header HF Spaces sets for the client IP? (Otherwise I'll dig into it during implementation.)
Note: this changes runtime behavior a bit (users behind the same IP would share a bucket), so worth calling out in the PR.