fix(research): return 503, not 500, when the events provider is at capacity - #828
Conversation
…pacity Social scrapes and event lookups share one Apify account. Measured 2026-08-11: 64GB max actor memory, 32 max concurrent jobs, and the Bandsintown actor takes about 4GB per run, so roughly 16 can be in flight across every caller. Past that Apify refuses to launch with "By launching this job you will exceed the memory limit of 65536MB". The handler's catch-all turned that into a 500 carrying Apify's own text. Two problems: a 500 tells callers their request is broken and to give up, when the request is fine and a retry would succeed; and it leaks provider internals to API consumers. Adds isApifyCapacityError and maps capacity rejections to 503 with a clean message. A failed actor run stays a 500, since that is a genuine upstream fault and callers should treat the two differently. Observed in production: a 67-artist roster sweep lost 2 artists to this. Reproduced at 4 failures in 20 concurrent requests, 0 in 16. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Bugbot is not enabled for your account, so this pull request was not reviewed. Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs. |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
📝 WalkthroughWalkthroughThe PR adds an Apify capacity-error classifier and uses it in the research events handler. Detected capacity errors now return a sanitized 503 response with retry guidance instead of a generic 500 response. ChangesApify capacity handling
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
lib/research/postResearchEventsHandler.ts (1)
78-88: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winReduce the handler size.
postResearchEventsHandlerspans Lines 34-94. Extract the provider call and error mapping into focused private helpers. This keeps request orchestration separate from provider failure classification and makes the 503/500 contract easier to test.As per coding guidelines, “Flag functions longer than 20 lines” and “Keep functions small and focused.” As per path instructions, “Keep functions under 50 lines.”
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@lib/research/postResearchEventsHandler.ts` around lines 78 - 88, Refactor postResearchEventsHandler so its provider invocation and error-to-response mapping move into focused private helper functions. Keep the handler responsible only for request orchestration, preserve the existing isApifyCapacityError 503 response and other 500 behavior, and ensure each function remains within the project’s size limits.Sources: Coding guidelines, Path instructions
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@lib/research/postResearchEventsHandler.ts`:
- Around line 78-88: Scope the isApifyCapacityError check in the post-research
events handler to errors originating from fetchBandsintownEvents, preventing
validation or artist lookup failures with matching text from returning 503.
Preserve the existing 500 behavior for those lookup errors and add a regression
test asserting HTTP 500.
---
Nitpick comments:
In `@lib/research/postResearchEventsHandler.ts`:
- Around line 78-88: Refactor postResearchEventsHandler so its provider
invocation and error-to-response mapping move into focused private helper
functions. Keep the handler responsible only for request orchestration, preserve
the existing isApifyCapacityError 503 response and other 500 behavior, and
ensure each function remains within the project’s size limits.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 72591d02-5e3d-43c8-8d27-5fe47bb8b9f2
⛔ Files ignored due to path filters (2)
lib/apify/__tests__/isApifyCapacityError.test.tsis excluded by!**/*.test.*,!**/__tests__/**and included bylib/**lib/research/__tests__/postResearchEventsHandler.test.tsis excluded by!**/*.test.*,!**/__tests__/**and included bylib/**
📒 Files selected for processing (2)
lib/apify/isApifyCapacityError.tslib/research/postResearchEventsHandler.ts
| // A saturated provider quota is a capacity condition, not a server fault: | ||
| // the caller's request is fine and retrying later will work. Reporting it | ||
| // as a 500 tells callers to give up, and echoing the provider's own text | ||
| // leaks our infrastructure to them. | ||
| if (isApifyCapacityError(error)) { | ||
| return errorResponse( | ||
| "Events provider is at capacity. Retry this request after a short delay.", | ||
| 503, | ||
| ); | ||
| } | ||
|
|
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Scope capacity detection to the Apify fetch.
This outer catch also handles validatePostResearchEventsRequest, getArtists, and getArtistBandsintownId. If one of those operations throws an error containing rate limit exceeded or another configured signature, the handler returns a misleading 503 stating that the Events provider is at capacity.
Apply isApifyCapacityError only around fetchBandsintownEvents, or require an Apify-specific error source before returning 503. Add a regression test for a lookup error with the same message and expect HTTP 500.
Suggested scoping
- const events = await fetchBandsintownEvents({
- bandsintownId,
- ...(validated.date && { date: validated.date }),
- });
+ let events: Awaited<ReturnType<typeof fetchBandsintownEvents>>;
+ try {
+ events = await fetchBandsintownEvents({
+ bandsintownId,
+ ...(validated.date && { date: validated.date }),
+ });
+ } catch (error) {
+ if (isApifyCapacityError(error)) {
+ return errorResponse(
+ "Events provider is at capacity. Retry this request after a short delay.",
+ 503,
+ );
+ }
+ throw error;
+ }
@@
- if (isApifyCapacityError(error)) {
- return errorResponse(
- "Events provider is at capacity. Retry this request after a short delay.",
- 503,
- );
- }
-📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| // A saturated provider quota is a capacity condition, not a server fault: | |
| // the caller's request is fine and retrying later will work. Reporting it | |
| // as a 500 tells callers to give up, and echoing the provider's own text | |
| // leaks our infrastructure to them. | |
| if (isApifyCapacityError(error)) { | |
| return errorResponse( | |
| "Events provider is at capacity. Retry this request after a short delay.", | |
| 503, | |
| ); | |
| } | |
| let events: Awaited<ReturnType<typeof fetchBandsintownEvents>>; | |
| try { | |
| events = await fetchBandsintownEvents({ | |
| bandsintownId, | |
| ...(validated.date && { date: validated.date }), | |
| }); | |
| } catch (error) { | |
| if (isApifyCapacityError(error)) { | |
| return errorResponse( | |
| "Events provider is at capacity. Retry this request after a short delay.", | |
| 503, | |
| ); | |
| } | |
| throw error; | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@lib/research/postResearchEventsHandler.ts` around lines 78 - 88, Scope the
isApifyCapacityError check in the post-research events handler to errors
originating from fetchBandsintownEvents, preventing validation or artist lookup
failures with matching text from returning 503. Preserve the existing 500
behavior for those lookup errors and add a regression test asserting HTTP 500.
POST /api/research/eventsreturns a 500 carrying Apify's own error text when the shared provider quota is saturated. Found by running a real 67-artist customer roster through the live endpoint: two artists were silently lost.Tracking issue: recoupable/chat#1954
What is actually happening
Social scrapes and event lookups run on one shared Apify account. Measured directly on 2026-08-11:
The Bandsintown actor takes roughly 4GB per run, so about 16 can be in flight at once across every caller. Past that, Apify refuses to launch:
Reproduced cleanly:
The failing artists are not special. Called individually they all return 200 in under 3s. It is purely a function of how many runs are in flight.
Why the current behaviour is wrong twice over
Both matter in practice. The agent that consumes this endpoint saw
HTTP 500 after retry, counted the artists as failures, and moved on. Neither had upcoming shows so nothing was lost that day, but a failed lookup and an artist with no shows are indistinguishable to the caller, which is the silent-wrong-answer shape this endpoint exists to avoid.The change
isApifyCapacityErrorrecognises the account-level rejections (memory limit, concurrent-run cap, rate limit) and the handler maps them to:{ "status": "error", "error": "Events provider is at capacity. Retry this request after a short delay." }A failed actor run stays a 500. That is a genuine upstream fault, and conflating "the provider is busy" with "the run broke" would remove the caller's ability to decide whether retrying is worth it. There is a test asserting the two stay distinct.
Verification
vitest run lib/research lib/apify lib/artistseslinton touched filesBoth units were written RED → GREEN: the detector failed on missing module, the handler test failed with
expected 500 to be 503.Two follow-ups this does not do
Retry-Afterheader. Genuinely useful for a capacity signal, but the sharederrorResponsehelper takes no headers and widening it touches every endpoint in the repo. Worth doing deliberately rather than as a side effect here.200/400/401/402/404and already omitted500; this adds503. The contract gap is now wider and should be closed in adocsPR, deferred per direction.The consuming task prompt has been updated separately to cap its own concurrency at 5, run its scrape pass and its events pass sequentially rather than in parallel, and back off exponentially on this response instead of retrying once.
🤖 Generated with Claude Code
Summary by cubic
Return 503 instead of 500 from
POST /api/research/eventswhen the provider is at capacity, so clients know to retry and we don’t leak provider internals.isApifyCapacityErrorand map them to 503 with a clean message.Written for commit 8322c28. Summary will update on new commits.
Summary by CodeRabbit