Route documents over 32,000 characters to chunklaya

[myxamediyar]

Inputs over MAX_CHARS (32,000 characters) are answered by chunklaya, our own long-document service (myxamediyar/chunklaya, serve/), when CHUNKLAYA_URL and CHUNKLAYA_TOKEN are set and CHUNKLAYA_ENABLED is "true". Without them, those requests keep answering 400 input_too_long, so the change is safe to deploy before the secrets exist.

Routing

• Only inputs over 32,000 characters are affected; everything at or under is untouched.
• Explicit model: "jev" keeps the 32,000 ceiling; model: "chunklaya" selects the service for shorter text.
• Results are labelled chunklaya/multilingual.
• Ceilings: 4,000,000 characters per input, 20 inputs per request; tier: "smart" is refused with bad_tier.
• Service refusals map to chunklaya_input (400), chunklaya_busy (429, Retry-After) and chunklaya_unavailable (503), in our own words. No fallback to Jev or the LLM chain.

Transport — a fourth Backend in src/jev.ts over the bearer transport Beam uses: URL and token from the environment, one document per request, 60 s deadline, no per-token provider cost. dimensions pack against its limits so all questions about a document go in one request.

Billing — zero rate in retail-rates.json (version 2026-09-23-chunklaya-v3), the spending price table and providerCallBound.

Surfaces — docs (LONG DOCUMENTS, parameters, error list), OpenAPI (model enum, error codes, limits), MCP enums, CLI (--model chunklaya), wrangler.example.toml, AGENTS.md.

Deploy — deploy.yml passes CHUNKLAYA_URL and CHUNKLAYA_TOKEN to the Worker when both are repository secrets, and omits them otherwise.

Checks — npm run typecheck clean; npm test 797 pass / 0 fail (8 new in test/chunklaya.test.ts); cd cli && node --test 27 pass.

🤖 Generated with Claude Code

Summary by CodeRabbit

• New Features
  • Added Chunklaya support for classifying long documents up to 4,000,000 characters.
  • Automatically routes oversized documents to Chunklaya when enabled and configured.
  • Added chunklaya as a supported model across the CLI, API, MCP tools, and classification workflows.
  • Added clear errors for invalid, busy, or unavailable Chunklaya requests.
  • Documented limits, configuration, and long-document behavior.
  • Added free-tier pricing support for Chunklaya.

[coderabbitai]

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 15f681cf-0ed0-4cca-b6f9-3e7fe39231ee

📥 Commits

Reviewing files that changed from the base of the PR and between b52c5f0 and d9e89a1.

📒 Files selected for processing (18)

• .github/workflows/deploy.yml
• AGENTS.md
• cli/classify.js
• src/cost.ts
• src/dimensions.ts
• src/docs.ts
• src/http/spending-classification.ts
• src/index.ts
• src/jev-observability.ts
• src/jev.ts
• src/mcp.ts
• src/openapi.ts
• src/retail-rates.json
• src/server/token-pricing.ts
• src/server/token-reservation.ts
• src/spending/policy.ts
• test/chunklaya.test.ts
• wrangler.example.toml

 _____________________
< Git gud or git out. >
 ---------------------
  \
   \   (\__/)
       (•ㅅ•)
       / 　 づ

✨ Finishing Touches

📝 Generate docstrings

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}