Skip to content

The Usage chart shows 4.2M tokens for the day the 250M ceiling was tripped — the daily series drops the cache columns, which are 98.2% of what the ceiling counts #547

Description

@serge-ivo

The owner's account of it

Six agents were stopped overnight by the 250M-token daily circuit breaker, twice, the second time at iteration 0 with 268M tokens counted (transcript quoted in #485). He then opened the Usage page to find the tokens. The page's daily chart, for the single most expensive day in the ledger, tops out at 4.2M tokens.

Measured

usage_summary on production, 2026-08-13.

The daily series for 2026-08-11 — the largest day in the whole ledger:

{ "date": "2026-08-11", "inputTokens": 2192612, "outputTokens": 2032986,
  "costMicros": 6457081537, "calls": 822 }

inputTokens + outputTokens = 4,225,598. That is what the chart plots. That day's notional value is $6,457.08.

The same window's totals, which DO carry the cache columns:

7d  input 5,907,475 · output 3,582,926 · cacheRead 1,278,844,405 · cacheWrite 14,548,104

So over the last seven days:

  • The chart's "Tokens" metric sums to 9,490,401.
  • The quantity accountUsageSince counts against the 250M ceiling sums to 1,302,882,910.
  • 137x apart. Cache reads alone are 98.2% of the counted total.

Live confirmation of the ceiling's own arithmetic, GET /v1/budget/limits, 2026-08-13:

"consumption": { "chargedMicros": 6763290, "tokens": 30710799 }

30.7M tokens in the trailing 24h, against a day whose chart-visible tokens are under 1M.

Mechanism — two places, one omission

The ceiling counts cache. workers/api/src/lib/usage.ts:643-645:

COALESCE(SUM(COALESCE(input_tokens, 0) + COALESCE(output_tokens, 0)
             + COALESCE(cache_read_tokens, 0) + COALESCE(cache_write_tokens, 0)), 0) AS tokens

The daily series does not carry cache. usage.ts:314:

daily: Array<{ date: string; inputTokens: number; outputTokens: number; costMicros: number; calls: number }>;

and usage.ts:386 builds each entry from only those four fields, although the UsageBucket behind it (dayMap) has already accumulated cacheReadTokens and cacheWriteTokens via bump() at usage.ts:330-331. The numbers are computed and then discarded on the way out.

The console mirrors the shape it is given. store/console/src/pages/Usage.tsx:10 declares interface Day with no cache fields, and :95:

const vals = daily.map((d) => (metric === "cost" ? d.costMicros : d.inputTokens + d.outputTokens));

Note this is not an oversight of principle elsewhere on the page: the totals strip (Usage.tsx:284) and the per-row TokenCell (:136-148) both include cache and both say so. It is only the time series — the one view that could answer "which day did I blow the ceiling, and on what?" — that drops it.

Why it composed into a bug

  1. [docs] Rewrite Job Application Assistant docs for JobApplyWorkflow #212 split cache out of input_tokens so the hit rate became visible and cost stopped being overstated. Correct.
  2. The ceiling was written to count every token regardless of payer (usage.ts:614-619) because a subscription runaway consumes tokens, not dollars. Also correct.

Composed: the ceiling counts a quantity that is 98% cache, and the only chart that could show it over time was written before the cache columns existed and never grew them. A user who trips the token breaker is sent to a page that contradicts it.

What to do, cheapest first

  1. Widen the daily row. Add cacheReadTokens / cacheWriteTokens to UsageSummary["daily"] and to the object built at usage.ts:386. Both values are already in b. Two fields, no query change, no migration.
  2. Make the chart's "Tokens" metric the ceiling's metric. Usage.tsx:95d.inputTokens + d.outputTokens + (d.cacheReadTokens ?? 0) + (d.cacheWriteTokens ?? 0). The tooltip at :111 should break it out (4.2M I/O + 850M cache) so the composition is legible, not just the magnitude.
  3. Then, and only then, draw the ceiling on it. A horizontal 250M/day rule on the token chart is what turns "you were stopped" into "here is the day, and here is why". Worth its own ticket; it is worthless before step 1 because the series it would sit above is off by two orders of magnitude.

Alternatives considered and rejected

Acceptance criteria

  • GET /v1/usage returns cacheReadTokens and cacheWriteTokens on every daily entry.
  • For this account, 2026-08-11's entry reports non-zero cache tokens and the chart's tokens metric for that day is within rounding of the sum of the four columns.
  • sum(daily[].{input,output,cacheRead,cacheWrite}) equals totals for the same range — assert it in usage.test.ts; today that identity silently fails for two of the four columns.
  • The chart tooltip states the I/O and cache split rather than one number.

Regression risk

The chart's y-axis will jump ~137x for any account with engine rows, which is the point, but it will also flatten the pre-engine days to invisibility. Check the denseDays zero-days still render as gaps rather than being lost in the new scale, and keep the "Cost" metric untouched — that one is already correct.

Measured via MCP usage_summary and GET /v1/budget/limits on production; the code paths are read from source.
Related: #212 (split cache out), #270 (what the estimate excludes), #485 (which ceiling to enforce), #543, #544.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    backendBackend / Worker / API workbugSomething isn't workingfrontendFrontend / UI work

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions