Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/tools-and-cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -194,7 +194,7 @@ Readings: the plugin's last readings, taken 2026-09-27 13:17:22 (14m ago); --ref
Pool: 93% used of 81x across 11 accounts
```

- **Snapshot-backed, not live.** The plugin polls `/wham/usage` for the pool status line and keeps the last readings in `oc-codex-multi-auth-tui-quota-overview.json` under the OpenCode state dir (`$OPENCODE_STATE_DIR`, else `$XDG_STATE_HOME/opencode` or `~/.local/state/opencode`). `limits` reports those readings; accounts with no snapshot entry (or a rotated token fingerprint) are read live. `--refresh` reads the whole pool live, and a full live read becomes the plugin's new snapshot. Nothing is written for `--tag` subsets, `--config-path` stores, or when the snapshot no longer describes the pool.
- **Snapshot-backed, not live.** The plugin polls `/wham/usage` for the pool status line and keeps the last readings in `oc-codex-multi-auth-tui-quota-overview.json` under the OpenCode state dir (`$OPENCODE_STATE_DIR`, else `$XDG_STATE_HOME/opencode` or `~/.local/state/opencode`). `limits` reports those readings; accounts with no snapshot entry (or a rotated token fingerprint) are read live. An account the plugin can no longer read is not read live either: when the poller keeps an account's last good reading because its credentials are dead (a refused refresh, an invalidated token, a deactivated workspace) it records since when and why, and the request path marks an account whose credentials were refused (`auth-failure` cooldown, counted while it runs or while the stored access token stays expired). `limits` shows such an account's last known figures under an `Error:` line carrying that reason (for example a refresh token that needs a new `opencode auth login`), leaves it out of the pool total, and exits 1. A transient failure (timeout, network error, 429, 5xx) does not mark an account; its older `Read:` time says the figures are old. `limits` never refreshes a token for an account with a snapshot entry; an account read live because it has none can have its token refreshed. `--refresh` reads the whole pool live, and a full live read becomes the plugin's new snapshot. Nothing is written for `--tag` subsets, `--config-path` stores, or when the snapshot no longer describes the pool.
- **Workspace names.** Business seats show their ChatGPT workspace name (owner-titled) via one `/wham/accounts/check` per login, cached in `oc-codex-multi-auth-workspace-names.json` beside the quota snapshot; lookup gives up after ~5s and a failure only drops the line.
- **Sorting.** `--sort usage|reset` judges each account by its governing window — the one with least headroom, and on ties the later reset. Accounts with no readable value sort last. Persist a default via `"limitsSort": { "by": "reset", "direction": "asc" }` in `~/.opencode/openai-codex-auth-config.json`.
- **Pool total.** `81x` is the sum of per-plan seat weights (see [plan allotments](plan-allotments.md)); the percentage is the weighted mean over exactly that sum, not a plain average. Plans with no published ratio weigh one baseline seat and print no `Nx` badge. Accounts with unreadable usage are excluded from both figures; `pool` is `null` in `--json` when nothing was readable. Both `used` and `left` percentages are emitted so `quotaDisplay` wording never changes the data.
Expand Down
85 changes: 83 additions & 2 deletions lib/codex-usage.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,10 @@ import { CODEX_BASE_URL, PLUGIN_NAME } from "./constants.js";
import {
createDeactivatedWorkspaceError,
createUsageRequestTimeoutError,
isDeactivatedWorkspaceErrorMessage,
isInvalidatedAuthTokenMessage,
} from "./error-sentinels.js";
import { CodexAuthError } from "./errors.js";
import { logWarn } from "./logger.js";
import {
DEFAULT_QUOTA_DISPLAY_MODE,
Expand Down Expand Up @@ -754,6 +757,69 @@ export function parseCodexUsagePayload(
};
}

/**
* Whether a failed usage read says the account's credentials are dead, as
* opposed to a timeout, a network error, a rate limit or an upstream outage.
* Only the first means the account cannot serve requests until someone logs
* in again: a refresh the token endpoint refused, an access token the backend
* reports invalidated, or a deactivated workspace.
*/
export function isCodexCredentialFailure(error: unknown): boolean {
if (error instanceof CodexAuthError && error.refreshFailureReason !== undefined) {
return !error.retryable;
}
const message = error instanceof Error ? error.message : undefined;
return isInvalidatedAuthTokenMessage(message) || isDeactivatedWorkspaceErrorMessage(message);
}

/**
* Decode the inside of a JSON string literal that may have been cut off
* mid-escape, as a bounded error body is.
*/
function decodeJsonStringFragment(raw: string): string {
for (const candidate of [raw, raw.replace(/\\(?:u[0-9a-fA-F]{0,3})?$/, "")]) {
try {
return JSON.parse(`"${candidate}"`) as string;
} catch {
// Try the fragment with a dangling escape removed.
}
}
return raw;
}

/**
* One readable line out of a failed request's error text. The OAuth refresh
* failure carries the endpoint's JSON body - pretty-printed, and cut to a
* bounded length before it reaches here, so often not parseable - which renders
* as a lone `{` on a report line. The human message inside it is what says
* what happened, so it is read out of the body, whole or truncated. Text that
* holds no such message is only collapsed onto one line.
*/
export function summarizeCodexErrorMessage(text: string, maxChars = 200): string {
const start = text.indexOf("{");
let summary: string | undefined;
if (start !== -1) {
const body = text.slice(start);
const match =
/"(?:message|error_description)"\s*:\s*"((?:[^"\\]|\\.)*)("?)/.exec(body) ??
/"error"\s*:\s*"((?:[^"\\]|\\.)*)("?)/.exec(body);
const message = match?.[1] === undefined ? undefined : decodeJsonStringFragment(match[1]).trim();
Comment thread
greptile-apps[bot] marked this conversation as resolved.
if (message) {
const complete = match?.[2] === '"';
const prefix = text.slice(0, start).trim().replace(/:$/, "");
const readable = complete ? message : `${message.replace(/\.*$/, "")}…`;
summary = prefix ? `${prefix}: ${readable}` : readable;
}
}
// Decoding turns an escaped `\u001b` into a live ESC, so control characters
// are dropped before the line can reach a terminal.
const line = (summary ?? text)
.replace(/[\u0000-\u001f\u007f-\u009f]/g, " ")
.replace(/\s+/g, " ")
.trim();
return line.length > maxChars ? `${line.slice(0, maxChars - 1)}…` : line;
Comment on lines +804 to +820

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 7daeda9: the captured message is decoded with JSON.parse as a string literal. For a body cut off mid-escape, a dangling \ or partial \uXXX is dropped first. Covered by the new summarizeCodexErrorMessage tests (\n, \", \u00e9, truncated escape).

}

/**
* Build a safe error message from a failed Codex backend response.
*
Expand Down Expand Up @@ -973,11 +1039,26 @@ export async function ensureCodexUsageAccessToken(params: {

const previousRefreshToken = params.account.refreshToken;
if (!previousRefreshToken) {
throw new Error("Cannot refresh: account has no refresh token");
throw new CodexAuthError("Cannot refresh: account has no refresh token", {
refreshFailureReason: "missing_refresh",
});
}
const refreshResult = await coordinatePersistedRefresh(params.account);
if (refreshResult.type !== "success") {
throw new Error(refreshResult.message ?? refreshResult.reason);
// Same transient rule as the request path's `refreshAndUpdateToken`,
// so a caller can tell a dead refresh token from a flaky network.
const statusCode =
typeof refreshResult.statusCode === "number" ? refreshResult.statusCode : undefined;
throw new CodexAuthError(refreshResult.message ?? refreshResult.reason ?? "token refresh failed", {
retryable:
refreshResult.reason === "network_error" ||
refreshResult.reason === "invalid_response" ||
(refreshResult.reason === "http_error" &&
statusCode !== undefined &&
(statusCode >= 500 || statusCode === 408 || statusCode === 429)),
refreshFailureReason: refreshResult.reason,
statusCode,
});
}
let refreshedCount = 0;
for (const storedAccount of params.storage.accounts) {
Expand Down
17 changes: 16 additions & 1 deletion lib/tui-quota-cache.ts
Original file line number Diff line number Diff line change
Expand Up @@ -385,6 +385,18 @@ export type TuiQuotaOverviewAccount = {
* build wrote, where the snapshot's time is all there is.
*/
fetchedAt?: number;
/**
* Set while the poller cannot read this account because its credentials
* are dead, and keeps its previous reading instead: when the latest such
* read failed, when they started failing, and why. The reading then
* describes the account as it was at `fetchedAt`, not as it is - a dead
* refresh token keeps failing, and its last reading would otherwise pass
* for a healthy, idle seat. A transient failure (timeout, network, 429,
* 5xx) sets none of these.
*/
readFailedAt?: number;
readFailedSince?: number;
readError?: string;
};

/**
Expand Down Expand Up @@ -420,7 +432,10 @@ function isTuiQuotaOverviewAccount(
(typeof value.resetCreditsApplicable === "number" && Number.isInteger(value.resetCreditsApplicable) && value.resetCreditsApplicable >= 0)) &&
Array.isArray(value.limits) &&
value.limits.every(isTuiQuotaLimit) &&
isOptionalFiniteNumber(value.fetchedAt)
isOptionalFiniteNumber(value.fetchedAt) &&
isOptionalFiniteNumber(value.readFailedAt) &&
isOptionalFiniteNumber(value.readFailedSince) &&
(value.readError === undefined || typeof value.readError === "string")
);
}

Expand Down
60 changes: 45 additions & 15 deletions lib/tui-quota-overview.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,15 @@ import {
ensureCodexUsageAccessToken,
fetchCodexUsage,
getUsageLeftPercent,
isCodexCredentialFailure,
hasUsageWindow,
parseCodexUsagePayload,
resolveCodexUsageAccountId,
summarizeCodexErrorMessage,
type CodexUsageSummary,
type LimitWindow,
} from "./codex-usage.js";
import { logDebug } from "./logger.js";
import { logDebug, maskString } from "./logger.js";
import type { QuotaOverviewAccount } from "./quota-overview.js";
import { loadAccounts, type AccountStorageV3 } from "./storage.js";
import {
Expand Down Expand Up @@ -98,10 +100,14 @@ export function toOverviewAccount(params: {
};
}

type OverviewFetchResult =
| { reading: TuiQuotaOverviewAccount }
| { error: string; credential: boolean };

async function fetchOverviewAccount(
storage: AccountStorageV3,
index: number,
): Promise<TuiQuotaOverviewAccount | undefined> {
): Promise<OverviewFetchResult | undefined> {
const account = storage.accounts[index];
if (!account) return undefined;
try {
Expand All @@ -110,7 +116,9 @@ async function fetchOverviewAccount(
account,
accessToken: credentials.accessToken,
});
if (!accountId) return undefined;
if (!accountId) {
return { error: "could not resolve account id (re-login may be required)", credential: true };
}
const usage = parseCodexUsagePayload(
await fetchCodexUsage({
accountId,
Expand All @@ -119,18 +127,23 @@ async function fetchOverviewAccount(
normalizeAccountErrors: true,
}),
);
return toOverviewAccount({
fingerprint: createUsageAccountFingerprint(account),
index: index + 1,
usage,
email: account.email,
label: account.accountLabel,
});
return {
reading: toOverviewAccount({
fingerprint: createUsageAccountFingerprint(account),
index: index + 1,
usage,
email: account.email,
label: account.accountLabel,
}),
};
} catch (error) {
logDebug(
`Failed to fetch pool quota for one account: ${(error as Error).message}`,
// The refresh endpoint's error body can echo token material, so the
// reason is masked before it is written to a shared cache file.
const message = maskString(
summarizeCodexErrorMessage(error instanceof Error ? error.message : String(error)),
);
return undefined;
logDebug(`Failed to fetch pool quota for one account: ${message}`);
return { error: message, credential: isCodexCredentialFailure(error) };
}
}

Expand All @@ -157,8 +170,8 @@ export async function fetchTuiQuotaOverview(params: {
chunk.map((index) => fetchOverviewAccount(storage, index)),
);
for (const [position, result] of results.entries()) {
if (result) {
accounts.push({ ...result, fetchedAt: now });
if (result && "reading" in result) {
accounts.push({ ...result.reading, fetchedAt: now });
continue;
}
// A failed fetch must not drop the account out of the snapshot: the
Expand All @@ -182,9 +195,19 @@ export async function fetchTuiQuotaOverview(params: {
createUsageAccountFingerprint(account),
);
if (previous) {
// Only dead credentials mark the reading as describing an account
// that cannot serve requests. A transient failure keeps whatever
// an earlier poll concluded; the older `fetchedAt` already dates it.
accounts.push({
...previous,
fetchedAt: previous.fetchedAt ?? cached?.fetchedAt,
...(result && "credential" in result && result.credential
? {
readFailedAt: now,
readFailedSince: previous.readFailedSince ?? previous.readFailedAt ?? now,
readError: result.error,
}
: {}),
});
carriedOver = true;
}
Expand Down Expand Up @@ -244,12 +267,19 @@ export function mergeOverviewWithLatestAccount(
return account;
}
merged = true;
// A response that came back after the latest failed poll proves the
// account works again. One from before it proves nothing about it.
const recovered =
account.readFailedAt === undefined || latest.fetchedAt > account.readFailedAt;
return {
...account,
planType: latest.planType ?? account.planType,
email: account.email ?? (latest.accountEmail?.trim() || undefined),
limits: latest.limits,
fetchedAt: latest.fetchedAt,
...(recovered
? { readFailedAt: undefined, readFailedSince: undefined, readError: undefined }
: {}),
};
});
return merged ? { ...snapshot, accounts } : snapshot;
Expand Down
55 changes: 48 additions & 7 deletions scripts/install-oc-codex-multi-auth-core.js
Original file line number Diff line number Diff line change
Expand Up @@ -1928,6 +1928,7 @@ async function runLimitsCommandInner(parsed, options = {}) {
accountId,
accessToken,
organizationId: account.organizationId,
normalizeAccountErrors: true,
}),
quotaDisplay,
);
Expand Down Expand Up @@ -2008,11 +2009,18 @@ async function runLimitsCommandInner(parsed, options = {}) {
const reading = cachedAccount
? toCachedLimitsReading(cachedAccount, usageMod, quotaDisplay)
: await readLive(account, index, entry);
poolMembers.push({
planType: reading.planType,
primary: reading.windows[0] ?? {},
secondary: reading.windows[1] ?? {},
});
const failure = cachedAccount && readPluginQuotaFailure(cachedAccount.account, account, now);
if (failure) {
// Its figures are last known, not capacity it can spend now.
entry.readFailure = failure;
failedCount += 1;
} else {
poolMembers.push({
planType: reading.planType,
primary: reading.windows[0] ?? {},
secondary: reading.windows[1] ?? {},
});
}
sortKeys.set(entry, readLimitsSortKeys(usageMod, reading.windows));
entry.source = reading.source;
entry.readAt = reading.readAt;
Expand All @@ -2028,7 +2036,9 @@ async function runLimitsCommandInner(parsed, options = {}) {
// response, so the message is redacted through the logger's token
// patterns before it reaches stdout, JSON output, or CI logs.
// Truncation alone does not protect bearer/JWT/refresh-token material.
entry.error = loggerMod.maskString(formatErrorForLog(error)).slice(0, 160);
entry.error = loggerMod.maskString(
usageMod.summarizeCodexErrorMessage?.(formatErrorForLog(error)) ?? formatErrorForLog(error).slice(0, 160),
);
failedCount += 1;
}
results.push(entry);
Expand Down Expand Up @@ -2171,6 +2181,31 @@ function findPluginQuotaReading(readings, account, usageMod) {
return { account: found, readAt: found.fetchedAt ?? readings.snapshot.fetchedAt };
}

/**
* What the plugin last knew had gone wrong with an account, from state it
* already holds - nothing here asks upstream. The poller keeps a failing
* account's last good reading and records when dead credentials made the reads
* fail; the request path marks an account whose credentials were refused. That
* mark counts while its cooldown runs, and after it only if the stored access
* token has also expired - no refresh has succeeded since, or it would have
* moved `expiresAt`. Either way the cached figures describe the account as it
* was, and it cannot serve requests now.
*/
function readPluginQuotaFailure(entry, account, now) {
if (Number.isFinite(entry.readFailedAt)) {
return {
since: Number.isFinite(entry.readFailedSince) ? entry.readFailedSince : entry.readFailedAt,
message: typeof entry.readError === "string" && entry.readError ? entry.readError : "the plugin could not read it",
};
}
const coolingDown = Number.isFinite(account.coolingDownUntil) && account.coolingDownUntil > now;
const tokenExpired = Number.isFinite(account.expiresAt) && account.expiresAt <= now;
if (account.cooldownReason === "auth-failure" && (coolingDown || tokenExpired)) {
return { since: undefined, message: "the plugin's last request with it was refused (auth failure)" };
}
return undefined;
}

/**
* A window nobody has drawn from reports "now plus the window" as its reset.
* The plugin now drops that reset, but a snapshot an older build wrote still
Expand Down Expand Up @@ -2325,7 +2360,7 @@ async function attachWorkspaceNames({ results, entryAccounts, entryWorkspaceIds,
let changed = false;
for (const entry of results) {
const workspaceId = entryWorkspaceIds.get(entry);
if (!workspaceId || known.has(workspaceId) || entry.error) continue;
if (!workspaceId || known.has(workspaceId) || entry.error || entry.readFailure) continue;
try {
const names = await lookup(entryAccounts.get(entry), entry.source === "live");
if (!names) continue;
Expand Down Expand Up @@ -2460,6 +2495,12 @@ function buildLimitsAccountRows(account, render, now, readings) {
rows.push(["Error", account.error]);
return rows;
}
if (account.readFailure) {
const since = Number.isFinite(account.readFailure.since)
? ` (failing since ${formatLimitsReadTime(account.readFailure.since, render, now)})`
: "";
rows.push(["Error", `${account.readFailure.message}${since}; last known figures below, left out of the pool total`]);
}
for (const limit of account.limits ?? []) {
rows.push([limit.name, formatLimitsPercent(limit, render)]);
const renewal = formatLimitsRenewal(limit, render, now);
Expand Down
Loading