Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,13 @@
# Repository agent instructions

## Hermes token analytics installation and upgrades

When a task asks you to install, set up, onboard, validate, repair, or upgrade `hermes-token-analytics`, you **must read and follow**:

- [`plugins/hermes-token-analytics/REQUIRED_INSTALLATION_CHECKLIST.md`](plugins/hermes-token-analytics/REQUIRED_INSTALLATION_CHECKLIST.md)

Do not declare a production installation complete after copying/enabling the plugin or after one successful manual sync. Unless the user explicitly requests a manual-only install, completion requires the wrapper, exactly one enabled recurring cron job, and a verified cron-triggered run. If a gateway restart is blocked from inside the gateway process, complete every independent scheduling step and report the restart as a remaining blocker.

<!-- intent-skills:start -->
# Skill mappings - load `use` with `npx @tanstack/intent@latest load <use>`.
skills:
Expand Down
18 changes: 11 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,22 @@
# Token usage analytics

> [!IMPORTANT]
> **Installing or upgrading the Hermes plugin?** Agents and operators must follow [`plugins/hermes-token-analytics/REQUIRED_INSTALLATION_CHECKLIST.md`](plugins/hermes-token-analytics/REQUIRED_INSTALLATION_CHECKLIST.md) from start to finish. A manual sync is not a production-ready install; the recurring cron path must also be created and verified.

## TL;DR

This repo is now a small monorepo:

- `apps/dashboard/` — the Cloudflare Workers + D1 analytics app
- `plugins/hermes-token-analytics/` — the Hermes-native plugin source that exports `state.db` rollups into the dashboard ingest endpoint
- `plugins/hermes-token-analytics/` — the Hermes-native plugin source that exports `state.db` rollups, including privacy-safe repository attribution, into the dashboard ingest endpoint
- `docs/` — shared operator and deployment documentation

The goal is simple: keep the dashboard, ingest contract, plugin source, and operational docs in one place instead of splitting the truth across repos.

For Hermes ingestion operations, the canonical operator guides are:
For Hermes ingestion operations:

- `docs/hermes-token-analytics-plugin.md`
- `docs/hermes-token-analytics-install-runbook.md`
- **Required install/upgrade procedure:** `plugins/hermes-token-analytics/REQUIRED_INSTALLATION_CHECKLIST.md`
- Background operations reference: `docs/hermes-token-analytics-plugin.md`

## What is in here

Expand All @@ -25,7 +30,7 @@ For Hermes ingestion operations, the canonical operator guides are:
- `docs/ai-build-prompts.md`
- `docs/cloudflare-deployment.md`
- `docs/hermes-token-analytics-plugin.md`
- `docs/hermes-token-analytics-install-runbook.md`
- `plugins/hermes-token-analytics/REQUIRED_INSTALLATION_CHECKLIST.md`
- `docs/hermes-sidecar-sync.md` (migration pointer to the plugin doc)

## Product direction
Expand Down Expand Up @@ -69,7 +74,6 @@ If a fresh worktree has no local D1 data yet, run `nub run cf:d1:bootstrap:local

## Hermes plugin source


The Hermes plugin source lives at:

- `plugins/hermes-token-analytics/`
Expand All @@ -79,7 +83,7 @@ Supporting files:
- install helper: `plugins/hermes-token-analytics/scripts/install-local-plugin.sh`
- tests: `plugins/hermes-token-analytics/tests/`
- operator guide: `docs/hermes-token-analytics-plugin.md`
- install runbook: `docs/hermes-token-analytics-install-runbook.md`
- required installation checklist: `plugins/hermes-token-analytics/REQUIRED_INSTALLATION_CHECKLIST.md`
- install helper also writes a legacy shim for `plugins/observability/token_analytics` in target Hermes checkouts

## CI / deploy model
Expand Down
48 changes: 48 additions & 0 deletions apps/dashboard/migrations/0003_repository_usage.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
CREATE TABLE repositories (
id TEXT PRIMARY KEY NOT NULL,
workspace_id TEXT NOT NULL,
repository_key TEXT NOT NULL,
display_name TEXT NOT NULL,
created_at INTEGER NOT NULL,
FOREIGN KEY (workspace_id) REFERENCES workspaces(id) ON DELETE CASCADE,
UNIQUE (workspace_id, repository_key),
UNIQUE (workspace_id, id)
);

CREATE INDEX repositories_workspace_idx ON repositories (workspace_id);

CREATE TABLE repository_daily_usage (
id TEXT PRIMARY KEY NOT NULL,
workspace_id TEXT NOT NULL,
repository_id TEXT,
usage_date TEXT NOT NULL,
environment TEXT NOT NULL,
attribution_status TEXT NOT NULL CHECK (attribution_status IN ('exact', 'cwd-derived', 'unknown')),
requests INTEGER NOT NULL,
total_tokens INTEGER NOT NULL,
input_tokens INTEGER NOT NULL,
output_tokens INTEGER NOT NULL,
cached_tokens INTEGER NOT NULL DEFAULT 0,
reasoning_tokens INTEGER NOT NULL DEFAULT 0,
estimated_cost_usd REAL NOT NULL,
created_at INTEGER NOT NULL,
FOREIGN KEY (workspace_id) REFERENCES workspaces(id) ON DELETE CASCADE,
FOREIGN KEY (workspace_id, repository_id) REFERENCES repositories(workspace_id, id) ON DELETE CASCADE
);

CREATE INDEX repository_daily_usage_workspace_date_repo_idx
ON repository_daily_usage (workspace_id, usage_date, repository_id);

CREATE TABLE repository_model_daily_usage (
id TEXT PRIMARY KEY NOT NULL,
repository_rollup_id TEXT NOT NULL,
model TEXT NOT NULL,
provider TEXT NOT NULL,
requests INTEGER NOT NULL,
tokens INTEGER NOT NULL,
estimated_cost_usd REAL NOT NULL,
FOREIGN KEY (repository_rollup_id) REFERENCES repository_daily_usage(id) ON DELETE CASCADE
);

CREATE INDEX repository_model_daily_usage_rollup_idx
ON repository_model_daily_usage (repository_rollup_id);
108 changes: 102 additions & 6 deletions apps/dashboard/src/db/schema.ts
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
import {
foreignKey,
index,
integer,
primaryKey,
real,
sqliteTable,
text,
uniqueIndex,
} from 'drizzle-orm/sqlite-core'

export const workspaces = sqliteTable('workspaces', {
Expand Down Expand Up @@ -43,7 +45,10 @@ export const dailyUsageRollups = sqliteTable(
createdAt: integer('created_at').notNull(),
},
(table) => [
index('daily_usage_rollups_workspace_date_idx').on(table.workspaceId, table.usageDate),
index('daily_usage_rollups_workspace_date_idx').on(
table.workspaceId,
table.usageDate,
),
index('daily_usage_rollups_environment_idx').on(table.environment),
],
)
Expand Down Expand Up @@ -78,10 +83,18 @@ export const publicModelPricingDaily = sqliteTable(
requestedModel: text('requested_model').notNull(),
sourceProvider: text('source_provider'),
sourceModel: text('source_model'),
inputMicroUsdPerMtok: integer('input_micro_usd_per_mtok').notNull().default(0),
outputMicroUsdPerMtok: integer('output_micro_usd_per_mtok').notNull().default(0),
cacheReadMicroUsdPerMtok: integer('cache_read_micro_usd_per_mtok').notNull().default(0),
cacheWriteMicroUsdPerMtok: integer('cache_write_micro_usd_per_mtok').notNull().default(0),
inputMicroUsdPerMtok: integer('input_micro_usd_per_mtok')
.notNull()
.default(0),
outputMicroUsdPerMtok: integer('output_micro_usd_per_mtok')
.notNull()
.default(0),
cacheReadMicroUsdPerMtok: integer('cache_read_micro_usd_per_mtok')
.notNull()
.default(0),
cacheWriteMicroUsdPerMtok: integer('cache_write_micro_usd_per_mtok')
.notNull()
.default(0),
resolved: integer('resolved', { mode: 'boolean' }).notNull().default(false),
fetchedAt: integer('fetched_at').notNull(),
sourceUrl: text('source_url').notNull(),
Expand All @@ -92,6 +105,86 @@ export const publicModelPricingDaily = sqliteTable(
],
)

export const repositories = sqliteTable(
'repositories',
{
id: text('id').primaryKey(),
workspaceId: text('workspace_id')
.notNull()
.references(() => workspaces.id, { onDelete: 'cascade' }),
repositoryKey: text('repository_key').notNull(),
displayName: text('display_name').notNull(),
createdAt: integer('created_at').notNull(),
},
(table) => [
uniqueIndex('repositories_workspace_key_idx').on(
table.workspaceId,
table.repositoryKey,
),
uniqueIndex('repositories_workspace_id_idx').on(
table.workspaceId,
table.id,
),
index('repositories_workspace_idx').on(table.workspaceId),
],
)

export const repositoryDailyUsage = sqliteTable(
'repository_daily_usage',
{
id: text('id').primaryKey(),
workspaceId: text('workspace_id')
.notNull()
.references(() => workspaces.id, { onDelete: 'cascade' }),
repositoryId: text('repository_id'),
usageDate: text('usage_date').notNull(),
environment: text('environment').notNull(),
attributionStatus: text('attribution_status', {
enum: ['exact', 'cwd-derived', 'unknown'],
}).notNull(),
requests: integer('requests').notNull(),
totalTokens: integer('total_tokens').notNull(),
inputTokens: integer('input_tokens').notNull(),
outputTokens: integer('output_tokens').notNull(),
cachedTokens: integer('cached_tokens').notNull().default(0),
reasoningTokens: integer('reasoning_tokens').notNull().default(0),
estimatedCostUsd: real('estimated_cost_usd').notNull(),
createdAt: integer('created_at').notNull(),
},
(table) => [
foreignKey({
columns: [table.workspaceId, table.repositoryId],
foreignColumns: [repositories.workspaceId, repositories.id],
name: 'repository_daily_usage_workspace_repository_fk',
}).onDelete('cascade'),
index('repository_daily_usage_workspace_date_repo_idx').on(
table.workspaceId,
table.usageDate,
table.repositoryId,
),
],
)

export const repositoryModelDailyUsage = sqliteTable(
'repository_model_daily_usage',
{
id: text('id').primaryKey(),
repositoryRollupId: text('repository_rollup_id')
.notNull()
.references(() => repositoryDailyUsage.id, { onDelete: 'cascade' }),
model: text('model').notNull(),
provider: text('provider').notNull(),
requests: integer('requests').notNull(),
tokens: integer('tokens').notNull(),
estimatedCostUsd: real('estimated_cost_usd').notNull(),
},
(table) => [
index('repository_model_daily_usage_rollup_idx').on(
table.repositoryRollupId,
),
],
)

export const toolDailyUsage = sqliteTable(
'tool_daily_usage',
{
Expand Down Expand Up @@ -123,7 +216,10 @@ export const issueEvents = sqliteTable(
metadataJson: text('metadata_json'),
},
(table) => [
index('issue_events_workspace_date_idx').on(table.workspaceId, table.usageDate),
index('issue_events_workspace_date_idx').on(
table.workspaceId,
table.usageDate,
),
index('issue_events_severity_idx').on(table.severity),
],
)
Loading
Loading