[llm] Add AI usage governance with provider contracts, pricing, and quota controls

[kang-heewon]

Summary

Croco의 LLM surface를 단순 모델 호출 abstraction에서 tenant-aware AI usage governance layer로 확장한다.

현재 llm-core는 provider/model abstraction을 제공하고, llm-metering은 token/cost metering과 OpenTelemetry GenAI bridge를 제공한다. 하지만 실제 OpenAI/Anthropic provider package는 아직 확인되지 않았고, pricing table은 코드에 고정되어 있다. 이번 이슈의 목표는 provider conformance, pricing registry, tenant quota, telemetry를 하나의 사용량 통제 경험으로 묶는 것이다.

Evidence

• packages/llm-core/src/libs/LlmService.ts:26-53 resolves models from a registry, emits usage events, and wraps generation in telemetry.
• packages/llm-core/src/libs/LlmService.ts:55-120 supports streaming with abort/backpressure handling.
• packages/llm-core/src/libs/LlmModel.ts:17-83 defines abstract model methods for generate, stream, generateObject, callTool, embed, and embedMany.
• packages/llm-metering/src/libs/LlmMeteringService.ts:19-49 defines usage event fields including tenantId, modelId, provider, usage, and idempotency key.
• packages/llm-metering/src/libs/LlmMeteringService.ts:51-125 records prompt tokens, completion tokens, and cost meters with pricing.
• packages/llm-metering/src/libs/decorators/AiMetered.ts:161-285 meters async iterable/generate/embed usage paths.
• packages/llm-metering/src/libs/LlmTelemetryBridge.ts:8-68 maps LLM usage into OpenTelemetry GenAI-style attributes/events.
• packages/llm-metering/src/libs/PricingTable.ts:3-86 contains hardcoded provider/model prices.
• packages/llm-metering/src/tests/LlmMeteringService.spec.ts:45-103 tests prompt/completion/cost meter recording.
• packages/llm-metering/src/tests/decorators/AiMetered.spec.ts:253-320 tests embedding metering and metering failure behavior.
• Local package scan found llm-core and llm-metering, but no first-party llm-openai or llm-anthropic provider package.

Tasks

• Stabilize provider conformance tests.
  • Cover generate, stream, generateObject, callTool, embed, and embedMany.
  • Include abort/error behavior and usage accounting expectations.
• Decide and add the first real provider package.
  • Candidate: @croco/llm-openai or another explicitly chosen provider.
  • Before implementation, confirm current SDK/API behavior against official provider docs because model APIs and pricing are time-sensitive.
  • Provider package must pass conformance tests and expose deterministic errors.
• Externalize pricing.
  • Replace hardcoded default-only pricing with a configurable/versioned pricing registry.
  • Keep any built-in pricing table clearly versioned or documented as sample data.
• Integrate tenant quotas.
  • Connect LLM meters to entitlements/metering for keys such as llm.prompt_tokens, llm.completion_tokens, and llm.cost_usd.
  • Quota exhaustion must fail clearly before or during usage according to a documented policy.
• Define metering failure policy.
  • Current behavior should be reviewed and made explicit: fail-closed, soft-fail, or configurable.
  • Tests must prove the selected behavior for generate, stream, and embed paths.
• Document GenAI telemetry.
  • Include attribute names, event behavior, and how to inspect traces.
• Add a generated-app AI usage example.
  • Prefer a SaaS example endpoint that performs one LLM call, records usage, and enforces tenant quota.

Acceptance Criteria

• A provider conformance suite exists and can be reused by future provider packages.
• At least one real provider package exists or the issue produces a documented, evidence-backed provider decision with follow-up scope.
• Pricing is configurable and no longer relies solely on stale hardcoded values.
• Tenant-level quota enforcement works for LLM token/cost meters.
• Metering failure behavior is explicit and covered by tests.
• Docs explain how to enable provider, metering, pricing, quota, and telemetry.

Suggested Validation

• pnpm test --filter=@croco/llm-core
• pnpm test --filter=@croco/llm-metering
• pnpm test --filter=@croco/entitlements-core
• pnpm test --filter=@croco/metering-core
• Provider package tests once added.
• pnpm typecheck

Non-goals

• Do not ship undocumented model prices as if they are always current.
• Do not add provider SDKs without checking current official docs first.
• Do not silently continue when metering/quota enforcement fails unless the chosen policy explicitly allows it and preserves failure evidence.