feat(loki): add the Grafana Loki drain adapter - #480
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
🦋 Changeset detectedLatest commit: d84c3f3 The changes in this PR will be included in the next version bump. This PR includes changesets to release 2 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
📝 WalkthroughWalkthroughAdded a Grafana Loki HTTP drain adapter with configurable authentication, labels, batching, retries, and timestamps. Added package exports, unit and end-to-end tests, a local Loki/Grafana sandbox, integration documentation, and release metadata. ChangesGrafana Loki adapter
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant Application
participant LokiDrain
participant LokiPushAPI
Application->>LokiDrain: Emit event or batch
LokiDrain->>LokiDrain: Group labels and order timestamps
LokiDrain->>LokiPushAPI: Send authenticated JSON payload
LokiPushAPI-->>LokiDrain: Return HTTP response
LokiDrain-->>Application: Complete or report failure
Possibly related PRs
Suggested labels: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 Trivy (0.72.0)Trivy execution failed: 2026-08-01T19:02:02Z FATAL Fatal error run error: fs scan error: scan error: scan failed: failed analysis: post analysis error: post analysis error: helm scan error: fs filter error: fs filter error: walk error range error: stat .nuxt/manifest/meta/eb87d1cf-97be-423d-aab2-3fea338616b7.json: no such file or directory: range error: stat .nuxt/manifest/meta/eb87d1cf-97be-423d-aab2-3fea338616b7.json: no such file or directory Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Thank you for following the naming conventions! 🙏 |
|
@coderabbitai review |
✅ Action performedReview finished.
|
525c910 to
5e5e23e
Compare
commit: |
`semantic-pull-request.yml` validates a PR title against the scope list as it exists on the base branch, so a PR that introduces a new subsystem cannot register its own scope — the check reads main's copy and rejects the title. #480 and #481 both fail on exactly that. Registering the two scopes ahead of the adapters unblocks them. Worth noting in `.agents/skills/create-adapter/SKILL.md`, which currently tells you to add the scope "in the same PR".
5e5e23e to
746ec44
Compare
|
@coderabbitai review |
✅ Action performedReview finished.
|
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@apps/docs/content/4.integrate/adapters/self-hosted/04.loki.md`:
- Line 255: Update the “Missing endpoint” documentation to state that
createLokiDrain() logs an error and does not block the request when
LOKI_ENDPOINT and endpoint are both absent; remove the inaccurate claim that the
drain skips silently.
In `@packages/evlog/test/adapters/loki.test.ts`:
- Around line 1-49: Update the Loki adapter tests to import and use the shared
mockFetch, getFetchJson, and getFetchHeaders helpers. Replace the local fetchSpy
setup in beforeEach and remove the bodyOf and headersOf helper functions,
adapting each test assertion to the shared helper APIs while preserving existing
request-body and header coverage.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 80b969b7-c9b1-4621-9bf5-b89a1476ba7e
⛔ Files ignored due to path filters (1)
packages/evlog/test/toolkit/__snapshots__/api-surface.test.ts.snapis excluded by!**/*.snap
📒 Files selected for processing (8)
.changeset/loki-adapter.mdapps/docs/content/4.integrate/adapters/01.overview.mdapps/docs/content/4.integrate/adapters/self-hosted/04.loki.mdapps/docs/skills/review-logging-patterns/SKILL.mdpackages/evlog/package.jsonpackages/evlog/src/adapters/loki.tspackages/evlog/test/adapters/loki.test.tspackages/evlog/tsdown.config.ts
77eef42 to
7d72c93
Compare
|
@coderabbitai review |
✅ Action performedReview finished.
|
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (1)
packages/evlog/test/adapters/loki.test.ts (1)
25-49: 📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick winUse the shared fetch test helpers instead of local mocking.
This test file still defines and uses local
fetchSpy,bodyOf, andheadersOfinstead of the shared helpers inpackages/evlog/test/helpers/fetch. A prior review already flagged this and requested importingmockFetch,getFetchJson, andgetFetchHeadersfrom that module and removing the local re-implementations. That request is still unaddressed in this version of the file.Replace
fetchSpysetup inbeforeEach, and replace allbodyOf(fetchSpy, ...)/headersOf(fetchSpy, ...)calls throughout the file with the shared helpers.Based on path instructions: "Use the helpers in
packages/evlog/test/helpers/, including drain spies, fake timers, fetch mocks, and framework-matrix helpers" and "Import real source helpers in tests; never re-implement those helpers in test code."🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/evlog/test/adapters/loki.test.ts` around lines 25 - 49, Update the Loki adapter tests to import and use mockFetch, getFetchJson, and getFetchHeaders from the shared fetch helpers module. Remove the local bodyOf, headersOf, and fetchSpy implementations, replace the beforeEach fetch setup with mockFetch, and update all bodyOf(fetchSpy, ...) and headersOf(fetchSpy, ...) usages to the shared helper calls.Source: Path instructions
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@apps/docs/content/4.integrate/adapters/self-hosted/04.loki.md`:
- Line 36: Update the setup prompt wording in the Grafana Loki drain adapter
documentation to use “evlog-wide events” instead of “evlog wide events,”
preserving the rest of the sentence.
In `@packages/evlog/src/adapters/loki.ts`:
- Around line 133-146: Replace the btoa-based Basic auth encoding in
buildHeaders with Node’s Unicode-safe Buffer.from(...).toString('base64')
approach, preserving the existing user/apiKey credential format and
Authorization header behavior. Confirm the package targets a Node/Nitro runtime
and check nearby adapters for btoa usage before making the focused change.
---
Duplicate comments:
In `@packages/evlog/test/adapters/loki.test.ts`:
- Around line 25-49: Update the Loki adapter tests to import and use mockFetch,
getFetchJson, and getFetchHeaders from the shared fetch helpers module. Remove
the local bodyOf, headersOf, and fetchSpy implementations, replace the
beforeEach fetch setup with mockFetch, and update all bodyOf(fetchSpy, ...) and
headersOf(fetchSpy, ...) usages to the shared helper calls.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: c49f700d-2859-4bb1-aff4-fdfee671f791
⛔ Files ignored due to path filters (1)
packages/evlog/test/toolkit/__snapshots__/api-surface.test.ts.snapis excluded by!**/*.snap
📒 Files selected for processing (8)
.changeset/loki-adapter.mdapps/docs/content/4.integrate/adapters/01.overview.mdapps/docs/content/4.integrate/adapters/self-hosted/04.loki.mdapps/docs/skills/review-logging-patterns/SKILL.mdpackages/evlog/package.jsonpackages/evlog/src/adapters/loki.tspackages/evlog/test/adapters/loki.test.tspackages/evlog/tsdown.config.ts
|
Both comments reviewed. 1. 2. The real defect behind the comment was still there though: |
921b1a4 to
6c29724
Compare
6c29724 to
46032d3
Compare
46032d3 to
0cf4f42
Compare
`createLokiDrain()` pushes wide events to Loki's push API and covers the three deployment shapes: self-hosted single-tenant (endpoint only), multi-tenant (`X-Scope-OrgID`), and Grafana Cloud (instance ID + token sent as HTTP Basic). Loki bills and indexes by label cardinality, so the adapter labels only `service`, `environment` and `level` by default and pushes the full wide event as a JSON log line — everything else stays queryable with `| json` without inflating the index. `labelFields` promotes extra low-cardinality fields; `labels` adds static deployment-wide ones. Object-valued fields are never promoted. Events sharing a label set are grouped into one stream and their entries sorted by timestamp, since Loki rejects out-of-order pushes within a stream. Built on `defineHttpDrain` + `sendEncodedDrainRequest`, so both the drain and `sendBatchToLoki` share one encoder. Tests use the shared `test/helpers/fetch` mocks rather than a local fetch spy.
0cf4f42 to
d84c3f3
Compare
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@apps/docs/content/4.integrate/adapters/01.overview.md`:
- Around line 389-394: Update the Grafana Loki zero-config environment example
to include the LOKI_TENANT_ID variable alongside LOKI_ENDPOINT and the Grafana
Cloud credentials, documenting the tenant ID required for multi-tenant
deployments.
In `@packages/evlog/test/e2e/docker-compose.yml`:
- Around line 17-18: Update the Docker Compose port mappings for the Grafana and
Loki services to bind explicitly to the loopback interface, changing the host
bindings for ports 3100 and 3001 while preserving their existing container
ports.
In `@packages/evlog/test/e2e/loki.e2e.ts`:
- Around line 43-46: Export a Loki adapter-owned header helper from the module
containing buildHeaders and toBasicCredentials, then update the E2E setup to
import and call that helper instead of constructing Authorization and
X-Scope-OrgID headers locally. Preserve the existing Basic, Bearer, and
tenant-header behavior while removing the duplicated raw btoa logic.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: bc47033b-c749-4aeb-a3cd-4fb94f09ac18
⛔ Files ignored due to path filters (1)
packages/evlog/test/toolkit/__snapshots__/api-surface.test.ts.snapis excluded by!**/*.snap
📒 Files selected for processing (15)
.changeset/loki-adapter.mdapps/docs/content/4.integrate/adapters/01.overview.mdapps/docs/content/4.integrate/adapters/hybrid/.navigation.ymlapps/docs/content/4.integrate/adapters/hybrid/01.loki.mdapps/docs/skills/review-logging-patterns/SKILL.mdpackage.jsonpackages/evlog/package.jsonpackages/evlog/src/adapters/loki.tspackages/evlog/test/adapters/loki.test.tspackages/evlog/test/e2e/README.mdpackages/evlog/test/e2e/docker-compose.ymlpackages/evlog/test/e2e/grafana/datasources.ymlpackages/evlog/test/e2e/loki.e2e.tspackages/evlog/test/e2e/seed.mjspackages/evlog/tsdown.config.ts
What
evlog/loki—createLokiDrain()pushes wide events to Loki's push API.The design decision that matters: cardinality
Loki indexes and bills by label cardinality — one stream per unique label combination. Labelling
requestIdwould create a stream per request and degrade or break an instance.So the adapter labels only
service,environment,levelby default, and pushes the full wide event as a JSON log line. Everything else stays queryable at read time:labelFieldspromotes extra low-cardinality fields (region,tier)labelsadds static deployment-wide labels (cluster: prod-eu)The docs page carries an explicit warning about this.
Ordering
Loki rejects out-of-order entries within a stream. The adapter groups events by label set and sorts each stream's entries by timestamp, so batched pushes (via
evlog/pipeline) are accepted.Auth resolution
user+apiKeyAuthorization: Basic base64(user:apiKey)— Grafana CloudapiKeyonlyAuthorization: Bearer <apiKey>tenantIdX-Scope-OrgID(independent of the above)Touchpoints
All 8 from
.agents/skills/create-adapter/SKILL.md, adjusted for the current docs layout (4.integrate/adapters/self-hosted/, not4.adapters/):src/adapters/loki.tstsdown.config.tsentrypackage.jsonexports+typesVersionstest/adapters/loki.test.ts— 26 testsapps/docs/content/4.integrate/adapters/self-hosted/04.loki.md.envblockapps/docs/skills/review-logging-patterns/SKILL.mddrain table + descriptionlokiadded to bothsemantic-pull-request.ymlandpull_request_template.mdVerification
pnpm run test— 1664/1664 pass (26 new)pnpm run lint— 0 errors (2 pre-existingmax-paramswarnings innitro-v3/plugin.ts)pnpm run typecheck— 26/26 tasks passpnpm run api:snapshot— adds the./lokisubpath onlyTests cover URL resolution (including an endpoint that already carries the push path), all four auth shapes, label promotion and the cardinality guard, stream grouping and timestamp ordering, empty batches, env resolution, missing-endpoint skip, and the drain never throwing on a failed push.
Summary by CodeRabbit