-
Notifications
You must be signed in to change notification settings - Fork 1
feat(hook-telemetry): marketplace-wide telemetry contract + markdown-formatter producer #5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
24a31c0
docs: add hook-telemetry envelope contract (schema_version 1.0)
kyle-sexton d5d74f9
feat(markdown-formatter): emit hook telemetry (schema_version 1.0)
kyle-sexton 937fc97
feat(markdown-formatter): resolve relative HOOK_TELEMETRY_SINK produc…
kyle-sexton f4467ed
fix(hook-telemetry): relax status to documented open string; tighten …
kyle-sexton a6eee7f
Merge remote-tracking branch 'origin/main' into feat/hook-telemetry-c…
kyle-sexton d4826cd
chore(hook-telemetry): conform to CI platform merged from main (#4)
kyle-sexton File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| # Hook Telemetry Contract — Changelog | ||
|
|
||
| Notable changes to the hook-telemetry envelope contract. The envelope is versioned by `schema_version` | ||
| (SemVer) and evolves independently of per-hook `data` schemas, which churn additively (README "Forward | ||
| compatibility"). Removal, rename, or type-change of a field is a major `schema_version` bump; a field is | ||
| marked deprecated here for one minor cycle before removal. | ||
|
|
||
| ## 1.0 — 2026-06-24 | ||
|
|
||
| Initial published contract. | ||
|
|
||
| - Common envelope: `schema_version`, `timestamp`, `hook`, `hook_event`, `status`, `duration_ms`, `data`. | ||
| - `status`: documented value set `ok | error | skipped | blocked` — a documented open string, not a closed | ||
| JSON-Schema enum (mirrors `hook_event`), so a future value never trips a schema-derived validator. | ||
| - First per-hook `data` schema: `markdown-format` (`tool`, `file`, `findings`). | ||
| - Sink path resolution: `HOOK_TELEMETRY_SINK` is a single executable path, absolute or relative to the | ||
| consuming repo root (producer-resolved). Relative is the portable, tracked-wiring form. Not an envelope | ||
| field, so no `schema_version` change. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,155 @@ | ||
| # Hook Telemetry Convention | ||
|
|
||
| A versioned, marketplace-wide contract for plugin hooks to emit structured execution telemetry. A hook | ||
| (the **producer**) emits one JSON envelope per run to a **sink** the consuming repo sets via | ||
| `HOOK_TELEMETRY_SINK`. The signal it carries — **this hook's own `duration_ms`, outcome, and findings** — | ||
| is what Claude Code's native OTEL cannot provide (CC reports an aggregate `total_duration_ms` across all | ||
| hooks and excludes third-party plugin content). | ||
|
|
||
| This directory is the source of truth: `envelope.schema.json` (common fields), `data/<hook>.schema.json` | ||
| (per-hook payload), `CHANGELOG.md` (version history), `examples/` (worked fixtures). | ||
|
|
||
| ## Mediator boundary | ||
|
|
||
| The producer and sink are decoupled — neither imports the other. The producer writes an envelope to stdin | ||
| of whatever `HOOK_TELEMETRY_SINK` names; the sink interprets it. This lets sinks be written independently | ||
| (medley maps the envelope into its own event store) and lets a plugin ship telemetry that simply no-ops | ||
| where no sink is configured. | ||
|
|
||
| - **`HOOK_TELEMETRY_SINK` unset → no-op.** The hook behaves exactly as it did before telemetry: same stdout, | ||
| same exit code. Telemetry is purely additive. | ||
| - **Fire-and-forget.** The producer dispatches the sink in the background and returns immediately; it never | ||
| waits on, nor fails because of, the sink. | ||
| - **Best-effort / lossy.** A slow or down sink drops the event silently. This contract is for observability | ||
| ("find the slow hook"), **not** for must-not-lose data. Do not build accounting, billing, or audit-of-record | ||
| on it. | ||
|
|
||
| ## Sink path resolution | ||
|
|
||
| `HOOK_TELEMETRY_SINK` is a **single executable path** — absolute, or relative to the consuming repo root. | ||
| The producer resolves it before dispatch: | ||
|
|
||
| - **Absolute** (POSIX `/…` or Windows `X:\` / `X:/`) — used as-is. | ||
| - **Relative** (e.g. `.claude/hooks/hook-telemetry-sink.sh`) — joined onto the consuming repo root the | ||
| producer already resolves for `data.file` (falling back to `$CLAUDE_PROJECT_DIR`); skipped fail-open if | ||
| neither anchor is available, since a drifted hook CWD would resolve it incorrectly. | ||
|
|
||
| Relative is the portable, team-shared wiring form. Claude Code injects `settings.json` `env` values | ||
| **literally** — no `${VAR}` expansion (that is a `.mcp.json`-only feature) — so a relative path committed in | ||
| `settings.json` is the only clone-portable, worktree-safe way to wire a sink without a per-machine absolute | ||
| path. To pass arguments, wrap the sink in a script: the value is exec'd as a single command. | ||
|
|
||
| ## The envelope (common fields) | ||
|
|
||
| Every event, from every hook, carries these seven fields. All are required and always present. | ||
|
|
||
| | Field | Type | Meaning | | ||
| |-------|------|---------| | ||
| | `schema_version` | string (SemVer) | Version of this envelope contract. | | ||
| | `timestamp` | string (RFC 3339, UTC) | Instant the hook finished. True UTC — the `Z` is not a local-time lie. | | ||
| | `hook` | string | Producer hook id, e.g. `markdown-format`. **Not** CC's `hook_name` (event:matcher). Keys data-schema discovery. | | ||
| | `hook_event` | string | The triggering event: `PostToolUse`, `SessionStart`, `ConfigChange`, `WorktreeCreate`, … A **free string**, not an enum — the event vocabulary grows, and custom events exist. | | ||
| | `status` | string | Universal execution outcome (documented value set, not a closed enum). See below. | | ||
| | `duration_ms` | integer (≥ 0) | **This hook's** runtime in milliseconds. Not CC's aggregate `total_duration_ms`. | | ||
| | `data` | object | Per-hook payload; always present (at minimum `{}`). See "Per-hook data". | | ||
|
|
||
| Naming is snake_case throughout and aligns with Claude Code's own field names where the concept matches | ||
| (`hook_event`), and deliberately diverges where it does not (`hook` ≠ `hook_name`, `duration_ms` ≠ | ||
| `total_duration_ms`) so a name never misleads. | ||
|
|
||
| ### `status` — the universal outcome (documented value set) | ||
|
|
||
| `ok | error | skipped | blocked` — a **documented open string, not a closed JSON-Schema enum** (same encoding | ||
| as `hook_event`, and for the same reason: the set grows). A closed enum would *reject* a future value at | ||
| validation time, contradicting the tolerate-unknown rule below. These four express the outcome of *any* hook, | ||
| validated against the full medley hook set (formatters, guards, audit, action hooks): | ||
|
|
||
| | Value | Meaning | | ||
| |-------|---------| | ||
| | `ok` | The hook ran and did its job. | | ||
| | `error` | The hook ran but failed internally. | | ||
| | `skipped` | The hook did not apply (no-op — wrong file type, disabled, nothing to do). | | ||
| | `blocked` | The hook intentionally blocked the operation (guard hooks: git-safety, branch-protection, secret-pattern-detection, …). Distinct from `error`. | | ||
|
|
||
| Domain detail — *what* a hook found, not *whether* it ran — lives in `data`, never in `status`. Lint | ||
| findings are `data.findings`, because "found issues" is specific to formatters and not a universal outcome. | ||
| `cancelled` is intentionally excluded: a killed hook cannot self-report. | ||
|
|
||
| ## Per-hook data | ||
|
|
||
| `data` carries everything not universal to all hooks. Its shape per producer is published at | ||
| `data/<hook>.schema.json`, **discovered by the envelope's `hook` value**. For `hook: "markdown-format"`, | ||
| read `data/markdown-format.schema.json` (`tool`, `file`, `findings`). | ||
|
|
||
| - **`data` is always present** — an object, possibly `{}`. | ||
| - **Generic sinks ignore `data`** and consume only the common envelope. | ||
| - **Unknown `hook`** (no matching data schema) → record the common fields, ignore `data`. Never hard-fail. | ||
|
|
||
| ## Forward compatibility (don't break clients) | ||
|
|
||
| The whole point of a published contract is that independently-written sinks keep working as producers evolve. | ||
| Two rules make that hold: | ||
|
|
||
| 1. **`data` evolves additive-only** — new keys may be added; existing keys are never silently removed, | ||
| renamed, or type-changed (those require a deprecation cycle and a major bump). | ||
| 2. **Consumers MUST ignore unknown keys AND MUST tolerate unknown enum values.** A sink reading an envelope | ||
| from a newer producer must skip keys it does not recognize, and must treat an unrecognized `status` (or | ||
| any future enum value) as a catch-all rather than crashing. Ignore-unknown-keys alone is not enough — | ||
| enum growth (e.g. a future `status`) needs the tolerate-unknown-enum rule too. | ||
|
|
||
| Both schema files set `additionalProperties: true`, which encodes only the **unknown-keys-tolerated** half of | ||
| these rules. The rest is **policy, enforced by review and the deprecation cycle below — not by the schema**: | ||
| `additionalProperties` says nothing about existing keys never being removed/renamed/type-changed, and a closed | ||
| `enum` would actively *reject* a new `status` value (which is why `status` is a documented open string, like | ||
| `hook_event`). Do not over-trust the schema as the enforcement boundary; it is the contract a reviewer reads. | ||
|
|
||
| ## Deprecation policy | ||
|
|
||
| A field is marked **deprecated** in `CHANGELOG.md` for one minor cycle before it is removed. Removal, rename, | ||
| or type-change of any field is a major `schema_version` bump. This gives consumers a release window to adapt | ||
| before a breaking change lands. | ||
|
|
||
| ## Versioning | ||
|
|
||
| `schema_version` (SemVer) versions the **envelope**. Per-hook `data` schemas churn independently under the | ||
| additive / ignore-unknown rules above and are **not** separately version-stamped (deferred as YAGNI). The | ||
| envelope version and a hook's `data` shape are decoupled on purpose. | ||
|
|
||
| A per-`data` version signal is **deferred, not designed out**: today a hook's `data` shape is discovered only | ||
| by the `hook` value, so a breaking `data` change would carry no version marker short of a whole-envelope major | ||
| bump. **Trigger** — when a producer first needs a *breaking* `data` change, add an optional per-payload schema | ||
| identifier (a `data_schema` URI, à la CloudEvents `dataschema`) rather than bumping the envelope. It is | ||
| additive (optional field), so it ships without breaking existing consumers. | ||
|
|
||
| ## Schemas are contract-docs, not machine-enforced | ||
|
|
||
| The JSON schemas here are **not machine-enforced** — no validator is wired into producer or sink. They are | ||
| the human-readable, reviewable contract; conformance is checked by hand and by `jq` required-key assertions | ||
| (producers and sinks each carry their own). Treat the schemas as the authority a reviewer reads, not a | ||
| runtime gate. | ||
|
|
||
| ## Adoption (adopt-by-copy) | ||
|
|
||
| The emit function is **co-located in each plugin's `hooks/hook-utils.sh`** — plugins are runtime-isolated | ||
| under `${CLAUDE_PLUGIN_ROOT}`, so there is no shared library to import. The first implementer | ||
| (`markdown-formatter`) carries its own copy. **The moment a second hook needs to emit, extract a canonical | ||
| copy and add a drift-check in the same change** — copy once, then consolidate, so the two copies never drift | ||
| unwatched. This mirrors the standards-repo "adopt by copy" seam. | ||
|
|
||
| ## Consuming (sink side) | ||
|
|
||
| A repo **subscribes** by setting `HOOK_TELEMETRY_SINK` (relative, committed in `settings.json`) to an | ||
| executable that reads one envelope on stdin and maps the common fields into its own store. The sink: | ||
|
|
||
| - consumes only the common envelope unless it specifically handles a given `hook`'s `data`; | ||
| - ignores unknown keys and treats an unrecognized `status` as a catch-all (see Forward compatibility); | ||
| - never crashes and never writes to stdout — it runs fire-and-forget, exec'd as a single command per event. | ||
|
|
||
| That is the whole consumer contract: any number of independently-written sinks can subscribe to the same | ||
| producers without coordinating with them or each other. | ||
|
|
||
| ## Implementers | ||
|
|
||
| | Producer | `hook` value | Data schema | | ||
| |----------|--------------|-------------| | ||
| | `markdown-formatter` plugin | `markdown-format` | `data/markdown-format.schema.json` | |
24 changes: 24 additions & 0 deletions
24
docs/conventions/hook-telemetry/data/markdown-format.schema.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| { | ||
| "$schema": "https://json-schema.org/draft/2020-12/schema", | ||
| "$id": "https://github.com/ghraw/melodic-software/claude-code-plugins/main/docs/conventions/hook-telemetry/data/markdown-format.schema.json", | ||
| "title": "markdown-format telemetry data", | ||
| "description": "Per-hook `data` payload for the markdown-format hook. Discovered from the envelope `hook` value \"markdown-format\". Evolves additive-only.", | ||
| "type": "object", | ||
| "required": ["tool", "file", "findings"], | ||
| "additionalProperties": true, | ||
| "properties": { | ||
| "tool": { | ||
| "type": "string", | ||
| "description": "Claude Code tool that triggered the hook (Write or Edit)." | ||
| }, | ||
| "file": { | ||
| "type": "string", | ||
| "description": "Path of the formatted markdown file, relative to the consuming repo root." | ||
| }, | ||
| "findings": { | ||
| "type": "array", | ||
| "items": { "type": "string" }, | ||
| "description": "Unfixable markdownlint violations remaining after --fix, one per line. Empty array = clean." | ||
| } | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,50 @@ | ||
| { | ||
| "$schema": "https://json-schema.org/draft/2020-12/schema", | ||
| "$id": "https://github.com/ghraw/melodic-software/claude-code-plugins/main/docs/conventions/hook-telemetry/envelope.schema.json", | ||
| "title": "Hook Telemetry Envelope", | ||
| "description": "Common envelope a marketplace plugin hook (producer) emits to a consumer-set sink. Versioned by schema_version. Per-hook detail lives in `data`, described by data/<hook>.schema.json keyed on the `hook` value. Contract-doc only: no validator is wired (see README).", | ||
| "type": "object", | ||
| "required": [ | ||
| "schema_version", | ||
| "timestamp", | ||
| "hook", | ||
| "hook_event", | ||
| "status", | ||
| "duration_ms", | ||
| "data" | ||
| ], | ||
| "additionalProperties": true, | ||
| "properties": { | ||
| "schema_version": { | ||
| "type": "string", | ||
| "description": "SemVer version of this envelope contract." | ||
| }, | ||
| "timestamp": { | ||
| "type": "string", | ||
| "format": "date-time", | ||
| "description": "RFC 3339 UTC instant the hook finished. True UTC (TZ=UTC) — the Z suffix is not a local-time lie." | ||
| }, | ||
| "hook": { | ||
| "type": "string", | ||
| "description": "Producer hook id, e.g. \"markdown-format\". NOT Claude Code's hook_name (event:matcher). Keys data-schema discovery: data/<hook>.schema.json." | ||
| }, | ||
| "hook_event": { | ||
| "type": "string", | ||
| "description": "Free string naming the triggering event (PostToolUse, SessionStart, ConfigChange, WorktreeCreate, ...). Deliberately not an enum." | ||
| }, | ||
| "status": { | ||
| "type": "string", | ||
| "description": "Universal execution outcome. Documented values: ok | error | skipped | blocked. Deliberately a documented open string, NOT a closed enum (mirrors hook_event) — the value set grows, and a closed enum would reject a future value, contradicting the tolerate-unknown rule. Consumers MUST treat an unrecognized value as a catch-all, never hard-fail. See README \"status\"." | ||
| }, | ||
| "duration_ms": { | ||
| "type": "integer", | ||
| "minimum": 0, | ||
| "description": "This hook's own runtime in milliseconds. NOT Claude Code's aggregate total_duration_ms." | ||
| }, | ||
| "data": { | ||
| "type": "object", | ||
| "additionalProperties": true, | ||
| "description": "Per-hook payload; always present (at minimum {}). Shape described by data/<hook>.schema.json. Generic sinks ignore this and consume only the common fields." | ||
| } | ||
| } | ||
| } |
15 changes: 15 additions & 0 deletions
15
docs/conventions/hook-telemetry/examples/markdown-format.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| { | ||
| "schema_version": "1.0", | ||
| "timestamp": "2026-06-23T22:37:29Z", | ||
| "hook": "markdown-format", | ||
| "hook_event": "PostToolUse", | ||
| "status": "ok", | ||
| "duration_ms": 142, | ||
| "data": { | ||
| "tool": "Write", | ||
| "file": "docs/foo.md", | ||
| "findings": [ | ||
| "docs/foo.md:12 MD013/line-length Line length [Expected: 80; Actual: 118]" | ||
| ] | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The new contract documents
schema_versionas SemVer, but the producer emits1.0, which is not a valid SemVer 2.0 version because it lacks the patch component. Any sink that uses a standard SemVer parser or range comparison for this public contract will reject or mis-handle the first producer envelopes; emit1.0.0or explicitly document a non-SemVer two-part versioning scheme.Useful? React with 👍 / 👎.