feat(hook-telemetry): marketplace-wide telemetry contract + markdown-formatter producer#5
Merged
Merged
Conversation
Publish the marketplace-wide hook-telemetry convention: a versioned public-API contract for plugin hooks (producers) to emit structured execution telemetry to a consumer-set sink (HOOK_TELEMETRY_SINK, fire-and-forget, opt-in; unset = no-op). Carries this hook's own duration_ms, outcome, and findings — the per-hook signal CC-native OTEL structurally cannot provide (it reports aggregate total_duration_ms and excludes third-party plugin content). docs/conventions/hook-telemetry/: - README.md: spec — mediator boundary, fire-and-forget, best-effort/lossy guarantee; 7-field common envelope; status enum ok|error|skipped|blocked; hook_event free string; per-hook data + data/<hook>.schema.json discovery keyed on hook; forward-compat (additive-only, ignore unknown keys AND tolerate unknown enum values); deprecation policy; SemVer versioning; schemas are contract-docs (not machine-enforced); adopt-by-copy. - envelope.schema.json: draft 2020-12; required = all 7 common fields; additionalProperties true; status enum (4); duration_ms integer >= 0. - data/markdown-format.schema.json: required [tool, file, findings]. - CHANGELOG.md: schema_version 1.0 initial. - examples/markdown-format.json: worked fixture, consistent with both schemas. Contract authoring only; no runtime code. markdown-formatter is the first implementer (emit lands Phase 2). Verified (Tier-0, all PASS): fixture shape (schema_version/status/hook_event/ duration_ms/data); envelope additionalProperties + 4-value status enum; README forward-compat/deprecation/not-machine-enforced text; all 3 JSON parse. Lint deferred — markdownlint-cli2 absent locally. Co-Authored-By: Claude <noreply@anthropic.com>
First implementer of the hook-telemetry convention. The markdown-format hook now emits one telemetry envelope per run to a consumer-set sink (HOOK_TELEMETRY_SINK), fire-and-forget. Additive and opt-in: sink unset = no-op, with stdout and exit code byte-identical to before. hook-utils.sh: add hook::emit_telemetry <hook_id> <hook_event> <status> <start_epoch> <data_json>. Opt-in guard (sink unset -> return 0) + fail-open (jq absent -> return 0). duration_ms from $EPOCHREALTIME (locale-safe: handles . and , separators, 10# guards octal). timestamp via TZ=UTC printf (true UTC). Envelope built with jq -n (stderr -> /dev/null, never touches fd1). Background dispatch with the sink's stdout AND stderr -> /dev/null so it cannot block CC's stdout read to the 15s hook timeout (the C1 fd1-inheritance blocker) nor leak into additionalContext. markdown-format.sh: capture start before formatting; parse tool_name for data.tool; compute repo-relative data.file (cygpath -lm on Windows Git Bash to reconcile drive-letter vs mount form, plain prefix-strip on Linux/macOS, raw-path fallback); call hook::emit_telemetry LAST on the three post-format paths (markdownlint absent -> skipped; clean -> ok, findings []; residual -> ok, findings = violation lines filtered to the " MD<n>/" pattern). Pre-format exits (ext gate, missing file, kill switch) do not emit. stdin read once (INPUT=$(cat)) and fed to both the file_path and tool_name parses. Tests (TDD): markdown-format.test.sh PASS=41/0 (adds sink-set envelope validity, sink-unset additive-safety, non-zero sink, slow-sink C1 timing, stdout-leak); hook-utils.test.sh PASS=23/0 (sourced-lib unit). Shellcheck clean. Verified Tier-0 end-to-end: schema-valid envelope lands in sink; no fd1 leak; sink-unset produces no envelope with identical stdout; hook returns ~1.4s under a 3s sink (non-blocking). Co-Authored-By: Claude <noreply@anthropic.com>
…er-side
Resolve a relative HOOK_TELEMETRY_SINK against the caller's repo root so the
sink path is clone-portable and worktree-safe when wired via a tracked
settings.json env value (CC injects env literally, no ${VAR} expansion).
- hook::emit_telemetry: optional 6th repo_root arg ($CLAUDE_PROJECT_DIR
fallback, fail-open skip if neither); absolute paths pass through.
- Exec changed from unquoted $SINK to quoted "$sink": the sink is now a single
executable path (wrap in a script to pass arguments).
- markdown-format.sh passes its file-derived REPO_ROOT at all 3 emit sites
(CC #9447: CLAUDE_PROJECT_DIR can be empty in plugin hooks).
- Contract README gains "Sink path resolution"; CHANGELOG notes it under 1.0.
No schema_version bump - envelope wire-format unchanged.
- Tests: stub-script sinks replace command-with-args sinks + 4 resolution
cases (hook-utils 27/0, markdown-format 41/0).
Co-Authored-By: Claude <noreply@anthropic.com>
…contract docs A closed JSON-Schema enum on `status` contradicted the contract's own "consumers MUST tolerate unknown enum values" rule: a validator code-generated from the published schema would reject a future status value (JSON Schema 2020-12 sec 6.1.2). Relax `status` to a documented open string (mirrors `hook_event`), matching OpenTelemetry's open-enum pattern (`error.type`). - envelope.schema.json: drop the closed `status` enum; keep type:string and list the documented values in the description. - README: correct the `additionalProperties: true` claim (it encodes only unknown-keys-tolerated, not never-remove/rename — the rest is review/policy); record a deferred-with-trigger note for per-data versioning (CloudEvents `dataschema` precedent); add a consumer/sink adoption section. - CHANGELOG: 1.0 `status` documented as an open string, not a closed enum. Pre-publish correction — no schema_version bump. Validated against CloudEvents, OpenTelemetry semantic conventions, and JSON Schema 2020-12 (primary sources). Co-Authored-By: Claude <noreply@anthropic.com>
Bring the pre-CI-era telemetry files into conformance with the lanes #4 added: - hook-utils.test.sh: git mode 100644 -> 100755 (it carries a shebang; the exec-bit lane requires shebang scripts be 100755, matching its sibling). - Reword "mis-parse"/"mis-resolve" to plain English in the README sink-path note and two hook-utils.sh comments; the typos lane flags the bare "mis" token. Comment-only; no behavior change. Verified: hook-utils 27/0, markdown-format 41/0, shellcheck clean, typos clean.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d4826cde27
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| FILE=$(hook::read_file_path) || exit 0 | ||
| # Capture $EPOCHREALTIME immediately after kill-switch so duration_ms covers | ||
| # the formatting work (pre-format exits below do not emit telemetry). | ||
| start=$EPOCHREALTIME |
There was a problem hiding this comment.
On hosts whose env bash is Bash <5 (notably stock macOS /bin/bash 3.2), EPOCHREALTIME is unset; with set -u, this assignment aborts before stdin is parsed, so every Markdown edit fails instead of the hook's documented always-exit-0 behavior, even when telemetry is not configured. Use a portable timestamp source or guard this with a fallback before enabling telemetry timing.
Useful? React with 👍 / 👎.
| # variable — never to fd1. | ||
| local envelope | ||
| envelope=$(jq -n \ | ||
| --arg schema_version "1.0" \ |
There was a problem hiding this comment.
The new contract documents schema_version as SemVer, but the producer emits 1.0, which is not a valid SemVer 2.0 version because it lacks the patch component. Any sink that uses a standard SemVer parser or range comparison for this public contract will reject or mis-handle the first producer envelopes; emit 1.0.0 or explicitly document a non-SemVer two-part versioning scheme.
Useful? React with 👍 / 👎.
This was referenced Jul 12, 2026
kyle-sexton
added a commit
that referenced
this pull request
Jul 12, 2026
Address two more Codex P2 findings plus the analogous coaching case: - codebase-detection case (#2): the eval repo has no real auth-flow module, so grounding would fail or reward inventing sources. Reframe to the routing + graceful-degradation invariant — resolve to codebase mode, discover from live files, and ask the user to point when no grounding exists rather than inventing repo sources. - primary-source case (#5) and coaching case (#4): a fresh topic invocation runs the mission interview before teaching, so expecting immediate content rewarded skipping the mission gate. Make both gate-aware — respect the new-workspace mission flow while asserting the grounding / one-question-coaching invariants.
kyle-sexton
added a commit
that referenced
this pull request
Jul 17, 2026
…opy through standards SSOT Corrector #4 reason-dont-recite (incumbency discipline): inherited content is evidence of what is, never self-justifying authority; a choice supported only by precedent earns first-principles re-derivation, not compliance. A distinct axis from do-your-research (acquire external evidence you lack — this questions internal evidence you inherited). SSOT: the consuming project's incumbency / first-principles rule, degrading to a portable baseline. A standards disagreement it surfaces routes upstream via follow-our-standards (routing only — that skill now owns "never silent deviation, never silent conformance"; no doctrine duplicated). point-dont-copy SSOT correction: its re-anchor step now routes through the org standards' reference-don't-duplicate (in-repo facts: literal/semantic, describe/use/expose roles, stable anchors) and documentation-and-citations (external facts: cite + fetch at read time, no upstream-inventory recap, citation placement) conventions instead of restating that doctrine. The skill keeps only its own pins — threshold two (tighter than the convention's three-or-more smell signal), point-at-public-contracts, no-capability- enumeration — and the not-a-copy carve-outs. Metadata, README, CHANGELOG updated; root catalog regenerated via scripts/generate-catalog.mjs. Corrector #5 (terseness) body authored and held pending its locked name. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> 🤖 Generated with [Claude Code](https://claude.com/claude-code)
kyle-sexton
added a commit
that referenced
this pull request
Jul 17, 2026
Corrector #5 tighten-your-output (name locked): terseness discipline — say markdown in fewer words with no semantic loss, write code in fewer lines when readability holds. Code side re-anchors the consuming org's simpler-code convention (named failure modes: speculative generality / wrong abstraction / YAGNI; constraints never traded for line count). Markdown-terseness has no standards convention yet (verified against the standards engineering conventions), so the skill flags that gap rather than inventing a rubric and routes batch prose work to a compress capability's semantic-diff safety net; batch code to simplify. Distinct from point-dont-copy: this removes wasted words from one copy, not a second copy of a fact. Completes the plugin at five correctors sharing one engine doc. Metadata, README, CHANGELOG updated; catalog regenerated. 4 warranted evals. All gates green (markdownlint, skill-quality 0/0 x5, evals schema, plugin validate --strict). Co-authored-by: Claude Fable 5 <noreply@anthropic.com> 🤖 Generated with [Claude Code](https://claude.com/claude-code)
This was referenced Jul 17, 2026
kyle-sexton
added a commit
that referenced
this pull request
Jul 19, 2026
Eval #4 exercised only the briefed-delegable open outcome. The closing invariant names status:needs-info as the other open outcome that must clear status:needs-triage, and its re-entry path differs (attention-view needs-info bucket on reporter reply, not the raw-marker bucket). Add eval #5 to prove an agent clears the raw marker on the needs-info outcome without orphaning the item. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
kyle-sexton
added a commit
that referenced
this pull request
Jul 20, 2026
…463) Two real findings from the PR review, both in the "Open linked PRs" adapter mechanic: 1. Missing word boundary — the keyword alternation had no lookbehind, so `fix` matched inside ordinary words (a body reading `prefixes #463` spuriously matched `fixes #463` and silently dropped the issue from the frontier). Add `(?<!\w)` (Oniguruma variable-width lookbehind, valid in jq). Verified: crafted `prefixes`/`suffix`/`unfixes`/`postfix` bodies now return false; real closing keywords still return true. 2. Inaccurate "truncation-safe per item" claim — `--search "<N> in:body"` is a coarse prefilter matching every PR mentioning the digit, so a small `<N>` (e.g. #5) can exceed the page cap and truncate the real closing PR (GitHub sorts by relevance, not recency). Raise the limit to 1000 (exceeds any realistic repo's open-PR count) and reword the prose to describe the prefilter/superset accurately instead of overclaiming. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
5 tasks
This was referenced Jul 22, 2026
This was referenced Jul 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A versioned, marketplace-wide hook-telemetry contract (a public API for plugin hooks to emit structured execution telemetry) plus
markdown-formatteras its first producer. A hook emits one JSON envelope per run to a consumer-set sink (HOOK_TELEMETRY_SINK, fire-and-forget); unset → no-op. The signal it carries — this hook's ownduration_ms, outcome, findings — is what Claude Code's native OTEL cannot provide (CC reports an aggregatetotal_duration_msand excludes third-party plugin content).Contents
docs/conventions/hook-telemetry/): envelope + per-hookdataJSON schemas, CHANGELOG, worked example.schema_version1.0.markdown-formatter):hook::emit_telemetryemits the envelope after formatting; relativeHOOK_TELEMETRY_SINKresolved producer-side against the caller's repo root (clone-portable, worktree-safe; works around CC #9447 emptyCLAUDE_PROJECT_DIRin plugin hooks).hook-utils27/0,markdown-format41/0.Design notes
statusis a documented open string, not a closed enum — a closed JSON-Schema enum would reject a future value, contradicting the contract's own "consumers MUST tolerate unknown values" rule (JSON Schema 2020-12 §6.1.2). Mirrorshook_eventand OpenTelemetry's open-enum pattern (error.type).stdout+stderr→/dev/null) — never blocks the hook, never touches itsadditionalContextchannel.Validation
.mdedit produced a telemetry record with an isolated per-hookduration_msand a repo-relative subject; fire-and-forget non-blocking and child-survives-reaping both confirmed live (plain&, nosetsidneeded).main(ci: onboard to the CI platform (ci-workflows + standards) #4 CI platform) and conforms to its lanes (shellcheck, exec-bit, typos all clean locally).Notes for reviewers
Consumer (sink) wiring is per-repo: a consumer sets a relative
HOOK_TELEMETRY_SINKin itssettings.jsonpointing at a script that maps the envelope into its own store. No consumer repo is named in the contract (platform stays decoupled from consumers).