docs(ai-adoption-ladder): native-vs-hook telemetry audit + decision record

[kyle-sexton]

Summary

Executes the WP2-deferred audit (#351) empirically and records the decision:

• Live probes on Claude Code 2.1.215 (headless sessions against this machine's otelcol-contrib collector): native emission declares no schemaUrl on any signal; OTEL_RESOURCE_ATTRIBUTES injection lands autonomy.work_item.url at RESOURCE scope on native session metrics (Pillar 2 satisfied natively, no hook code); an inbound TRACEPARENT env value is ignored — fresh root trace (Pillar 3 not satisfiable natively).
• Decision — hybrid, native-first: session signals ride native emission with the join attribute env-injected by the dispatching surface; the contract-authored wrapper span keeps owning causal joining; native session signals join the chain query-side via the Pillar-2 attribute. No hook-emission migration is pending because the conforming session path was already native.
• Version-pinned load-bearing facts + three revisit triggers recorded; the ladder hub links the record under a post-delivery decision-records section.

Acceptance criteria from #351: field-by-field comparison verified against live session capture ✔; decision recorded in docs/topics/ai-adoption-ladder/ with revisit triggers ✔.

Related

• WP2 delivery PR feat(autonomy): telemetry contract + setup slice, conformance check, live-proven demo (WP2) #343 (contract under audit)
• Fleet dogfood wiring for WP2 telemetry and WP3 return accounting #352 (fleet dogfood wiring — will use the env-injection mechanism this audit proved)

Closes #351

🤖 Generated with Claude Code

[claude]

Claude finished @kyle-sexton's task in 1m 51s —— View job

---

PR Review

• Gather context and read changed files
• Review docs/topics/ai-adoption-ladder/native-vs-hook-telemetry-audit.md
• Review docs/topics/ai-adoption-ladder/index.md changes
• Assess correctness, completeness, and alignment with CLAUDE.md

---

This is a documentation-only PR adding an empirical decision record. Overall the content is accurate, well-structured, and appropriately narrow in scope.

What I verified

Contract accuracy — I read plugins/autonomy/reference/telemetry.md in full and checked each Pillar mapping in the audit table:

• Pillar 1 (schema_url): The contract says contract-authored emissions must declare the schema URL pin, but native tool output is "consumed as-is" and the native-surface principle forbids rewriting it. The audit's verdict ("Already handled: native output is consumed as-is") correctly reflects this. ✓
• Pillar 2 (join attribute at RESOURCE scope): The contract says RESOURCE-scope on agent-session emission. The audit confirms OTEL_RESOURCE_ATTRIBUTES injection lands the attribute at RESOURCE scope. Verdict "SATISFIED natively" is correct. ✓
• Pillar 3 (inbound traceparent): The contract (line 65–68 of telemetry.md) says "conforming headless agent surfaces read inbound trace context from their environment." The audit finds Claude Code 2.1.215 ignores TRACEPARENT and emits a fresh root trace. Verdict "NOT satisfied natively" is correct. ✓

Relative link: ../../../plugins/autonomy/reference/telemetry.md from docs/topics/ai-adoption-ladder/ resolves correctly to plugins/autonomy/reference/telemetry.md (file confirmed to exist). ✓

Revisit triggers: All three are precisely scoped to the conditions that would invalidate this decision — native trace context joining, a native schemaUrl declaration, and injection regression. No gaps. ✓

Index.md change: The new "Post-delivery decision records" section is correctly placed below the work-package table and follows the existing formatting conventions. ✓

One minor nit (non-blocking)

Pillar 2 verdict implies a precondition that isn't visible in the table.

The verdict "SATISFIED natively" via environment injection by the dispatching surface is accurate, but "SATISFIED" is conditional — the dispatching surface must actually inject OTEL_RESOURCE_ATTRIBUTES at launch. The decision section covers this, but a reader skimming only the table could read the verdict as "this just works, nothing to wire." A parenthetical note in the verdict like "(requires injection by dispatcher)" would make the precondition scannable without digging into the decision body.

This is cosmetic — the decision section makes the requirement clear. Not requesting a change, just flagging.

Verdict

Clean, well-reasoned decision record. Content matches the live empirical findings as described, maps correctly to the contract pillars, and the hybrid-native-first conclusion is the right call given the Pillar 3 gap. The revisit triggers mean this record has a defined exit path rather than being open-ended.

Ready to merge.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 046ac006d9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[claude]

Claude finished @kyle-sexton's task in 1m 52s —— View job

---

PR Review (follow-up — c76dfab reconciliation)

• Load tools and initialize
• Read changed files at HEAD
• Verify Codex P1 fix: telemetry.md Pillar 3 correction
• Verify Codex P1 fix: ci-otlp-artifact.md template correction
• Check audit doc reconciliation note
• Check plugin.json version bump + CHANGELOG.md entry

---

This review covers the reconciliation commit (c76dfab) added after the initial review. The Codex P1 concern was valid and the fix is complete and correct.

Codex P1 — Pillar 3 contract vs native-surface reality

telemetry.md (L65–76)

The old text said "conforming headless agent surfaces read inbound trace context from their environment" — binding native surfaces to a guarantee they don't honour. The new text correctly scopes the requirement to CONTRACT-AUTHORED emissions only, explicitly documents that a native surface may start a fresh root trace, and states that native session signals attach query-side via the Pillar 2 attribute. Native inbound-context adoption is now a migration trigger, not an assumption. The fix is precise and doesn't overstate or understate what the probe proved. ✓

ci-otlp-artifact.md trace-context section (L35–54)

The old heading said "so its spans join the tree" — implying all agent spans, including native ones, would trace-propagate. The new text changes "its spans" to "every contract-authored emission in that step", adds a callout explaining that native CLI emissions may ignore TRACEPARENT and start a fresh root, documents the OTEL_RESOURCE_ATTRIBUTES injection as the query-side join mechanism, and keeps TRACEPARENT export for forward compatibility. This is the right wiring model. ✓

Audit doc — reconciliation note

Decision point 4 explicitly names the contract-vs-probe contradiction and records that both the telemetry contract and the CI template were corrected in this change. An adopter reading the audit doc can trace the full provenance of the decision. ✓

Plugin version + CHANGELOG

Version bumped 0.7.1 → 0.7.2 in plugin.json ✓. CHANGELOG entry is accurate: "No emission or checker behavior changes" is correct — the conformance checker behavior is unchanged; only the prose description of what the contract requires was wrong. ✓

One minor nit (non-blocking, pre-existing)

The session env block comment at ci-otlp-artifact.md L92–94 still reads:

# Session spans are beta: both flags below are required for the session to
# join the trace tree

After this change, "join the trace tree" means query-side attachment, not direct trace-context joining — a reader hitting this comment after reading the new trace-context section above it will infer the right meaning, but in isolation it implies direct trace propagation. This comment predates this PR and is out of scope here; flagging as a future cleanup candidate.

Verdict

The Codex P1 finding is fully resolved. The contract, template, and audit doc are now internally consistent: wrapper-authored spans join by trace context, native session spans attach query-side via the Pillar 2 join attribute. No gaps remain.

Ready to merge.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c76dfabd2c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[claude]

Claude finished @kyle-sexton's task in 2m 49s —— View job

---

PR Review (follow-up — f038a44 / open Codex P2)

• Load tools
• Read changed files at HEAD
• Check SKILL.md lines 110-117 for Codex P2 finding — confirmed open
• Compute and verify fix
• Post final review

---

Codex P2 — SKILL.md trace-context claim (open, not yet addressed)

The Codex P2 finding posted against c76dfabd2c is valid and still unaddressed at HEAD (f038a44).

plugins/autonomy/skills/setup/SKILL.md:114 currently says:

"Headless -p sessions inherit TRACEPARENT/TRACESTATE from the environment"

The audit proves the opposite: native sessions ignore inbound trace context and start a fresh root. The same correction was already applied to both telemetry.md (Pillar 3 text) and ci-otlp-artifact.md (trace-context section) in commit c76dfab, but SKILL.md was missed. An adopter running /autonomy:setup reads this step before looking at the CI template; the contradictory prose leaves them with the wrong mental model of what the step actually wires.

Required fix — replace lines 114–115:

-   the official monitoring doc). Headless `-p` sessions inherit `TRACEPARENT`/`TRACESTATE`
-   from the environment; interactive sessions deliberately ignore inbound trace context.
+   the official monitoring doc). Headless `-p` sessions may ignore inbound
+   `TRACEPARENT`/`TRACESTATE` — current native surfaces do (verified 2.1.215); the session
+   attaches query-side through the `autonomy.work_item.url` resource attribute. Interactive
+   sessions deliberately ignore inbound trace context.

Fix this →

I prepared and committed this fix locally but was unable to push (insufficient write permission). Apply the diff above directly to docs/351-native-vs-hook-telemetry-audit.

---

Everything else

All other aspects of the PR are correct — the audit table, decision points, revisit triggers, index.md placement, telemetry.md Pillar 3 correction, and ci-otlp-artifact.md template are internally consistent and accurate. The f038a44 changelog-parity-baseline cleanup is clean and unrelated to the docs content.

Blocked on the SKILL.md fix above before merging.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f038a4430e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[claude]

Claude finished @kyle-sexton's task in 3m 17s —— View job

---

PR Review (follow-up — aa5164a)

• Read changed files at HEAD (aa5164a)
• Check SKILL.md Codex P2 status — resolved
• Check design-threads.md Codex P2 status — still open
• Post findings

---

What changed in aa5164a

This commit re-probed Pillar 3 with CLAUDE_CODE_ENHANCED_TELEMETRY_BETA=1 and found that the enhanced-telemetry surface DOES honor inbound TRACEPARENT — session spans carry the injected traceId and parent to the injected span. The prior finding ("native ignores TRACEPARENT") was true of the DEFAULT surface only; the beta surface behaves as the original design-threads record claimed.

All four documents that were changed update consistently:

• Audit table (native-vs-hook-telemetry-audit.md:17) — Pillar 3 verdict correctly nuanced to "Beta-gated. Default surface ignores … With CLAUDE_CODE_ENHANCED_TELEMETRY_BETA=1 the same probe JOINS." ✓
• telemetry.md Pillar 3 (lines 65–76) — Scoped to contract-authored emissions, explains beta behavior as bonus not dependency. ✓
• ci-otlp-artifact.md trace-context section — Explanation is accurate; template correctly exports CLAUDE_CODE_ENHANCED_TELEMETRY_BETA=1 before the agent step. ✓
• Decision point 2 — "DEFAULT native session does not join the trace, and the beta path that does is explicitly non-load-bearing." ✓

Previous blocking finding — SKILL.md — resolved

The prior Claude review (f038a44) blocked on SKILL.md lines 114–115:

"Headless -p sessions inherit TRACEPARENT/TRACESTATE from the environment"

The commit resolves this cleanly: the claim was written about the beta path and stays accurate because the CI template the setup skill generates explicitly exports CLAUDE_CODE_ENHANCED_TELEMETRY_BETA=1 (ci-otlp-artifact.md:96) before the headless step. The SKILL.md step and the template form a coherent whole — in the wired setup, the inherit claim is empirically true. The unblocking rationale is sound. ✓

Open finding — design-threads.md:234–236 — not yet addressed (P2)

docs/topics/ai-adoption-ladder/design/design-threads.md:234–236 still reads:

3. W3C TRACEPARENT propagates across trigger → CI → agent session: one causal tree.
   Verified: headless agent sessions inherit TRACEPARENT natively. Unification by context
   propagation, not sink merging.

The word "natively" is now incorrect — it implies the default surface without any opt-in, which the audit proves is wrong (fresh root on the default surface). The correct reading is: behind CLAUDE_CODE_ENHANCED_TELEMETRY_BETA=1. Every other document in this PR has been updated to reflect the default-vs-beta distinction; this design record was not. This file is identified in index.md as the causal-tree contract record, so a reader consulting it gets the pre-audit claim as a current-state assertion.

Suggested fix — append a SHARPENED note inline with the existing pattern used at line 230:

 3. W3C TRACEPARENT propagates across trigger → CI → agent session: one causal tree.
-   Verified: headless agent sessions inherit TRACEPARENT natively. Unification by context
-   propagation, not sink merging.
+   SHARPENED 2026-07-20 (native-vs-hook audit, #351): headless sessions join natively only
+   behind `CLAUDE_CODE_ENHANCED_TELEMETRY_BETA=1` (verified 2.1.215); default surface starts
+   a fresh root. Contract-authored wrapper emission is the stable causal-chain leg; the beta
+   path is a bonus. Query-side join via `autonomy.work_item.url` is the stable fallback.
+   See `docs/topics/ai-adoption-ladder/native-vs-hook-telemetry-audit.md`.

Fix this →

Verdict

The audit finalization in aa5164a is correct and internally coherent. The SKILL.md blocking concern from the prior review is legitimately resolved — the beta-path rationale is sound and the CI template confirms it. Only design-threads.md:234–236 remains outstanding: the "natively" claim is a stale empirical assertion in the governing design record and should carry the same SHARPENED annotation the rest of the PR establishes.

One targeted fix away from ready to merge.

[claude]

Claude finished @kyle-sexton's task in 1m 52s —— View job

---

PR Review (follow-up — 7e26885)

• Load tools and initialize
• Read SKILL.md lines 110–120 — verify Codex P2 fix (trace-context inherit claim)
• Read design-threads.md lines 228–240 — verify Codex P2 fix (design-thread inherit claim)
• Confirm no remaining open findings
• Post final review

---

Both open Codex P2 findings from the previous review are resolved. Here's what I verified at HEAD.

SKILL.md — Codex P2 (trace-context inherit claim)

plugins/autonomy/skills/setup/SKILL.md:113–117

The old text stated unconditionally that headless sessions inherit TRACEPARENT/TRACESTATE. The new text reads:

"Headless -p sessions inherit TRACEPARENT/TRACESTATE from the environment only under the enhanced-telemetry beta — the default surface starts a fresh root and joins query-side via the resource attribute (verified empirically) — and interactive sessions deliberately ignore inbound trace context."

This is precise: the beta path inherits (which the re-probe in aa5164a confirmed is accurate), the default surface does not, and the query-side join mechanism is named. ✓

design-threads.md — Codex P2 (T6 causal-tree record)

docs/topics/ai-adoption-ladder/design/design-threads.md:234–238

The old text stated "Verified: headless agent sessions inherit TRACEPARENT natively." The new text cites the audit, scopes the claim to the enhanced-telemetry opt-in, documents the query-side fallback for the default surface, and names the wrapper span as the stable causal leg until graduation. ✓

Internal consistency check

• Audit doc Pillar 3 verdict (native-vs-hook-telemetry-audit.md:17): "Satisfied natively only behind the enhanced-telemetry beta, whose span shapes the setup slice deliberately does not depend on — the contract-authored wrapper emission remains the stable conforming leg until spans graduate." ✓
• telemetry.md Pillar 3 (reference/telemetry.md:65–76): Binds propagation to contract-authored emissions; notes empirically that a default native surface starts a fresh root (with opt-in as the exception); states native sessions attach query-side. ✓
• All four changed surfaces (audit doc, telemetry contract, CI template, SKILL.md, design-threads.md) now describe the same model consistently.

Pre-existing nit (unchanged, still out of scope)

ci-otlp-artifact.md:94–95 comment "Session spans are beta: both flags below are required for the session to join the trace tree" — "join the trace tree" still implies direct propagation rather than query-side attachment, but this was already flagged by the previous reviewer as pre-existing and out of scope for this PR. Worth a follow-up cleanup.

Verdict

All previously flagged open findings are resolved. The audit table, decision section, telemetry contract, CI template, SKILL.md, and design-threads.md now describe a single internally consistent model: wrapper-authored spans own the causal join, native session signals attach query-side via the Pillar 2 resource attribute on the default surface, and the beta path adds direct span parenting as a non-load-bearing bonus.

Ready to merge.