feat(autonomy): telemetry contract + setup slice, conformance check, live-proven demo (WP2)#343
Conversation
The T6 contract as tool-agnostic normative text: semconv v1.43.0 pin with schema_url on every emission (registry-cited, never copied), the autonomy.work_item.url join attribute (normalized canonical item URL, resource-scoped on sessions / span-scoped on CI, selection rule, 1:1-lease granularity guarantee, join-epoch limitations, deferred immutable-ID trigger, confidentiality clause, governed autonomy.* extension), one causal tree via W3C context propagation with the interactive exclusion, three sink classes with the file-artifact free default, native-surface principle, telemetry-is-not-return boundary. Co-Authored-By: Claude Fable 5 (1M context) <noreply@anthropic.com>
… phase 2) Telemetry slice in setup: existing-stack wiring, file-artifact free default (pipeline-span JSON-lines writer + ephemeral per-job collector into one artifact directory, interactive coverage via a local collector store), OTEL_RESOURCE_ATTRIBUTES join-attribute wiring for dispatched sessions (vendor honoring verified against the official monitoring doc), paid sinks advisory-first with metered-cost caveats, additive telemetry section in the schema-versioned binding. Ships the emission-conformance check (schemaUrl pin + normalized join attribute over OTLP JSON-lines, lowerCamelCase encoding), surface-class snippet templates, three new eval cases, v0.2.0 manifest + README. Co-Authored-By: Claude Fable 5 (1M context) <noreply@anthropic.com>
…2 phase 3) Scratch-repo demo executed the full free-default path: a trigger stub emitted its own root span and propagated TRACEPARENT (observed hop), the template writer emitted the pipeline span, and a live headless agent session inherited the context — DuckDB over the single artifact directory shows 1 distinct trace across 4 spans, and cost/token metrics carry autonomy.work_item.url at resource scope string-identical to the pipeline span. Empirical contract correction from the run: native agent-session emission declares no schemaUrl, so the pin now binds contract-authored emissions (native surfaces consumed as-is) and the conformance check requires at least one pinned declaration per output set instead of one per line. Sanitized fixture (trigger + pipeline + scrubbed session cost metrics) committed as the WP3 demo interface. Co-Authored-By: Claude Fable 5 (1M context) <noreply@anthropic.com>
…e 4) Co-Authored-By: Claude Fable 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8c4cb15bc7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Claude finished @kyle-sexton's task in 3m 12s —— View job PR Review — feat(autonomy): WP2 telemetry contract + setup slice
SummaryThe telemetry contract is well-designed — the three-pillar structure (standard OTLP, work-item join attribute, W3C causal tree), native-surface principle, and "telemetry is not return" boundary are all sound. The conformance script handles the native-vs-contract-authored distinction correctly (tolerating absent Two issues need fixing before this is ready to merge, and two more are worth addressing now. Critical1. Template doesn't show how to extract the parent span ID from The pipeline-span writer uses The committed fixture ( # Parse inbound trigger context when present
if [ -n "${TRACEPARENT:-}" ]; then
IFS='-' read -r _ver trace_id trigger_span_id _flags <<< "$TRACEPARENT"
else
trace_id="<generated 32-hex>"
trigger_span_id=""
fi
span_id="<generated 16-hex>"And the JSON shape needs a conditional 2. Conformance check validates join attribute globally, not per emission
if (joinAttributeHits === 0 && linesChecked > 0) {
findings.push(`no occurrence of ${JOIN_ATTRIBUTE} anywhere in the checked output`);
}This passes when a pipeline span carries A tighter approach: require the join attribute on every Important3. Fixture span count doesn't match PR description The PR description states "1 distinct trace across 4 spans (trigger + pipeline + 2 session spans)" and "DuckDB over the single artifact directory." The committed fixtures have the trigger span (1) and pipeline span (1) — but If the session spans were omitted intentionally (e.g., because 4. OTEL_RESOURCE_ATTRIBUTES=autonomy.work_item.url=<canonical-item-url>This assignment silently drops any existing OTEL_RESOURCE_ATTRIBUTES="${OTEL_RESOURCE_ATTRIBUTES:+$OTEL_RESOURCE_ATTRIBUTES,}autonomy.work_item.url=<canonical-item-url>"Minor5. No eval case for conformance-check failure
6. The description now mentions telemetry prominently (WP2's primary deliverable) but the What works well
|
…k, span fixture The pipeline-span writer now consumes inbound TRACEPARENT for both the shared trace ID and parentSpanId (joining the trigger's tree instead of rooting a second one); the conformance check requires the join attribute on EVERY emission, not once globally; the committed fixture gains a scrubbed agent-session span line so the >=3-span single-tree demo is reproducible from the fixture alone (verified: 1 trace, 4 spans, conformance OK). Exec bit set on the conformance script (hygiene gate). Co-Authored-By: Claude Fable 5 (1M context) <noreply@anthropic.com>
|
Claude finished @kyle-sexton's task in 3m 13s —— View job PR Re-Review — feat(autonomy): WP2 telemetry contract + setup slice (c56e742)
Re-review summaryThe three issues flagged as Critical (×2) and Important (#3) in the previous round are all correctly fixed in Previously critical — confirmed fixedFix #1: TRACEPARENT propagation The shell snippet correctly splits TRACEPARENT on Fix #2: Per-emission join attribute enforcement The check now fails any emission line where Fix #3: Fixture span count The fixture now contains a
One trace, four spans, single continuous parent chain. PR description claim now verifiable from the fixture alone. ✅ Still unaddressed — Important#4 — OTEL_RESOURCE_ATTRIBUTES=autonomy.work_item.url=<canonical-item-url>Any adopter whose CI platform already sets OTEL_RESOURCE_ATTRIBUTES="${OTEL_RESOURCE_ATTRIBUTES:+$OTEL_RESOURCE_ATTRIBUTES,}autonomy.work_item.url=<canonical-item-url>"This is particularly relevant for teams adopting the "existing-stack" sink class (eval 6), who are exactly the adopters most likely to have an existing Still open — Minor#5 — No conformance-failure eval case
#6 —
What's cleanThe Overall: the two blocking issues from the previous review are resolved. Issue #4 is a real footgun for adopters and worth a follow-up fix before or shortly after merge; #5 and #6 can reasonably live in a follow-up. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c56e742028
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Claude finished @kyle-sexton's task in 2m 32s —— View job PR Re-Review — feat(autonomy): WP2 telemetry contract + setup slice (fc1f8ca)
Re-review summaryThe merge commit New Codex findings — assessedCodex P1 — Valid, Important
The In practice, the file-artifact free default (collector Minimal fix — move for (const entry of block) {
const entryHits = { count: 0 }; // per resource-entry, not per line
checkAttributeList(entry.resource?.attributes, `${where} resource`, entryHits);
for (const scope of ...) {
for (const item of ...) {
checkAttributeList(item.attributes, ..., entryHits);
}
}
if (entryHits.count === 0) {
findings.push(`${where}: resource entry carries no ${JOIN_ATTRIBUTE} attribute`);
}
totalHits += entryHits.count;
}The return value and the per-line Codex P2 — Valid, Minor
const NORMALIZED_URL = /^https:\/\/[^\s?#]+[^\s?#/]$/;This regex accepts structurally invalid URLs: function isValidNormalizedUrl(v) {
try {
const u = new URL(v);
return u.protocol === "https:" && u.hostname !== "" &&
u.search === "" && u.hash === "" && !v.endsWith("/");
} catch { return false; }
}Non-blocking, but the regex flaw undermines the normalization guarantee. Worth addressing in this PR or a follow-up. Carried over — still open#4 — OTEL_RESOURCE_ATTRIBUTES=autonomy.work_item.url=<canonical-item-url>Still a plain assignment. Any adopter with an existing OTEL_RESOURCE_ATTRIBUTES="${OTEL_RESOURCE_ATTRIBUTES:+$OTEL_RESOURCE_ATTRIBUTES,}autonomy.work_item.url=<canonical-item-url>"Previously open — Minor (still open)#5 — No conformance-failure eval case: #6 — What looks cleanAll three previously-blocking/critical fixes ( Verdict: Codex P1 and issue #4 are both real gaps at the contract enforcement and adoption layers respectively. Neither blocks merge on its own, but two open Important items in the same PR is worth one more fix pass before landing. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fc1f8ca846
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Claude finished @kyle-sexton's task in 2m 10s —— View job PR Re-Review — feat(autonomy): WP2 telemetry contract + setup slice (ccff787)
Re-review summary
Still open — should fix before mergeSession env block variables are not exported CLAUDE_CODE_ENABLE_TELEMETRY=1
OTEL_METRICS_EXPORTER=otlp
OTEL_LOGS_EXPORTER=otlp
OTEL_EXPORTER_OTLP_ENDPOINT=http://127.0.0.1:4318
OTEL_RESOURCE_ATTRIBUTES=autonomy.work_item.url=<canonical-item-url>These five assignments are shell-local. Without The export CLAUDE_CODE_ENABLE_TELEMETRY=1
export OTEL_METRICS_EXPORTER=otlp
export OTEL_LOGS_EXPORTER=otlp
export OTEL_EXPORTER_OTLP_ENDPOINT=http://127.0.0.1:4318
export OTEL_RESOURCE_ATTRIBUTES="${OTEL_RESOURCE_ATTRIBUTES:+$OTEL_RESOURCE_ATTRIBUTES,}autonomy.work_item.url=<canonical-item-url>"Still open — Important (can follow up)Per-resource join attribute validation
Fix: move Still open — MinorCodex P2 — URL regex accepts invalid hostnames
Issue #5 — No conformance-failure eval case
Issue #6 — WP2 makes telemetry the primary new deliverable; What looks cleanAll three previously-blocking fixes from Verdict: One fix before merge (session env block not exported — silently breaks all telemetry for adopters following the template). The per-resource join validation fix is a real correctness gap in the enforcement surface worth addressing soon after. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ccff787538
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…ported env Address the review wave on the conformance checker and CI template: - Require the join attribute per OTLP resource entry, not per JSON line — a joined pipeline entry in an export batch must not vouch for an unjoined session entry beside it - Validate the join value by parsing (WHATWG URL) with a round-trip equality requirement, rejecting parser-repaired values like an empty host and malformed ports a prefix regex accepted - Validate scope-level schemaUrl declarations against the pin, not just resource-entry ones - Export the session telemetry variables in the CI template so the launched agent process actually inherits them Co-authored-by: Claude Fable 5 (1M context) <noreply@anthropic.com>
|
Claude finished @kyle-sexton's task in 2m 11s —— View job PR Re-Review — feat(autonomy): WP2 telemetry contract + setup slice (f22220c)
Re-review summaryAll four targeted fixes in Confirmed fixedSession env block now exports all variables All five variables now carry Per-entry join attribute validation
URL parsing via
Scope-level
Still open — Important (carry-forward)
export OTEL_RESOURCE_ATTRIBUTES=autonomy.work_item.url=<canonical-item-url>The The safe form: export OTEL_RESOURCE_ATTRIBUTES="${OTEL_RESOURCE_ATTRIBUTES:+$OTEL_RESOURCE_ATTRIBUTES,}autonomy.work_item.url=<canonical-item-url>"Not blocking on its own; carried over from previous rounds. Still open — Minor#5 — No conformance-failure eval case
#6 —
What's cleanThe Verdict: All blocking items are now resolved. The remaining OTEL_RESOURCE_ATTRIBUTES clobber is a real adoption footgun worth a follow-up fix, but with the |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f22220cbc7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Metrics and logs alone leave the session out of the trace tree; the beta span export needs CLAUDE_CODE_ENHANCED_TELEMETRY_BETA=1 plus OTEL_TRACES_EXPORTER=otlp, as the setup slice already documents. Co-authored-by: Claude Fable 5 (1M context) <noreply@anthropic.com>
|
Claude finished @kyle-sexton's task in 1m 28s —— View job PR Re-Review — feat(autonomy): WP2 telemetry contract + setup slice (6f91836)
Re-review summary
Confirmed fixedCodex P1 — Beta trace export enabled # Session spans are beta: both flags below are required for the session to
# join the trace tree; without them only metrics/logs are emitted.
export CLAUDE_CODE_ENHANCED_TELEMETRY_BETA=1
export OTEL_TRACES_EXPORTER=otlpBoth flags are present, exported, and accompanied by an inline comment explaining why both are required. The comment precisely describes the consequence of omitting them ("only metrics/logs are emitted"), giving adopters enough context to make an informed decision if their fleet policy restricts beta surfaces. The complete session env block now has all seven variables exported — every value the agent process needs to participate in the trace tree. ✅ Still open — Important (carry-forward from three rounds)
export OTEL_RESOURCE_ATTRIBUTES=autonomy.work_item.url=<canonical-item-url>The export OTEL_RESOURCE_ATTRIBUTES="${OTEL_RESOURCE_ATTRIBUTES:+$OTEL_RESOURCE_ATTRIBUTES,}autonomy.work_item.url=<canonical-item-url>"Still open — Minor#5 — No conformance-failure eval case
#6 —
What's cleanThe inline comment on the beta flags is exactly right — it explains the consequence, not just the requirement, which is more useful when an adopter's fleet policy gates beta features. The session env block now comprehensively covers all seven variables a conforming headless session requires. All conformance-script fixes from prior rounds ( Verdict: All blocking items across the full review series are resolved. This PR is mergeable. The |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6f91836303
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| for (const scope of entry.scopeSpans ?? entry.scopeMetrics ?? entry.scopeLogs ?? []) { | ||
| checkSchemaUrl(scope.schemaUrl, `${where} scope`, tally); | ||
| for (const item of scope.spans ?? scope.metrics ?? scope.logRecords ?? []) { | ||
| checkAttributeList(item.attributes, `${where} ${item.name ?? "record"}`, hits); |
There was a problem hiding this comment.
When an agent-session resourceSpans entry has no autonomy.work_item.url on its resource but one individual span has the attribute, this increments the shared hit count and the checker exits successfully. That accepts an export which violates the telemetry contract's resource-scope requirement for agent sessions, leaving other session spans (and any resource-level signals) without the work-item join while setup can still declare the emitting state conformant. Validate the attribute at the required scope for agent-session entries rather than allowing span attributes to satisfy the resource check.
Useful? React with 👍 / 👎.
…ven demo (WP3) (#356) ## Summary WP3 return-accounting package (T5), on the merged WP2 telemetry contract (#343): - **Convention doc** `plugins/autonomy/reference/return-accounting.md`: three-layer data model (machine/deterministic, human-attested, agent/LLM), the two Boris questions verbatim as the human fields (counterfactual would-have-done-anyway + manual-effort band), six-band effort scale `<1h / 1-4h / 4h-1d / 1d-1w / 1w-1mo / >1mo` (user-locked), capture at the task boundary (work-item close / PR merge) as tracker-resident marker records, autonomous-class scope only, agents never estimate the return fields. - **Guided-setup capture slice**: close-boundary templates + marker-record shape, idempotent at-close capture, attestation upsert. - **WP1 PLAN Phase 6 → [DONE]** (fleet binding doc merged in melodic-software/standards#196). ## Live demo evidence (executed this effort) Throwaway private repo `kyle-sexton/autonomy-demo-scratch` issue #1: unattested marker record written at close; a re-fired close stayed idempotent (count 1, no downgrade); attestation upsert recorded with platform actor + source citation; DuckDB join against the WP2 committed OTLP fixture matched the join key string-identically (`autonomy.work_item.url`). Near-duplicate audit: the capture slice composes the work-item tracker seam and the WP2 telemetry contract — no second telemetry or tracker surface created. No linked issue ## Related - #343 (WP2 telemetry — join-key contract this package consumes) - melodic-software/standards#196 (WP1 Phase 6 fleet binding) - #330 (WP2+WP3 plans) 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5 (1M context) <noreply@anthropic.com>
…ils, routines, runner design pack) (#359) ## Summary Architect-round Plans for the four remaining ai-adoption-ladder work packages, on the merged WP1-WP3 foundations (#330, #343, #356): - **WP4 trigger-dispatch** (`docs/topics/ai-ladder-wp4-trigger-dispatch/PLAN.md`): T1 contract — four signal-surface classes (`tracker-vcs-event`/`temporal`/`agent-internal`/`channel-feed`), six adapter obligations, signal envelope (`signal.*` marker record incl. `signal.work_class` stamped from the security surface), kick + drain dispatch through the one work-item queue entrypoint, `check-signal-envelope.mjs` enforcement surface, scratch-repo demo. - **WP5 guardrails** (`ai-ladder-wp5-guardrails/PLAN.md`): T2+T3+#241 — matrix hub + 4 leaves, two-surface binding split (security binding in settings-as-code home, `executor_class`, admission decision table + classification rules), `check-security-binding.mjs` with eval-mode promotion-ceiling semantics, probe-before-bind isolation slice, free-tier CI positive probe + fail-closed demo. - **WP6 routines** (`ai-ladder-wp6-routines/PLAN.md`): T7 — catalog hub (39 classes) + 10 v1 leaves (dead-code sweep excluded per D4 correction), contract-owned catalog-to-matrix mapping rules, setup slice with detect-diff-reconcile of existing schedulers, novel-class derivation probe. - **WP7 runner design pack** (`ai-ladder-wp7-runner/PLAN.md`): T4 graduation — spine + 8 seam specs, lifecycle state model, two-family stop criteria, terminal-handoff escalation with severity/ack knobs, topology seam map. ZERO build artifacts; build stays trigger-gated; birth decisions USER-RESERVED. Merge order pinned: WP4 before WP5 (absent admission binding fail-closes to human-gated — no ungoverned window), WP6 after both, WP7 last. Review: WP4+WP5 stress-tested by a fresh-context reviewer (14 findings, all folded — headline: work-class classification obligation + `executor_class` on the security binding). WP6+WP7 stress-tested by a second fresh-context pass (10 findings, FIX-THEN-SHIP both, all folded — headline: routine→class mappings land on the WP5 security surface, never the repo-local agent-writable section; D2 gains a hybrid DET+AGT branch, a `not-a-routine` catalog flag, and provenance/structural axes reaching C4/C5; WP7's escalation severity refinement adds the WP5 schema+validator edits it requires as genuinely additive optional keys, C5→L3 cited from the matrix cell, vendor-hosted merge cap restated on the cloud-backend path). No linked issue ## Related - #330 (contract set + WP1-WP3 plans) - #343 (WP2 telemetry), #356 (WP3 return accounting) 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5 (1M context) <noreply@anthropic.com>
…elope check (WP4) (#372) WP4 trigger-dispatch package (T1), per the merged PLAN (all four phases complete on this branch). No linked issue ## Related - #359 (WP4–WP7 plans, incl. this package's PLAN as merged) - #356 (WP3 return accounting — the marker-record precedent the envelope serialization follows) - #343 (WP2 telemetry — the trace-tree obligation this contract imports) ## What ships - **`plugins/autonomy/reference/trigger-dispatch.md`** (new contract): four signal-surface classes (`tracker-vcs-event` / `temporal` / `agent-internal` / `channel-feed`), two recorded attributes (provenance, transport incl. `push-lifecycle` expiry fail-close + poll-detector backstop), six class-generic adapter obligations (normalize+enqueue only; layered idempotent dedup with create-then-reconcile and a live-duplicate drain guard; provenance + durable raw link; traceparent propagation; admission enforcement at the seam with absent-binding fail-close; closed-loop ack), security-surface work-class stamping with verified `agent-internal` parent provenance and higher-of class resolution, the schema-versioned signal envelope (marker `<!-- autonomy:signal:v1 -->`), kick + drain dispatch with the one-entrypoint invariant and execution-surface attestation, executor surface classes with security-owned `executor_class`. - **Setup trigger/dispatch slice** (`skills/setup/SKILL.md`): discovery-first per-class surface interview, DIY-floor kick + hourly drain as reviewable changes, plan-gated surfaces advisory with cost, execution-surface recording with the attestation caveat stated, fail-closed absent-admission wiring, additive `triggers` binding section (shared `surfaces` map shape that later scheduling sections reuse). - **Templates**: `templates/trigger-adapters.md` (per-class adapter shapes, six obligations inline), `templates/ack-reply.md` (one-line ack, item URL first). - **Enforcement**: `scripts/check-signal-envelope.mjs` — validates the envelope marker record (required keys, enums, W3C traceparent, agent-internal `signal.parent_item` as normalized canonical URL, deterministic raw-link origin branching via the binding's surfaces maps), exit contract 0/1/2 mirroring `check-emission-conformance.mjs`. - Five new setup evals; plugin 0.4.0; README + catalog. ## Conforming-path demo (scratch repo, live) Run on `kyle-sexton/autonomy-demo-scratch` (private throwaway) with the work-items capability's tracker seam installed (`tools/work-item-tracker/` copied per its documented install path) and a repo-local binding carrying the `triggers` section. No admission binding present → the fail-closed path is what the demo proves. No paid service touched. 1. **Stub signal**: label `autonomy:trigger` applied to source issue [#2](kyle-sexton/autonomy-demo-scratch#2); labeled-event id `28172185501` fetched from the issue timeline (surface-native unique event id → `signal.identity`). 2. **Dedup pre-search** for the identity: `[]` (no existing item). 3. **Enqueue via the seam** (`work-item-tracker.sh create-item`): queue item [#3](kyle-sexton/autonomy-demo-scratch#3) created carrying the full envelope — class/transport/provenance, identity, raw link (`https://github.com/ghapi/repos/kyle-sexton/autonomy-demo-scratch/issues/events/28172185501`), traceparent `00-dab5c21d6389b65c950f6b13f14af7e3-de1b63c53960913b-01` (the trigger-hop trace context observed on the enqueue record), NO `signal.work_class` (no classification rules bound → unclassified), label `needs-human`. 4. **Create-then-reconcile re-search**: exactly one item (#3) matches the identity — no duplicate to close. 5. **Ack echo** on the source surface: [comment on #2](kyle-sexton/autonomy-demo-scratch#2 (comment)) — `Queued as kyle-sexton/autonomy-demo-scratch#3 (autonomy: tracker-vcs-event signal)`. 6. **Envelope conformance**: `check-signal-envelope.mjs <item-3-body> --binding .claude/autonomy/binding.json` → `Signal-envelope conformance OK: 1 envelope(s) checked.` exit 0. 7. **Drain claims via the seam lease** (autonomous drain mode, the ONE entrypoint): `work-item-tracker.sh claim github:kyle-sexton/autonomy-demo-scratch#3` → lease acquired (`holder: kyle-sexton`, lease comment `5014153239`), exit 0. 8. **Fail-closed refusal, audited live**: admission check before execution found NO admission binding → autonomous execution REFUSED, [refusal comment on #3](kyle-sexton/autonomy-demo-scratch#3 (comment)), claim released; final state `OPEN`, labels `["needs-human"]`, assignees `[]`. This also exercises the unclassified → human-gated path (no security binding → no `signal.work_class`). No second dispatch path was exercised anywhere: the event kick and the drain both funnel into the seam lease. ## Near-duplicate audit The dispatch entrypoint composes the work-item queue capability's EXISTING autonomous drain mode and race-safe seam lease — no second claim or dispatch mechanism was created anywhere in this package. The adapter templates normalize and enqueue only; the drain-side live-duplicate guard and the create-then-reconcile rule reuse the queue's own primitives. ## Gates - `scripts/validate-plugins.sh` exit 0; `node scripts/validate-plugin-contracts.mjs` exit 0 (vendor deny-list clean over `reference/`); `node scripts/generate-catalog.mjs --check` in sync. - `claude plugin validate plugins/autonomy --strict` pass; skill-quality `check-skill.sh setup` PASS (2 pre-existing WARNs: line soft-target, gotchas surface); `evals.json` valid against the bundled schema (ajv draft2020). - markdownlint 0 errors; typos clean; lychee (offline) 0 errors. Full plugin test suite rides CI (local run exceeds the shell timeout; prior-session finding). - Checker exit contract verified with fixtures: no-args → 2 + usage; valid tracker/temporal-local → 0; temporal without `--binding`, ci-cron origin with `file:` link, unrecorded surface, agent-internal missing `signal.parent_item` → 1 each. ## Boris check The 2→3 cell ("break up your work into loops and routines; let Claude kick off Claude") is instantiated as governed queue + kick; the step-3 trap is honored (audit trail before scale — envelope, ack, lease, refusal are all tracker-resident audit records; no agent-count machinery); the step-1 guardrail is untouched; the telemetry trace tree is the kick-off audit trail (traceparent on the enqueue record). Fresh-docs mandate: GitHub Actions events reference and the Claude Code headless reference re-fetched this session before the SKILL.md edit (cited in the Phase 2 commit). <details> <summary>PLAN.md (as merged, tags advanced)</summary> See `docs/topics/ai-ladder-wp4-trigger-dispatch/PLAN.md` on this branch — all four phase tags [DONE], decisions table and stress-test summary carried in the file. </details> 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…#377) WP5 guardrail-enforcement package (T2 + T3 + the security-review policy resolution), per the merged PLAN. Phases 1–3 complete; Phase 4's positive probe runs in this PR's CI (transcript below after the run). No linked issue ## Related - #359 (WP4–WP7 plans, incl. this package's PLAN as merged) - #372 (WP4 trigger-dispatch — the admission seam this package's policy content binds; envelope/classification cross-links land here) - #343 (WP2 telemetry — the promotion-evidence base and trace-tree payloads) ## What ships - **Guardrail hub** (`reference/guardrails.md`): five-class × five-column matrix (T3 verbatim), glance-layer routing to four leaves, D8 permission-posture note, escalation event classes + org-bound routing + work-item/trace payload over the governed queue (no second channel). - **Leaves** (`reference/guardrails/`): `isolation-ladder.md` (L0–L3, attendance × input-provenance axes, rejected trigger-source axis with falsification reason, L2 unattended floor, fail-closed no-silent-degrade), `work-classes.md` (risk bundles; C1 repo-read-only with queue writes permitted; C5→L3 floor; promotion/demotion discipline with shipped default predicates — C2 auto-merge ≥20 completions/≥14 days/100% gates/0 reverts, C3 ai-review ≥30 advisory/0 missed-blocking; demotion one-event-suffices), `security-review.md` (two layers, per-class blocking knobs as floors — no override escape on verification), `admission-policy.md` (3-axis decision table with `"*"` wildcard + most-specific-wins, equal-specificity tie = invalid binding, shipped defaults C1/C2 autonomous within caps regardless of provenance, C3 per-item human, C4/C5 human-gated; caps `autonomous_concurrency: 1`, `items_per_run: 3`). - **Security-binding schema** (`skills/setup/schemas/guardrails-security-binding.schema.json`): draft 2020-12, `additionalProperties: false`, fail-closed semantics in-schema; `executor_class`, `dispatch_posture`, per-surface `isolation_bindings` (level → substrate + `probe_evidence` + `runtime_markers`), merge/verification knobs, `promotion_state`, `escalation_routes`, `admission` (classification + rules + caps). - **Semantic validator** (`skills/setup/scripts/check-security-binding.mjs`, exit 0/1/2): matrix merge caps, vendor-hosted human-gate cap, marker non-emptiness + pairwise joint-satisfiability ambiguity rejection, class-aware per-surface isolation verdicts (C5 requires L3), missing-L2 error under `autonomous-enabled` vs declared verdict under `human-gated-only`, non-promotable promotion entries rejected, admission weakening requires `override_justification` (admission-only escape), promotion-gated knobs require their ratified `promotion_state` entry, equal-specificity ties invalid, evaluation mode resolving EFFECTIVE promotion state against contrary evidence without writing the binding. 91 binding fixtures plus 67 transcript and 2 evidence files, each negative producing exactly one finding. Note: egress-probe targets must be multi-label DNS names or public literal IPs — encoded loopback/private forms are normalized and rejected. - **Setup guardrail slice** (SKILL.md): two-surface resolution (security binding in settings-as-code home behind an agent-unwritable org-policy-home bootstrap; repo-local non-security remaps), substrate detection per surface, probe-before-bind live validation, detect-diff-reconcile against existing org guardrail surfaces, security review folded in (one slice, no near-duplicate capability), paid SKUs advisory + cost surfaced, fail-closed no-L2. Plus `templates/isolation-probe.md` and five new evals; plugin 0.5.0. ## Live-validation demonstration (Phase 4) **Positive probe** — this PR carried a one-off workflow (`wp5-isolation-probe-demo.yml`, since removed per the PLAN; evidence persists in the run log below) that PROVISIONED a genuine egress-denied boundary on the hosted runner (nested `docker run --network none` container) and executed the probe recipe INSIDE it, with a networked-outer-job control step. Transcript from the PR run: From [run 29675620833](https://github.com/melodic-software/claude-code-plugins/actions/runs/29675620833): ```text outer-egress: HTTP 200 (networked, as expected) egress-denial: PASS - external fetch failed inside the boundary (exit 4) credential-absence: PASS - every tested credential path absent inside the boundary (exit 3) captured live transcript: "schema_version": "1", "probed_at": "2026-07-19T05:56:39Z", "assertions": { "egress_denied": { "host": "example.com", "exit_code": "4", "outcome": "denied" }, "credentials_absent": { "path": "$HOME/.ssh,$HOME/.netrc,$HOME/.docker/config.json", "exit_code": "3", "outcome": "absent-or-denied" } }, "outer_context_networked": true Security binding OK: live-evidence/demo-binding.json - surface "ci-pool-a": C1 eligible, C2 eligible, C3 eligible, C4 eligible, C5 blocked (requires L3, proven L2) ``` The final line is `check-security-binding.mjs` exit 0 against a binding whose `probe_evidence` is the transcript captured seconds earlier in the same job — the checker passes only against live-captured, identity-matched evidence. (The recorded transcript predates the capture-contract fields that later review hardening added to the checker — per-target credential exit codes, host-side expansions, transport outcomes, and the outer-context reachability/existence codes on both assertions; it is preserved verbatim as the run's historical record.) **Negative path** — `check-security-binding.mjs` on the missing-L2 `autonomous-enabled` fixture (exit 1): ```text - isolation_bindings: no bound surface reaches L2 under dispatch_posture autonomous-enabled — autonomous dispatch is blocked fail-closed; compliant paths: bind an L2-capable substrate (whole-process OS-sandbox wrap, default-deny-egress container) on an execution surface, or declare dispatch_posture human-gated-only ``` **Declared posture** — the same missing-L2 shape under `human-gated-only` validates (exit 0) with the blocked-dispatch verdict reported as the DECLARED posture, not a defect. **Demotion evaluation** — promoted-C2 binding + contrary evidence (exit 0, binding file unchanged): ```text - C2-auto-merge: bound promoted -> effective unpromoted — ceiling lowered by contrary evidence (reverted-merge at 2026-07-10T12:00:00Z) WITHOUT modifying the binding; demotion files an escalation item on route "queue:escalations/demotion" requesting the human-ratified binding update ``` ## Near-duplicate audit Security review is folded into the ONE guardrail slice of guided-setup (no separate security-review capability); escalation composes the governed queue itself via the trigger contract's acknowledgment symmetry — no second claim, dispatch, or escalation channel was created anywhere in this package. ## Gates `validate-plugins.sh`, `validate-plugin-contracts.mjs` (vendor deny-list clean over `reference/`), catalog `--check` in sync, markdownlint 0 errors, typos clean, lychee offline 0 errors, `claude plugin validate --strict` pass, skill-quality `check-skill.sh setup` PASS (2 pre-existing WARNs), evals.json schema-valid (ajv draft2020), actionlint clean on the demo workflow. Full plugin test suite rides CI. Validator exercised over the full fixture set with expected exit codes and single-finding assertions. ## Boris check Step-3 guardrails cell covered (agent sandboxing = isolation ladder; automatic code + security review = verification layers; token/model management = cost tiers); the step-4 sentence ("enforcing the right guardrails for each type of work") is instantiated as the matrix; the step-3 trap honored (human-ratified promotion, automatic evaluation-time demotion); step-1 guardrail untouched. Auto-mode classifier tuning + allowlist distribution explicitly deferred with named triggers. <details> <summary>PLAN.md (as merged, tags advanced)</summary> # ai-ladder-wp5-guardrails ## Brief ### TLDR Guardrail-enforcement package (T2 + T3 + #241 instance): one guardrail contract shaped as a progressive-disclosure hub (matrix doc as index/spine) with on-demand leaf docs, an org-binding seam split by governance sensitivity, evidence-gated promotion with automatic demotion, and the guardrail slice of guided-setup (detect → bind → live-validate → fail-closed). ### Goal Any adopting org can bind the five-class guardrail matrix to its own instances and get per-work-class enforcement — isolation floor, verification layers, merge policy, cost tier, escalation — with the security-sensitive bindings outside agent blast radius, promotion earned on queryable evidence, and no silent degrade anywhere. ### Locked decisions | # | Decision | |---|---| | D1 | ONE guardrail contract, hub-and-leaves progressive disclosure: matrix doc = index/spine (table + one-line class/column definitions, glance layer resolves the common question); leaf docs load on demand — e.g. isolation-ladder levels, per-class detail + promotion triggers, security-review policy, admission policy (examples, not a fixed list; split at architect). Plus the guardrail slice of guided-setup. Both in the capability-distribution home. Fleet materializations = /work-items backlog. | | D2 | Org-binding seam: contract-owned schema, one logical binding, two governance surfaces split by sensitivity. Security-sensitive axes (isolation substrates per level, merge policy, per-layer blocking knobs, promotion-trigger state, escalation routes, admission policy) bind in the settings-as-code home — outside the blast radius of agents working in the consuming repo (agent-writable guardrail binding = bypass channel). Non-security mappings (class→label strings, cost-tier→model names) may bind repo-locally per the work-item-tracker binding pattern. Layered resolution: org-policy-home defaults → settings-as-code per-repo binding → repo-local non-security remaps; contract defines resolution order, materialization at architect. Fail-closed on absent/invalid security-sensitive binding; documented defaults only for non-security axes. | | D3 | Promotion/demotion discipline: contract defines the trigger SHAPE — an evidence predicate over queryable telemetry (verification outcomes per the telemetry contract are the evidence base). Promotion = human-ratified knob flip, recorded as a reviewable change on the governance surface — never automatic. Demotion = automatic fail-closed on contrary evidence (gate failure, human-reverted merge), re-earn from there. Org binds threshold values; suggested defaults at architect. Near-term promotable cells: C2 auto-merge, C3 AI-review advisory→blocking (#241); C4/C5 merge never promotes. | | D4 | Sandbox-ladder setup slice: detect available substrates per level per machine surface → bind level→substrate in the security binding → live-validate BEFORE recording (empirical probe, e.g. denied-egress smoke test inside the boundary; a binding lands only after the sandbox provably blocks) → fail-closed verify (no L2 substrate → autonomous dispatch blocked for that surface, compliant paths named — no silent degrade). | | D5 | #241 three-part resolution folds into the matrix structure, nothing ships separate: security-review policy contract (two layers, per-layer blocking knob) = verification-column leaf; security-review setup = part of the one guardrail slice of guided-setup (near-duplicate capability ban); the adjustment/audit layer generalizes matrix-wide — guided-setup always detect-diff-reconcile against existing org guardrail surfaces (sandbox configs, branch protections, review workflows), never greenfield-assume, never silently overwrite. | | D6 | Escalation column sharpened from DIRECTIONAL: contract defines escalation event classes (gate failure, verification divergence, admission rejection, demotion event, structural-class plan approval, untrusted-provenance always), routing obligation (org-bound routes in the security binding), and payload (work-item ref + trace link, one causal tree). Mechanism reuses the governed queue: escalation lands as a human-gated work item + optional channel notification via the trigger-contract acknowledgment symmetry — no second channel. Only interactive escalation UX stays deferred; trigger: WP7 runner design pack. | | D7 | Admission-policy content (the trigger contract's admission seam enforces it; this package owns it): decision-table shape — signal-surface class × initiator provenance × work class → disposition (autonomous-eligible / human-gated / audited rejection) — plus caps (autonomous concurrency, per-run items). Shipped defaults: C1/C2 autonomous-eligible within caps regardless of provenance (blocking agent provenance would sever the agent-kicks-off-agent loop), C3 per-item human admission, C4/C5 human-gated. Provenance is recorded input to gating, never trusted as isolation; caps bound total autonomous fan-out. | | D8 | Permission posture: attended ergonomics (auto-mode classifier tuning, safe-command allowlist distribution) is NOT a matrix column — unattended runs replace per-action prompts with the whole-process boundary; the ladder leaf carries a one-line permission-posture note (L1 = attended ergonomics tier, L2+ = the boundary is the control). Classifier tuning + allowlist distribution = backlog seeds with trigger, not silent drops. | ### Constraints - Any fleet repo or vendor name in normative contract text is a defect; vendor names appear only as marked examples and in binding docs. - Security-sensitive bindings never live where the agents they govern can edit them. - Fail-closed everywhere: unavailable substrate, absent/invalid security binding, contrary promotion evidence — block and name the compliant path, never degrade silently. - No new cost by default: free-path scanners and substrates default; entitlement-gated tools (e.g. paid code-scanning SKUs) are advisory + explicit opt-in with cost surfaced. - One queue, one escalation channel (the queue itself); no second claim, dispatch, or escalation mechanism. - Boris-alignment is the standing acceptance criterion (no step-skipping, trust before scale). ### Acceptance criteria - Matrix doc's glance layer alone answers "what governs class X" — leaf docs only for depth (progressive-disclosure convention conformance). - Binding schema names the two governance surfaces + layered resolution order; security axes fail closed when unbound. - Promotion is human-ratified + evidence-gated; demotion is automatic; both leave a reviewable audit trail on the governance surface. - Guided-setup slice detect-diff-reconciles existing guardrail surfaces, live-validates isolation before binding, and has zero paid dependencies on its default path. - Admission decision-table defaults match the matrix class defaults verbatim; agent provenance is not blocked by default; caps are org-bindable. - Boris check: step-3 guardrails cell covered (agent sandboxing = ladder; automatic code + security review = verification layers; token/model management = cost tiers); step-4 sentence ("enforcing the right guardrails for each type of work") instantiated as the matrix; step-3 trap honored (human-ratified promotion, automatic demotion); step-1 guardrail untouched. - Auto-mode classifier tuning + allowlist distribution explicitly out-of-scope with named triggers, not dropped. ### Captured assumptions - Verification-outcome telemetry (WP2 contract) is queryable at promotion-evaluation time; until wired, promotion evaluation is manual over the same evidence definition. - Free-path deterministic scanners remain available on the default path; paid code-scanning SKUs stay entitlement-gated (re-verified: private-repo CodeQL requires a paid license). - The work-item-tracker binding precedent (contract-owned schema, org-supplied values, stop-and-report on invalid binding) remains the fleet's binding idiom. ### Out-of-scope (deferred with triggers) - Interactive escalation UX — trigger: WP7 runner design pack. - Auto-mode classifier tuning + safe-command allowlist distribution — trigger: 1→2 residue sweep / standards allowlist-distribution work (backlog seeds). - Cost ENFORCEMENT (hard spend caps) — trigger: 3→4 transition work; cost tiers here are policy vocabulary only. - Fleet materializations (binding instances, workflow gates, scanner wiring) — /work-items backlog post-graduation. ### Deferred questions - Exact class/level/column/attribute tokens + binding schema fields — `/architect` (with the plugin naming pass). - Leaf-doc file split + glance-layer wording — `/architect`. - Promotion-trigger default values (run counts, windows) + demotion evidence set — `/architect`. - Setup-slice probe mechanics per substrate class — `/architect`. - Admission decision-table serialization format — `/architect`. ## Plan Recommendation-locked this round under the user's standing pre-authorization (same basis as the WP2/WP3 round): hub doc `reference/guardrails.md` with leaf docs under `reference/guardrails/` (`isolation-ladder.md`, `work-classes.md`, `security-review.md`, `admission-policy.md` — the D1 example split, adopted); class tokens `C1`–`C5` and level tokens `L0`–`L3` carried verbatim from T3/T2; security binding = a contract-owned JSON document in the settings-as-code home (`schema_version` from `"1.0"`); non-security remaps = an ADDITIVE `guardrails` section of the WP1 repo-local binding; admission decision table serialized as JSON inside the security binding. Prerequisite: the WP4 implementation PR merged (`reference/trigger-dispatch.md` exists — the admission leaf cites its seam; the interim is safe via WP4's fail-closed absent-binding clause). ### Phase 1: Guardrail hub + leaf docs [DONE] | File | Action | What changes | |---|---|---| | `plugins/autonomy/reference/guardrails.md` | Create | The hub/spine (D1): the five-class × five-column matrix table imported from T3 unchanged (C1 read-only / C2 mechanical / C3 scoped / C4 structural / C5 untrusted-provenance × min-isolation / verification / merge policy / cost tier / escalation), one-line class and column definitions, and the glance-layer rule — this table alone answers "what governs class X"; every deeper question routes to a named leaf. Permission-posture note per D8 (one line: L1 = attended ergonomics tier; L2+ = the boundary is the control). Escalation event classes per D6 (gate failure, verification divergence, admission rejection, demotion, structural-class plan approval, untrusted-provenance always) with the routing obligation (org-bound routes in the security binding) and payload (work-item ref + trace link); mechanism = the governed queue itself (human-gated item + optional channel notification via the trigger contract's acknowledgment symmetry — no second channel). Boris step-4 sentence cited as the matrix's source framing. Zero vendor/fleet names. | | `plugins/autonomy/reference/guardrails/isolation-ladder.md` | Create | T2 ladder normative text: L0–L3 level definitions, attendance × input-provenance axes, the falsified trigger-source axis recorded as rejected (with reason), L2 unattended floor, fail-closed rule where L2 unavailable (block + name compliant paths, no silent degrade), permission-posture line per D8. Instances org-supplied; free-path substrate CLASSES named as marked examples only. | | `plugins/autonomy/reference/guardrails/work-classes.md` | Create | Per-class detail: risk-property bundle per class (blast radius, reversibility, provenance, verifiability) — stating explicitly that C1 "read-only" scopes REPO surfaces and permits governed-queue/tracker writes (work-item filing, no repo mutation; the WP6 stress-test's F8 disambiguation) and that C5's min-isolation cell is the L3 floor WP7's C5-dispatch gate cites, per-cell promotion triggers (D3 shape: evidence predicate over queryable telemetry per the telemetry contract; promotion = human-ratified knob flip recorded as a reviewable change on the governance surface; demotion = automatic fail-closed on contrary evidence, re-earn from there). Suggested default predicates (org-bindable values): C2 auto-merge — ≥ 20 autonomous C2 completions over ≥ 14 days with 100% deterministic-gate pass and 0 human-reverted merges; C3 AI-review advisory→blocking — ≥ 30 advisory reviews with 0 human-confirmed missed-blocking findings. C4/C5 merge never promotes. Demotion evidence set: any post-merge gate failure, any human-reverted merge, any verification divergence — one event suffices. | | `plugins/autonomy/reference/guardrails/security-review.md` | Create | The #241 security-review policy contract as the verification-column leaf: two layers (deterministic scanners + AI security review), per-layer blocking knob (advisory/blocking per class, bound on the governance surface), free-path scanners default, paid SKUs advisory + explicit opt-in with cost surfaced. | | `plugins/autonomy/reference/guardrails/admission-policy.md` | Create | The D7 admission-policy content the WP4 seam enforces: decision-table shape (signal-surface class × initiator provenance × work class → disposition `autonomous-eligible` / `human-gated` / `audited-rejection`) + caps (`autonomous_concurrency`, `items_per_run`). Shipped defaults verbatim from D7: C1/C2 autonomous-eligible within caps regardless of provenance; C3 per-item human admission; C4/C5 human-gated; provenance recorded, never trusted as isolation. Default cap values (org-bindable): `autonomous_concurrency: 1`, `items_per_run: 3` — conservative Boris trust-before-scale floor. Serialization: the `admission` object of the security binding (Phase 2 schema). | **Sanity Check:** - Hub glance test: `grep -c '| C[1-5] |' plugins/autonomy/reference/guardrails.md` = 5 (matrix rows present) and hub contains zero level-definition prose (`grep -c 'microVM' plugins/autonomy/reference/guardrails.md` = 0 — depth lives in leaves) - `grep -c 'audited-rejection' plugins/autonomy/reference/guardrails/admission-policy.md` ≥ 1 - `grep -ci 'fail-closed' plugins/autonomy/reference/guardrails/isolation-ladder.md` ≥ 1 - Promotion defaults present: `grep -c '20 autonomous C2 completions' plugins/autonomy/reference/guardrails/work-classes.md` ≥ 1 and `grep -c '14 days' plugins/autonomy/reference/guardrails/work-classes.md` ≥ 1 - Vendor+fleet deny-list sweep exit 0; lychee lane passes ### Phase 2: Binding schema (two governance surfaces) [DONE] | File | Action | What changes | |---|---|---| | `plugins/autonomy/skills/setup/schemas/guardrails-security-binding.schema.json` | Create | Contract-owned JSON Schema for the security binding document that lives in the settings-as-code home (outside agent blast radius per D2): `schema_version` (const "1.0"), `executor_class` (`self-operated` \| `vendor-hosted` — the merge-policy gating input, security-surface data per the trigger contract), `dispatch_posture` (`autonomous-enabled` \| `human-gated-only`, default `autonomous-enabled` — an org may intentionally record its security settings while keeping every dispatch human-gated; the PR-review finding that a missing L2 entry must not conflate that deliberate posture with an invalid binding), `isolation_bindings` keyed by EXECUTION SURFACE (org-named surface id — e.g. the CI pool, a self-run host class; the PR-review finding that a flat level→substrate map lets one surface's L2 binding satisfy the check while a different dispatch surface has no boundary at all) → level token → substrate instance id + probe evidence ref + `runtime_markers` (the per-surface TRUSTED identifying markers the trigger contract's dispatch seam matches platform-attested runner context against — a key/value set drawn from metadata the executing workload cannot forge, e.g. attested runner-pool/label identity; serialized HERE, on the agent-unwritable security surface, so surface attestation never falls back to the repo-local recorded id), `merge_policy` (per class, capped per matrix), `verification_blocking` (per layer per class), `promotion_state` (per PROMOTABLE cell only: state + ratifying change ref + evidence window), `escalation_routes` (event class → org route), `admission` (classification rules — the signal→work-class mappings the trigger contract's adapters stamp from, e.g. label→class for tracker-vcs-event — plus decision-table rules with a `"*"` wildcard per axis and most-specific-wins precedence (full triple > two axes > one axis > default), optional per-rule `override_justification`, and caps `autonomous_concurrency` / `items_per_run`). `additionalProperties: false`; fail-closed semantics documented in-schema (absent/invalid security binding blocks autonomous dispatch). | | `plugins/autonomy/skills/setup/scripts/check-security-binding.mjs` | Create | Validates a security-binding document against the schema + semantic rules the schema cannot express: merge-policy caps never exceed the matrix (C4/C5 human always; `executor_class: vendor-hosted` caps every class at human-gated); every surface bound under `dispatch_posture: autonomous-enabled` must declare non-empty `runtime_markers` (else the dispatch seam could never attest the actual surface and every dispatch would fail closed as unattestable), and marker sets must be PAIRWISE UNAMBIGUOUS under joint satisfiability — two conjunctive predicates are compatible (and the binding REJECTED) unless they require CONFLICTING values for at least one shared key, since a runtime context carrying the union of two non-conflicting predicates matches both (subset/identity checks alone miss e.g. `{pool: blue, region: us}` vs `{pool: blue, os: linux}`); the dispatch seam still requires EXACTLY ONE matching surface at runtime, failing closed on zero or multiple matches — the validator-level joint-satisfiability rejection is what keeps that runtime rule from turning validly bound fleets into ambiguity outages; isolation verdicts are PER SURFACE and CLASS-AWARE: each bound surface resolves, per work class, eligible (its bound level meets that class's min-isolation matrix cell — L2 is the floor for any autonomous dispatch, C5 requires L3) or blocked — admission/executor resolution must name the surface a dispatch would run on AND the item's work class, and verifies the surface meets the class's minimum at dispatch time, so an L2-only surface is never selected for a C5 item (even after human admission or a later policy change) and an unbound surface is blocked even when a sibling surface is bound; under `dispatch_posture: autonomous-enabled` it is an ERROR when NO bound surface reaches L2 (fail-closed, compliant paths named) — under `human-gated-only` the binding validates and the verdict reports blocked autonomous dispatch as the declared posture, not a defect; a `promotion_state` entry for a non-promotable cell (C4/C5 merge) is rejected; admission defaults not weaker than shipped defaults without the rule's `override_justification` (declared in the schema, so a justified override validates); every bound isolation level carries probe evidence. Plus an EVALUATION mode: given a binding + an evidence source, resolves the EFFECTIVE promotion state — the bound knob is a ceiling; live contrary evidence (gate failure, reverted merge, verification divergence) lowers it at evaluation time without writing the binding (automatic demotion has no write-back actor by design: the binding is agent-unwritable; the demotion event additionally files an escalation item requesting the human-ratified binding update). This evaluation is a LIVE-PATH OBLIGATION, not a demo artifact: the admission seam and the merge disposition MUST resolve the effective state (querying the promotion-evidence telemetry) before every autonomous dispatch/merge decision — reading the raw `promotion_state` alone is non-conforming, and unavailable evidence telemetry fail-closes to the unpromoted default — the checker's evaluation mode exists to exercise the same resolution mechanically. Exit contract 0/1/2. | | `plugins/autonomy/skills/setup/SKILL.md` | Modify | Document the two-surface split + layered resolution order (org-policy-home defaults → settings-as-code per-repo security binding → repo-local non-security remaps in the WP1 binding's additive `guardrails` section: class→label strings, cost-tier→model names) — INCLUDING the security binding's resolvable locator: the org-policy home carries the repo→security-binding-document registry the dispatch seam resolves the binding through when settings-as-code is a separate repository — and for SECURITY resolution the org-policy-home identity itself must come from an agent-unwritable bootstrap, never the repo-local binding (the binding seam's known limitation that `org_policy_home` may persist repo-locally is tolerable for non-security defaults only; a repo-writable pointer would let an agent redirect the whole chain to a forged policy repository with a forged registry, binding, and matching runtime markers): the seam pins the org-policy-home identity from org-level platform configuration outside repo blast radius (an org-level setting/variable repo agents cannot write) or the executor's trusted deployment config, and any security resolution that would depend on a repo-writable pointer — or an unresolvable locator — fail-closes autonomous dispatch with the compliant path named. Fail-closed on absent/invalid security binding; documented defaults only for non-security axes. | **Sanity Check:** - `check-security-binding.mjs` exit 0 on a fixture-valid document, exit 1 on each negative fixture: C4 merge_policy `auto`, `executor_class: vendor-hosted` with any non-human-gated merge row, `promotion_state` entry for a C4 cell, missing probe evidence, missing L2 under `dispatch_posture: autonomous-enabled`, admission weakened without `override_justification`; exit 0 with a blocked-dispatch verdict on the missing-L2 + `human-gated-only` fixture (deliberate posture, not a defect) (fixtures under `evals/fixtures/`) - Evaluation mode: a promoted-C2 binding + contrary-evidence fixture resolves the EFFECTIVE state demoted (exit output asserts the ceiling was lowered without modifying the binding file) - Schema declares `executor_class` and `override_justification` (`grep -c 'executor_class' …schema.json` ≥ 1; `grep -c 'override_justification' …schema.json` ≥ 1) - Schema `additionalProperties: false` present (`grep -c '"additionalProperties": false' …schema.json` ≥ 1) - `claude plugin validate --strict` exit 0 ### Phase 3: Guided-setup guardrail slice [DONE] Extends the `setup` skill: detect → bind → live-validate → fail-closed (D4), always detect-diff-reconcile against existing org guardrail surfaces (D5 — sandbox configs, branch protections, review workflows), never greenfield-assume, never silently overwrite. | File | Action | What changes | |---|---|---| | `plugins/autonomy/skills/setup/SKILL.md` | Modify | Guardrail slice: detect available substrates per level per machine surface; bind level→substrate into the security binding; LIVE-VALIDATE before recording — the empirical probe per substrate class: denied-egress smoke test inside the boundary (a network fetch to a well-known external host MUST fail) + host-credential-path read attempt (MUST be absent/denied); a binding lands only after the probe transcript proves the boundary; probe evidence ref recorded in the binding. Fail-closed verify: no L2 substrate on a surface → autonomous dispatch blocked for that surface, compliant paths named. Security-review slice folded here per D5 (one guardrail slice, no separate near-duplicate capability). Paid scanner SKUs advisory + opt-in with cost surfaced. | | `plugins/autonomy/skills/setup/templates/isolation-probe.md` | Create | Probe recipe per substrate class (container / OS-sandbox wrap / VM-microVM), parameterized, vendor names as marked examples only: egress-denial probe command shape, credential-absence probe shape, expected-failure assertions, transcript capture shape. | | `plugins/autonomy/skills/setup/evals/evals.json` | Modify | Add guardrail-slice cases: substrate detection interview, probe-before-bind ordering, fail-closed no-L2 path, detect-diff-reconcile against a pre-existing branch-protection surface, paid-SKU advisory refusal-to-default. | | `plugins/autonomy/README.md` + `plugins/autonomy/.claude-plugin/plugin.json` | Modify | Capability list + description + minor version bump. | **Sanity Check:** - `/skill-quality:check` + `validate-evals` pass - `grep -c 'probe' plugins/autonomy/skills/setup/templates/isolation-probe.md` ≥ 3 - `grep -ci 'detect-diff-reconcile' plugins/autonomy/skills/setup/SKILL.md` ≥ 1 - Fleet-name sweep exit 0 ### Phase 4: Live-validation demonstration + gates [DONE] Acceptance probe, BOTH paths mandatory: (positive) run the Phase 3 probe recipe against a REAL, PROVISIONED egress-denied boundary — the CI job on this public repo (free minutes) launches a nested no-network container via the container runtime present on the hosted Linux runner image (network mode none / internal-only, no host env or secrets passed in) and executes the probe INSIDE it; the outer job keeps normal networking, so the probe's egress-denial assertion is satisfied by a genuine boundary the job itself created — never by the bare runner (which has outbound access and would fail the assertion) and never by a hardcoded "denied" — transcript shows denied egress + absent credentials inside the boundary, proving the probe DETECTS a genuine boundary; (negative) the fail-closed path is demonstrated by validating an `autonomous-enabled` binding with a MISSING L2 entry and observing the blocked-dispatch error; the deliberate `human-gated-only` posture validates separately with its blocked-dispatch verdict reported as declared, not as a defect. A fixture security binding referencing the positive probe's evidence passes `check-security-binding.mjs`; the demotion evaluation fixture (Phase 2) runs here as part of the demo record. Then the full gate roster (WP2 Phase 4): `validate-plugins.sh`, `run-plugin-tests.sh`, `validate-plugin-contracts.mjs`, markdown/typos/lychee, `claude plugin validate --strict`, catalog regen. Near-duplicate audit statement: security review folded into the one guardrail slice (D5); escalation composes the governed queue — no second channel created. **Sanity Check:** - Positive probe transcript (provisioned no-network container on the CI runner: egress denied + credentials absent inside; outer job networked) attached to the PR body - Negative path: `check-security-binding.mjs` exit 0 on the demo binding, exit 1 on the missing-L2 `autonomous-enabled` fixture, exit 0 + declared blocked-dispatch verdict on the `human-gated-only` fixture - Demotion evaluation output (effective state lowered, binding file unchanged) in the demo record - All gate scripts exit 0; catalog in-sync - Near-duplicate audit statement present in the PR body ## Blast radius MEDIUM-HIGH — one plugin's files, but this contract governs every autonomous dispatch decision (admission), every isolation binding, and the promotion path to the closed loop; security-posture content raises the review bar. Fully git-revertible; the security binding itself ships as schema + fixtures only (no live org binding lands in this package). ## Stress-test summary Fresh-context plan review (WP4+WP5 batch): 14 findings, verdict FIX-THEN-SHIP, all folded. WP5's share — F1b (HIGH, cross-package): the hosted human-merge-gate cap keyed on an executor class the schema never carried → `executor_class` added to the security schema and enforced by the check; F9 (MED): `override_justification` was required by the check but undeclared under `additionalProperties: false` → declared in the admission-rule sub-schema; F10 (MED): automatic demotion had no write-back actor against an agent-unwritable binding → eval-time evidence-gating (bound state is a ceiling; live contrary evidence lowers the effective state; the demotion event files an escalation item requesting the human-ratified update), with an evaluation-mode fixture in the check and the demo; F11 (MED): the fail-closed pivot alone never proved the probe detects a real boundary → positive probe mandatory against a free-tier CI container, negative path demonstrated separately; F12 (LOW): fragile `grep -c '20'` → full-phrase grep; F13 (LOW): 3-D admission table gains a `"*"` wildcard + most-specific-wins precedence, and promotion entries for non-promotable cells are rejected; F14 (LOW, symmetry): the D8 backlog seeds are echoed in Open questions. The classification-rules half of F1a lands here as the `admission.classification` schema content (the WP4 contract's adapters stamp from it). ## Execution shape Fully sequential 1 → 2 → 3 → 4 — Phase 2's schema serializes Phase 1's admission leaf; Phase 3 wires Phase 2's binding; Phase 4 probes Phase 3's recipe. Cross-package: after the WP4 implementation PR (admission seam exists to cite). | Phase | Surface | Basis | |---|---|---| | 1 | main-session | normative security-policy authoring, tightly coupled T2/T3 imports | | 2 | main-session | schema + semantic-validator judgment | | 3 | main-session | setup-skill judgment, probe design | | 4 | main-session | live probe + gate runs | ## Open questions - Fleet materializations (live org security binding, workflow gates, scanner wiring) — /work-items backlog post-merge (Brief out-of-scope, triggers recorded). - Interactive escalation UX — WP7 runner design pack (Brief trigger). - D8 backlog seeds (auto-mode classifier tuning + safe-command allowlist distribution) — carried in the Brief with named triggers; filed via the work-items flow. ## Decisions made (gate-passed) | Decision | What it changes in the plan | Basis (evidence) | |---|---|---| | Leaf split: 4 leaves under `reference/guardrails/` | Phase 1 file set | Brief D1 names exactly these four as the example split; each maps to one matrix column/axis | | Hub-subdirectory layout (first `reference/` subdir) | Phase 1 paths | Progressive-disclosure hub needs its leaves grouped; flat names would couple hub prose to 4 sibling filenames | | Promotion defaults: C2 ≥20 runs/≥14 days/100% gates/0 reverts; C3 ≥30 advisory/0 missed-blocking | work-classes leaf | D3 requires suggested defaults at architect; values conservative per Boris trust-before-scale (org-bindable) | | Cap defaults `autonomous_concurrency: 1`, `items_per_run: 3` | admission leaf + schema | D7 requires caps; floor values bound total fan-out at the trust-earning stage | | Security binding as JSON + JSON Schema + semantic validator | Phase 2 deliverables | WP1 binding precedent (schema-versioned JSON, stop-and-report on invalid); schema alone cannot express matrix caps | | `executor_class` + classification rules live on the security surface; promotion is eval-time evidence-gated (ceiling semantics) | Phase 2 schema + check | Stress-test F1b/F10: gating inputs must sit outside agent blast radius; an agent-unwritable binding cannot receive an automated demotion write | | Demo pivots to fail-closed path when no container substrate exists | Phase 4 | D4 makes fail-closed the required behavior — the pivot demonstrates the contract, not a skipped demo | [FALLBACK — confirm or override] `check-security-binding.mjs` as a NEW deliverable (Phase 2) — invented beyond the Brief on the WP2/WP4 enforcement-surface precedent. Flag if unwanted. ## Handoff to implementation ### User-approval gates - The binding-check script above is [FALLBACK] — surface before authoring if contested. - Any change to the shipped admission defaults or matrix cells during implementation → STOP (these are T3/D7 user-locked content, not implementation discretion). - Any scope expansion beyond the four phases re-enters `/architect review`. ### Execution shape ([EXEC-SHAPE] tagged) Sequential 1→4, all main-session (table above). PLAN.md phase tags advance in the same commit as each phase; live probe per Phase 4 with the documented fail-closed pivot. ### Mechanical work Commit per phase on the implementation branch (suggest `feat/autonomy-guardrails`); gates re-run in full at Phase 4; PR body carries the probe transcript + near-duplicate audit statement + this PLAN in a `<details>` block at close-out. </details> 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
## Summary Effort-end housekeeping for the AI-adoption-ladder effort: all seven work packages are merged, so the per-WP topic slices come out per the close-out convention (publish the PLAN, then prune the slice). - WP4–WP7 PLANs were already spliced into their PR bodies at each close-out (#372, #377, #600, #676). - WP1–WP3 PLANs are now published the same way into #333, #343, and #356 (each merged PR body gained a `PLAN (published at effort-end close-out)` details block), so every slice's content has a durable home on its merged PR before the directory is removed. - The WP map in `docs/topics/ai-adoption-ladder/index.md` now records delivery PRs instead of live slugs. The effort hub itself (index, design evidence, research docs) stays. No content is lost: every deleted PLAN.md is byte-recoverable from git history and readable in its merged PR's details block. ## Related No linked issue (effort-end housekeeping; the WP map issue #239 is already closed). - #239 (WP map: AI adoption ladder — the effort this closes out) - #333 / #343 / #356 (WP1–WP3 delivery PRs, PLANs published into their bodies in this close-out) - #372 / #377 / #600 / #676 (WP4–WP7 delivery PRs, PLANs published at their own close-outs) Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…ger row (#698) ## Summary D1 sweep for #353: the WP1–WP7 PLAN out-of-package notes (mined from the merged PR bodies #333/#343/#356/#372/#377/#600/#676) contained 31 deferred notes; every note that named no tracking issue and had no contract-recorded trigger now has a durable home: - **README roadmap** gains four trigger-gated rows: fleet guardrail materializations (WP5), fleet routine stand-up + existing-scheduler reconciliation (WP6), vendor-binding capability templates (WP6), cost enforcement / hard spend caps (WP5, 3→4 trigger). - **Trigger register** gains the second-binding-consumer cross-repo drift check (WP1). - **`reference/return-accounting.md`** records the per-work-class precision-graduation deferral beside its band-stability rule (WP3). - **CHANGELOG.md created** for the autonomy plugin (first entry; 0.1.0–0.7.0 history stays in the WP PR bodies) + version bump to 0.7.1 — starts the CHANGELOG-parity posture #663 gates on. All other mined notes were already covered: tracked issues (#351, #352, #694–#697), contract-recorded triggers (telemetry immutable-ID, return-accounting expansion/revisit), delivered sibling WPs, resolved `/architect` questions, or the user-held dormant triggers (runner build T4, L3 backend, merge serialization, mid-run interrupt, org-enablement, cross-team). Full disposition table lands as a comment on #353 at close. Documentation only — no contract semantics change. ## Related - #685 (effort-end prune that moved the PLAN records into the merged PR bodies) - #663 (CHANGELOG-parity gate this PR's new CHANGELOG.md starts satisfying) - WP delivery PRs mined: #333, #343, #356, #372, #377, #600, #676 Closes #353 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…ecord (#699) ## Summary Executes the WP2-deferred audit (#351) empirically and records the decision: - **Live probes on Claude Code 2.1.215** (headless sessions against this machine's `otelcol-contrib` collector): native emission declares no `schemaUrl` on any signal; `OTEL_RESOURCE_ATTRIBUTES` injection lands `autonomy.work_item.url` at RESOURCE scope on native session metrics (Pillar 2 satisfied natively, no hook code); an inbound `TRACEPARENT` env value is ignored — fresh root trace (Pillar 3 not satisfiable natively). - **Decision — hybrid, native-first**: session signals ride native emission with the join attribute env-injected by the dispatching surface; the contract-authored wrapper span keeps owning causal joining; native session signals join the chain query-side via the Pillar-2 attribute. No hook-emission migration is pending because the conforming session path was already native. - Version-pinned load-bearing facts + three revisit triggers recorded; the ladder hub links the record under a post-delivery decision-records section. Acceptance criteria from #351: field-by-field comparison verified against live session capture ✔; decision recorded in `docs/topics/ai-adoption-ladder/` with revisit triggers ✔. ## Related - WP2 delivery PR #343 (contract under audit) - #352 (fleet dogfood wiring — will use the env-injection mechanism this audit proved) Closes #351 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Summary
WP2 (
ai-ladder-wp2-telemetry) implementation, phases 1–4 per its approved PLAN (docs/topics/ai-ladder-wp2-telemetry/PLAN.md, #330):reference/telemetry.md): OTLP pinned to semconv v1.43.0 withschema_urlon contract-authored emissions (native tool output consumed as-is — empirically corrected during the demo); theautonomy.work_item.urljoin attribute (normalized canonical item URL, selection rule, resource-scope on sessions / span-scope on CI, 1:1-lease granularity guarantee, join-epoch limitations, confidentiality clause, governedautonomy.*extension); one causal tree via W3C context propagation with the interactive exclusion; three sink classes with the file-artifact free default; native-surface principle; telemetry-is-not-return boundary.OTEL_RESOURCE_ATTRIBUTESjoin wiring (vendor honoring verified against the official monitoring doc), paid sinks advisory-first, additivetelemetrybinding section.scripts/check-emission-conformance.mjs): pinnedschemaUrl(OTLP JSON lowerCamelCase), join-attribute presence + normalization — the contract's enforcement surface.Conforming-path demo (executed live)
Trigger stub emitted its own root span and propagated TRACEPARENT → template writer emitted the pipeline span → a real headless agent session inherited the context with the join attribute at resource scope. Results: DuckDB over the single artifact directory shows 1 distinct trace across 4 spans (trigger + pipeline + 2 session spans);
claude_code.cost.usage/token.usagecarryautonomy.work_item.urlstring-identical to the pipeline span; conformance check exit 0. Sanitized fixture committed (skills/setup/evals/fixtures/otlp-demo/) as the WP3 demo interface. Demo capture used an equivalent OTLP/HTTP-JSON receiver (same JSON-lines output shape as the template's collector fileexporter) — the adopter template keeps the collector.Audits
No linked issue
Related
docs/topics/ai-ladder-wp2-telemetry/PLAN.md(docs(topics): ai-adoption-ladder contract set + WP1-WP3 plans #330); WP1: feat(autonomy): autonomy plugin — category, topology contracts, guided-setup v0 (WP1) #333🤖 Generated with Claude Code
PLAN (published at effort-end close-out)
ai-ladder-wp2-telemetry
Brief
TLDR
Telemetry unification contract (T6): every execution context — interactive session, CI,
autonomous runner — emits standard OTLP pinned to the OTel CI/CD + VCS semantic conventions,
plus one contract-owned custom attribute carrying the work-item ID (the return-accounting join
key), with W3C TRACEPARENT propagation forming one causal tree per triggered chain. Sink
binding is deployment-owned and out of the contract. WP2 ships the contract doc and the
telemetry slice of guided setup.
Goal
Any adopting org can reach the emitting state — all three execution contexts producing
semconv-conformant OTLP with the work-item join attribute and propagated trace context — via
guided setup, with zero paid dependencies on the default path, against whatever observability
stack the org already has (or the free file-artifact default when it has none).
Locked decisions
schema_urlon every emission. RC status accepted; upstream renames are handled as reviewed migrations, never silent. Exact version number resolved at build time against the live releases page.Constraints
fleet bindings live only in the binding-seam instance doc.
return fields stay in WP3.
trace-tree joining is a headless/CI/runner property — the contract must not promise it for
interactive contexts.
(standard OTel into whatever stack exists), no step-skipping, trust before scale.
Acceptance criteria
pinned version +
schema_url, never hand-copied lists.join key without reopening this contract.
default path, as reviewable changes, with no paid dependency.
Captured assumptions
changes (evidence: githubreceiver v1.37.0 alignment).
same-day verbatim captures in durable homes (Unify session and CI verification telemetry? #247 resolution comment, design-threads
T5/T6). Re-fetch if any quote beyond those captures becomes load-bearing.
attributes, not beta span trees.
("go with your recommendation"), each grounded in same-day primary-sourced research.
Out-of-scope (deferred with triggers)
WP2 build lands.
contract text exists to audit against.
classes (existing stack / file default / opt-in backend), never vendor picks.
Deferred questions
/architect./architect(verify live at build)./architect(verify live; communityactions unofficial).
/architect.Plan
Interview-locked this round: join-key attribute namespace
autonomy.work_item.*(OTelapp-name-prefix form; reverse-DNS structurally excluded by the fleet-name ban; migrates to any
future upstream work-item namespace per D3); join-key value = the item's canonical web URL,
attribute
autonomy.work_item.url(globally unique across repos/trackers, derivable on bothsides; native short IDs collide multi-repo).
Prerequisite: WP1 implementation merged (
plugins/autonomy/exists withreference/docs,skills/setup/, and the contract-validator fleet-name gate). All phases sit on a fresh branchcut after that merge; WP3's PR follows this package's PR (its convention doc cites this
contract — dead-link avoidance, same ordering rule as WP1's standards PR).
Phase 1: Telemetry contract doc [DONE]
plugins/autonomy/reference/telemetry.mdv1.43.0withschema_url: https://opentelemetry.io/schemas/1.43.0on every emission; attributes cited by registry reference, never hand-copied lists; RC caveat + reviewed-migration policy for upstream renames (D4). (2) The work-item join attribute: single custom attributeautonomy.work_item.url, value = the work item's canonical web URL in normalized form (the tracker's own canonical item URL: https scheme, no trailing slash, no query string or fragment) — string equality is the join operation, so the normalization rule is normative. Selection rule: the key is always the WORK ITEM's URL, never a PR/change URL; a change closing N items yields N per-item associations; an agent session keys on its leased item. Attribute SCOPE pinned per context: RESOURCE-scope on agent-session emission (so cost/token metrics and session spans all carry it), span-scope on the CI pipeline/task spans. Granularity guarantee: conforming autonomous dispatch runs ONE leased work item per emitting session/process (the trigger-layer lease contract is the guarantor); multi-item batch sessions get session-granular cost only — stated as a known limitation, not silently misattributed. Known join-epoch limitations stated normatively: repo rename/transfer and tracker migration change the canonical URL; the join is query-time, so sinks MAY remap historical values; a secondary immutable-ID attribute is deferred with trigger (rename churn proves material). Confidentiality clause: the attribute value inherits the confidentiality class of the referenced repo/tracker — any sink or artifact carrying it must enforce access controls at least as strict as the item's home. No other custom namespace is defined by THIS contract; a sibling capability contract in the same home MAY define its own additions under the sameautonomy.*prefix and governance (reviewed contract changes, no-parallel-schema rule) — minimality binds this contract, it does not forbid governed extension. No-parallel-schema migration rule if upstream ships a tracker namespace; contract migrations (including the semconv-Stable rename wave when cicd/vcs graduate) are reviewed changes owned by the contract home, trigger recorded in the plugin's trigger register. (3) W3C TRACEPARENT propagation trigger → CI → agent session forms one causal tree; headless/CI-only property — interactive contexts explicitly excluded from inbound trace joining. Sink binding out of contract: three sink classes only (existing observability stack / OTLP-file-artifact free default with query-on-read / opt-in network backend), adapter obligations per class stated as shape, no vendor picks. Native-surface principle (D7): prefer each tool's native telemetry export; reimplementations of native surfaces are non-conforming. Telemetry-is-not-return boundary note (T5 seam). Zero vendor/fleet names — tool-specific mechanics live in SKILL.md.Sanity Check:
grep -c 'opentelemetry.io/schemas/1.43.0' plugins/autonomy/reference/telemetry.md≥ 1grep -c 'autonomy.work_item.url' plugins/autonomy/reference/telemetry.md≥ 1reference/telemetry.md(same token list as WP1 Phase 3)grep -ci 'traceparent' plugins/autonomy/reference/telemetry.md≥ 1grep -o 'cicd\.[a-z.]*' plugins/autonomy/reference/telemetry.md | wc -l≤ 4 (attributes named only as illustrative citations, never enumerated)grep -c 'no trailing slash' plugins/autonomy/reference/telemetry.md≥ 1 (URL normalization rule present)node scripts/validate-plugin-contracts.mjsexit 0; lychee lane passesPhase 2: Guided-setup telemetry slice [DONE]
Extends the WP1
setupskill (discovery-first per D6/WP1 D7); all wiring lands as reviewablechanges; paid sinks advisory + explicit opt-in with cost surfaced. First work item — fresh-docs
mandate (repo CLAUDE.md): re-fetch the official skills + settings + monitoring docs before
editing SKILL.md or citing vendor env vars.
plugins/autonomy/skills/setup/SKILL.mdOTEL_RESOURCE_ATTRIBUTEScarryingautonomy.work_item.urlfor work-item-dispatched sessions — verify the vendor honors it at implementation) alongside TRACEPARENT behavior and beta-trace flags; all vendor env vars documented here, not inreference/. Records the telemetry binding (sink class, endpoint/artifact path, semconv pin) as an ADDITIVE section of the WP1 schema-versioned binding — absent-section tolerance stated; no schema major bump for additive sections.plugins/autonomy/skills/setup/scripts/check-emission-conformance.*(name/extension per repo script conventions)schemaUrlkey, since the OTLP JSON encoding uses lowerCamelCase field names (resourceSpans,schemaUrl), never the proto snake_case — join attribute present where required, value normalization holds. This is the contract's only enforcement surface — without it adopter drift is undetectable.plugins/autonomy/skills/setup/evals/evals.jsonplugins/autonomy/skills/setup/templates/(or the skill's existing snippet home from WP1 impl)plugins/autonomy/README.mdplugins/autonomy/.claude-plugin/plugin.jsonSanity Check:
/skill-quality:check+validate-evalspass (skills_root=plugins/autonomy/skills)claude plugin validate --strictexit 0grep -ci 'file-artifact' plugins/autonomy/skills/setup/SKILL.md≥ 1 andgrep -ci 'opt-in' plugins/autonomy/skills/setup/SKILL.md≥ 1grep -c 'OTEL_RESOURCE_ATTRIBUTES' plugins/autonomy/skills/setup/SKILL.md≥ 1validate-plugin-contracts.mjs) exit 0Phase 3: Conforming-path demonstration [DONE]
Acceptance-criterion probe: one path demonstrates a single trace tree trigger → CI → agent
session on the free default, zero paid dependencies. Scratch consumer repo (NOT this repo),
mirroring WP1's migration-gate step 9: a stub trigger step GENERATES its own root span and
propagates TRACEPARENT (the trigger hop is observed, not seeded); the Phase 2 writer snippet
emits the pipeline span into a single artifact directory; a headless agent session inherits
TRACEPARENT with
OTEL_RESOURCE_ATTRIBUTESset, its OTLP captured by the Phase 2 ephemeralcollector into the SAME artifact directory; the Phase 2 conformance check runs against the
combined output; query via the DuckDB
otlpextension (live-verified 2026-07-18). A smallsanitized OTLP fixture from this run is COMMITTED (plugin test-fixture home) as the durable
artifact the WP3 demo consumes — the PR-body transcript is evidence, not the interface.
Sink-class note: this demo exercises the FILE-ARTIFACT FREE DEFAULT class end-to-end. The
fleet's claude-ops collector/DuckDB pipeline is the existing-stack class's conforming
instance and is deliberately NOT used here — the scratch repo must not depend on fleet
plugins; no standing capability is created (near-duplicate audit rationale).
Sanity Check:
trace_idcovering ≥ 3 spans (trigger root + pipeline + agent session)autonomy.work_item.urlequal (string-identical) to the pipeline span's attribute valuecheck-emission-conformanceexit 0 over the demo outputPhase 4: Gates [DONE]
Full in-repo gate run (same roster as WP1 Phase 5):
scripts/validate-plugins.sh,scripts/run-plugin-tests.sh,node scripts/validate-plugin-contracts.mjs,markdown/typos/lychee lanes,
claude plugin validate --strict, catalog regen check.Near-duplicate audit statement: the telemetry slice composes claude-ops observability
(fleet's conforming sink instance) rather than duplicating it.
Sanity Check:
node scripts/generate-catalog.mjsreports in-syncBlast radius
MEDIUM — ~8 files in one plugin, but the contract doc constrains WP3–WP6, every future
adopter, and the join key WP3 depends on; new convention + cross-cutting observability
triggers match. Fully git-revertible; automated gates cover the shared surfaces.
Stress-test summary
Step 3 fresh-context plan review: 11 findings (1 CRITICAL — the cost-join was undemonstrable:
attribute OTel scope unpinned,
OTEL_RESOURCE_ATTRIBUTESunwired, demos joined against azero-cost pipeline span; all folded). Step 4
/devils-advocate: 3 CRITICAL / 4 HIGH /4 MEDIUM / 1 LOW — folded: process-granularity guarantee + batch limitation; ephemeral
per-job collector closes the session-capture gap; join-key selection rule +
rename/transfer/tracker-move limitations + deferred immutable-ID trigger (user-locked: keep
URL); emission-conformance check + migration ownership; confidentiality clause;
observed-not-seeded trigger hop; committed durable fixture; consistent private-repo cost
caveats. Residual accepted:
OTEL_RESOURCE_ATTRIBUTESreaching the vendor's metric exporteris verify-at-implementation (divergence escalates); RC semconv pin accepted per D4.
Execution shape
Fully sequential 1 → 2 → 3 → 4 — Phase 2 cites Phase 1's contract; Phase 3 exercises Phase 2's
snippets and conformance check; Phase 4 gates the authored tree. Parallel saving immaterial
(small volume, tightly coupled). Cross-package: this PR merges before the WP3 PR (WP3's doc
cites this contract; WP3's demo consumes this package's committed fixture); both wait on the
WP1 implementation PR.
Open questions
work-items flow after this package lands (out of package per D1; trigger recorded).
Decisions made (gate-passed)
reference/telemetry.mdreference/) + WP1's three shipped namesv1.43.0now, re-verified at build[FALLBACK — confirm or override] Emission-conformance check script as a NEW deliverable
(Phase 2) — invented beyond the Brief to close the stress-test governance gap (HIGH #6:
pinned contract with no enforcement surface). Flag if unwanted.
Handoff to implementation
User-approval gates
contested.
OTEL_RESOURCE_ATTRIBUTESnot honored by the vendor's metric exporter at implementation →STOP and re-surface (the cost-join design depends on it; do not improvise an alternative).
/architect review.Execution shape ([EXEC-SHAPE] tagged)
Sequential 1→4, all main-session (table above). PLAN.md phase tags advance in the same commit
as each phase; scratch-repo demo per Phase 3; divergence escalation applies to every phase.
Mechanical work
Commit per phase on the implementation branch (suggest
feat/autonomy-telemetry); gates re-runin full at Phase 4; commits via the repo's commit conventions; PR body carries the demo
transcript + near-duplicate audit statement + this PLAN in a
<details>block at close-out.