feat: absorb boris-video digest — run-e2e config surface, gap report, recording tier, routines provenance#845
Conversation
…tion Approved plan for absorbing vetted ideas from the "4 Levels of Agentic Coding" video digest: run-e2e enrichment (config surface, gap report, recording tier), confirm delegation touch-up, routines.md thin-pointer clause. Coordination comment posted on #778. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QdCmaHa15bwUTh9sy7Rfdz
The live-app delegation section now names run-e2e's subagent-isolated drive loop, optional recording/session-artifact evidence tier, and structured gap report on prerequisite failure. /verify supplementary path and stage/verdict machinery unchanged. Patch entry [0.3.2]. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QdCmaHa15bwUTh9sy7Rfdz
Routine instruction content lives in a version-controlled, reviewable artifact; the stored prompt is a thin pointer. Pasted-prose prompts are non-compliant (no history, invisible drift). Cloud-routine and Desktop scheduled-task mappings appear as illustrative deployment-owned bindings, consistent with the Hosting stance. Minor entry [0.8.0]. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QdCmaHa15bwUTh9sy7Rfdz
run-e2e gains the consumer config surface .claude/testing/e2e.md (recording: video|gif|off default off; browser_mode: headed|headless default headless) with an explicit read path, per-key merge, provenance reporting, and plumbing of resolved values to the playwright executor. Keys are owned by the new context/e2e-config.md (policy-vs-mechanics boundary, prompt-over-config precedence ladder). The prerequisite hard-fail keeps its STOP and now also writes a structured verification-environment gap report. The drive loop is subagent-isolated; verification delegates to bundled /verify when present (CC >=2.1.145). Evidence contract adds an optional recording tier and session-artifacts rows. Eval #1 asserts the new prerequisite-failure shape. Implementers table gains the surface row. Minor entry [0.3.0]. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QdCmaHa15bwUTh9sy7Rfdz
Aligns manifests with the CHANGELOG entries from the boris-video-absorption phases; PLAN phase tags updated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QdCmaHa15bwUTh9sy7Rfdz
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
Claude finished @kyle-sexton's task in 2m 54s —— View job PR Review — feat/boris-video-absorptionTasks
OverviewSolid, well-structured absorption. Defaults are non-breaking, semver magnitudes are correct, and the three CHANGELOG entries accurately describe the changes. The PLAN.md's stated sanity checks all pass against the actual files. Three findings worth addressing before merge: Findings1 — Raw
|
|
Claude finished @kyle-sexton's task in 2m 57s —— View job Security Review — PR #845
ScopeThis PR is 491 additions / 5 deletions across 16 files — all markdown prose and JSON manifests. No GitHub Actions workflows, no executable code, no secret handling, no dependency changes. Finding 1 — Unpinned live-branch URL references inside AI skill instructionsSeverity: SUGGESTION Two newly-added files instruct the AI to consult an external URL for layering-contract mechanics:
Both reference: The URL resolves to the live This is qualitatively different from a markdown hyperlink for human readers: inside an AI skill document, a "see [URL]" with a full Suggested fix: replace Alternatively, reference the file via Finding 2 — Consumer config surface as AI-readable prompt-injection vectorSeverity: SUGGESTION The new surface This risk is present for every consumer-config surface in this marketplace (it is inherent to AI-readable config), so it is not newly introduced by this PR. The finding is noted here because this PR adds a new surface, expanding the attack area. The closed scalar values and short key-set limit the viable injection window somewhat, but the merge step reads the full file text rather than parsing a strict schema. No code change is required to "fix" this — the architecture is AI-skill design; the mitigations are operational (validate consumer config files in CI, keep surfaces narrow and schema-validated). Noting it so the team is aware the new surface inherits this property. No other security issues found
|
The plugin contract gate bans vendor names in reference/ contracts; the illustrative bindings now name surface classes (cloud scheduling surface, desktop scheduling surface) instead of products and vendor-coupled paths. CHANGELOG wording aligned. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QdCmaHa15bwUTh9sy7Rfdz
|
Claude finished @kyle-sexton's task in 2m 48s —— View job Security Review — PR #845Tasks
Scope16 files — all markdown skill documents and This synchronize trigger adds one commit beyond the prior security review ( Finding 1 — Unpinned live-branch URL inside AI-executable skill instructionsSeverity: SUGGESTION Two files added by this PR embed a
The Risk is moderated because the URL resolves within the same repository (no third-party trust involved), but the semver isolation the plugin's version number implies is violated. Suggested fix: replace Or drop the hyperlink entirely and reference the path as prose ( Finding 2 — New consumer config surface is an AI-readable prompt-injection vectorSeverity: SUGGESTION The new surface This risk is inherent to any AI-readable config surface and is not unique to this PR; the closed scalar values ( No code change is required. Existing mitigations (narrow key-set, defaults off) are correct. Teams operating in high-trust-boundary environments may wish to validate consumer config files in CI before use. Delta commit —
|
|
Claude finished @kyle-sexton's task in 1m 41s —— View job PR Review — feat/boris-video-absorptionTasks
Latest commit assessmentThe most recent commit ( Findings — three prior items remain unaddressedThe three findings from the earlier review are all still present in the current HEAD. Noting them here for the record since they were not addressed in the intervening push. 1 — Raw live-branch URL couples installed plugin to a specific GitHub hostname (Moderate)Both files still reference: This is a live 2 — Old GIF instruction in
|
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QdCmaHa15bwUTh9sy7Rfdz
|
Claude finished @kyle-sexton's task in 1m 35s —— View job PR Review — feat/boris-video-absorptionTasks
Latest commit assessment
Open findings — still unaddressed at HEADAll three findings raised in prior reviews remain open in the current HEAD. Summarized below with current file + line permalinks. 1 — Raw live-branch URL inside AI-executable skill instructions (Moderate)Two files still link directly to Inside an AI skill document a 2 — Pre-existing GIF instruction not reconciled with new Recording tier (Low)e2e.md lines 128–131 (pre-existing, untouched by this PR):
This PR added a Recording tier section (lines 52–60) that makes 3 — Handoff
|
|
Claude finished @kyle-sexton's task in 2m 4s —— View job Security Review — PR #845Tasks
Scope16 files changed — all markdown skill documents, The latest commit in this sync ( Finding 1 — Unpinned live-branch URL in AI-executable skill instructionsSeverity: IMPORTANT Two files added by this PR embed a This is not merely a human-readable link. In an AI skill document, a complete
The convention doc is not inside the plugin, so a relative path cannot reach it. Replacing the hyperlink with prose — naming the path Finding 2 — New consumer config surface is an AI-readable prompt-injection vectorSeverity: SUGGESTION
Exploitation requires write access to a consumer repo's This risk is inherent to the AI-readable config-surface architecture this marketplace uses, not unique to this PR. It is noted here because this PR adds a new surface, expanding the attack area. No code change is required for this finding; teams operating in high-trust-boundary environments may wish to validate consumer config files in CI before use. No other security issues found
Delta commit —
|
…y trial (#871) ## What Flips the last open phase tag in `docs/topics/boris-video-absorption/PLAN.md` — Phase 8 (post-merge trial) — to `[DONE]`, with a short trial record. ## Trial summary (medley, 2026-07-21, local-only) - Config surface resolved with per-key provenance (local overlay `.claude/testing/e2e.local.md`; team + user-global layers absent) - `recording: video` honored — WebM artifact on disk (gitignored in consumer repo) - Induced prereq failure (run before orchestrator start) → hard STOP + structured verification-environment gap report - Evidence table carried session artifacts (recording path, playwright session ID, transcript pointers) - Bonus: explicit "run this headed" prompt beat the overlay's `browser_mode: headless` - Operator reviewed the recording Docs-only; no plugin content changes. ## Related No linked issue. Related: #845 (phases 1–7 of the same PLAN, merged). Co-authored-by: Claude <noreply@anthropic.com>
Summary
Absorbs the vetted ideas from the "4 Levels of Agentic Coding" video digest (Boris Cherny "Steps of AI Adoption" walkthrough) into the plugin estate, per the approved plan at
docs/topics/boris-video-absorption/PLAN.md./testing:run-e2egains the consumer config surface.claude/testing/e2e.md(recording: video|gif|offdefaultoff;browser_mode: headed|headlessdefaultheadless) with explicit read path, per-key merge + provenance reporting, and plumbing to the playwright executor; keys owned by newcontext/e2e-config.md(policy-vs-mechanics boundary; prompt > config precedence). Prerequisite hard-fail keeps STOP and adds a structured verification-environment gap report. Drive loop subagent-isolated; presence-gated/verify-first delegation (CC >=2.1.145). Evidence contract adds optional recording tier + session artifacts. Eval chore: initialize marketplace scaffold #1 updated./verification:confirmdelegation wording acknowledges the enriched run-e2e; stages/verdict machinery untouched.reference/routines.mdgains the instruction-provenance clause — routine instruction content lives in a version-controlled artifact, stored prompt is a thin pointer; surface mappings are illustrative deployment-owned bindings.docs/conventions/consumer-config-layering/README.md.Coordination: zero file overlap with the ignition/enforcement lanes (#778 comment posted; ignition Phases 2–3 merged via #810). Defaults preserve current consumer behavior.
Related
No linked issue: this PR closes no GitHub issue. Related, not closed:
Verification
check-skill.sh: run-e2e PASS, confirm PASS (pre-existing advisory warnings only)evals/evals.json: valid against bundled schema (ajv, draft2020)PLAN.md
PLAN — boris-video-absorption
Brief
Absorb the vetted ideas from the Ray Amjad "4 Levels of Agentic Coding"
video digest (research base:
~/.work/youtube-watch/the-4-levels-of-agentic-coding-how-to-sh-XLA-sTSJ-Wc/)into the plugin estate. Interview-locked contracts (this session):
testing:run-e2e(driver) +verification:confirm(judge) keep their division of labor.delegation (presence-gated, CC ≥2.1.145); structured missing-environment
gap report on prerequisite failure (hard-fail stays, plus actionable
report); video/session-artifact evidence tier.
headless) are layered config knobs per
docs/conventions/consumer-config-layering/— NOT hardcodes.plugins/autonomy/reference/routines.mdgains a thin-pointer clause:routine/scheduled-task prompts on any surface point to
version-controlled files; pasted prose non-compliant.
~/.claude/scheduled-tasks/(exists, live task) proposed for dotfilestracking via the dotfiles repo's add-dotfile flow.
verification self-improvement loop (trigger: accumulated recorded
runs); bounded build→review→fix loop (trigger: post-Phase-II
enforcement rail); Claude Tag enablement (trigger: post-8/3 cutover +
Enterprise admin check).
Chronicle adoption, lifecycle reordering.
trace to the doc's step 1→2/2→3 recipes or native CC features.
Success: PRs merged with skill-quality + markdownlint green; every
adopted behavior traceable to research findings; zero collision with the
primary session's lanes (ignition Phases 2–3 merged via #810; no pending
branch touches target files — verified).
Standards grounding
docs/conventions/consumer-config-layering/README.md— three layers(user-global / team / local overlay), additive-preferred merge; new
config surface must declare keys in the owning plugin's docs and point
at the convention for layering.
docs/CATALOG-TAXONOMY.md+docs/PLUGIN-PHILOSOPHY.md— extend-over-add, seam cooperation presence-gated, fresh-eyes delegation.
resolution before merge,
--body-file -for PR bodies).Plan
Phase 1: Lane setup [DONE]
origin/main, branchfeat/boris-video-absorption(sibling-worktree convention).
docs/topics/boris-video-absorption/PLAN.md(this file) +design/design-resolution.md(Tier B early-exit: no new types; one newconfig surface whose shape is resolved here — see Decisions).
touched, no overlap with ignition/enforcement lanes.
git worktree listshows the new worktree on the newbranch;
gh issue view 778shows the comment; PLAN.md exists in theworktree at
docs/topics/boris-video-absorption/PLAN.md.Phase 2: run-e2e enrichment [DONE]
Files:
plugins/testing/skills/run-e2e/SKILL.md,plugins/testing/skills/run-e2e/context/e2e.md,plugins/testing/README.md, newplugins/testing/skills/run-e2e/context/e2e-config.md(key owner doc),plugins/testing/skills/run-e2e/evals/evals.json(eval #1 asserts theprerequisite-failure behavior this phase changes — update
unconditionally),
plugins/testing/CHANGELOG.md(Keep a Changelog entry;minor bump — new behaviors),
docs/conventions/consumer-config-layering/README.md(Implementers-tablerow for the new surface — the convention tracks conformance, not
assumes it).
.claude/testing/e2e.md(folder-form name perconvention: surface identity = whole path relative to
.claude/).Keys:
recording: video | gif | off(defaultoff— evidence contractscreenshots stay the floor),
browser_mode: headed | headless(defaultheadless, operator overrides for visibility). Per-key override
semantics. Layering resolution cites the convention doc — no
restatement.
resolution instructions — anchor at repo root, read all three layers
(user-global → team → local overlay), merge per-key, report which layer
supplied each effective value. Plumbing: resolved
browser_modeand
recordingare passed to the executor (/playwright:playwrightinvocation args / gif_creator choice) — keys without plumbing are the
documented ai-briefing deviation; not repeating it.
boundary — run-e2e owns capture POLICY (what evidence, which format,
visibility preference); playwright owns MECHANICS (how the browser
runs). Keys are policy inputs run-e2e resolves and passes through.
explicit session prompt always wins ("run this headed" overrides
browser_mode: headless). e2e-config.md states this: prompt >local overlay > team > user-global > bundled default.
subagent; orchestrator consumes evidence paths only); /verify-first
delegation note (presence-gated on bundled /verify, CC ≥2.1.145, with
existing orchestrator fallback unchanged).
verification-environment gap report (what's missing, what the operator
must provide — keys/CLIs/environments; the "what do you need to verify
this e2e" elicitation) written to the run's evidence output.
via playwright CLI preferred for long flows; GIF via gif_creator for
short demos — driven by the
recordingkey) + session artifacts row(recording path, session ID, transcript pointer) in the evidence table.
rg -c "e2e-config|\.claude/testing/e2e\.md" plugins/testing/skills/run-e2e/SKILL.md≥ 1;
rg -c "gap report" plugins/testing/skills/run-e2e/SKILL.md≥ 1;rg -c "recording" plugins/testing/skills/run-e2e/context/e2e.md≥ 2;read-path present:
rg -c "layer" plugins/testing/skills/run-e2e/SKILL.md≥ 2 ANDrg -c "browser_mode" plugins/testing/skills/run-e2e/SKILL.md≥ 2(resolution + plumbing sites); Implementers-table row:
rg -c "testing/e2e" docs/conventions/consumer-config-layering/README.md≥ 1; CHANGELOG entry present; eval chore: initialize marketplace scaffold #1 updated
(
rg -c "gap" plugins/testing/skills/run-e2e/evals/evals.json≥ 1);markdownlint exit 0 on changed files;
/skill-quality:check run-e2ePASS.
Phase 3: confirm delegation touch-up [DONE]
Files:
plugins/verification/skills/confirm/SKILL.md,plugins/verification/README.md(only if wording drifts),plugins/verification/CHANGELOG.md(patch bump — wording touch-up).recording/session evidence tier + gap report); /verify stays named as
the supplementary live-run path with unchanged presence gate. No
structural change to stages or verdict machinery.
rg -c "gap report|recording" plugins/verification/skills/confirm/SKILL.md≥ 1; stage structure unchanged
(
rg -c "Stage" plugins/verification/skills/confirm/SKILL.mdequalbefore/after); markdownlint exit 0.
Phase 4: routines.md thin-pointer clause [DONE]
Files:
plugins/autonomy/reference/routines.md,plugins/autonomy/CHANGELOG.md(minor bump — new normative clause).(reviewer risk docs(playbook): document the local plugin development loop #7): the NORMATIVE clause is surface-agnostic — "a
routine's instruction content lives in a version-controlled, reviewable
artifact; the stored prompt is a thin pointer to it; pasted-prose
prompts are non-compliant (no history, invisible drift)". The
surface-specific mappings (cloud routines → committed
.claude/skills/, documented transfer path; Desktop scheduled tasks →~/.claude/scheduled-tasks/<task>/SKILL.mdunder dotfiles) appear onlyas ILLUSTRATIVE binding examples, marked as deployment-owned bindings
per the doc's own Hosting stance. Cite research
(
routines-versioning-deep-dive.md: native prompt versioningdocumented-absent). If the clause reads better in
wiring-vs-advisor.mdat build time, the builder flags it back to themain thread rather than deciding — placement objection is the known
review risk.
rg -c "thin pointer|version-controlled" plugins/autonomy/reference/routines.md≥ 2; agnosticism preserved:
rg -c "deployment-owned" plugins/autonomy/reference/routines.mdcount increases by ≥1; markdownlint exit 0; CHANGELOG entry present.
Phase 5: digest-slice deferred items + through-line [DONE]
Files (home slice, not repo):
~/.work/youtube-watch/the-4-levels-of-agentic-coding-how-to-sh-XLA-sTSJ-Wc/recommendations/menu.md,README.md(same slice).named triggers; no-go unchanged). Add through-line note: Boris doc →
interview locks → PLAN → PRs.
rg -c "deferred" menu.md≥ 3; each adopted itemcarries a PR/branch reference.
Phase 6: dotfiles tracking proposal [TODO — user gate]
add-dotfileflow for~/.claude/scheduled-tasks/(dir exists with liveautonomy-demo-hourly-drain/). Flow owns chezmoi mechanics; neverchezmoi applyfrom agent context.scheduled-tasks source state, or an explicit operator decline recorded
in PLAN.
Phase 7: verify + ship [TODO]
/skill-quality:checkper touched skill, eval validation.<details>; codex threadsresolved before merge; close-out per
/planning:plan close-out(slice prune + pointer).
gh pr viewshows mergeable; allphase tags [DONE].
Phase 8: trial run (post-merge validation) [TODO]
change in a consumer repo: config surface resolved + reported,
recording produced per key, gap report exercised (remove a prereq
deliberately), evidence table carries session artifacts.
on induced prereq failure; operator reviews the recording.
Blast radius
LOW-MEDIUM. Markdown skill/contract text + one new additive config
surface; zero runtime code; three plugins touched; consumers unaffected
by default (recording defaults off, browser_mode default preserves
current behavior). Cross-repo edge: dotfiles proposal (user-gated).
Stress-test summary
Fresh-context plan-reviewer ran against live repo. 3 CONFIRMED-GAPs
folded: (1) phantom config surface — read-path resolution + executor
plumbing now mandated in P2 with sanity greps; (2) Implementers-table
row added to P2 files; (3) CHANGELOG entries added to P2/P3/P4 with
SemVer magnitudes (testing minor, verification patch, autonomy minor).
Risks addressed: key-ownership boundary framed policy-vs-mechanics in
e2e-config.md; routines.md clause re-phrased surface-agnostic with
bindings as illustrative examples; eval #1 update made unconditional.
Collision re-verified clean (no open PR overlap). Formal
/devils-advocate skipped: blast radius LOW-MEDIUM, no
infra/CI/dependency triggers.
Execution shape
Wave A (parallel, file-disjoint, one worktree, agents never touch git):
Wave B (sequential, main thread): 5 (home slice), 6 (user-gated
cross-repo), 7 (verify + ship — main thread only, empirical checks +
git).
Phase 1 precedes all. Scope fences: each builder gets an ALLOWED list =
its phase's files; FORBIDDEN = PLAN.md, other phases' files, any git
command. Cost: 3 builders vs sequential saves modest wall-clock; fallback
= run 2→3→4 sequentially in main thread on any fence violation.
Open questions
None blocking — all interview-locked.
Handoff to implementation
User-approval gates
Execution shape ([EXEC-SHAPE] tagged)
.work/first; materialized todocs/topics/inside the worktree at Phase 1 (avoids writing to thecanonical checkout the primary session owns).
above; sequential fallback documented.
rationale) unless review load argues for a split at ship time.
Mechanical work
phase 2–4, Phase 5 lives outside the repo (no commit), Phase 7 ship.
phase before its commit; empirical greps per Sanity Checks.
remaining phases in main thread.
🤖 Generated with Claude Code
https://claude.ai/code/session_01QdCmaHa15bwUTh9sy7Rfdz