Skip to content

Audit the bundled configuration skill against the code - #70

Merged
khalilgharbaoui merged 1 commit into
masterfrom
skill-audit
Oct 1, 2026
Merged

khalilgharbaoui merged 1 commit into
masterfrom
skill-audit

Conversation

@khalilgharbaoui

Copy link
Copy Markdown
Owner

Audit of skills/claude-code-plugin/SKILL.md against the source, as asked: read every
factual statement, find the code that decides it, fix the statement where it disagreed.
Identifier coverage was already complete; this is about what the skill says about those
identifiers. No runtime code changed.

What was wrong

Section What it said What the code does Source
Recipe "Change when idle workers are freed" "The default is thirty minutes: that long after a turn ends ... the conversation's claude process exits" Idle eviction is off unless idleProcessTimeoutMs is set. DEFAULT_IDLE_PROCESS_TIMEOUT_MS = 0, and resolveIdleProcessTimeoutMs maps an unset option onto it src/session-manager.ts:195
Same recipe "0 keeps workers until the 8-process LRU cap" The cap is 16 src/session-manager.ts:208
Troubleshooting, truncated answer with many chats "skips the round when all 8 are busy; the 30-minute idle timer spares a busy worker too" 16, and there is no default idle timer to spare anything src/session-manager.ts:208, :195
Options table, idleProcessTimeoutMs "(16 processes...)" Correct, and it contradicted the two rows above. The file stated both numbers src/session-manager.ts:208
Recipe "opencode 2" "provider.claude-code.settings is the native spelling" opencode 2's own key is providers.claude-code.settings, plural. The plugin reads four places, lowest first: provider.<id>.options, provider.<id>.settings, providers.<id>.settings, then the plugin entry's own options, which wins src/v2.ts:265-277
Recipe "opencode 2" "accounts may also sit in the plugin entry's own options" Any option may, and the plugin entry wins over all three config spellings src/v2.ts:271-276
Ground rule 6 "do not use the known-broken question form to configure itself" The question round-trip is verified, and the same file says so two sections later. The reason to leave Question alone is that it disables Claude's own AskUserQuestion AGENTS.md questions section, src/types.ts:579-591
Options table, accountFailover "Triggered only by a rejected rate_limit_event or the two known account-limit error texts" Also opens on five account-level failure kinds (authentication_failed, oauth_org_not_allowed, account_on_hold, verification_required, billing_error). The recipe lower down already had this; the option row did not src/account-failover.ts:106-112
Options table, permissionPreset "hand-combining the five options around it" vs the recipe's "the preset replaces all four" Three different relationships: four options replaced, proxyTools filtered, extraDisallowedTools unioned src/permission-presets.ts:148-204
Options table, turnStats listed four token figures and omitted denials The line also carries a permission-denial count src/turn-stats.ts:104-107
Proxy tool table, task_batch "Separate task calls were measured serial on CLI 2.1.258" The recorded measurement is 2026-09-06, two 8-second calls, second request 7 ms after the first resolved; no CLI version was part of it src/proxy-mcp.ts:439-453
"Verify and diagnose", /claude-code-doctor usage "usage adds a Plan usage section" The section is always printed; without the argument it prints one line saying how to fill it src/doctor.ts:318-337
"Verify and diagnose", doctor field list omitted turnStats and the last-stderr block, and did not say a deadline-free call shows none All three are in formatDoctorReport src/doctor.ts:219, 263, 339-350
"Version requirements" four floors, presented as one kind of thing Four are flag gates (2.1.142, 2.1.220, 2.1.258, 2.1.263), and 2.1.258 also gates --restricted, which the read-only preset depends on and which the list never mentioned. 2.1.284 (Sonnet 5.5) was missing entirely src/cli-version.ts:117-168, src/cli-events.ts:807-810
Skill bridge recipe "The package also registers its skill directory with opencode's skills.paths" True on opencode 1.x only; opencode 2 has no config hook, so the bundled skill is registered through the skill domain instead src/skill-bridge.ts:583-600, src/v2.ts:505-521
Skill bridge recipe "~/.claude/skills and ~/.agents/skills ... obey OPENCODE_DISABLE_EXTERNAL_SKILLS and OPENCODE_DISABLE_CLAUDE_CODE_SKILLS" OPENCODE_DISABLE_EXTERNAL_SKILLS drops both; OPENCODE_DISABLE_CLAUDE_CODE_SKILLS drops ~/.claude/skills alone src/skill-bridge.ts:238-243
Env table, OPENCODE_WORKTREE unqualified opencode 1.x layering only; the V2 layering walks to the filesystem root with no worktree boundary src/mcp-bridge.ts:523
Background subagents implied the plugin reads OPENCODE_EXPERIMENTAL_BACKGROUND_SUBAGENTS The plugin never reads it. It reads whether opencode's advertised task schema carries a background property, and on opencode 2 it does not ask at all src/proxy-mcp.ts:668-676
"How a proxied call ends" "Only deadline-free calls (task, task_batch) can reach it" Any tool the operator sets to 0 in proxyToolTimeoutMs is deadline-free too src/proxy-mcp.ts:182-203
Six troubleshooting rows "Plugin older than this fix / this release" Each is a shipped version. Pinned to 0.19.0 (is_error forwarding), 0.20.0 (deleted-session release, mid-turn eviction), 0.31.0 (last-call context usage), 0.32.0 (interactive per-call usage), 0.33.0 (interactive result shape, transcript realpath) git describe --contains on each change
"Upgrade the plugin" "Some opencode versions freeze latest in the package cache" The recorded measurement is unconditional: a plain restart never re-resolves @latest; removing that directory and fully relaunching is what picks a version up AGENTS.md (h #g23)

AGENTS.md was wrong too

AGENTS.md (h #g62) said "Cap is 8". The code says 16. PR #36 proposed 8 alongside a
30-minute default idle timer; dfb82d5 reverted both at merge on 2026-09-19, one day
after the history entry was written, and the condense in #47 carried the old wording
across. Fixed in AGENTS.md, with the evidence in a new docs/agents-history.md #g179
in the format of #g178. README.md was right about this throughout.

What I changed beyond the corrections

  • New ### How a proxied call ends section. The whole proxy-deadline lifecycle (what
    ends a call, the three timers, the busy-recheck, the two "still waiting" log lines) was
    buried inside ### Background subagents, where an agent looking up a timeout will not
    find it. Moved verbatim in content, split into paragraphs and a list, and placed
    directly after the proxy tool table. No fact dropped.
  • "Which major each recipe is for" note under ## Recipes, since every fragment uses
    the 1.x spelling that 2.x also reads. Absorbed the duplicated "fragments belong inside
    the options object" sentence that sat in one recipe only.
  • Version requirements became a table with the two floors that were missing and a line
    saying which entries are flag gates and which are model floors.
  • Env table gained CLAUDE_CODE_PROMPT_CACHE_TTL, OPENCODE_EXPERIMENTAL_BACKGROUND_SUBAGENTS,
    OPENCODE_DISABLE_EXTERNAL_SKILLS, OPENCODE_DISABLE_CLAUDE_CODE_SKILLS.
  • New troubleshooting row for the first-turn MCP race recorded in #g178 (a server still
    connecting when the turn plans its spawn is correctly read as not enabled, and a reused
    process keeps its first MCP config). Marked as not a config fault and not fixable by an
    option, which is what a diagnosing agent needs to be told.
  • task_batch's input contract (tasks array, at least two items, three required string
    fields, validated before queueing) added, because the model-facing failure is silent
    otherwise.
  • Two more log lines added to the grep list (evicting LRU claude process,
    background subagent gate); every line named in that list was checked to exist in src/.

What I removed

Only the claims above, each replaced by what the code does. One claim was removed without
replacement: "(measured: 2.0.11 loaded a plugin listed under plugin)". The fact that
opencode 2 reads the plugin key is recorded (docs/agents-history.md #g39), but that
specific per-version measurement is not, so the citation now points at the recorded fact.

README fixes

Line Was Now
Comparison table "17 ids auto-registered" 18. defaultModels has 18 entries
opencode 2 section "Its native spelling is provider.claude-code.settings" providers.claude-code.settings, plus the full four-layer precedence

Everything else in the README that overlaps this audit was checked and is correct,
including the 16-process cap, the idle default, Sonnet 5/5.5 at $2/$10, the 2.1.284
floor, the paused Agent SDK credit, and the Plan usage section being always present.

Test changes

test-configure-skill.ts gained six mechanical guards for classes of drift this audit
found, all derived from imported constants or parsed out of the source:

  • stated defaults in the options table against the code (proxyTools against
    DEFAULT_PROXY_TOOL_NAMES, compactionModel against DEFAULT_COMPACTION_MODEL,
    idleProcessTimeoutMs against DEFAULT_IDLE_PROCESS_TIMEOUT_MS, skipPermissions
    against the ?? true in src/index.ts, cliPath), plus a negative assertion that the
    skill never claims a default idle timer again;
  • every process-count figure against MAX_ACTIVE_PROCESSES;
  • the proxy deadline defaults against PROXY_DEFAULT_TIMEOUT_MS,
    PROXY_PER_TOOL_DEFAULT_TIMEOUT_MS and MAX_PROXY_TIMEOUT_MS;
  • the two watchdog defaults against the literals in src/turn-state.ts;
  • the set of N× price multipliers against the registry's, both directions;
  • every MODEL_CLI_FLOORS model and version.

Mutation-checked rather than assumed: changing 16 to 8, the question deadline to 45 min,
the compaction model, the Opus 5.5 multiplier and the start-watchdog default each failed
the suite, and the file was restored byte-identical afterwards.

No new test file, so package.json is unchanged.

Checks

npm run typecheck                  TYPECHECK_EXIT=0
npm test > /tmp/lane-skill.log     EXIT=0
  tests 986 / pass 986 / fail 0 / cancelled 0 / skipped 0 / todo 0
npm run build                      BUILD_EXIT=0  (ESM + d.ts, no errors)
npm pack --dry-run                 91.9kB skills/claude-code-plugin/SKILL.md, 6 files

The skill still ships and is discoverable in the built package.

Not verified

  • No live opencode or claude process was started, so nothing here is a live probe. Every
    statement is tied to source, to a recorded measurement in AGENTS.md /
    docs/agents-history.md, or removed.
  • The opencode 2 settings precedence is read off configuredSeedSettings; it was not
    re-run against a 2.x sandbox.
  • I did not re-verify the dated external claims the skill inherits (Anthropic's pricing
    page, the paused Agent SDK credit, the February 2026 third-party auth ban). They match
    the code comments and README.md and were left as they stand.
  • The README's own phrasing of the two skill-disable env vars is vague in the same way the
    skill's was, but it is not a contradiction, so I left it rather than widening the diff.

@khalilgharbaoui
khalilgharbaoui merged commit f2a760e into master Oct 1, 2026
@khalilgharbaoui
khalilgharbaoui deleted the skill-audit branch October 1, 2026 08:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant