Skip to content

fix(verification): neutralize .NET/PowerShell-flavored examples in agnostic skills#492

Merged
kyle-sexton merged 2 commits into
mainfrom
fix/424-verification-agnostic-examples
Jul 19, 2026
Merged

fix(verification): neutralize .NET/PowerShell-flavored examples in agnostic skills#492
kyle-sexton merged 2 commits into
mainfrom
fix/424-verification-agnostic-examples

Conversation

@kyle-sexton

Copy link
Copy Markdown
Contributor

Summary

The verification plugin's confirm and measure skills are sold as ecosystem-agnostic, but their illustrative examples leaned .NET/C#/PowerShell, and two "how to check" cells assumed a Unix shell (wc -l) against the cross-platform "never assume Bash" contract. A non-.NET consumer was handed a stack-specific illustration as the universal path. This patch neutralizes the examples while keeping the guidance concrete and useful.

Fix

  • skills/confirm/context/fix.md — reproduction-test example now uses a generic descriptive name (returns_null_when_user_is_deleted, "name it per your framework's convention") and a generic failure ("the reported failure (e.g. a null-dereference crash)") instead of UserService_ShouldReturnNull_WhenUserIsDeleted / NullReferenceException.
  • skills/measure/context/metrics.md — coupling row counts "import/dependency declarations (import/require/using, package or project references)" instead of using/ProjectReference; coverage row points to "your test runner's output" instead of dotnet test; both wc -l sites are now stated shell-neutrally (wc -l on POSIX/Git Bash, Measure-Object -Line in PowerShell), mirroring the both-forms pattern established in fix: state shell requirements with a Windows path across 8 plugins #331.
  • skills/measure/context/performance.md — metric "how to measure" cells point to "your test runner", "your build tool", "your platform's profiler", and "your benchmark harness", with the shell timer given for both shells (time on POSIX/Git Bash, Measure-Command in PowerShell) instead of dotnet test/dotnet-counters/BenchmarkDotNet as the only path.
  • The named marketplace-plugin evidence pointers (dotnet-*:*) at the bottom of both measure files are unchanged — those are real named plugins used as evidence sources, not universal-path illustrations, and verification: .NET/C#/PowerShell-flavored examples and Unix-shell assumptions in stack-agnostic skills (low severity) #424 deliberately did not flag them.
  • verification bumped 0.2.20.2.3 (docs = patch) with a CHANGELOG entry.

Verification

Gates run in this Windows environment against the changed markdown; all clean, nothing suppressed:

  • markdownlint-cli2 v0.23.0 (markdownlint v0.41.0) (repo-pinned) on the four changed .md files — Summary: 0 error(s).
  • typos-cli 1.44.0 on the four changed .md files — exit 0, no output.
  • editorconfig-checker v3.8.0 on the changed .md files plus plugin.json — exit 0, no findings.
  • plugin.json re-parsed as valid JSON; version reads 0.2.3.

Related

Closes #424. Sibling stack-bias issues in the same work-readiness sweep, same pattern: #430, #429, #428 (all testing plugin).

🤖 Generated with Claude Code

…nostic skills

The confirm/measure skills are sold as ecosystem-agnostic, but their
illustrative examples leaned .NET/C#/PowerShell and one assumed a Unix
shell (`wc -l`) against the cross-platform "never assume Bash" contract.
Neutralize them so a non-.NET consumer isn't handed a stack-specific
illustration as the universal path: generic reproduction-test name and
failure, import/dependency-declaration counting instead of
`using`/`ProjectReference`, "your test runner"/"your benchmark harness"
instead of `dotnet test`/BenchmarkDotNet, and shell-neutral line counting
(`wc -l` on POSIX/Git Bash, `Measure-Object -Line` in PowerShell). The
named marketplace-plugin evidence pointers (`dotnet-*`) are unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011V31qpAHP3jfs76B9d5Rfo
@claude

claude Bot commented Jul 19, 2026

Copy link
Copy Markdown

Claude finished @kyle-sexton's task in 1m 49s —— View job


PR Review — fix(verification): neutralize .NET/PowerShell-flavored examples in agnostic skills

  • Gather context
  • Read changed files
  • Review for correctness, alignment, and quality
  • Post findings

Verdict: Approve — clean, focused, well-reasoned patch. The changes are correct and do exactly what the PR description says they do. No blocking issues.


Summary

This is a documentation-only patch (5 files, 26 additions / 11 deletions) that neutralizes .NET/C#/PowerShell-flavored illustrative examples in the verification plugin's confirm and measure skills. I checked each changed file against the REVIEW.md blocking criteria — none of the security/authorization/tenancy/atomicity gates apply here. Below are observations from the content review.


✅ What's done well

fix.md — test name neutralization (line 52)
Replacing UserService_ShouldReturnNull_WhenUserIsDeleted (xUnit/MSTest PascalCase convention) with returns_null_when_user_is_deleted (name it per your framework's convention) is the right move. The parenthetical is a good way to acknowledge that snake_case itself is just one convention (Python/Ruby/Rust), without leaving the example unlabeled.

Dual-shell pattern is applied consistently (metrics.md L14, performance.md L14)
Both wc -l / Measure-Object -Line and time / Measure-Command pairs follow the pattern established in #331. They appear in every location where a shell command was previously implied as Unix-only.

Marketplace plugin evidence pointers left intact (metrics.md L69–73, performance.md L61–66)
The dotnet-*:* plugin references at the end of both files are real named marketplace plugin pointers, not illustrative examples. Leaving them untouched is the correct decision.

Version and CHANGELOG are consistent
Patch bump (0.2.20.2.3) is correct for a docs-only change. The CHANGELOG entry accurately describes what changed and explicitly calls out what did not change (the dotnet-* plugin pointers).


🟡 Nit — parenthetical inside code fence

fix.md line 52:

Reproduction test: returns_null_when_user_is_deleted (name it per your framework's convention)

Embedding (name it per your framework's convention) inside the code fence is slightly unusual — code fences typically show actual output or commands, not inline meta-commentary. A first-time reader might initially read the parenthetical as part of the test name. The surrounding prose context makes the intent clear enough, but a comment-style delimiter (e.g. # ... or a prose note before the fence) would be unambiguous. Not worth a change on its own, but worth noting.


🟡 Nit — table cell verbosity in metrics.md

metrics.md line 14:

Line counter (wc -l on POSIX/Git Bash, Measure-Object -Line in PowerShell), git diff --stat

The "How to check" cell is now quite long. These files are AI-readable instruction documents rather than rendered markdown for end users, so this is low impact. Still, if a future edit passes through, splitting the dual-shell note to a separate prose sentence below the table would improve scannability.


No concerns on REVIEW.md blocking criteria

All six blocking checks (object-level auth, tenant-scoped data access, secrets, injection, audit logging, atomicity) are inapplicable — this is a documentation-only change with no code paths, data access, or external input handling.

…t name

Address claude[bot] review nit: the "(name it per your framework's
convention)" parenthetical inside the code fence could be misread as part
of the test name. Move it to a trailing `#` comment delimiter so the
meta-note is unambiguously commentary, not the identifier.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@claude

claude Bot commented Jul 19, 2026

Copy link
Copy Markdown

Claude finished @kyle-sexton's task in 1m 26s —— View job


PR Review — fix(verification): neutralize .NET/PowerShell-flavored examples in agnostic skills

  • Gather context
  • Read changed files
  • Review for correctness, alignment, and quality
  • Post findings

Verdict: Approve — the follow-up commit cleanly resolves the one nit from the prior review, and the full patch is well-scoped and correct.


What changed since the last review

Commit 8c06888 addresses the nit about the parenthetical meta-note inside the code fence:

-Reproduction test: returns_null_when_user_is_deleted (name it per your framework's convention)
+Reproduction test: returns_null_when_user_is_deleted  # name it per your framework's convention

The # delimiter is the right call. A bare parenthetical inside an unlanguaged code fence could be read as part of the identifier; a # reads unambiguously as a comment in the majority of languages that use snake_case test names (Python, Ruby, Rust). The fix is minimal and targeted.


Full patch — residual observations

CHANGELOG date field (CHANGELOG.md L6)
The Keep a Changelog spec encourages ## [0.2.3] - YYYY-MM-DD, but all prior entries in this file omit the date too (## [0.2.2], ## [0.2.1], etc.). The new entry is consistent with the established pattern — no action needed.

git show <base>:<file> pipe in metrics.md (L24)
The prose now reads: "line count from git show <base>:<file> piped to a line counter — wc -l on POSIX/Git Bash, Measure-Object -Line in PowerShell". This is accurate — git show is cross-platform but the pipe operator has a different form in PowerShell (| vs. |... actually the same). The wording is correct; the split is only in the downstream command. Minor but fine.

REVIEW.md blocking criteria — all six inapplicable to a docs-only patch: no object-level authorization, no tenant-scoped data, no secrets, no injection surface, no audit-log requirements, no multi-file atomic write.


Summary

Both commits are clean. The original neutralization was well-reasoned; the follow-up precisely resolves the single nit. Ready to merge.

@kyle-sexton

Copy link
Copy Markdown
Contributor Author

Thanks for the review — verdict noted as Approve. Classification of the two 🟡 nits:

# Finding Classification Evidence Reacted
1 fix.md L52 — (name it per your framework's convention) parenthetical inside the code fence could be misread as part of the test name VALID — fixed Moved the meta-note to a trailing # comment delimiter (returns_null_when_user_is_deleted # name it per your framework's convention) so it reads unambiguously as commentary, not the identifier. Fixed in 8c06888. 👍
2 metrics.md L14 — "How to check" cell verbosity from the dual-shell note ACKNOWLEDGED (no action) The finding is scoped to "if a future edit passes through" and is low-impact by your own read; these are AI-readable instruction docs, not rendered end-user markdown. Splitting the dual-shell note out of the table cell would reduce the table's self-containedness, so leaving it. No behavior/clarity regression. 👍

Head is now 8c06888; all checks green, merge state CLEAN. Not merging (merge gate on).

@kyle-sexton
kyle-sexton merged commit 2277c98 into main Jul 19, 2026
15 checks passed
@kyle-sexton
kyle-sexton deleted the fix/424-verification-agnostic-examples branch July 19, 2026 17:03
kyle-sexton added a commit that referenced this pull request Jul 20, 2026
…ls (#531) (#609)

## Summary

Skills declared ecosystem/forge/tracker-agnostic ship bare hardcoded
stack/forge/branch/tracker defaults because agnosticism was asserted in
prose and
never enforced mechanically — the dominant review-churn class, re-caught
by the
external reviewer PR after PR (branch/remote hardcodes, seam-conditional
eval,
.NET-flavored examples). This adds a fail-closed CI lane that
mechanically detects
that coupling instead of paying for it at review time again and again.

Detection only — this PR builds the gate. Fixing the existing violations
stays
with the member issues below.

## Fix

A new `portability-lint` lane (`.github/workflows/ci.yml`) wired into
the required
`ci-status` aggregate, plus a repo-local detector:

- **`scripts/check-skill-portability.sh`** — scans skill files for
coupling tokens.
On a PR it scans only the skill files the change **touches** (mirroring
the
`skill-quality-gate` changed-diff pattern), so enabling a token class
prevents
**new** coupling without red-lining pre-existing violations — main's
push event
scans nothing (self-test is the push path), and existing hits wait for
their
  owning follow-up fix or the file's next edit.
- **`scripts/skill-portability-tokens.txt`** — the token list as
external,
extensible **data** (not logic buried in bash), so a reviewer re-catch
is a
one-line data edit. Seeded with **one active class** — the
branch/default-branch
hardcode (`origin/main` / `origin/master`) — per the ratified
one-token-class-at-
a-time rollout; further classes (`dotnet`/`Clean Arch`,
`raw.githubusercontent`,
bare `gh` tracker calls) are staged as commented entries with
enable-triggers.
- **Never-skip shape**: the job is unconditional; only the PR-diff step
is
event-gated, and a self-test step runs first so a broken detector can
never mask
  a real violation behind a green gate.

**Design decisions** (resolved per the ratified plan, not invented):

- *How a skill declares agnosticism scope* — **no new frontmatter
field**. A skill
is agnostic by default (the Design boundary already binds every plugin),
so the
gated set is the files a change touches. A hit is excused three ways,
all
reviewer-visible comments (reusing the silent-skip gate's
annotated-exemption
shape): an auto-recognized detection-first / presence-gated use; a
per-site
`portability-ok: <reason>`; or a whole-file `portability-scope:
<reason>`
declaring an inherent narrower boundary (the
forge-locked-under-a-neutral-name
case). This distinguishes **guarded** refs (fine) from **bare** ones, as
the
  guarded-forward-ref disposition requires.
- `docs/PLUGIN-PHILOSOPHY.md` gains one doctrine sentence extending the
existing
  declared-narrower-boundary allowance from OS platform to the
  forge/ecosystem/tracker axis.

No `plugins/<name>/` directory is touched, so no version bump /
CHANGELOG entry is
needed (matching the CI-gate precedent).

## Verification

- **Self-test suite** (`scripts/check-skill-portability.test.sh`, runs
in CI):
`PASS=12 FAIL=0` — covers bare-token FAIL with file:line,
detection-ladder pass,
same-line and comment-block-above `portability-ok`, annotation non-leak
past
intervening code, whole-file `portability-scope`, staged tokens staying
inactive
under the shipped list, fail-closed exit 2 on missing token list /
invalid base
  ref, the vendor/evals/`*.test.sh` exclusion set, and empty scope.

- **Catches a real violation, passes a real legit use** (the
bare-vs-guarded
  discrimination that drives this class), on live corpus files:

  ```
$ scripts/check-skill-portability.sh --paths
plugins/work-items/skills/track/actions/start.md
COUPLING: plugins/work-items/skills/track/actions/start.md:61:
origin/(main|master) -> ... git checkout -b <type>/<N>-<slug>
origin/main ...
COUPLING: plugins/work-items/skills/track/actions/start.md:65:
origin/(main|master) -> ... git checkout -b <type>/<N>-<slug>
origin/main ...
  (exit 1)

$ scripts/check-skill-portability.sh --paths
plugins/review/skills/fanout/SKILL.md
No unexcused coupling tokens in 1 skill file(s). # detection-first
origin/HEAD ladder → guarded
  (exit 0)
  ```

- **Calibrated against the full corpus** (`--all`): the active branch
class flags
exactly 3 genuine bare hardcodes in 2 files (owned by member issues;
changed-file
scoping keeps them off main) and correctly passes the detection-ladder
uses in
  `review/*` and excludes evals/test fixtures.

- **Changed-file CI path proven** against a real base ref (16 changed
skills gated,
all clean). Local checks green: `shellcheck` (repo `.shellcheckrc`),
`shfmt`,
  `actionlint`, comment-residue detector (T1=T2=T3=0), `typos`.

Closes #620

Part of #531 (umbrella stays open — stages remain; child #620 carries
this PR's shipped scope per the #603 pattern).

## Related

- Member coupling instances (this is prevention; it does not block
them): #404
#405 #406 #408 #410 #412 #415 #416 #418 #421 #422 #423 #428 #429 #432
#438 #439
  #441 #442
- #445 — sibling CI-gate backlog (mechanical conformance only;
explicitly not
  coupling)
- #412 — guarded-forward-ref disposition the gate honors (guarded ≠
bare)
- #441 — declared narrower-scope exemption the `portability-scope`
mechanism serves
- #467 — the branch/remote review-churn the seed active class targets
- #453 — seam-conditional eval re-catch of this class
- #491 #492 — .NET-flavored-example hand-fixes a future staged class
would obviate
- #611 — post-green review finding (annotation-carry pending_annot
doesn't distinguish inline vs block HTML comments), deferred out of this
PR's scope

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01KyLj6oaFVTE6xFuoYCC2KC

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

verification: .NET/C#/PowerShell-flavored examples and Unix-shell assumptions in stack-agnostic skills (low severity)

1 participant