Skip to content

docs: audit agent instructions and skills for accuracy and brevity - #292

Merged
webern merged 8 commits into
mainfrom
claude/audit-agent-instructions-skills-o4s0sl
Jul 4, 2026
Merged

docs: audit agent instructions and skills for accuracy and brevity#292
webern merged 8 commits into
mainfrom
claude/audit-agent-instructions-skills-o4s0sl

Conversation

@webern

@webern webern commented Jul 3, 2026

Copy link
Copy Markdown
Owner

Summary

To get the most out of limited Fable preview, I asked it to take a pass over the repository looking to improve its legibility to coding agents on a few dimensions:

  • skill tightness and organization
  • instructions
  • documenting weird code including TODOs where it looks like a bug
  • creating copilot instruction mds for pull requests

AI Summary

What the audit found and fixed:

  • Wrong instructions: gen/AGENTS.md pointed at a nonexistent make target (test-cpp), AGENTS.md claimed make test runs everything and banned anonymous namespaces "anywhere" while 45 files contain them, the README hello-world example did not compile (Result<int> needs .value()), and several skills described a repo state that is months stale (baseline "pins one file" vs ~140, "segno is never written", a worked bug example that is already fixed).
  • Bloat: skills trimmed by ~140 net lines (arch lost its microservices boilerplate, duplicated test-run and PR-template text now has one owner), and the third retelling of the audit-tool story now points at audit/README.md instead.
  • New guidance where agents actually look:
    • AGENTS.md "mx::api conventions": new absent-able fields use std::optional; absent-able enums keep an unspecified first enumerator (including the ternary Bool); no new -1/is...Specified patterns and no mass-migration. Codifies the convention from the feat(api): support lv (let-ring) ties #286 review discussion.
    • docs/ai/design/api-design-principles.md: seven principles for how the api re-shapes MusicXML (stateless ticks, containment over label fields, one-fact-one-field, no neighbor-dependent meaning, denormalized state, quarantined edge cases, ignorable fidelity knobs), with a digest inlined in AGENTS.md.
    • Six path-scoped Copilot review instruction files under .github/instructions/.
  • Code comments at verified traps (tie/tied dual emit, 1/1 time-modification sentinel, undertie bytes, id seed) and TODOs marking six suspected bugs found during the audit: arranger/publisher overwrite lyricist, a && that drops explicit normal-type, an off-by-one in test filename stripping, silent normal-type loss for tuplets without tuplet notations, ambiguous __ path flattening in the classifier, and the dead DurationData::isTied field. Each TODO is an issue candidate; happy to file them.

Testing

  • clang-format dry-run clean on all touched C++ files (comment-only changes; no behavior)
  • python3 -m py_compile on touched Python
  • Every factual claim in the edited docs was verified against the Makefile, CI workflow, and source before editing; stale-skill claims were re-checked in code (e.g. emitSegno exists, baseline length, batch_plan output)

References


Generated by Claude Code

webern added 5 commits July 3, 2026 06:34
- AGENTS.md: quickstart, mx::api presence/absence conventions (std::optional
  for new fields, unspecified enumerators for enums incl. Bool), accurate
  'make test' scope, soften anonymous-namespace rule to match reality,
  condense Docker/audit/corert sections, fix typos
- gen/AGENTS.md: fix nonexistent 'make test-cpp' target, unify commit policy
- gen/README.md: point at gen/AGENTS.md instead of duplicating commands
- README.md: fix non-compiling createFromScore example (Result<int>.value()),
  ARGS filter example, 'make dev' description
- audit/data READMEs, Makefile, CI: fix skill-name references, add classify.py
  to layout table, fix garbled sentences, drop stale 829 pin count comment
- code: add TODOs for 5 suspected bugs (arranger/publisher clobber lyricist,
  time-mod normal-type && condition, filename off-by-one, silent normal-type
  drop for tuplets without <tuplet> marks, ambiguous __ path flattening) and
  9 clarifying comments at verified confusing spots
Corrects stale claims and trims redundancy across .claude/skills per the
instructions audit; a follow-up commit completes the remaining skill fixes.
- skills: complete audit fixes (arch trimmed of irrelevant generic sections,
  stale examples replaced, optionality guidance in add-feature step-2 now
  matches the std::optional / unspecified convention)
- docs/ai/design/api-design-principles.md: seven principles describing how
  mx::api re-shapes MusicXML (stateless time, containment over labels,
  one-fact-one-field, no neighbor-dependent meaning, denormalized state,
  quarantined edge cases, ignorable fidelity knobs), referenced from
  AGENTS.md and the mx-api-add-feature skill
- NoteData.h: correct stale tie comment (writer emits both tie and tied)
- DurationData.h: TODO on dead isTied field
Six .github/instructions/*.instructions.md files gated by applyTo globs:
public api headers (presence/absence conventions, equality macros), impl
(read/write symmetry, silent-drop and sentinel contracts), generated core
(never review or hand-edit generated code), gen (cardinal rules), repo-wide
C++ (anonymous namespaces, formatter-owned layout), and corpus/tests
(pinned counts, markers, baseline discipline).
Agents reliably read AGENTS.md (SessionStart hook) but often skip linked
docs; carry the seven rules inline, keep the full doc for examples and
per-rule tests.
@webern webern added non-breaking fixes or implementation that do not require breaking changes ai Issues opened by, or through, a coding agent. labels Jul 3, 2026 — with Claude
@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown

Coverage report

Core-dev coverage src/private/mx/core/

Metric Coverage Covered / Total
Lines 77.8% 28487 / 36624
Functions 74.3% 6349 / 8550
Branches 50.6% 22632 / 44725

API coverage src/private/mx/{api,impl,utility}/

Metric Coverage Covered / Total
Lines 78.7% 6101 / 7754
Functions 64.3% 2091 / 3250
Branches 47.8% 5182 / 10834

Core HTML report | API HTML report

Commit 124e3d2314d151486e2b03cb7a42b8b582481255.

@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown

gen-quality gen/

gen-quality: 84.5 / 100   (floor 84.5, +0.0)

  structure     86.5  x0.50   [fn 90.5 / file 82.6]
  cyclomatic    88.4  x0.25
  cognitive     76.6  x0.25

  409 functions across 31 files, 7702 lines (largest file 1044)
  max cc 56  max cognitive 44  max fn loc 152

Worst offenders (top 5 per axis; full lists in score.json):
  cyclomatic gen/xsd/analyze.py:311     report                             56
  cyclomatic gen/plates/build.py:956    _validate_config_against_ir        35
  cyclomatic gen/press/context.py:145   plate_context                      34
  cyclomatic gen/__main__.py:46         _ir                                23
  cyclomatic gen/tests/test_ir.py:102   _check_references                  20
  cognitive  gen/xsd/analyze.py:311     report                             44
  cognitive  gen/ir/resolve.py:119      flat_elements                      40
  cognitive  gen/tests/test_ir.py:102   _check_references                  38
  cognitive  gen/press/context.py:145   plate_context                      37
  cognitive  gen/xsd/analyze.py:207     _sccs                              37
  size       gen/xsd/analyze.py:311     report                             152
  size       gen/press/context.py:145   plate_context                      96
  size       gen/plates/build.py:533    _value_plate                       89
  size       gen/plates/build.py:956    _validate_config_against_ir        89
  size       gen/ir/resolve.py:119      flat_elements                      78

Commit 124e3d2314d151486e2b03cb7a42b8b582481255.

arch -> mx-architecture, grill-me -> mx-grill-me, open-mx-issue ->
mx-open-issue, open-mx-pr -> mx-open-pr, project -> mx-project,
questions -> mx-questions. Cross-references updated; open-issue/open-pr
descriptions gain trigger phrases (report a bug, submit as PR). AGENTS.md
layout notes the convention.
@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown

Coverage report

Core-dev coverage src/private/mx/core/

Metric Coverage Covered / Total
Lines 77.8% 28487 / 36624
Functions 74.3% 6349 / 8550
Branches 50.6% 22632 / 44725

API coverage src/private/mx/{api,impl,utility}/

Metric Coverage Covered / Total
Lines 78.7% 6101 / 7754
Functions 64.3% 2091 / 3250
Branches 47.8% 5182 / 10834

Core HTML report | API HTML report

Commit 4cda961ba1756f5905628b9ee04c21a71296ed00.

@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown

gen-quality gen/

gen-quality: 84.5 / 100   (floor 84.5, +0.0)

  structure     86.5  x0.50   [fn 90.5 / file 82.6]
  cyclomatic    88.4  x0.25
  cognitive     76.6  x0.25

  409 functions across 31 files, 7702 lines (largest file 1044)
  max cc 56  max cognitive 44  max fn loc 152

Worst offenders (top 5 per axis; full lists in score.json):
  cyclomatic gen/xsd/analyze.py:311     report                             56
  cyclomatic gen/plates/build.py:956    _validate_config_against_ir        35
  cyclomatic gen/press/context.py:145   plate_context                      34
  cyclomatic gen/__main__.py:46         _ir                                23
  cyclomatic gen/tests/test_ir.py:102   _check_references                  20
  cognitive  gen/xsd/analyze.py:311     report                             44
  cognitive  gen/ir/resolve.py:119      flat_elements                      40
  cognitive  gen/tests/test_ir.py:102   _check_references                  38
  cognitive  gen/press/context.py:145   plate_context                      37
  cognitive  gen/xsd/analyze.py:207     _sccs                              37
  size       gen/xsd/analyze.py:311     report                             152
  size       gen/press/context.py:145   plate_context                      96
  size       gen/plates/build.py:533    _value_plate                       89
  size       gen/plates/build.py:956    _validate_config_against_ir        89
  size       gen/ir/resolve.py:119      flat_elements                      78

Commit 4cda961ba1756f5905628b9ee04c21a71296ed00.

Job display names now state the suites executed (gen gates, api mxtest +
round-trip, core corert/unit/xmllint/probes, Go+C corert targets,
coverage, macOS core+api, swift build-only); the coverage and macos names
previously claimed core-only while also running api suites. Job keys are
unchanged. Comments note what each job deliberately skips.
@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown

Coverage report

Core-dev coverage src/private/mx/core/

Metric Coverage Covered / Total
Lines 77.8% 28487 / 36624
Functions 74.3% 6349 / 8550
Branches 50.6% 22632 / 44725

API coverage src/private/mx/{api,impl,utility}/

Metric Coverage Covered / Total
Lines 78.7% 6101 / 7754
Functions 64.3% 2091 / 3250
Branches 47.8% 5182 / 10834

Core HTML report | API HTML report

Commit 6485b6e81b62e528ea776ee82d0e12e665bca53a.

@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown

gen-quality gen/

gen-quality: 84.5 / 100   (floor 84.5, +0.0)

  structure     86.5  x0.50   [fn 90.5 / file 82.6]
  cyclomatic    88.4  x0.25
  cognitive     76.6  x0.25

  409 functions across 31 files, 7702 lines (largest file 1044)
  max cc 56  max cognitive 44  max fn loc 152

Worst offenders (top 5 per axis; full lists in score.json):
  cyclomatic gen/xsd/analyze.py:311     report                             56
  cyclomatic gen/plates/build.py:956    _validate_config_against_ir        35
  cyclomatic gen/press/context.py:145   plate_context                      34
  cyclomatic gen/__main__.py:46         _ir                                23
  cyclomatic gen/tests/test_ir.py:102   _check_references                  20
  cognitive  gen/xsd/analyze.py:311     report                             44
  cognitive  gen/ir/resolve.py:119      flat_elements                      40
  cognitive  gen/tests/test_ir.py:102   _check_references                  38
  cognitive  gen/press/context.py:145   plate_context                      37
  cognitive  gen/xsd/analyze.py:207     _sccs                              37
  size       gen/xsd/analyze.py:311     report                             152
  size       gen/press/context.py:145   plate_context                      96
  size       gen/plates/build.py:533    _value_plate                       89
  size       gen/plates/build.py:956    _validate_config_against_ir        89
  size       gen/ir/resolve.py:119      flat_elements                      78

Commit 6485b6e81b62e528ea776ee82d0e12e665bca53a.

Comment thread .claude/skills/mx-open-pr/SKILL.md
Comment thread .github/instructions/api-headers.instructions.md
Comment thread .github/instructions/cpp.instructions.md Outdated
- mx-open-pr template gains a Human Summary section for the maintainer
  to fill in
- api-headers review instructions: comments should serve humans and
  coding agents, crisp and focused on the surprising
- cpp review instructions: comments state constraints and effects hard
  to ascertain from the code
@webern
webern merged commit e50e33a into main Jul 4, 2026
7 checks passed
@webern
webern deleted the claude/audit-agent-instructions-skills-o4s0sl branch July 4, 2026 06:31
@github-actions

github-actions Bot commented Jul 4, 2026

Copy link
Copy Markdown

Coverage report

Core-dev coverage src/private/mx/core/

Metric Coverage Covered / Total
Lines 77.8% 28487 / 36624
Functions 74.3% 6349 / 8550
Branches 50.6% 22632 / 44725

API coverage src/private/mx/{api,impl,utility}/

Metric Coverage Covered / Total
Lines 78.7% 6117 / 7776
Functions 64.3% 2091 / 3252
Branches 47.9% 5206 / 10866

Core HTML report | API HTML report

Commit 41f7c9f3b46c8d45c1a96c51094b8a1edc92553f.

@github-actions

github-actions Bot commented Jul 4, 2026

Copy link
Copy Markdown

gen-quality gen/

gen-quality: 84.5 / 100   (floor 84.5, +0.0)

  structure     86.5  x0.50   [fn 90.5 / file 82.6]
  cyclomatic    88.4  x0.25
  cognitive     76.6  x0.25

  409 functions across 31 files, 7702 lines (largest file 1044)
  max cc 56  max cognitive 44  max fn loc 152

Worst offenders (top 5 per axis; full lists in score.json):
  cyclomatic gen/xsd/analyze.py:311     report                             56
  cyclomatic gen/plates/build.py:956    _validate_config_against_ir        35
  cyclomatic gen/press/context.py:145   plate_context                      34
  cyclomatic gen/__main__.py:46         _ir                                23
  cyclomatic gen/tests/test_ir.py:102   _check_references                  20
  cognitive  gen/xsd/analyze.py:311     report                             44
  cognitive  gen/ir/resolve.py:119      flat_elements                      40
  cognitive  gen/tests/test_ir.py:102   _check_references                  38
  cognitive  gen/press/context.py:145   plate_context                      37
  cognitive  gen/xsd/analyze.py:207     _sccs                              37
  size       gen/xsd/analyze.py:311     report                             152
  size       gen/press/context.py:145   plate_context                      96
  size       gen/plates/build.py:533    _value_plate                       89
  size       gen/plates/build.py:956    _validate_config_against_ir        89
  size       gen/ir/resolve.py:119      flat_elements                      78

Commit 41f7c9f3b46c8d45c1a96c51094b8a1edc92553f.

webern added a commit that referenced this pull request Jul 12, 2026
## Human Summary

I asked Fable, while I still have access to it, to improve Agent
performance with better instructions putting them in places more likely
to be read and incorporating some of my latest pet-peeves.

## Summary

Agents were not reading docs/ai/design/api-design-principles.md, and
several recurring review
themes were written down nowhere. This PR moves the doctrine to where
agents actually look and
captures the missing rules, without growing AGENTS.md (it shrinks by 8
lines).

- New mx-api-doctrine skill whose description says it MUST be read
before designing or changing
anything in the api layer. It contains the seven design principles
(moved verbatim from the
  deleted doc) plus the previously unwritten doctrine:
- Audience: the api is for people creating music and building,
modifying, and interpreting
MusicXML files. Round-tripping is how we develop the api, not what it is
for. Header comments
explain notation semantics and how MusicXML encodes them;
"source"/"preserved" framing is
allowed only on true fidelity knobs. No documenting what is not modeled,
no banner comments
    (from review feedback on #326, #333, #321).
- Failure model: misuse is made unrepresentable or degrades to a
documented fallback. UB must be
unreachable from the public interface, nothing throws across the api
boundary (never for a
failed precondition), and Result stays quarantined at the
DocumentManager I/O boundary.
- Choice types: the TimeChoice / MarkDataChoice pattern for either/or
payloads (copy-returning
accessors, default-constructed wrong-kind fallback, auto-collapse,
common case kept prominent
    in its own header), from the #321 redesign and the #330 review.
- AGENTS.md digest replaced with a pointer to the skill; the add-feature
skill and the Copilot
api-headers review instructions repointed and extended with the same
rules.
- Deleted the mx-architecture skill: generic architect boilerplate that
wasn't helping. Its few
mx-specific facts live elsewhere (gen design in gen/DESIGN.md; the
error-handling stance is now
  concrete doctrine in the new skill).

Net -33 lines repo-wide.

## Testing

- [x] No dangling references: grep for api-design-principles and
mx-architecture is clean
- [x] Docs/skills only; no C++ or build changes, so no test suites apply

## References

- Related: #292 (created the principles doc and the AGENTS.md digest
this replaces)
- Captures review feedback from #321, #326, #330, #333
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai Issues opened by, or through, a coding agent. non-breaking fixes or implementation that do not require breaking changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant