Skip to content

Retire the serena tooling without retiring what the runs saw - #515

Merged
MongLong0214 merged 1 commit into
devfrom
retire-serena
Aug 11, 2026
Merged

Retire the serena tooling without retiring what the runs saw#515
MongLong0214 merged 1 commit into
devfrom
retire-serena

Conversation

@MongLong0214

Copy link
Copy Markdown
Owner

Closes #514. The owner no longer uses the serena agent tooling, so its configuration comes out of the repository.

Two kinds of mention, handled differently on purpose.

Removed — configuration for a tool that is gone

.gitignore entries, two benchmark task fixtures, and the guard in test/manifest.test.ts. Nothing here describes anything that happened; it describes what the repository expects to find, and it no longer expects to find it.

Kept — what the recorded runs actually saw

bench/results/ is untouched: 58 files still name .serena, because that is what was on disk during those runs.

bench/PREREGISTRATION.md and bench/README.md keep it too, where they report what a run encountered — with the retirement and its date noted beside them. A preregistration exists to fix claims before the numbers exist. Rewriting it to describe a noise-immunity check against a directory no recorded run ever met would make it describe a run that never happened, which is exactly the defect class docs/SELF-AUDIT.md catalogues.

Where the prose states a rule for future runs, it is generalised. That rule was never about serena specifically — it was about agent tooling dropping files into a workspace, which the next tool will do too.

The record

r-strayserena was not edited. A record lives in the commit that declared it; retirement is a new record that names the old one and supersedes it.

Verification

git grep serena -- bench/results          58 files — evidence intact
git grep serena -- . ':!bench/results'    6 lines — all historical prose

Package and bench typechecks clean, bench/verify.mjs passes, dist unchanged, 65 cases pass across the manifest and README suites.

The owner no longer uses the serena agent tooling, so the configuration that
existed for it goes with it: the .gitignore entry, the manifest-test guard,
and the task-YAML comments that named its noise. Where prose states a rule
for future runs, the rule now names the class — agent tooling that drops
files into a workspace — rather than one tool.

The evidence keeps naming the directory. bench/results/ is untouched: the
transcripts carry `.serena` because that is what was on disk when those runs
ran. bench/PREREGISTRATION.md and bench/README.md keep reporting it where
they report what a run saw, with a note that the tooling was retired on
2026-08-11 — rewriting them around a directory no recorded run encountered
would make the preregistration describe a run that never happened, which is
the defect class docs/SELF-AUDIT.md catalogues (#514). The released CHANGELOG
entry is unchanged; a new one records the retirement.

r-strayserena justified the ignore entry by the directory coming back in
every worktree the server was pointed at. The server is no longer pointed at
any, so the record no longer carries its weight and is superseded rather
than edited.

Supersedes: r-strayserena
Ruled-out: rewriting the evidence documents to match the retirement | the transcripts and the preregistration report what the recorded runs saw, and a published claim the evidence does not support is the defect class docs/SELF-AUDIT.md exists to catalogue
Ruled-out: keeping the ignore entry and the test guard as cheap insurance | the record that justified them said the directory comes back while the tool runs; it no longer runs, and a guard for a tool nobody uses guards nothing
Ruled-out: editing r-strayserena in place | a record lives in the commit that declared it; retirement is a new record that names the old one
Limit: the preregistration fixes claims before numbers exist, so what it says about the calibration cannot move to match later tree state
Warn: bench/results/ and the historical prose in bench/README.md and bench/PREREGISTRATION.md name `.serena` on purpose -- that is what the recorded runs saw; do not clean those mentions up
Blast: local
Undo: easy
Certainty: firm
Record-Id: r-retireserena
Verified: both tsconfigs typecheck clean; the second build leaves dist/ unchanged; bench/verify.mjs passes; the manifest, readme and readme-order suites pass 65/65
Verified: git grep finds `.serena` only under bench/results/ and in historical prose; nothing operational references it
@github-actions

Copy link
Copy Markdown

CommitLore — record lint

Trailers: clean — 1 commit in origin/dev..c1171ef60c18c6826884f0737adfc5cea8318b4f
Active constraints: 46 limits · 113 ruled-out · 44 warnings — from 61 records over 7 changed paths

Active constraints for the paths this PR touches

Limits (46)

  • r-retireserena c1171ef — the preregistration fixes claims before numbers exist, so what it says about the calibration cannot move to match later tree state
  • r-rel071 af8e0ab — 0.7.0 stays published with its notes amended to name the defect at the top; retracting a tag people may already have installed trades a known-bad version for an unknown one
  • r-rel070 d4a4d8b — the README's behaviour claim now rests on M5 while the generated numbers block beneath it still publishes M4, which is The README's generated numbers block still publishes M4; M5 measured the thing the README leads with #480 rather than a release-time edit
  • r-numgate b770054 — the README's behaviour claim and the generated block below it now describe different studies until The README's generated numbers block still publishes M4; M5 measured the thing the README leads with #480 lands
  • r-readmem5 6d04c0b — the README now leads its behaviour claim with a [claim]-tier number while shipping a [directive] tier nobody has measured, and that gap will widen until something measures it
  • r-clog070 172fa3d — the entry stays under ## Unreleased and names no version, because the version bump belongs to the release commit and a changelog that pre-announces a number can be wrong about it
  • r-benchscope 67f4375 — nothing checks the shape of the eight metric-row files. This gate names them and steps over them, and bench/deterministic/types.ts is the only definition that family has -- there is no JSON schema for it, so drift on that side is still invisible
  • r-benchscope 67f4375 — the pre-provenance exemption reads started_at, which is data on the row rather than a fact about the file. A row that misreported it would be held to the shorter list of requirements; that is a deliberate falsification rather than the omission this fixes, and nothing here detects it
  • r-4e7c19 6d4e828turn_usage carries the provider's tokenizer; every figure in bench/TOKEN-LEDGER.md is in the product's own ceil(chars / 4) proxy. A future W4 must be reported beside the floor, never added into it, until both sides are on one tokenizer
  • r-4e7c19 6d4e828content_blocks says a turn called a tool, not which tool or on what — the tool_use block's name and input are discarded, so reproposed counts correct behaviour as failure — measure rejected work, not rejected mentions #141's rejected-path tool actions stay uncounted
  • r-tokenledger 298e47c — both sides are byte-derived token proxies under CHARS_PER_TOKEN=4, so the ratio cancels a uniform tokenizer error and does not cancel a differential one between prose and diffs
  • r-ledgerresult bc31c90 — both sides are byte-derived proxies under CHARS_PER_TOKEN=4 rather than a provider tokenizer, so the ratio cancels a uniform error and not a differential one between diff text and prose
  • r-ledgerresult bc31c90 — break-even in reads assumes reads land on the evaluation set the way the delivery run's per-path average describes, and real editing concentrates on a few files
  • r-rel060 e999b9d — the install one-liner in all four READMEs now points at a tag that does not exist until this is tagged, so the window between merging to main and pushing v0.6.0 is one where the documented install is broken
  • r-release051 19810d2 — the hook is written at install time, so no release repairs a repository that already has one; every release touching hook behaviour has to restate what does
  • r-toolstate334 f4e89ef — a check that only inspects a declared list cannot see an undeclared file, and in a clone-is-the-distribution model the undeclared file ships anyway
  • r-release050 ad402c7 — the hook is written at install time, so a corrected release never reaches a repository that already has one; every release fixing hook behaviour has to say what repairs an existing install
  • r-rel041notes 71efe1f — 0.4.1 makes the installer honest about a verification it cannot complete rather than fixing the kill, so an upgrading user may still see the unverified message instead of a version
  • r-rel040notes 5d57a72 — the 26.3-point density gap quoted in the notes is measured at this head and will drift with merge volume; it is illustrative of the denominator problem rather than a stable figure
  • r-notes030 a289ca5 — the density denominator is named here and in the handoff, not in the harness that emits it, so the next run reproduces the same ambiguity
  • r-recov140 da5a210 — the current M4 pool supplies no eligible episodes; confirmation requires at least 24 of at most 30 independently annotated candidates
  • r-m4basis 5e2d2cb — the guard question stays unanswered until the exposure instrument is verified and M4 is rerun on it
  • r-m4withdraw e5f9b73 — the guard question is now unanswered rather than answered null
  • r-expomerge1 d6ad014 — M4's existing rows have no exposure field and must read as unknown, not as not-exposed — backfilling by inference would erase the finding
  • r-f61a2c 9114cf0 — the matcher remains deterministic and lexical; no embedding or semantic service is available to distinguish paraphrases
  • r-exposure52 ba69411 — legacy JSONL artifacts predate model and guard-exposure fields | their absence remains unknown and is never inferred or backfilled
  • r-fix92dupid 7f41a6e — cross-references between two blocks declared by the same commit (a Follows:/Supersedes: naming a sibling block's id) are still reported as dangling rather than resolved against the sibling -- unchanged from before this fix, and called out in validate.ts's own comment as future work
  • r-fix93pkg 9c4a396 — package.json remains a development artifact (build, typecheck, dependency floor) -- it is not read as a distribution manifest by anything in this repository
  • r-distrace88 d118a73 — the fix insulates bench-ablation.test.ts from the race; it does not remove the underlying design (four test files independently, redundantly rebuilding one shared dist/ in their own beforeAll). A fifth file doing the same thing, or a future check elsewhere that also depends on dist/'s mid-run stability, can still race the same way.
  • r-parsemulti 6d39d25parse has no git-commit context (no sha, no notes mirror) — its identityCollision check is local to the one message being parsed and cannot detect a Record-Id that collides with something already committed elsewhere in history the way context's fold does.
  • r-multirec01 92aeb24 — parseRecordBlocks only recognizes a non-final block by its declared Record-Id, so an unidentified inherited record beyond the first stays recoverable in the plan that computed it but not in a later re-parse of stored text; squash-preserve orders unidentified blocks last so the common case (at most one) is unaffected.
  • r-multirec01 92aeb24 — multi-block reference checking (Follows:/Supersedes:) does not resolve one block's reference against a sibling block declared by the same commit; each block is still checked against every earlier commit in history.
  • r-exit065 e545dee — any new command's exit codes must be drawn from SPEC §10, not invented locally
  • r-m4n560 081d858 — M4 treatment-arm data did not exist when this sample size was fixed
  • r-4c73a2 1d82a5bcode inherits the limit of any lexical surface — an alternative implemented under another name is invisible, one named in a string literal is counted; both were true of artifacts
  • r-5c07d2 9ec184a — the pre-registration is append-only once collection starts, so this lands as an appended correction rather than an edit to section 11
  • r-3e8a41 2f0a8a0 — scoping is not implemented, so off-path records reach every arm
  • r-2f8b90 a376808claude -p inherits the operator's MCP servers, settings, CLAUDE.md, skills and hooks unless every one is explicitly disabled
  • r-9c2f74 d653153 — the ablation arms cannot discriminate on these fixtures -- no-grade and no-lifecycle are byte-identical to the treatment in 9 of 10 tasks, because the seeds carry one reconstructed record and one task with a lifecycle trailer between them
  • r-9c2f74 d653153 — the harness assembles its own projection rather than calling the shipped injector, so what is measured is the harness's rendering of the records, not src/core/inject.ts (issue B-08 · Replace the benchmark harness injector with the actual src/core/inject.ts #36)
  • r-3f7a29 49817dc — reconstruction reads text written before the protocol existed, so the evidence is thinner than a harvest and the discard rate is expected to be high
  • r-5c8b31 60ddc39 — the agent CLI exposes no in-flight turn limit, so a per-task turn budget can only ever be observed with this driver
  • r-0b7c44 d2b2ce3 — a command is only real once --help names it, because that is where users look before they read source
  • r-6c2b95 5da793c — the CLI holds no API key, so any real driver runs through the user's own agent session and cannot be exercised in this environment
  • r-c0f4e2 3d249cd — npm gitlore is held by an active same-domain CLI, so the owner's first-choice name was not available
  • r-b2e7f1 00d348d — Parsing must delegate to git interpret-trailers -- reimplementing the block rules would drift from the rest of the git ecosystem

Ruled out (113)

  • r-retireserena c1171ef — rewriting the evidence documents to match the retirement | the transcripts and the preregistration report what the recorded runs saw, and a published claim the evidence does not support is the defect class docs/SELF-AUDIT.md exists to catalogue
  • r-retireserena c1171ef — keeping the ignore entry and the test guard as cheap insurance | the record that justified them said the directory comes back while the tool runs; it no longer runs, and a guard for a tool nobody uses guards nothing
  • r-retireserena c1171ef — editing r-strayserena in place | a record lives in the commit that declared it; retirement is a new record that names the old one
  • r-rel071 af8e0ab — deleting the v0.7.0 tag and re-cutting it | rewriting a published ref breaks every clone that already fetched it, and the release notes can carry the truth without that
  • r-rel071 af8e0ab — reading the flag with a nullish check and documenting the commander default | the documentation would sit in this file while the defect sits in every install, which is the arrangement that produced this
  • r-rel070 d4a4d8b — bumping the manifests first and the README pins after | the readme suite pins them to package.json, so the intermediate commit is one where CI is red and the documented install is wrong
  • r-rel070 d4a4d8b — rebuilding dist for the version change | the version is read at runtime, and a rebuild would move the digest every M5 row records without changing a byte of behaviour
  • r-numgate b770054 — repointing README_SOURCES at M5 inside the release | its own comment explains why the declaration is deliberate, and writing that note in a hurry would produce exactly the careless publication it was built to stop
  • r-numgate b770054 — dropping the rates too and linking everything | the two rates are the claim, and a README that states a behaviour result without its magnitude sends every reader to a second document to learn what was measured
  • r-readmem5 6d04c0b — putting the limits in docs/evidence.md and linking them | a reader who stops at the table has then read a claim without its scope, and the scope is what makes the claim survivable
  • r-readmem5 6d04c0b — rounding 6.7x into the headline | the ratio is arithmetic on two small counts and moves fast with either; the two rates and the interval are what the study actually bounds
  • r-clog070 172fa3d — grouping by conventional-commit type | the reason a reader opens this file is "what breaks and what is now possible", and feat:/fix: sorts by the author's vocabulary rather than by that question
  • r-benchscope 67f4375 — A declared list of result files, in the gate or in a manifest | opt-in under another name, and a file left off it is silently ungated -- which is precisely the failure bench/verify.mjs is not run by anything, so the result schema drifted unnoticed #392 reports
  • r-benchscope 67f4375 — Validating only the three files that pass today | a description of the present rather than a rule, and it goes stale the first time a matrix is added or the schema moves
  • r-benchscope 67f4375 — A naming convention on results filenames | the runner's default output is an ISO stamp plus hex and the metric writers' is a family name plus an ISO stamp, but every run-record file in the tree was named by hand through --out (t702-m2, m5-off-design-20-tasks), so the convention is enforced nowhere and cannot carry a gate
  • r-benchscope 67f4375 — A marker file dropped beside each gated result | still a step somebody has to remember, and a forgotten step is the defect being fixed
  • r-benchscope 67f4375 — Widening result.schema.json to describe both row families | they share no required field, so the union would accept a run record with no task and a metric row with no metric, which is a schema that rejects nothing
  • r-benchscope 67f4375 — Making harness_commit and dist_digest optional in the schema | weakens the gate for every future row in order to admit six historical files, and Make per-turn token usage observable, and say what it still cannot price #390's drift was an additionalProperties failure that a laxer schema catches later rather than sooner
  • r-benchscope 67f4375 — Skipping the six pre-provenance files whole | cheaper to write, but it stops checking types, patterns and the closed property set on 380 rows in order to excuse two absent fields
  • r-4e7c19 6d4e828 — turning per-turn capture on by default | it changes the CLI invocation, and a harness flag that quietly rewrote how a run is observed would make old and new rows two different things under one name
  • r-4e7c19 6d4e828 — summing the usage on assistant events | they repeat once per content block and their output_tokens is a pre-generation snapshot; that reading is wrong by 397 of 403 on the one probe that checked it
  • r-4e7c19 6d4e828 — writing turn_usage on every row with nulls when uninstrumented | an always-null field is worse than no field, and absent already reads as "not instrumented" everywhere else on this row
  • r-4e7c19 6d4e828 — retaining the whole stream to parse at the end | partial messages emit an event per chunk, so a long run would hold tens of megabytes to produce a few hundred integers
  • r-tokenledger 298e47c — Remeasuring decision_delivery inside this harness | it would price the read side on a corpus that already includes the commits adding this measurement, and docs/evidence.md would cite two corpora for one ratio
  • r-tokenledger 298e47c — Reporting a single write-side accounting | issue The break-even figure does not say which token accounting it uses, and both accountings are defensible #138 established that a break-even which does not say what it counted is underspecified, so the diff term is reported both charged and free
  • r-tokenledger 298e47c — Estimating the drafting turn from the 112 recorded runs | those rows carry a session total from --output-format json with no per-turn attribution, so any split across turns would be invented
  • r-ledgerresult bc31c90 — Leading with the 99.4% reduction against the whole-repository dump | it is the largest number in the table and its denominator is a route no agent can run, which is the figure this market publishes
  • r-ledgerresult bc31c90 — Omitting the code-only row because it has no percentage | a reduction table that drops the case where the product is the expensive option is the thing the method document was written to avoid
  • r-ledgerresult bc31c90 — Publishing a single write-side accounting | The break-even figure does not say which token accounting it uses, and both accountings are defensible #138 established that a break-even which does not say what it counted is underspecified, so the diff term is reported both charged in full and charged at nothing
  • r-rel060 e999b9d — 0.5.2 | three observable behaviour changes and one narrowed acceptance rule are not a patch, and an upgrader reading the number would not go looking
  • r-rel060 e999b9d — Publishing the delivery figure in the README | the measurement is one corpus and one repository, and the README already carries more numbers than a first-time reader can weigh; docs/evidence.md and bench/ hold it
  • r-release051 19810d2 — Waiting for 0.5.2 to carry index ingests any key: value line as a trailer; doctor reports 106 records where git has 0, and context serves commit subjects to the agent #335 | false decision context reaching an agent is the inversion of the product claim, and a stable release should not hold it
  • r-release051 19810d2 — 0.6.0 | no command is added and nothing breaks; calling it minor would hide a correctness fix behind a feature number
  • r-release051 19810d2 — Filtering Verified: because a report listed it as noise | it is vocabulary, and separating a release-note use from a record use needs a guess that can discard real records
  • r-toolstate334 f4e89ef — Asserting the clone contains no unexpected path | the legitimate tree is large and grows, so the whitelist would fail on every honest addition and be widened without being read
  • r-toolstate334 f4e89ef — Forbidding every dotted directory | some are deliberate -- .github and .claude-plugin are the product -- and a rule that cannot tell those apart is not a rule
  • r-toolstate334 f4e89ef — Asserting only against .serena | it names the instance rather than the class, and the next one will have a different name
  • r-release050 ad402c7 — 0.4.2 | a new command and a platform support claim are not a patch, and calling them one hides both from anyone reading version numbers to decide whether to upgrade
  • r-release050 ad402c7 — 1.0.0 | nothing here breaks an existing install, and the milestone that closed is Gate B rather than a stability commitment
  • r-release050 ad402c7 — Describing Windows as supported without the caveat | the word is the one most likely to be read as more than it is, and the release that earns it is the right place to bound it
  • r-nobin284 ab2f08f — Keeping the binary classification arm for the installer's wrapper | the wrapper is a shell script that execs node, so trusting it by name would re-admit any extensionless executable called commitlore while the bundle it runs is the thing with an interpreter to check
  • r-nobin284 ab2f08f — Relaxing the dogfood Evidence check to allow a missing path | it exists to catch a citation nobody can follow, and dropping the check to accommodate a deletion would retire the guarantee instead of correcting the question it asked
  • r-nobin284 ab2f08f — Leaving src/core/paths.ts alone because ADR-0026's inventory did not name it | it was the only remaining code that read files out of a compiled artifact, so leaving it would have left the invariant false while the test asserting it passed
  • r-nobin284 ab2f08f — Keeping /dist/commitlore in gitignore in case a binary returns | an ignore rule for an artifact nothing produces is a claim that something might, and the ADR says otherwise
  • r-hookfailopen 16963dd — asserting the fail-open property against the working tree instead of a clone | this suite exists to test what a real install gets, and a working-tree assertion would pass while a clean clone was broken, which is the failure mode the suite was written to catch
  • r-hookfailopen 16963dd — covering this inside test/inject.test.ts with the other fail-open cases | those exercise the CLI, and this branch is reached only when no CLI can be resolved, so it belongs with the manifest suite that runs the script itself
  • r-rel041notes 71efe1f — folding this into the next feature release | the documented install is broken for upgraders now, and every day it stays tagged is a day the one-liner in the README fails for them
  • r-rel041notes 71efe1f — moving the v0.4.0 tag to the fixed commit | a tag users may already have fetched must not change under them; the fix gets its own version
  • r-rel040notes 5d57a72 — describing 0.4.0 as making agents safer | no study here measures agent behaviour, and the recovery protocol is registered and unrun; the release delivers a loop, not an outcome
  • r-rel040notes 5d57a72 — omitting guard's figures from the release notes because they are unflattering | the notes are read by people deciding whether to trust the tool, and a quality limit hidden at release time is discovered in use
  • r-notes030 a289ca5 — editing the dated benchmark artifact to restate its denominator | it is a provenance-stamped snapshot of one run, and rewriting a past measurement to read better today is the failure this project exists to prevent
  • r-notes030 a289ca5 — tagging first and correcting the notes after | the tag is what users read
  • r-recov140 da5a210 — coding behaviour as the primary outcome | three studies failed at different downstream layers that CommitLore does not control
  • r-m4basis 5e2d2cb — keeping the non-delivery wording as a hedged claim | it asserts something I never established, and a verdict that overstates its own evidence is the defect this repository exists to catch
  • r-m4withdraw e5f9b73 — retracting the dataset or calling M4 invalid | the data is valid and its provenance is clean; what it measured was not the treatment, and those are different words
  • r-f61a2c 9114cf0 — raising the guard threshold | the frozen corpus puts every true positive in the middle band, so a higher cutoff removes signal without repairing score order
  • r-f61a2c 9114cf0 — dropping keyword evidence | long proposals and diffs dilute Jaccard, while one-token alternatives such as Redis still need lexical containment
  • r-exposure52 ba69411 — compute a treatment effect with unknown guard exposure | an old row that never recorded whether the guard reached the run cannot distinguish no treatment from an ignored treatment
  • r-m4docs1 32d5bf1 — keeping the withdrawal notice and only landing the verdict document | bench/report.ts already draws this line -- a provenanced dataset that still shows a withdrawal is a hard failure in check-readme-numbers.mjs (checked here), not a style choice left open
  • r-m4vrd01 21d8bf3 — re-cutting the result by task after seeing it | PREREGISTRATION.md §4 forbids subset analysis regardless of how the result came out, and applies with more force to the run built to give the hypothesis a fair matrix
  • r-rel0200b 5870b35 — leaving "21 defects" and fixing only the GitHub Release body | CHANGELOG.md is the permanent record; a release-notes-only fix would leave the canonical file quoting a number two independent checks had already disproven
  • r-rel0200b 5870b35 — re-deriving a fresh defect count from scratch instead of verifying the given one | gh issue list --label bug,security undercounts (misses no-label and epic-labeled defect issues like doctor probed dist/cli.js while distribution ships dist/commitlore.mjs; CI red #40-56), so a bottom-up recount would have been less reliable than confirming the specific claim (Guard fires wrong 5 times out of 8 — precision 37.5% at the shipped threshold #61/commitlore-query skill teaches agents an invocation whose documented property is false #69 reopened) against the issue tracker directly, which matched
  • r-rel0200a a074754 — bumping ci.yml's "v0.1.0 was published with zero attached assets" comments | those describe a historical fact about the actual v0.1.0 release, not a version this project declares; the check they document (releases/latest/download/SHA256SUMS returning 200) is written to start exercising the real path automatically the day any release ships assets, v0.2.0 included, with no workflow edit
  • r-rel0200a a074754 — touching docs/adr/ADR-0001-scope-v010.md, docs/tickets/release.md, bench/VERDICT-M1.md, HANDOFF.md, bench/ROUTE-GAP.md | planning and historical-record prose that names v0.1.0 as a past decision or measurement subject, not a live version carrier
  • r-rel0200a a074754 — changing test/mcp.test.ts's CommitLore-Version: 0.1.0 fixture trailer | that's protocol-version content inside a synthetic seed commit (what an old commit's trailer looked like), unrelated to and never asserted against package.json's version
  • r-instci99a 318b9a2 — pre-installing curl/jq/tar/git in the containers before running install.sh | would hide exactly the gap this work exists to find — Debian ships neither curl nor wget by default, and asserting that failure cleanly is the point of the bare debian:stable-slim step, not an obstacle to route around
  • r-instci99a 318b9a2 — publishing a -musl release target so Alpine could install successfully | a release.yml/build-matrix change, not an install.sh or CI-verification fix; out of scope here (DO NOT: no Docker in the release build matrix) and orthogonal to making the existing failure clear instead of a raw crash
  • r-instci99a 318b9a2 — heuristically detecting musl before download (checking for /lib/ld-musl-*.so.1) | executing the freshly extracted, already checksum-verified binary directly is a stronger, more general check — it catches any reason the binary cannot run on this machine, not only the one musl signature this repository happens to know the path for
  • r-instci99a 318b9a2 — a matrix job splitting debian/alpine | the two images need different assertions (debian: bare failure, then success once curl is added; alpine: one clean failure, nothing more to add) and share one staged release + HTTP server; one job with named, ordered steps keeps that shared setup and reads linearly in the log
  • r-fix92dupid 7f41a6e — flagging every duplicate sha in a group regardless of payload, matching parse's labelRecordBlocks exactly | validate's own collision-check array pairs repositoryRecords (already carrying the message's last block once) with a per-block candidate; an unconditional duplicate-sha rule would flag the message's own single last block against its own re-derived copy, a false positive fixed instead by building ownRecords once per message rather than loosening the predicate past correctness
  • r-fix92dupid 7f41a6e — reverting core/query.ts's collectRows dedup key back to sha+source+seq once findIdCollisions was fixed | that key was already wrong on its own terms (it does not match the trailers table's unique index), and leaving it in place would keep silently dropping a commit's second record block from any scoped-path context query, collision or not
  • r-fix90shape 7f5766d — keeping the parent-count signal and also computing it for --message-file via .git/MERGE_HEAD | that only exists while a merge is in progress (the commit-msg hook's own scenario), not when re-validating an already-made merge commit's extracted message -- which is the issue's own reproduction and stays unreconciled under that approach
  • r-fix93pkg 9c4a396 — repointing bin at dist/commitlore.mjs instead of removing it | that entry exists only to serve a package-manager install ADR-0011 already replaced with a git clone; repointing it keeps maintaining a path to an install mode that will never happen
  • r-fix93pkg 9c4a396 — also removing files now | the issue's own "After B-09 · Single static binary — remove the Node runtime dependency #39" note groups bin/files/dependencies as one cleanup once the static binary lands; files alone is inert without a registry rather than actively broken, so doing it now means doing it twice
  • r-distrace88 d118a73 — consolidating the four files' redundant tsc rebuilds into one vitest globalSetup | would remove the race for every file at once and is the more complete root-cause fix, but touches five files' bootstrap plus global config, and mcp.test.ts's beforeAll already tolerates unrelated in-flight tsc errors elsewhere in src/ (OWNED_SOURCES filtering) in a way a single shared build step cannot preserve without its own redesign — bigger blast radius than this issue asked for.
  • r-distrace88 d118a73 — raising bench-ablation's timeout, globally or per-test | proven not to be a timeout problem at all: execFileSync blocks synchronously, so vitest's timeout mechanism cannot even engage against it (verified with a probe test); the actual failure is the runner subprocess legitimately exiting 1.
  • r-parsemulti 6d39d25 — expanding findIdCollisions/index-db to flag a same-message duplicate id generally | reaches into query.ts's lifecycle fold and every consumer route (context, guard, stale) for a case bug-issue-89 only asked parse to surface; the smaller, requested fix is in parse alone.
  • r-multirec01 92aeb24 — minting a fresh Record-Id for an inherited record that never declared one | no other code path in this project invents identity; Record-Id is always author-declared, and doing it here would be new scope beyond what the issue asked for
  • r-multirec01 92aeb24 — a CI step comparing a PR's commits against its post-merge squash commit, as the primary mechanism for finding 1 | needs a GitHub API dependency this tool takes nowhere else, and can only run after the squash is already pushed; doctor catches the same mistake locally, before push, when the source branch is still in refs/heads
  • r-plugman51 db8e2a1 — PreToolUse hook fires twice per edit | Claude Code's plugin loader dedups a manifest.hooks path that resolves (via realpath) to the already-loaded hooks/hooks.json before merging it, so the hooks array is never doubled; the redeclaration instead surfaced as a hook-load-failed PluginError under the loader's default strict mode
  • r-exit065 e545dee — leaving guard's inversion in place and only documenting it | guard and inject are the two commands wired into hooks and CI, so a script written against guard today is already trusting the number; a documented inversion is still an inversion, and it gets load-bearing the longer it sits. The surface is one boolean per command right now — cheaper to fix once than to explain forever.
  • r-m4n560 081d858 — n = 14 per arm | detecting only a two-thirds reduction is not a practically meaningful minimum effect
  • r-6a29d5 31b1524 — keeping the leaking briefs and noting the caveat in the verdict | a null from a saturated design is uninterpretable — the effect could be absent, or both arms could have been told the answer
  • r-4f71c8 74f7f39 — re-running M1's matrix with more seeds | 98 per arm at the observed base rate is 294 runs to answer a question a better instrument answers in 72
  • r-4f71c8 74f7f39 — selecting the three tasks that did produce variance and reporting them | that is the post-hoc subset §4 forbids, and it is the specific dishonesty this file exists to prevent
  • r-2b58d4 4842356 — exempting datasets written before the fields existed | it is one line and it deletes the guarantee
  • r-8e15c2 70f6bcf — hashing only the files the entry point transitively imports | it is the same judgement call that produced this bug, made with more machinery
  • r-9f52d7 2cd02e7 — reporting M3 with a caveat | a treatment that changed mid-run is a confound, not a limitation, and §4 forbids salvaging a result by narrowing the question after the fact
  • r-9f52d7 2cd02e7 — stratifying rows by the dist they ran against | it would compare two arms across two products with n≈50 and no registered hypothesis for either half
  • r-4a17d9 68d4c92 — guard versus the control as the primary test | it would confound the route with whether records exist at all, and the interesting question is which route works better once they do
  • r-4a17d9 68d4c92 — a blocking guard arm | GUARD-CANNOT-BLOCK measured true and false positives in one score band; the only precision-safe threshold catches one in five
  • r-8e04c1 5640748 — re-running M1-b's design at a larger n | it would buy precision on a delivery path the product does not use, which is the same mistake ROUTE-GAP identified and a bigger version of it
  • r-4c73a2 1d82a5b — re-scoring the recorded transcripts and reporting the result | the labels would be chosen with the outcome known, which §4 forbids; the recorded runs were used only to check the surface behaves as intended
  • r-4c73a2 1d82a5b — moving violation_if to code as well | it is instrumented-only, and changing an unregistered metric alongside a registered one makes it unclear which change any difference belongs to
  • r-0d4b81 8005227 — a longer quickstart that demonstrates context, limits, ruled-out, warnings and stale | an agent calls those itself once the MCP server is registered, so listing them teaches the human a workflow that is not theirs
  • r-3e8a41 2f0a8a0 — off-path records that advocate the ruled-out option | with no scoping they reach all three arms, raising re-proposal everywhere and compressing the grading and lifecycle contrasts the set is built to isolate
  • r-2f8b90 a376808 — accept the inherited environment and document it | a memory server between runs is a confound, not a caveat
  • r-2f8b90 a376808 — fail hard when the isolation flags are missing | an older CLI should still produce a run, as long as the log says it was not controlled
  • r-9c2f74 d653153 — resume the pilot into the same file | a new process would load the edited code and create the mixing that had not happened
  • r-9c2f74 d653153 — run the ablation arms as they stand | three nulls from comparing identical inputs read as "these guarantees do not matter"
  • r-9c2f74 d653153 — keep the tasks that showed an effect and rewrite only the rest | the property is the criterion, not the direction of the result
  • r-3f7a29 49817dc — repair a draft that fails verification | backfill's source material is weak enough that a repair loop would mostly be inventing
  • r-3f7a29 49817dc — write reconstructed records into commit messages | history rewriting is irreversible and reaches every existing clone
  • r-3f7a29 49817dc — post a fresh comment per push | it turns the signal into noise and the check gets muted
  • r-5c8b31 60ddc39 — keep turns and explain it in prose | the JSONL outlives the prose, and whoever reads the rows later will not have it
  • r-5c8b31 60ddc39 — drop the turn budget since it cannot be enforced | the overrun is still the signal that a run went off the rails
  • r-6c2b95 5da793c — skip the dry-run driver | then nothing exercises the harness until a key exists, and the first real run debugs the harness instead of measuring anything
  • r-6c2b95 5da793c — emit dry-run rows without a marker | indistinguishable from measurements the moment they leave the terminal
  • r-c0f4e2 3d249cd — GitLore published as git-lore | the binary and search results still collide with the existing gitlore tool
  • r-c0f4e2 3d249cd — keep Annals | the sound problem does not decay, and with code near zero this is the cheapest moment the project will ever have
  • r-c0f4e2 3d249cd — rename code and spec first, documents later | the drift window makes every artifact written in it wrong
  • r-b2e7f1 00d348d — line-matching Key: prefixes | prose containing a colon line parses as a record and feeds agents false context (verified B3)
  • r-b2e7f1 00d348d — content-hash Record-Id | must survive rebase and squash, which a hash of the commit does not

Warnings (44)

  • r-retireserena c1171ef (claim) — bench/results/ and the historical prose in bench/README.md and bench/PREREGISTRATION.md name .serena on purpose -- that is what the recorded runs saw; do not clean those mentions up
  • r-rel071 af8e0ab (claim) — any option declared with a non-nullish default has the same shape, and nothing checks for it -- this fix repairs one site, not the class
  • r-rel070 d4a4d8b (claim) — between the promotion merge and the tag push, every README install one-liner points at a tag that does not exist -- tag promptly and confirm the URL rather than assuming
  • r-numgate b770054 (claim) — the gate matches p =, a pp figure and Fisher exact, so a future statistic in another shape -- an odds ratio, a chi-square -- would pass it and reintroduce the drift
  • r-readmem5 6d04c0b (claim) — the sentence this replaces was the honest one for four releases, and its replacement will need the same treatment the day a task set nobody designed for discrimination gives a different answer
  • r-clog070 172fa3d (claim) — the fixed list cites issue numbers rather than restating each defect, so a reader without network access gets a list of numbers
  • r-4e7c19 6d4e828 (claim) — this closes the attribution half of blocker B only. W4 is still unmeasured, W3 is still unrecoverable for this corpus, and no figure in TOKEN-LEDGER.md section 9 changes
  • r-4e7c19 6d4e828 (claim) — section 5 of TOKEN-LEDGER.md is pre-registration and was appended to, not edited — the floor has to stay visibly declared in advance
  • r-nobin284 ab2f08f (claim) — an extensionless COMMITLORE_BIN is now ignored rather than honoured -- the hook falls through to the recorded install and to PATH, so a caller who pointed it at the wrapper still gets a working hook, by a different route
  • r-hookfailopen 16963dd (claim) — this branch is what keeps a hook failure from blocking an edit -- if it ever needs to change, the replacement must still exit 0 and write nothing to stdout
  • r-recov140 da5a210 (claim) — freeze independent gold before encoding any CommitLore record or the episode is invalid
  • r-release030 510d70e (claim) — Tagging and publishing remain separate release steps.
  • r-f61a2c 9114cf0 (claim) — keyword-strength is IDF-weighted keyword mass multiplied by unweighted distinctive-token coverage; do not restore the old identity label
  • r-exposure52 ba69411 (claim) — guard exposure is instrumented in the benchmark hook adapter, not by changing guard scoring | M4 rows remain unexposed and metrics refuses their effect estimate
  • r-rel0200a a074754 (claim) — scripts/commitlore-bootstrap.sh is orphaned -- no hooks.json entry invokes it, and its npm-install strategy contradicts ADR-0011. It still carries a live version default, now bumped for consistency, but nothing exercises it. Worth a follow-up issue: either wire it up correctly or delete it.
  • r-instci99a 318b9a2 (claim) — the new binary-execution guard's die() message names musl/glibc specifically because that is the only way this repository's own release actually fails to execute today (glibc-only "-gnu" targets, verified against Alpine). If a -musl target is ever published, or the binary can fail to execute for an unrelated reason, this message will misdiagnose it — revisit the wording if that guard ever needs to distinguish causes.
  • r-fix90shape 7f5766d (claim) — a message whose first line happens to start with one of these templates on a non-merge, non-CommitLore commit would also be excused if its trailer-shaped final paragraph were entirely unknown-key -- the same shape of risk bug-issue-76's original parent-count gate carried for genuine merge commits, now traded for one that is at least computable the same way everywhere
  • r-distrace88 d118a73 (claim) — COMMITLORE_BENCH_DIST_DIR is read once at module load (bench/hooks-settings.ts DIST_DIR is a top-level const) — set it before importing the module, not after.
  • r-parsemulti 6d39d25 (claim) — context and validate still do not catch two commit-sourced blocks in the current message sharing one Record-Id unless a notes mirror also disagrees (findIdCollisions requires a notes-sourced record in the group) — parse's check is intentionally independent, not a call into that function, so fixing this in context/validate is separate work.
  • r-multirec01 92aeb24 (claim) — index-db.ts SCHEMA_VERSION is now 2 for the added block column — bump it again, not the shape in place, the next time the trailers table changes.
  • r-plugman51 db8e2a1 (claim) — manifest.hooks/manifest.mcpServers in plugin.json exist only for files beyond hooks/hooks.json and .mcp.json; pointing either at its own conventional path reintroduces a hook-load-failed PluginError (hooks) or a no-op re-merge (mcpServers) — test/manifest.test.ts guards both
  • r-exit065 e545dee (claim) — guard's swap is a breaking change for anything scripted against the old exit 2 for "matched" — see CHANGELOG Unreleased/Breaking
  • r-m4n560 081d858 (claim) — 46/60 is the aggregate over all ten candidates; the eight qualifying tasks total 43/48
  • r-6a29d5 31b1524 (claim) — briefs get shorter and harder to write, and work will be redone that a leaking brief would have got right the first time. That redo cost is the measurement, which is uncomfortable and correct
  • r-4f71c8 74f7f39 (claim) — the prediction that real rejections raise the base rate may be wrong. If fewer than six tasks qualify, M4 does not proceed — a matrix of two tasks is a result about two tasks
  • r-4f71c8 74f7f39 (claim) — a positive result licenses one sentence, that records delivered before an edit reduce revival of a rejected approach. It does not license "makes agents better" or any figure about time saved. This project withdrew one set of numbers for running ahead of its evidence
  • r-2b58d4 4842356 (claim) — this leaves the README with no measured numbers at all until M3-b runs. That is the honest state and it is also a worse first impression. The alternative was publishing numbers produced by a binary nobody recorded
  • r-8e15c2 70f6bcf (claim) — this is the second time in two commits that a fix for the M3 failure did not actually cover the M3 failure. The first was caught by testing the fix against the incident rather than against its own tests
  • r-9f52d7 2cd02e7 (claim) — this is the fourth defect this week of one shape — the thing being verified and the thing actually in play had drifted apart. A check that read a decision's inputs; a doctor that probed the developer's artifact instead of the user's; a scanner keyed to one field of a record whose every field was a surface; and now a benchmark reading a binary its own operator was rebuilding
  • r-4a17d9 68d4c92 (claim) — both hooked arms pay a subprocess per edit. M2 put all three of its timeouts in its hooked arm, and a timeout truncates a run, which flatters whichever arm suffers it — stated before the numbers exist
  • r-8e04c1 5640748 (claim) — M2's numbers do not continue M1-b's — different delivery path, so they are three measurements and not a series
  • r-4c73a2 1d82a5b (claim) — SURFACES in task-loader.ts and MatchSurface in types.ts are two lists of the same thing — adding a surface means editing both
  • r-0d4b81 8005227 (claim)claude mcp add commitlore -- commitlore mcp is Claude Code's syntax — other MCP clients register a stdio server their own way
  • r-8b41e6 0bbad66 (claim) — the payload share and the fixture share are not interchangeable — the payload adds framing and drops what grading and lifecycle withhold, so a reader who swaps one for the other will be wrong by a small, plausible margin
  • r-5c07d2 9ec184a (claim) — the off-path share here is measured over trailer-block characters in the fixtures, not over the assembled payload — the two differ by framing and by what grading and lifecycle withhold, so do not quote one for the other
  • r-3e8a41 2f0a8a0 (claim) — the ablation set is a different synthetic repository from bench/tasks, so its commitlore-on arm must never be pooled with the primary matrix's
  • r-2f8b90 a376808 (claim) — if the runner log carries the "does not support" warning, the rows are not environment-controlled -- do not compare them to rows that are
  • r-9c2f74 d653153 (claim) — after the measurement, check that git status is clean and the recorded sha is still HEAD -- an edit mid-run breaks reproducibility silently, and that check is the only thing that catches it
  • r-3f7a29 49817dc (claim) — every backfilled record is Provenance: reconstructed, which the trust model always renders as a claim -- do not add a path that lets a draft override that field
  • r-5c8b31 60ddc39 (claim) — a driver that gains a real turn limit should stop emitting over-turns and start emitting an enforced label -- do not reuse over-turns for something the harness actually stopped
  • r-6c2b95 5da793c (claim) — never cite a row with simulated:true -- README numbers come from bench/results logs and those rows are not results
  • r-c0f4e2 3d249cd (claim) — ADR-0008 and ADR-0009 keep the literal string Annals on purpose -- mechanical substitution there destroys the decision trail
  • r-c0f4e2 3d249cd (claim) — the residual grep for lore_query reports a false positive because commitlore_query contains it as a substring, so check the prefix
  • r-b2e7f1 00d348d (claim) — spec/fixtures and spec/contract-cases are the contract -- prose in SPEC.md explains them, it does not define them

git log --follow accepts exactly one pathspec, so renames are not followed for 7 paths; query one path at a time to follow its rename chain

Trailer violations fail this check. Active constraints are informational — they are what the repository already decided, not a verdict on this PR.

@MongLong0214
MongLong0214 merged commit d3ba6ca into dev Aug 11, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant