Skip to content

init's clean-summary test fails intermittently on node 22, and the probe misreports the cause #192

Description

@MongLong0214

Authority binding — the planning set is merged. Canonical authority is now dev at a333e76f59493ec90e100d1d44c833d7d9e5f093 (PR #220 merged). Exact set: 25 issues #192#213, #215, #216, #219, one ticket each. Evidence bound to any earlier candidate head is void — re-establish the baseline on this dev head before citing a number. Historical evidence SHAs quoted below are left as recorded.

Ticket: T-1030 in docs/tickets/release.md (planning candidate: PR #220a333e76)
Milestone: M5 · Product loop (Gate A) · Gate: A
Baseline to preserve: 45 test files, 1500 passed, 1 skipped; both typechecks exit 0.
Evidence policy: evidence is exact-head bound; an unknown cause is reported as unknown rather than guessed.

One responsibility. This ticket corrects diagnostic honesty; it does not claim to fix the intermittent root cause.


Owns

  • src/commands/doctor.ts — the hook-failure diagnostic at line 453 (at dd6dfe2), specifically the template string the hook fails when git's PATH carries no node: ${said}

Depends on

  • Nothing — this is a standalone honesty fix

Forbidden scope

  • Do NOT fix the intermittent hook failure itself — this ticket owns the diagnostic message only
  • Do NOT modify the release gate, merge anything, or tag anything
  • Do NOT touch src/mcp/server.ts, src/hooks/, or any hook registration logic
  • Do NOT alter exit codes or the check() function signature
  • A root-cause fix for the intermittency is NOT guaranteed by this ticket

RED test

  • File: test/doctor.test.ts (new or addition to existing)
  • Reason: a test simulates a hook that exits non-zero with stderr containing a node stack trace (e.g., at Object.<anonymous> (/path/dist/mcp/server.js:49:1)) and asserts the diagnostic message does NOT claim "git's PATH carries no node". The test must fail on dd6dfe2 because the current code unconditionally interpolates ${said} into the sentence "the hook fails when git's PATH carries no node: ..." regardless of whether the stderr indicates a missing-node problem or a runtime error.

Minimum GREEN

  1. The diagnostic distinguishes at least two cases:
    • stderr suggests node is missing (e.g., node: not found, ENOENT, no stack trace): message says the hook cannot find node
    • stderr contains a node stack trace or other runtime error: message says the hook ran but failed, and quotes the first line of stderr without asserting a cause
  2. If the cause cannot be determined from stderr, the message says so explicitly rather than guessing
  3. The 'ok' path message is unchanged
  4. The diagnostic never names a cause it has not verified from the stderr content

AC <-> test

AC Test assertion Traces to
Node-missing case diagnosed correctly stderr "sh: node: not found" → message mentions missing node #192
Runtime-error case does not claim missing node stderr with stack trace → message does NOT contain "carries no node" #192
Unknown case names uncertainty stderr with unexpected content → message contains "cause unclear" or equivalent #192
OK path unchanged hook exits 0 → message unchanged Forbidden scope

Commands

scope command expected
focused npx vitest run test/doctor.test.ts pass, including new assertions
full npx vitest run 45 files, 1500+ passed, 1 skipped
release npx tsc --noEmit && npx tsc --noEmit -p bench/tsconfig.json both exit 0
manual node dist/commitlore.mjs doctor in a repo with a working hook ok message
LIVE_NA Cannot reproduce #192's intermittent failure on demand — the diagnostic honesty is testable via mock, the intermittency itself is not intermittent by nature

Evidence invalidation

  • Bound to HEAD dd6dfe27ff295c497d73f3cf64fbbd4d75ad5d15. If src/commands/doctor.ts line 453 region changes on another branch first, rebase and re-verify line targeting.

Stop / escalate

  • If the hook-failure diagnostic is used in more places than line 453, audit all call sites before changing the pattern
  • If determining "node is missing" vs "node threw" requires parsing more than the first line of stderr, document the heuristic explicitly and accept false-uncertain over false-confident

Safety checks

check response
fail-closed If the heuristic cannot determine the cause, it says "cause unclear" — never fabricates a diagnosis
wrong-target Only the diagnostic string at ~line 453; a diff touching the hook execution logic itself is wrong
ambiguity The exact current string is quoted above; the replacement must be strictly more honest
timeout Single-file change; no build time risk
partial state The diagnostic is a single code path; atomic
privacy stderr may contain file paths — acceptable to quote as diagnostic context
prompt injection stderr comes from a subprocess, not user text input; but it may contain attacker-controlled content if the hook processes untrusted input — quote only the first line, do not eval

Completion evidence

  • git diff src/commands/doctor.ts shows only the diagnostic string change
  • npx vitest run test/doctor.test.ts passes with all case-distinction assertions
  • Full test suite: 45 files, 1500+ passed, 1 skipped
  • npx tsc --noEmit exits 0

Original incident evidence

test/init.test.ts — "formats a human-readable report with a clean summary line when nothing needs attention" — failed once on node 22 and passed on the same commit in a run started four seconds later. It blocked the release PR's checks until a re-run.

Evidence

Both runs are on dev at 4add1d9, same tree:

run started node result
30492464436 21:27:51Z 22 success
30492468520 (job 90713414577) 21:27:55Z 22 failure

Node 24 passed in both. Subsequent runs on dd6dfe2 passed on both versions, so nothing has been fixed — the failure has simply not recurred.

What the failure was

The test asserts the clean summary line. doctor --fix, step 4 of 4, reported two failures instead of none:

fail    commit-msg hook — installed at /tmp/commitlore-init-format-clean-U99jXx/.git/hooks/commit-msg;
        commitlore.bin: .../dist/cli.js; commitlore.node: /opt/hostedtoolcache/node/22.23.1/x64/bin/node;
        outcome: the hook fails when git's PATH carries no node:
        file:///home/runner/work/commitlore/commitlore/dist/mcp/server.js:49
fail    hook runtime — the hook fails when git's PATH carries no node:
        file:///home/runner/work/commitlore/commitlore/dist/mcp/server.js:49

init: 4/4 steps ran, 1 need(s) attention — doctor --fix (see detail above)

Expected init: 4/4 steps completed cleanly.

What is worth noting

The probe message is the hook fails when git's PATH carries no node: <detail>, and the detail it printed is a node stack location inside dist/mcp/server.js:49 — which is a different failure from "no node on PATH". src/commands/doctor.ts:453 interpolates whatever the hook said into a sentence that has already decided what the cause was:

`the hook fails when git's PATH carries no node: ${said || `exit ${String(run.status)}`}`

So the diagnostic cannot distinguish "node is missing" from "the hook ran node and node threw", and reports the first either way. That is the same defect class as #183, #186 and #187 — a check that answers without saying what it could not see — and it is the reason this failure cannot be diagnosed from the log alone.

What I did not determine

  • Why dist/mcp/server.js:49 throws, or whether the truncation is in the probe or in the log. The full stderr from the hook is not in the run output.
  • Whether the trigger is node 22 specifically or a race in the sandbox setup that node 22's timing makes more likely. Two data points on one node version do not separate those.
  • Whether it can affect a real install, or only the test's temporary repository.

Found while verifying PR #184's checks, not by reproducing it locally — it has not been reproduced locally at all.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions