Skip to content

Assert the secret-history gate is real, and stop discarding flawed_regex - #790

Merged
hyperpolymath merged 1 commit into
mainfrom
fix/secret-history-coverage
Sep 15, 2026
Merged

hyperpolymath merged 1 commit into
mainfrom
fix/secret-history-coverage

Conversation

@hyperpolymath

Copy link
Copy Markdown
Owner

What this does

Phase 1c of the hypatia campaign, and the first arm of the owner's "2 then 1"
sequencing ruling — finish arming the scanner at source before spending fleet PRs.

Three changes, all zero-PR: hypatia is resolved by git ls-remote … HEAD at
runtime by hypatia-scan-reusable.yml, so these reach every consumer on its next
scan. Severities were chosen by measurement, not by taste.

1. The secret-history coverage rule

A deduplicated local census (579 raw roots → 573 distinct repos after deduping
on git rev-parse --git-common-dir) found 409 hypatia consumers, 398
secret-scanner callers, and 11 repos with no secret-history gate at all
.

Rather than reimplement history scanning in Elixir alongside the existing pinned,
checksum-verified gitleaks rail, this asserts the gate exists and can actually
see history
:

Rule Severity Condition
missing_secret_history_scan :medium No rail call, no direct scanner. Fires on the 11.
secret_scan_without_history :low A scanner exists but is --no-git only (:add_history_pass) or fed by a checkout without fetch-depth: 0 (:set_fetch_depth_zero).

:medium keeps it visible at the default threshold while staying below the
reusable's blocking high/critical refusal
— the 11 repos get a visible finding,
and none gets a permanently-red required check with no PR to review.

It deliberately does not assert secrets: inherit. The rail's own header still
requires it, but lines 44-49 record that #500 replaced the gitleaks action with a
pinned binary and that the guidance "became wrong when the binary replaced it —
but it was left in place."
Probing for it would false-positive fleet-wide.

2. GS008 — shallow-clone instrument health

A history rule that sees no history is a fake gate by construction, so the
coverage rule needs a companion reporting on the scan environment. GS008 fires
when .git/shallow exists, marking every history-dependent finding in that report
as vacuous rather than as a pass.

⚠️ The anchor is load-bearing and counter-intuitive. file: must not be
.git/shallow, however naturally that reads. .git/ is in @universal_excludes
(scanner_suppression.ex), so a finding anchored there is silently deleted by
the path filter between the rule module and the CLI output
. Measured: with that
anchor the rule was a complete no-op on a real git clone --depth 1 while
passing a full-clone test perfectly. The exclusion exists to avoid scanning files
inside .git/; it also eats findings about it. Anchored at . (matching
GS005/GS007) and pinned by a dedicated regression test.

:medium not :high because the condition is controlled by one shared line in
standards
— at :high, a single regression there would redden every consumer
simultaneously. It is a filesystem probe rather than
git rev-parse --is-shallow-repository because an unguarded System.cmd raises
ErlangError :enoent — the exact defect that got secret_scanner_verification.ex
deleted.

3. Restore the discarded flawed_regex findings

Pre-existing defect, found while reading. flawed_regexes was computed at the top
of audit/3 and summed into flawed_regex_count:, but was never added to the
findings: chain
— so the rule ran on every scan in the estate, reported a count
nothing consumes, and discarded every finding it produced. cli.ex's own
normalizer comment names flawed_regex as a source it handles, so the omission was
accidental, not a deliberate mute. Gate-safe: the rule emits :low.

Verification

  • Four-arm planted-positive control on GS008 — repo as-is (silent) · real
    git clone --depth 1 (fires) · git fetch --unshallow (silent again) · and
    arm 4 against the escript being replaced, where none of the new findings appear.
  • All five new-or-restored emitters were EXECUTED, not merely compiled. test/
    is path-suppressed under @training_corpus_paths, so a fixture placed there would
    have faked every arm; planted fixtures live outside it and ran at --severity info
    (info is rank 5 — --severity low would have hidden them).
  • Real-estate validation: 10 of the 11 predicted gap repos emit exactly one
    missing_secret_history_scan. The 11th (me-dialect) was explained, not waved
    away
    — a concurrent writer added a pinned trufflehog after the census ran, and
    direct invocation confirms the rule correctly returns secret_scan_without_history
    / :set_fetch_depth_zero instead. 8 of 8 rail-calling control repos stay silent.
  • No new fleet noise: the whole secret_scan_without_history class is invisible
    at the default severity threshold.
  • mix test — 1585 tests, 2 failures: the same two pre-existing
    ResearchExtensionsTest re001_missing_harden_runner failures present on main
    without these changes. Zero regressions.
  • Escript soundness gate 14/14, rc=0. mix format --check-formatted clean.
    Compiles clean under --warnings-as-errors.

Known limit, stated rather than worked around

No soundness-manifest entries were added for the new rules. An absence finding
names no file, and the manifest matcher requires a real fixture file — contorting
the anchor to satisfy the gate would be worse than documenting the limit.
missing_secret_history_scan does execute inside the soundness scan and is
message-checked by the placeholder assertion.

🤖 Generated with Claude Code

https://claude.ai/code/session_011YTttTxnPc1V2sATvzBDN7

Three changes, all at source. hypatia is resolved by `git ls-remote ... HEAD`
at runtime by the scan reusable, so these reach every consumer on its next
scan with zero PRs. Severities were chosen by measurement, not by taste.

1. `check_secret_history_coverage/1` (workflow_audit.ex)

A deduplicated local census on 2026-09-15 -- 579 raw roots collapsing to 573
distinct repos after deduping on `git rev-parse --git-common-dir` -- found 409
hypatia consumers, 398 secret-scanner callers, and 11 repos with no
secret-history gate of any kind. Rather than reimplement history scanning in
Elixir alongside the existing pinned, checksum-verified gitleaks rail, this
asserts the gate exists and can actually see history:

  - `missing_secret_history_scan` (:medium) -- no rail call and no direct
    scanner. Fires on the 11.
  - `secret_scan_without_history` (:low) -- a direct scanner exists but is
    restricted to `--no-git`, or is fed by a checkout without `fetch-depth: 0`.
    Two actions, `:add_history_pass` and `:set_fetch_depth_zero`.

:medium keeps it visible at the default threshold while staying below the
reusable's blocking high/critical refusal: the 11 repos get a visible finding
and none gets a permanently-red required check with no PR to review.

It deliberately does NOT assert `secrets: inherit`. The rail's own header still
requires it, but lines 44-49 record that #500 replaced the gitleaks action with
a pinned binary and that the guidance "became wrong when the binary replaced it
-- but it was left in place." Probing for it would false-positive fleet-wide.

2. GS008 shallow-clone probe (git_state.ex)

A history rule that sees no history is a fake gate by construction, so the
coverage rule needs a companion that reports on the scan environment. GS008
fires when `.git/shallow` exists, marking every history-dependent finding in
that report as vacuous.

The anchor is load-bearing and counter-intuitive. `file:` MUST NOT be
`.git/shallow`, however naturally that reads: `.git/` is in
`@universal_excludes` (scanner_suppression.ex), so a finding anchored there is
silently deleted by the path filter between the rule module and the CLI output.
Measured -- with that anchor the rule was a complete no-op on a real
`git clone --depth 1` while passing a full-clone test perfectly. The exclusion
exists to avoid scanning files *inside* `.git/`; it also eats findings *about*
it. Anchored at `.` (matching GS005/GS007) and pinned by a regression test.

:medium, not :high, because the condition is controlled by one shared line in
`standards`. At :high a single regression there would redden every consumer
simultaneously. It is a filesystem probe rather than
`git rev-parse --is-shallow-repository` because an unguarded `System.cmd`
raises `ErlangError :enoent` -- the exact defect that got
`secret_scanner_verification.ex` deleted.

3. Restore the discarded `flawed_regex` findings (workflow_audit.ex)

Pre-existing, found while reading. `flawed_regexes` was computed at the top of
`audit/3` and summed into `flawed_regex_count:`, but was never added to the
`findings:` chain -- so the rule ran on every scan in the estate, reported a
count nothing consumes, and discarded every finding it produced. The cli.ex
normalizer's own comment names `flawed_regex` as a source it handles, so the
omission was accidental, not a deliberate mute. Gate-safe: the rule emits
:low, below both the blocking threshold and the default `--severity medium`.

Verification

Four-arm planted-positive control on GS008, including a real
`git clone --depth 1` and `git fetch --unshallow`, plus arm 4 against the
escript being replaced (none of the new findings appear). All five new or
restored emitters were EXECUTED against planted fixtures at `--severity info`,
not merely compiled -- `test/` is path-suppressed, so a fixture placed there
would have faked every arm.

Real-estate validation: 10 of the 11 predicted gap repos fire exactly one
`missing_secret_history_scan`; the 11th (me-dialect) was explained, not waved
away -- a concurrent writer added a pinned trufflehog after the census ran, and
direct invocation confirms the rule correctly returns `secret_scan_without_history`
/ `:set_fetch_depth_zero` instead. 8 of 8 rail-calling control repos stay silent,
and the whole class is invisible at the default severity.

1585 tests, 2 failures -- the same two pre-existing `ResearchExtensionsTest`
re001 failures present on main without these changes. Soundness gate 14/14,
rc=0. Compiles clean under --warnings-as-errors.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011YTttTxnPc1V2sATvzBDN7
@coderabbitai

coderabbitai Bot commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

Next included review available in 14 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 8c3d2eec-5786-4106-82d8-2a9e1d7d8a81

📥 Commits

Reviewing files that changed from the base of the PR and between d250140 and 3793344.

📒 Files selected for processing (3)
  • lib/rules/git_state.ex
  • lib/rules/workflow_audit.ex
  • test/rules/secret_history_coverage_test.exs

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@hyperpolymath
hyperpolymath merged commit a3b31ed into main Sep 15, 2026
26 checks passed
@hyperpolymath
hyperpolymath deleted the fix/secret-history-coverage branch September 15, 2026 08:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant