Assert the secret-history gate is real, and stop discarding flawed_regex - #790
Merged
Merged
Conversation
Three changes, all at source. hypatia is resolved by `git ls-remote ... HEAD`
at runtime by the scan reusable, so these reach every consumer on its next
scan with zero PRs. Severities were chosen by measurement, not by taste.
1. `check_secret_history_coverage/1` (workflow_audit.ex)
A deduplicated local census on 2026-09-15 -- 579 raw roots collapsing to 573
distinct repos after deduping on `git rev-parse --git-common-dir` -- found 409
hypatia consumers, 398 secret-scanner callers, and 11 repos with no
secret-history gate of any kind. Rather than reimplement history scanning in
Elixir alongside the existing pinned, checksum-verified gitleaks rail, this
asserts the gate exists and can actually see history:
- `missing_secret_history_scan` (:medium) -- no rail call and no direct
scanner. Fires on the 11.
- `secret_scan_without_history` (:low) -- a direct scanner exists but is
restricted to `--no-git`, or is fed by a checkout without `fetch-depth: 0`.
Two actions, `:add_history_pass` and `:set_fetch_depth_zero`.
:medium keeps it visible at the default threshold while staying below the
reusable's blocking high/critical refusal: the 11 repos get a visible finding
and none gets a permanently-red required check with no PR to review.
It deliberately does NOT assert `secrets: inherit`. The rail's own header still
requires it, but lines 44-49 record that #500 replaced the gitleaks action with
a pinned binary and that the guidance "became wrong when the binary replaced it
-- but it was left in place." Probing for it would false-positive fleet-wide.
2. GS008 shallow-clone probe (git_state.ex)
A history rule that sees no history is a fake gate by construction, so the
coverage rule needs a companion that reports on the scan environment. GS008
fires when `.git/shallow` exists, marking every history-dependent finding in
that report as vacuous.
The anchor is load-bearing and counter-intuitive. `file:` MUST NOT be
`.git/shallow`, however naturally that reads: `.git/` is in
`@universal_excludes` (scanner_suppression.ex), so a finding anchored there is
silently deleted by the path filter between the rule module and the CLI output.
Measured -- with that anchor the rule was a complete no-op on a real
`git clone --depth 1` while passing a full-clone test perfectly. The exclusion
exists to avoid scanning files *inside* `.git/`; it also eats findings *about*
it. Anchored at `.` (matching GS005/GS007) and pinned by a regression test.
:medium, not :high, because the condition is controlled by one shared line in
`standards`. At :high a single regression there would redden every consumer
simultaneously. It is a filesystem probe rather than
`git rev-parse --is-shallow-repository` because an unguarded `System.cmd`
raises `ErlangError :enoent` -- the exact defect that got
`secret_scanner_verification.ex` deleted.
3. Restore the discarded `flawed_regex` findings (workflow_audit.ex)
Pre-existing, found while reading. `flawed_regexes` was computed at the top of
`audit/3` and summed into `flawed_regex_count:`, but was never added to the
`findings:` chain -- so the rule ran on every scan in the estate, reported a
count nothing consumes, and discarded every finding it produced. The cli.ex
normalizer's own comment names `flawed_regex` as a source it handles, so the
omission was accidental, not a deliberate mute. Gate-safe: the rule emits
:low, below both the blocking threshold and the default `--severity medium`.
Verification
Four-arm planted-positive control on GS008, including a real
`git clone --depth 1` and `git fetch --unshallow`, plus arm 4 against the
escript being replaced (none of the new findings appear). All five new or
restored emitters were EXECUTED against planted fixtures at `--severity info`,
not merely compiled -- `test/` is path-suppressed, so a fixture placed there
would have faked every arm.
Real-estate validation: 10 of the 11 predicted gap repos fire exactly one
`missing_secret_history_scan`; the 11th (me-dialect) was explained, not waved
away -- a concurrent writer added a pinned trufflehog after the census ran, and
direct invocation confirms the rule correctly returns `secret_scan_without_history`
/ `:set_fetch_depth_zero` instead. 8 of 8 rail-calling control repos stay silent,
and the whole class is invisible at the default severity.
1585 tests, 2 failures -- the same two pre-existing `ResearchExtensionsTest`
re001 failures present on main without these changes. Soundness gate 14/14,
rc=0. Compiles clean under --warnings-as-errors.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011YTttTxnPc1V2sATvzBDN7
Contributor
|
Warning Review limit reachedNext included review available in 14 minutes. View limit detailsLimit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Advanced Run ID: 📒 Files selected for processing (3)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
Phase 1c of the hypatia campaign, and the first arm of the owner's "2 then 1"
sequencing ruling — finish arming the scanner at source before spending fleet PRs.
Three changes, all zero-PR: hypatia is resolved by
git ls-remote … HEADatruntime by
hypatia-scan-reusable.yml, so these reach every consumer on its nextscan. Severities were chosen by measurement, not by taste.
1. The secret-history coverage rule
A deduplicated local census (579 raw roots → 573 distinct repos after deduping
on
git rev-parse --git-common-dir) found 409 hypatia consumers, 398secret-scanner callers, and 11 repos with no secret-history gate at all.
Rather than reimplement history scanning in Elixir alongside the existing pinned,
checksum-verified gitleaks rail, this asserts the gate exists and can actually
see history:
missing_secret_history_scan:mediumsecret_scan_without_history:low--no-gitonly (:add_history_pass) or fed by a checkout withoutfetch-depth: 0(:set_fetch_depth_zero).:mediumkeeps it visible at the default threshold while staying below thereusable's blocking high/critical refusal — the 11 repos get a visible finding,
and none gets a permanently-red required check with no PR to review.
It deliberately does not assert
secrets: inherit. The rail's own header stillrequires it, but lines 44-49 record that #500 replaced the gitleaks action with a
pinned binary and that the guidance "became wrong when the binary replaced it —
but it was left in place." Probing for it would false-positive fleet-wide.
2. GS008 — shallow-clone instrument health
A history rule that sees no history is a fake gate by construction, so the
coverage rule needs a companion reporting on the scan environment. GS008 fires
when
.git/shallowexists, marking every history-dependent finding in that reportas vacuous rather than as a pass.
:mediumnot:highbecause the condition is controlled by one shared line instandards— at:high, a single regression there would redden every consumersimultaneously. It is a filesystem probe rather than
git rev-parse --is-shallow-repositorybecause an unguardedSystem.cmdraisesErlangError :enoent— the exact defect that gotsecret_scanner_verification.exdeleted.
3. Restore the discarded
flawed_regexfindingsPre-existing defect, found while reading.
flawed_regexeswas computed at the topof
audit/3and summed intoflawed_regex_count:, but was never added to thefindings:chain — so the rule ran on every scan in the estate, reported a countnothing consumes, and discarded every finding it produced.
cli.ex's ownnormalizer comment names
flawed_regexas a source it handles, so the omission wasaccidental, not a deliberate mute. Gate-safe: the rule emits
:low.Verification
git clone --depth 1(fires) ·git fetch --unshallow(silent again) · andarm 4 against the escript being replaced, where none of the new findings appear.
test/is path-suppressed under
@training_corpus_paths, so a fixture placed there wouldhave faked every arm; planted fixtures live outside it and ran at
--severity info(
infois rank 5 —--severity lowwould have hidden them).missing_secret_history_scan. The 11th (me-dialect) was explained, not wavedaway — a concurrent writer added a pinned trufflehog after the census ran, and
direct invocation confirms the rule correctly returns
secret_scan_without_history/
:set_fetch_depth_zeroinstead. 8 of 8 rail-calling control repos stay silent.secret_scan_without_historyclass is invisibleat the default severity threshold.
mix test— 1585 tests, 2 failures: the same two pre-existingResearchExtensionsTestre001_missing_harden_runnerfailures present onmainwithout these changes. Zero regressions.
mix format --check-formattedclean.Compiles clean under
--warnings-as-errors.Known limit, stated rather than worked around
No soundness-manifest entries were added for the new rules. An absence finding
names no file, and the manifest matcher requires a real fixture file — contorting
the anchor to satisfy the gate would be worse than documenting the limit.
missing_secret_history_scandoes execute inside the soundness scan and ismessage-checked by the placeholder assertion.
🤖 Generated with Claude Code
https://claude.ai/code/session_011YTttTxnPc1V2sATvzBDN7