fix(ci): the invisible-character gate never matched anything - #78
Conversation
MEASURED 2026-08-27: this gate's pattern caught 0 OF 6 invisible-character test
cases. It has never detected an NBSP, zero-width space, BOM, soft hyphen, bidi
override or word joiner.
ROOT CAUSE: the pattern used UTF-8 BYTE sequences (\xc2\xa0) while grep -P
matches CHARACTERS. Bytes c2 a0 are ONE character U+00A0; \xc2\xa0 asks for TWO
characters, U+00C2 then U+00A0, which is never present.
grep -P '\xc2\xa0' -> miss
grep -P '\x{a0}' -> MATCH
Only \x00 worked, being single-byte in both readings.
FIXED: codepoint escapes; C0 control characters \x01-\x08,\x0B,\x0C,\x0E-\x1F
added (TAB/LF/CR excluded); and grep -a, without which grep skips any NUL-bearing
file as binary.
The C0 range matters: a stray BACKSPACE byte made a workflow unparseable in
developer-ecosystem, so it never ran, and this linter called it clean.
Canonical fix: hyperpolymath/empty-linter#70. 1 file(s) here.
VERIFIED: YAML re-parsed, and the corrected pattern was confirmed to catch a real
NBSP before the change was kept.
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review. 📜 Recent review details⏰ Context from checks skipped due to timeout. (25)
🔇 Additional comments (1)
📝 WalkthroughSummary by CodeRabbit
WalkthroughThe workflow now detects invisible characters with Unicode code-point patterns. It also uses ChangesInvisible-character gate
Estimated code review effort: 1 (Trivial) | ~5 minutes Merge Risk: 🔵 Low · up to The workflow’s invisible-character check is corrected for several Unicode cases, but its BOM pattern may make the check exit with an error instead of matching BOMs on the CI runner. The PR is otherwise localized and mergeable with explicit owner verification of BOM handling. Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Full details: Description checkExplanation The description explains the root cause, the implemented changes, and verification results. It does not follow the repository template headings or complete the required checklist, but it provides sufficient technical context. Full details: Linked Issues checkExplanation The change addresses codepoint escapes, C0 controls, and grep -a for [ Resolution Add the separate leading-BOM check, update the compiled linter and related configuration where applicable, apply the corrected pattern to all required estate copies, and provide evidence for the required detection and non-regression cases, including the backspace-corrupted workflow, clean files, and legitimate whitespace. Full details: Docstring CoverageExplanation No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0 files. (1 skipped: 1 unsupported.)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Up to standards ✅🟢 Issues
|
There was a problem hiding this comment.
Pull Request Overview
The pull request successfully updates the invisible-character gate to use Unicode codepoint escapes and expands the detection range to include C0 control characters. These changes resolve the immediate issue of the gate failing to match intended characters.
However, a significant risk remains: using grep -P with Unicode escapes can cause the command to fail silently when encountering files with invalid UTF-8 sequences. Since stderr is redirected to /dev/null, these files will bypass the check entirely. Additionally, the current implementation is inefficient for large repositories as it spawns a new process for every file.
Finally, while the fix was verified locally, the lack of automated regression tests within the repository makes this gate vulnerable to future regressions.
About this PR
- Regression tests for the invisible-character detection are currently missing from the repository. To prevent future regressions and ensure the gate remains functional, consider adding a set of test files containing the targeted invisible and control characters to the test suite.
Test suggestions
- Verify detection of Non-Breaking Space (U+00A0) using codepoint escape
- Verify detection of Zero-Width Space (U+200B) using codepoint escape
- Verify detection of C0 control character like Backspace (\x08)
- Verify that a file containing a NUL byte (\x00) is processed and the byte is detected instead of the file being skipped
Prompt proposal for missing tests
Consider implementing these tests if applicable:
1. Verify detection of Non-Breaking Space (U+00A0) using codepoint escape
2. Verify detection of Zero-Width Space (U+200B) using codepoint escape
3. Verify detection of C0 control character like Backspace (\x08)
4. Verify that a file containing a NUL byte (\x00) is processed and the byte is detected instead of the file being skipped
TIP Improve review quality by adding custom instructions
TIP How was this review? Give us feedback
| -o -name '*.idr' -o -name '*.zig' -o -name '*.v' -o -name '*.jl' \ | ||
| -o -name '*.gleam' -o -name '*.hs' -o -name '*.ml' -o -name '*.sh' \) \ | ||
| -exec grep -Prl "$PATTERNS" {} \; > /tmp/empty-lint-results.txt 2>/dev/null | ||
| -exec grep -aPrl "$PATTERNS" {} \; > /tmp/empty-lint-results.txt 2>/dev/null |
There was a problem hiding this comment.
🟡 MEDIUM RISK
This command has logic and performance issues. Using grep -P with Unicode escapes causes grep to fail on files containing invalid UTF-8 sequences; because 2>/dev/null is used, these failures are silent and the files bypass the gate. Furthermore, the -r flag is redundant when find provides paths, and using + instead of \; significantly improves performance.
Consider using byte-based patterns (e.g., \xc2\xa0) with LC_ALL=C to ensure all files are scanned regardless of encoding:
| -exec grep -aPrl "$PATTERNS" {} \; > /tmp/empty-lint-results.txt 2>/dev/null | |
| -exec grep -aPl "$PATTERNS" {} + > /tmp/empty-lint-results.txt 2>/dev/null |
Measured 2026-08-27: this gate caught 0 of 6 invisible-character test cases. It has never detected an NBSP, zero-width space, BOM, soft hyphen, bidi override or word joiner.
Root cause
The pattern used UTF-8 byte sequences (
\xc2\xa0) whilegrep -Pmatches characters. Bytesc2 a0are one character U+00A0;\xc2\xa0asks for two, U+00C2 then U+00A0 — never present.Only
\x00worked, being single-byte in both readings. The gate ran, passed, and could not see what it exists to see.Fixed
\x01-\x08,\x0B,\x0C,\x0E-\x1Fadded (TAB/LF/CR excluded)grep -a— without it grep skips any NUL-bearing file as binaryThe C0 range matters: a stray backspace byte made a workflow unparseable in
developer-ecosystem, so it never ran — and this linter called it clean.Canonical fix: hyperpolymath/empty-linter#70. 1 file(s) here.
Verified: YAML re-parsed, and the corrected pattern was confirmed to catch a real NBSP before the change was kept.