fix(testing): add ignored-test audit script + weekly cron + policy (#839)#881
Conversation
) Adds visibility for #[ignore]-gated tests: - `ci/audit_ignored_tests.py` walks the workspace and emits CSV (default) or markdown for every #[ignore] / #[ignore = "..."] attribute. Stdlib only; designed to be run via `uv run --no-project python ...`. - `.github/workflows/audit-ignored-tests.yml` runs weekly (Mondays 00:00 UTC) plus on workflow_dispatch, rewriting tracking issue #880 with the current inventory in markdown. - CLAUDE.md gains an "Ignored-test policy" section (after the TDD block) requiring a reason string on every #[ignore] that cites either a tracking issue (#NNN) or a hardware/toolchain requirement. First-seen dates are intentionally deferred — this PR captures the current-state inventory; baseline diffing is a follow-up. Local smoke test (workspace `main`): script reports 39 findings, 20 of which lack a reason string (policy violation per the new policy). Tracking issue: #880 Closes #839 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Warning Review limit reached
Next review available in: 21 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Summary
Adds visibility for
#[ignore]-gated tests across the workspace:ci/audit_ignored_tests.py— stdlib-only Python script that walks the workspace, parses every#[ignore]/#[ignore = \"...\"]attribute, and emits either CSV (default) or markdown. Example CSV row:.github/workflows/audit-ignored-tests.yml— weekly cron (0 0 * * MON) +workflow_dispatch. Runs the script with--markdownand rewrites the body of tracking issue tracking: ignored-test inventory (driven by #839) #880 viagh issue edit. Permissions:issues: write.CLAUDE.md— new "Ignored-test policy (process: ignored-test policy + bitrot audit (#826 followup) #839)" section (placed after the existing TDD block). Every#[ignore]MUST have a reason string referencing either a tracking issue (#NNN) or a hardware/toolchain requirement.First-seen-date tracking is intentionally deferred to a follow-up — this PR ships current-state inventory.
Inventory at time of PR
Run against
main: 39 findings, of which 20 lack a reason string (policy violations under the new policy).Test plan
uv run --no-project python ci/audit_ignored_tests.py --csv | headproduces a well-formed CSV with headerfile,test_name,reason,line.uv run --no-project python ci/audit_ignored_tests.py --markdownproduces a GitHub-flavored table with summary counts.#[ignore]followed by an inline// comment(e.g.crash_decoder.rs:743)..cargo/,target/).gh workflow run audit-ignored-tests.ymlavailable for early validation.Tracking issue: #880
Closes #839
🤖 Generated with Claude Code