fix(ci): diagnostic ratchet skips cards not supported in baseline - #3571
Conversation
The D-09 diagnostic-count ratchet in coverage-regression-check.sh classified a newly-emitting diagnostic (e.g. target-fallback) as a real regression whenever a card's parse_details differed from the baseline — but it never applied the support-delta section's documented guard that "cards absent from the baseline are skipped (new cards don't count as regressions)." During heavy MTGJSON churn (the MSH/MSC Marvel sets + OM2, releasing 2026-06-26) cards are swapped in/out while total_cards stays flat, so the count-based new-card allowance (curr_total - base_total) reads 0 even though brand-new cards are present by identity. A new card has a null baseline parse_details, which reads as "changed" and is miscounted as a real regression. Likewise a card that was already unsupported cannot regress via a diagnostic-category change. Fix: restrict the newly-affected set to cards that were supported in the baseline (mirrors the support-delta guard). Absent and already-unsupported cards are skipped. The genuine supported->unsupported case is still caught here (was_supported true) and independently by the engine bucket. Verified: against a churn baseline, target-fallback +3 real -> +1; against a CI-shaped baseline (the new preview cards unsupported), +3 all honesty-only, exit 0. Engine-regression bucket still reports 0.
There was a problem hiding this comment.
Code Review
This pull request updates the coverage regression check script to exclude cards that were not supported in the baseline from the newly-affected set, preventing false positive regressions. The feedback suggests simplifying the jq expression by leveraging native truthiness instead of explicit boolean comparisons and default fallbacks.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| ($b.cards // [] | map({key: .card_name, value: (.supported == true)}) | from_entries) as $bsup | | ||
| [ | ||
| ($c.diagnostics // {} | keys[]) as $cat | | ||
| ($c.diagnostics[$cat] // 0) as $cc | | ||
| ($b.diagnostics[$cat] // 0) as $bc | | ||
| select($cc > $bc) | | ||
| ($b | cards_emitting($cat)) as $base_cards | | ||
| ($c | cards_emitting($cat)) as $curr_cards | | ||
| (($curr_cards - $base_cards) | unique) as $newly | | ||
| # Only cards supported in the baseline can regress (see comment above): | ||
| # absent / already-unsupported cards are skipped, not counted. | ||
| (($curr_cards - $base_cards) | unique | map(select($bsup[.] // false))) as $newly | |
There was a problem hiding this comment.
[MEDIUM] Simplify the jq expression by leveraging jq's native truthiness.
Why it matters: Using .supported directly and filtering with select($bsup[.]) is more idiomatic and readable than explicit boolean comparisons and default fallbacks.
Suggested fix: See the code suggestion below.
| ($b.cards // [] | map({key: .card_name, value: (.supported == true)}) | from_entries) as $bsup | | |
| [ | |
| ($c.diagnostics // {} | keys[]) as $cat | | |
| ($c.diagnostics[$cat] // 0) as $cc | | |
| ($b.diagnostics[$cat] // 0) as $bc | | |
| select($cc > $bc) | | |
| ($b | cards_emitting($cat)) as $base_cards | | |
| ($c | cards_emitting($cat)) as $curr_cards | | |
| (($curr_cards - $base_cards) | unique) as $newly | | |
| # Only cards supported in the baseline can regress (see comment above): | |
| # absent / already-unsupported cards are skipped, not counted. | |
| (($curr_cards - $base_cards) | unique | map(select($bsup[.] // false))) as $newly | | |
| ($b.cards // [] | map({key: .card_name, value: .supported}) | from_entries) as $bsup | | |
| [ | |
| ($c.diagnostics // {} | keys[]) as $cat | | |
| ($c.diagnostics[$cat] // 0) as $cc | | |
| ($b.diagnostics[$cat] // 0) as $bc | | |
| select($cc > $bc) | | |
| ($b | cards_emitting($cat)) as $base_cards | | |
| ($c | cards_emitting($cat)) as $curr_cards | | |
| # Only cards supported in the baseline can regress (see comment above): | |
| # absent / already-unsupported cards are skipped, not counted. | |
| (($curr_cards - $base_cards) | unique | map(select($bsup[.]))) as $newly | |
The D-09 diagnostic-count ratchet in coverage-regression-check.sh classified
a newly-emitting diagnostic (e.g. target-fallback) as a real regression
whenever a card's parse_details differed from the baseline — but it never
applied the support-delta section's documented guard that "cards absent from
the baseline are skipped (new cards don't count as regressions)."
During heavy MTGJSON churn (the MSH/MSC Marvel sets + OM2, releasing
2026-06-26) cards are swapped in/out while total_cards stays flat, so the
count-based new-card allowance (curr_total - base_total) reads 0 even though
brand-new cards are present by identity. A new card has a null baseline
parse_details, which reads as "changed" and is miscounted as a real
regression. Likewise a card that was already unsupported cannot regress via a
diagnostic-category change.
Fix: restrict the newly-affected set to cards that were supported in the
baseline (mirrors the support-delta guard). Absent and already-unsupported
cards are skipped. The genuine supported->unsupported case is still caught
here (was_supported true) and independently by the engine bucket.
Verified: against a churn baseline, target-fallback +3 real -> +1; against a
CI-shaped baseline (the new preview cards unsupported), +3 all honesty-only,
exit 0. Engine-regression bucket still reports 0.