Make a test name matching nothing an error, and drop the dead CI jobs - #70
Merged
Merged
Conversation
test_manager.py reported OK when a requested name selected zero test, so a CI job kept on being green long after its test had been renamed or deleted. * test_manager.py: TestFinder now remembers what was explicitly asked for (restrict_to) and can tell which of those expressions selected none of the collected tests (unmatched_expressions, using collected_possibility to recognise a selection made by function, class, module or directory name). run() reports them and the script exits 1. A plain name selecting nothing is an error; a regular expression is only a warning, since it is allowed to select nothing, and the bulk modes (no name given, -p U / -p A) are untouched. When the name still exists elsewhere under tests/, the error says where, since that is usually a wrong -p option. * Remove the CI jobs whose test does not exist anymore: - test_read_madgraph4_proc_card: deleted in 9772921 - test_v4_heft: disabled (notest_) in 360366a, v4 not supported anymore - test_eft_running_nlo: disabled (notest_) in 6dc8390. The surviving LO test_eft_running is already run by acceptancetest_madevent.yml - test_tt_semi / test_zh: deleted with madweight in 9d3c959, which did remove those two jobs; a later merge brought them back - testIO_write_cpp_go_process_cc_file / testIO_write_dec_multiprocess_files: deleted with the pythia8 export tests in c83282d, which cleaned unittest.yml but not IOtest.yml * aloha.yml: 'testIO_aloha.*-e' was a missing space, so no testIO_aloha test was ever selected (none exists) and the -e exclusion was inert, running test_short_mssm_subset_creation in the batch it is meant to be kept out of. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
oliviermattelaer
added a commit
that referenced
this pull request
Aug 19, 2026
main brought 8 commits: the SubProcesses parallel build (#67), the dead CI test names and the test_manager change that makes a name matching nothing an error (#70), the aloha TMP eval namespace (#69), a more robust cppauto resolution and a compile log for the mg7 SubProcesses build. Four conflicts, all resolved towards main where it restructured something and towards this branch where it only renamed: - mg7/launch.py: main moved the per-device resolution out of the loop into resolve_cppauto_backend(), with real error handling, and builds the libraries up front. Its version is taken whole; the inline quick fix this branch had renamed no longer exists. - aloha.yml: this branch renamed the launcher, main fixed the test selector (dropping a testIO_aloha.* that never existed and a missing space that made the -e exclusion inert). Both kept. - acceptancetest.yml: main deletes three jobs naming tests that no longer exist; this branch had only renamed the launcher inside them. Deletion kept, which matters now that a name matching nothing is an error. - madgraph_interface.py: main changed the detection file from madmatrix.mk to madmatrix_standalone.mk, because after its restructure both exporters write madmatrix.mk and only the standalone one writes the other. Its file name with this branch's format name, otherwise a plain mg7 output is detected as standalone. main is written in the backend vocabulary this branch renames, and two of those files merged without a conflict: - launch.py arrived with seventeen cppauto references (the function and its name, the make invocation, the regex, the messages), translated to cpu. - madmatrix.mk gained BLDAVXS = cppnone cppsse4 ... from the parallel build. The bldavxs target happened to survive, since $(subst cpp,,cppnone) still gives bldnone, but BLDBACKENDS feeds commonlib.% which runs BACKEND=$* directly, so bldcommonlib died with Invalid backend BACKEND='cppnone': supported backends are ... 'cpu' that is, the dispatcher the parallel build routes through. BLDAVXS now holds the new names and the bld* targets are mapped explicitly, because they keep their historical short names (bldnone, not bldcpu_scalar) and three tests invoke make bldnone directly. Checked: bldall and bldcommonlib resolve to cpu_scalar and cpu_128b on this host with no invalid backend, and a full mg7 run goes from devices = ["cpu"] through "Device 'cpu' resolved as 'cpu_128b'" to events, with main's new compile_subprocesses.log in place. Unit suite 901 tests with only the two known pre-existing failures. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
tests/test_manager.pyreportedOKwhen a requested name selected zero tests, so a CI job kept being green long after its test had been renamed or deleted.test_manager.py
TestFindernow remembers what was explicitly asked for (restrict_to) and can tell which of those expressions selected none of the collected tests (unmatched_expressions, usingcollected_possibilityto recognise a selection made by function, class, module or directory name).run()reports them and the script exits 1.-p U/-p A) are untouched.tests/, the error says where, since that is usually a wrong-poption.Dead CI jobs removed
acceptancetest_28test_read_madgraph4_proc_cardacceptancetest_36test_v4_heftnotest_) in 360366a, v4 not supported anymoreacceptancetest_48test_eft_running_nlonotest_) in 6dc8390 -- the surviving LOtest_eft_runningis already run byacceptancetest_madevent.ymlacceptancetest_65/acceptancetest_66test_tt_semi/test_zhIOtest_cpp_writetestIO_write_cpp_go_process_cc_file,testIO_write_dec_multiprocess_filesunittest.ymlbut notIOtest.ymlaloha.yml
testIO_aloha.*-ewas a missing space, so notestIO_alohatest was ever selected (none exists) and the-eexclusion was inert, runningtest_short_mssm_subset_creationin the very batch it is meant to be kept out of.Validation
Rebased from
claude/fix-dead-ci-test-names(one conflict, inIOtest.yml, where main added theIOtest_fd_gaugejob next to the removedIOtest_cpp_writeone -- both kept/removed as intended). The change content is otherwise identical to the original commit.On this branch:
tests/.test_manager.pyinvocation in.github/workflows/*.ymlwas replayed against the newunmatched_expressionscheck (including themadspin_parallel.ymlmatrix expansion): no surviving job trips the new error. The only skipped invocation is the regex incheck_xsec_processes_mg7.yml, which is warn-only by design../tests/test_manager.py test_test_finder -t0-> 6 tests, OK, exit 0../tests/test_manager.py test_read_madgraph4_proc_card -pA -t0-> error reported, exit 1.🤖 Generated with Claude Code