Skip to content

tests: define aloha TMP temporaries in an explicit eval namespace (fix 9 errors on python 3.13+) - #69

Merged
oliviermattelaer merged 1 commit into
mainfrom
claude/vigorous-kalam-20f954
Aug 17, 2026
Merged

oliviermattelaer merged 1 commit into
mainfrom
claude/vigorous-kalam-20f954

Conversation

@oliviermattelaer

Copy link
Copy Markdown
Contributor

Symptom

Nine tests in tests/parallel_tests/test_aloha.py error out locally with NameError: name 'TMP0' is not defined (or TMP2, ...):

./tests/test_manager.py -p P -l WARNING 'test_aloha.*'
Ran 124 tests ... FAILED (errors=9)
  • testLorentzObject: test_short_expand_veto, test_short_part_spin32propagator, test_short_spin2propagator4
  • test_aloha_creation: test_short_aloha_FFT2, test_short_aloha_FFV, test_short_aloha_FFVP1N, test_short_aloha_FFV_MG4, test_short_aloha_ZPZZ, test_short_use_of_library_spin2

Root cause

Not an aloha bug. Aloha's stringification is correct, and the writers already materialise the temporaries properly (AbstractRoutine.contracted / .fct, built in create_aloha.define_simple_output from KERNEL.reduced_expr2).

The tests stringify an aloha expression and eval() the result. Those strings refer to the TMP.../FCT... temporaries aloha introduces for common subexpressions (aloha_lib.Computation.add_expression_contraction), so each test first defines them with a bare

for name, cexpr in aloha_lib.KERNEL.reduced_expr2.items():
    exec('%s = %s' % (name, cexpr))

That only ever worked by accident. Inside a function, exec() with no explicit namespace writes into the dict returned by locals(). Before PEP 667, CPython cached that dict on the frame and refreshed it from the fast locals without deleting extra keys — so the TMP0 written by exec() survived, and the later eval(), which reads the same cached dict, still saw it.

PEP 667 (python 3.13) makes locals() return an independent snapshot of the function scope. The exec() write goes into a throwaway dict, the next eval() gets a fresh snapshot without TMP0, and it raises NameError.

Minimal reproducer, independent of this repo:

def f():
    exec('TMP0 = 3 + 4')
    return eval('TMP0')

python3.9 gives 7. python3.14 gives NameError: name 'TMP0' is not defined.

So these tests were always fragile — they leaned on a CPython implementation detail — and only now trip. It is worth a real fix rather than writing it off as an environment artefact, because every interpreter from 3.13 onwards hits it.

Why CI is green

.github/workflows/aloha.yml has no setup-python step, so ./tests/test_manager.py (shebang #!/usr/bin/env python3) runs on the ubuntu-24.04 runner's default python3, which is 3.12 — one release before PEP 667. The test selection is identical to the local one; only the interpreter differs.

The fix

Build the namespace explicitly and pass it to both exec() and eval(), which behaves the same on every version. A small module-level helper seeds it from the module globals plus the test's own locals:

def eval_scope(local_vars):
    scope = dict(globals())
    scope.update(local_vars)
    return scope

and each affected site becomes scope = eval_scope(locals()), then exec(..., scope) and eval(..., scope). This is the idiom the generated write_param_card.py files in this repo already use (exec(..., {"cmath": cmath}, local_vars)).

Only the nine failing tests are touched, and the surrounding try/except structure of each site is preserved exactly, so the pre-3.13 behaviour is reproduced rather than changed.

No assertion was loosened. The tests still compare against the same hardcoded MG4 reference values and the same cross-checks between routines.

Verification

./tests/test_manager.py -p P -l WARNING 'test_aloha.*' (124 tests), run on both interpreters available locally:

python 3.9.6 (pre-PEP-667, proxy for CI's 3.12) python 3.14.6 (PEP 667)
before OK FAILED (errors=9)
after OK OK

That 2x2 is the evidence for the diagnosis: the interpreter is the only variable, and the fix is version independent.

Liveness checks, so the green is not vacuous:

  • The temporaries really are generated and really are consumed — test_short_aloha_ZPZZ builds TMP0..TMP7, test_short_aloha_FFV_MG4 builds TMP0/TMP1.
  • Negative control: perturbing the test_short_aloha_ZPZZ reference by one unit makes it fail with AssertionError: 178727040j != 178727041j — i.e. the eval() is genuinely producing the right value through the temporaries.

Full unit suite (-p U): 892 tests, only the two known pre-existing failures (testIO_UnitProcOutputIOTests, order-dependent; test_DensityMatrixObservables22, missing scipy). No regression.

Note on a nearby, separate issue

.github/workflows/aloha.yml:51 is missing a space before -e, so the test_short_mssm_subset_creation exclusion is inert. That is not touched here — a fix already exists on claude/fix-dead-ci-test-names (63466ec45), which has not yet landed on main.

🤖 Generated with Claude Code

Nine tests in tests/parallel_tests/test_aloha.py stringify an aloha
expression and eval() the result. Those strings refer to the TMP.../FCT...
temporaries aloha introduces for common subexpressions, so each test first
defines them with a bare exec('TMPn = ...').

That only ever worked by accident. The assignment landed in the dict handed
out by locals(), which CPython cached per frame, so the later eval() -- which
reads the same dict -- still saw it. PEP 667 (python 3.13) makes locals()
return an independent snapshot of the function scope, so the assignment is
discarded and the eval() raises NameError: name 'TMP0' is not defined.

Build the namespace explicitly instead and pass it to both exec() and eval(),
which behaves the same on every version. This is the idiom the generated
write_param_card.py files already use.

Evidence, `./tests/test_manager.py -p P 'test_aloha.*'` (124 tests):

                     python 3.9 (pre-PEP-667)   python 3.14 (PEP 667)
    before             OK                         9 errors
    after              OK                         OK

CI is green because .github/workflows/aloha.yml pins no interpreter and
ubuntu-24.04 defaults to python 3.12, which predates PEP 667.

No assertion was loosened: the tests still compare against the same hardcoded
MG4 reference values, and they fail if those values are perturbed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@oliviermattelaer
oliviermattelaer merged commit d388566 into main Aug 17, 2026
490 checks passed
@oliviermattelaer
oliviermattelaer deleted the claude/vigorous-kalam-20f954 branch August 17, 2026 16:29
oliviermattelaer added a commit that referenced this pull request Aug 19, 2026
main brought 8 commits: the SubProcesses parallel build (#67), the dead CI test
names and the test_manager change that makes a name matching nothing an error
(#70), the aloha TMP eval namespace (#69), a more robust cppauto resolution and
a compile log for the mg7 SubProcesses build.

Four conflicts, all resolved towards main where it restructured something and
towards this branch where it only renamed:

- mg7/launch.py: main moved the per-device resolution out of the loop into
  resolve_cppauto_backend(), with real error handling, and builds the libraries
  up front. Its version is taken whole; the inline quick fix this branch had
  renamed no longer exists.
- aloha.yml: this branch renamed the launcher, main fixed the test selector
  (dropping a testIO_aloha.* that never existed and a missing space that made
  the -e exclusion inert). Both kept.
- acceptancetest.yml: main deletes three jobs naming tests that no longer
  exist; this branch had only renamed the launcher inside them. Deletion kept,
  which matters now that a name matching nothing is an error.
- madgraph_interface.py: main changed the detection file from madmatrix.mk to
  madmatrix_standalone.mk, because after its restructure both exporters write
  madmatrix.mk and only the standalone one writes the other. Its file name with
  this branch's format name, otherwise a plain mg7 output is detected as
  standalone.

main is written in the backend vocabulary this branch renames, and two of those
files merged without a conflict:

- launch.py arrived with seventeen cppauto references (the function and its
  name, the make invocation, the regex, the messages), translated to cpu.
- madmatrix.mk gained BLDAVXS = cppnone cppsse4 ... from the parallel build.
  The bldavxs target happened to survive, since $(subst cpp,,cppnone) still
  gives bldnone, but BLDBACKENDS feeds commonlib.% which runs BACKEND=$*
  directly, so bldcommonlib died with

      Invalid backend BACKEND='cppnone': supported backends are ... 'cpu'

  that is, the dispatcher the parallel build routes through. BLDAVXS now holds
  the new names and the bld* targets are mapped explicitly, because they keep
  their historical short names (bldnone, not bldcpu_scalar) and three tests
  invoke make bldnone directly.

Checked: bldall and bldcommonlib resolve to cpu_scalar and cpu_128b on this
host with no invalid backend, and a full mg7 run goes from devices = ["cpu"]
through "Device 'cpu' resolved as 'cpu_128b'" to events, with main's new
compile_subprocesses.log in place. Unit suite 901 tests with only the two known
pre-existing failures.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant