mg7: default histograms, plots with bands, and user defaults for the launch switches

[oliviermattelaer]

Stacked on #188 (base fix-output-formats), because the first commit edits what that PR does. Please merge #188 first; the diff here is only the six commits on top of it.

1. Launch switches: user defaults, and the LHE auto-revert restored

#188 turned the silent output_format = 'lhe' override into an error. Put back: asking for a shower in the launch question is enough, a user should not have to know that it constrains output_format.

The switch that made that override visible (analysis defaulting to ExRoot whenever exrootanalysis_path is set — its default value even when nothing is installed) stays off, and gets a proper home instead: input/default_switch.txt, the switch counterpart of input/default_run_card_lo.dat.

analysis = MadAnalysis5
shower   = Pythia8

Read by ControlSwitch, so one file serves the LO madevent, NLO aMC@NLO and mg7 questions; each ignores the keys it does not have. Entries go through the usual validation and consistency resolution — an unavailable program is reported and skipped, and an NLO fixed_order = ON still drags the shower default off. Verified on a real LO and a real NLO output.

Along the way, three things that were quietly broken:

• the run_card default files were read from MG5DIR/internal/, a directory that does not exist — input/default_run_card_{lo,nlo}.dat and input/default_run_card_mg7.toml were therefore never applied (the only test of the feature monkeypatches the path);
• only default_run_card_lo.dat was checked before materialising all three, so an installation predating the mg7 one never received it;
• AskRun.get_allowed_reweight returned None on its first call.

2. [histograms]: a default set of plots per process

An empty section nobody filled in. The observable names are built from the final state, so RunCardMG7.create_default_for_process now writes them at output time: the pt and eta of every final-state particle, the invariant mass of every pair, the partonic sqrt(s), and the distribution of the event weight.

[histograms]
t_1-pt.min = 0.0      t_1-pt.max = 500.0    t_1-pt.bin_count = 50
jet-eta.min = -5.0    jet-eta.max = 5.0     ...
t_1-t_2-pair_mass ... sqrt_s ... weight

A particle goes under a predefined group (jet, lepton, ...) when the run time cannot tell its members apart: MadGraph merges a group's flavours into one subprocess and only the merged representative reaches the observables, so p p > l+ l- is lepton, not e and mu (which would silently select nothing). Anything else gets a [multiparticles] group of its own (t = [6, -6]). Multiplicity is the smallest count over subprocesses — jet_2 in a subprocess with one jet is a hard error in madspace's build_indices, not an empty histogram.

Ranges are order-of-magnitude guesses from the collider energy, or from the decaying particle's mass for a 1 → N width. On p p > t t~ j at 13.6 TeV every histogram holds the cross section with under 1.2% in the overflow bin.

weight is the one key that is not an observable of the momenta. madspace gained EventHistogramSpec::from_weight for it: the event weight over the reference weight (the cross section), so the axis does not depend on the process — an unweighted sample is a spike at 1, a partially unweighted one shows its spread.

In the launch question, set histograms OFF removes them all and set histograms default puts the generated ones back.

3. Plots, with the bands, in Events/<run>/plots

Every run now draws one PDF per observable and says where:

combining done, 00:00:00.10 wall, 00:00:00.15 cpu
7 plot(s) stored in Events/run_02/plots

Read straight from the event-sample histograms rather than through an HwU file, which is what buys the bands: the scale envelope and the PDF uncertainty are already computed per bin by madspace, while histograms.py has to rebuild the PDF band from the member weights and needs the LHAPDF python module to do it — not installed everywhere, and broken on python 3.14. Each plot is dσ/dx with its statistical errors, the bands behind it, and a second panel with the bands relative to the nominal. The y axis goes logarithmic when the spectrum falls by more than a factor 50.

[run] make_plots, true by default, turns it off. matplotlib is imported lazily and only inside plots.py: a run without it says so once and carries on.

4. MADatLO.HwU (opt-in)

For comparison with NLO: the same distributions in the format aMC@NLO writes as MADatNLO.HwU, so madgraph/various/histograms.py plots and compares them with no conversion step. [run] write_hwu, false by default — it is a second copy of numbers info.json already carries.

The variation columns are labelled muR=1.00 muF=2.00 / PDF= 331901 (and dyn=3 muR=...), which is what makes histograms.py read them as an envelope and a band; the envelope it rebuilds from them is bin-by-bin the one madspace computed. The group a weight belongs to decides its label, not the variation's own fields — a PDF set's central member carries mur = muf = 1 and would otherwise be written as a scale variation, hiding it from the band.

Two conventions applied on the way out, both of them where a converter goes wrong: HwU bins hold dσ/dx, so every column is divided by the bin width; and HwU has no under/overflow bins, so the first and last entries of the madspace arrays are dropped, as an aMC@NLO analysis does.

5. Which python runs a run, and a quiet stdout

bin/generate_events trusted #! /usr/bin/env python3. A run needs the packages of the environment MadGraph7 was started from — matplotlib, the LHAPDF bindings, madspace — and the python3 the PATH resolves to often has none of them; on one setup here the plots were silently skipped for exactly that reason. The launcher now pins sys.executable and, because a shebang is only a default, re-executes itself through that interpreter when it finds it is running somewhere else.

The path is deliberately not resolved through its symlinks: in a virtual environment sys.executable points at the environment, and the base interpreter it links to does not see its site-packages — taking the realpath handed the run a python without matplotlib. A vanished interpreter is ignored rather than fatal, and the gridpack, which is the portable artefact, keeps its /usr/bin/env python3.

matplotlib narrates every font it embeds in a PDF at DEBUG level — 28549 lines for one run here, once anything lets DEBUG through the root logger. Its loggers are pinned to WARNING before the import, since matplotlib logs its own start-up while being imported.

Testing

• Full unit suite: 2130 tests, 1 failure — the pre-existing test_DensityMatrixObservables22 (No module named 'scipy').
• madspace: 1584 passed, 11 skipped for a missing PDF set. A madspace rebuild is needed for the weight histogram (python madspace/install.py --source).
• New: 12 for the switch defaults, 13 for the histogram generator, 10 for the HwU writer (which read the result back with madgraph.various.histograms rather than checking the text against itself), 12 for the plots, 7 for the launcher re-exec, 7 in test_mg7_launch.py for the switches.
• End to end on real outputs: LO e+ e- > mu+ mu-, NLO p p > e+ e- [QCD], mg7 p p > t t~ and p p > t t~ j.

🤖 Generated with Claude Code

[oliviermattelaer]

Superseded by #195, which merges this branch in (default histograms, plots/HwU output, input/default_switch.txt, restored LHE auto-switch). On top of it, #195 adds [run] weighted_histograms (default off) and postprocessing_histograms (default on), and fixes the scan-summary means, which counted the under/overflow bins as in-range bins.