MadSpin: read mg7's decay pools as .npy again, and stop splitting them

[oliviermattelaer]

Reinstates output_format = lhe_npy for decay_generator = mg7, which T145 dropped in favour of LHE.

The objection was wrong, and no sidecar is added

T145's reason for LHE was that a pool has to survive being handed to another process by path alone carrying the channel's partial width, which an LHE <init> block does and a .npy does not -- and that supplying it for a .npy would mean a width sidecar in the owner->waiter publish contract, "the one protocol surface where a publish/open race can live".

Both hops were checked directly:

1. generation fork to parent. Already carries the width explicitly: _generate_decay_entry writes {'files', 'width', 'channel_widths'} into its JSON and _generate_decays hands it back to _reader_from_paths as cross. Predates this change.
2. refill owner to waiters. Nothing has to be published. Note the width here is not unused, as was suggested: get_decay_from_file weights the choice between a pdg's channels by channels[key].cross, and after a refill the reader it reads that off is the refill slice. But a channel's partial width does not change between generations of its pool, so the reader that just ran dry already has it and _worker_refill passes it on in process. ms_refill.gen stays the only thing on disk a waiter looks at.

The .npy removes the split rather than optimising it

A numpy pool is memory-mapped and randomly addressable, so worker i of N takes rows i, i+N, ... by index arithmetic. No per-worker file to write, no other worker's events to parse past. _StridedEvents over LHE text has to next() over them, which is precisely why the pool had to be dealt out into one file per worker first.

This makes PR #86 unnecessary. #86 taught madspace's combine_to_lhe to write N LHE files from C++ so MadSpin would not have to split one; its own author measured 0.34 s / 10k and 1.11 s / 100k and judged that not worth a compiled dependency. With .npy there is no split at all, so there is nothing left for a multi-file writer to optimise -- and none is needed for .npy either.

The three call sites

blocker	real?	what was done
_open_refill_slice builds EventFile(path) directly	yes	dispatches on suffix; takes the channel's width from its caller
_count_lhe_events returns 0 on a .npy	yes, and the nastiest	renamed _count_pool_events, dispatches, has a test. The silent zero capped the owner's slice at zero events so it refilled on its first draw for ever -- it presents as a refill loop, not a type error
_split_pool_round_robin on npy sources	moot	never reached for a numpy pool; now says so instead of raising UnicodeDecodeError three frames inside the LHE parser

_refill_pool_paths tells the layouts apart by what the backend actually left in the refill directory rather than by a flag. _StridedEvents and _LimitedEvents were re-checked on an NpyDecayPool and do still work unchanged (T144's measurement holds), but native striding replaces the former.

Measured

ttbar_full, PA, seed 42, 18 cores, warm, back to back, same production sample. mg7/LHE is 554d8e981 itself, so this is the same tree either side of the format.

	madevent	mg7/LHE	mg7/.npy	npy vs LHE
10 000	35.13 s	20.45 s	19.69 s	-3.7%
100 000	89.08 s	38.10 s	35.74 s	-6.2%

Medians of 3 (10k) and 6 (100k). One 100k pass was slow for all three backends (madevent's me_generation went 9.6 to 15.1 s in the same pass); dropping it gives 38.10 to 35.38 s, -7.1%. Spread otherwise 0.4 s at 10k, 0.7 s at 100k.

By phase at 100k: decay_mg7_split 1.1 s to 0, decay_mg7_launch 14.2 to 12.6 s (madspace writes binary instead of formatting text), decay_loop 15.4 to 14.7 s.

Price: peak RSS 259 to 308 MiB at 100k, and a .npy pool is ~35% larger on disk (every event padded to the longest). Both transient.

Physics agrees -- BR within 0.2% of madevent's 0.917999 throughout.

Refill exercised deliberately

The benchmark triggers no refill at either size, so it was forced: with MADSPIN_OWNER_UNDERSIZE=0.9 the 10k run drove 14+ generations through _worker_refill / _owner_generate / _open_refill_slice, and every Events/ms_refill_<gen> came back holding exactly one events.npy and nothing beside it. BR 0.917343, cross section 462.667 pb.

Tests

MadSpin unit suite 570 to 579, all green. The LHE-pool test class is replaced by npy equivalents, plus new classes for the strided reader, the _count_pool_events silent zero, the refill width hop, and the splitter guard.

🤖 Generated with Claude Code