chore(eval): cross-session memory eval + the OptMem evaluation it settled

[justrach]

No behaviour change. Adds the harness that answers a question neither existing one can: after a session ends, can a fresh one recover what the first one learned?

live-ab measures token economics within a run; golden proves byte-identity of rendering. Neither crosses a session boundary, and neither exercises compaction more than incidentally. This validates #441, #391 and #411 under the conditions they were built for — and it reports 12/12 cold recall through 13 compactions with 0 fabricated tokens on v0.0.242.

The OptMem answer: not adopting

Evaluated because OptMem solves a real gap — a navigable hierarchical index over an append-only log — that #441's flat transcript does not.

	base (v0.0.242)	+ OptMem
cold recall	12/12	8/12
steps completed	12/12	8/12 — budget exhausted
calls per completed step	3.2	7.5
input tokens	198,864	332,925 (+67%)
compactions	13	21

Its retention was flawless — 8 of 8 it stored survived the session boundary. It did not forget; it ran out of budget at step 8 of 12. The mechanism is the whole result: the base agent persisted with python3 probe.py step 9 | tee -a .probe-token-chain, folding durability into a tool call it was already making at zero marginal cost, while OptMem needs a separate memo note per fact plus a wake and prompted compressions. That is the #440 doctrine reached by the model unprompted, without a subsystem.

The tool is recorded, not vendored: the comparison arm needs LH_MEMO_BIN pointing at a binary the operator supplies.

Four design properties, three found the hard way

1. Tokens are erased from disk when printed — grep -r afterwards finds nothing. Without this the eval measures re-reading, and every arm scores full marks by reading the file again.
2. Steps are chained, not numbered. Measured, not assumed: a predictable 1..N sequence gets batched into single turns — 14 steps produced 9 api calls and 3 compactions. Chained, the same 14 produced 45 calls and 15 compactions.
3. GRAFF_COMPACT_PCT, not GRAFF_CONTEXT. The latter applies only to models whose window graff cannot look up, so against a known-window model it is silently ignored and the run compacts zero times while appearing to work.
4. Filler stays under the Tool results return handles by default above a size threshold (filesystem as the namespace) #440 handle threshold, so it accumulates in context rather than spilling to a handle.

Scoring separates three things

correct (matched against its own label), loose (present anywhere), and fabricated (label carrying the wrong value). loose caught a run scoring 0/14 exact that had actually retained all 14 — a labelling artifact, not amnesia. fabricated exists because on a cold recall an agent that invents plausible values is worse than one that admits it cannot recover them, and a hits-only score hides that.

An invalid round, recorded deliberately

The first attempt was single-session. Both arms scored 18/18 and OptMem looked like +10.7% cost for no benefit — tidy, plausible, and entirely unfounded: memo note was called zero times and LOG.txt was empty. Two findings survive it: the model ignored instructions marked "mandatory" despite a correct AGENTS.md and an executable at the documented path, which is an adoption risk for any memory layer depending on the model volunteering to write; and a single-session test cannot evaluate a cross-session tool.

The general lesson, and why memo_notes/memo_log_lines are in the record: check the mechanism was exercised before believing the score.

Caveats stated in the results file

n=1 per arm, so deltas are directional. The 60-call cap is a chosen parameter — OptMem would likely have finished without it — though the 2.3× per-step cost ratio is cap-independent. Both arms were told a later session would need the data, which does real work: in the un-hinted round the agent persisted nothing at all.