Harden the paired-analysis, PowerPlan, and BCa final benchmark gate

[Punisheroot]

Context

docs/BENCHMARKING.md defines the final economic claim as a powered one-sided
paired study with a 95% BCa interval for the paired cost ratio entirely below
1.0. The current gate models calibration/holdout splits, repetitions, pair
seeds, the two built-in routes, and FrontierDirect versus NeedleMiss.

Today, estimate_required_pairs runs inside final-gate evaluation and requires
at least two valid positive calibration ratios per route. The public campaign
contains only one calibration task and one repetition per route, so it cannot
produce a valid power estimate. At the same time, #6 previously required its
holdout count to come from this issue while this issue required a schedule from
#6, creating a circular contract.

The current observation validation can also omit unmatched or failed pairs,
accept schedule identities not bound to an expected campaign, and calculate a
favorable ratio from incomplete evidence. Power planning and final evaluation
must become two explicit, digest-bound stages.

Expected outcome

A deterministic calibration stage validates exact economic pairs and emits an
immutable versioned PowerPlan containing the required holdout pair count for
each route. The sealed-corpus protocol in #6 freezes a final schedule that references this plan digest.
The final gate then verifies that every scheduled task, route, split,
repetition, seed, and arm is present exactly once before calculating quality,
economics, or a BCa interval.

Calibration is used only for power planning. Holdout observations are used only
for the final claim. Missing, failed, non-positive, duplicated, synthetic-only,
or mismatched evidence fails closed instead of being filtered or replaced.

Scope

• Define a canonical, versioned PowerPlan artifact produced from validated
  calibration observations. Bind corpus/campaign identity, calibration input
  digest, route, baseline/treatment arms, pair-key contract, observed log-ratio
  mean and standard deviation, one-sided alpha, target power, required pairs,
  estimator revision, and artifact digest.
• Add a dedicated power-planning entry point that accepts calibration data only.
  Require complete exact FrontierDirect/NeedleMiss pairs with positive,
  finite costs and at least two valid ratios per claimed route. Reject missing
  counterparts, duplicate arms, mismatched seeds, route/split contamination,
  infrastructure failures, quality failures, and synthetic data presented as
  real calibration evidence.
• Keep the current one-sided alpha = 0.05 and target power 0.90 planning
  semantics unless a separately reviewed statistical change is made. A plan is
  deterministic for identical canonical inputs and records why planning failed
  when variance, effect direction, or sample count is insufficient.
• Make Build the sealed holdout corpus protocol and evaluator boundary #6's frozen final schedule reference the exact PowerPlan digest and
  required pair count for each route. The final gate must reject a missing,
  stale, mismatched, superseded, or synthetic-only plan and any schedule count
  smaller or larger than the frozen plan.
• Strengthen observation validation around a canonical key containing corpus,
  schedule, task, route, split, repetition, pair seed, and arm. Require exactly
  one scheduled observation or explicit failure record for every identity; do
  not infer identities from list order.
• Require exactly one baseline and one treatment observation for every economic
  pair and every scheduled FinalArm::ALL observation for real holdout tasks.
  Reject omissions, extras, duplicates, mismatched seeds, and cross-route or
  cross-split pairing before aggregation.
• Do not filter infrastructure-failed or quality-failed observations into a
  smaller favorable sample. Preserve each failure in the report and make it a
  route failure. Non-positive or absent costs cannot form a ratio.
• Compute the final paired ratio and BCa interval from complete valid holdout
  pairs only. Preserve the existing quality, stale-hit, exact-zero-worker,
  partial-recomputation, zero-main-discovery, and upper_bound < 1.0 gates.
• Record the power-plan digest, required and observed pair counts, calibration
  diagnostics, schedule digest, bootstrap seed/resample count, interval, and
  every validation failure in the machine-readable report.
• Add deterministic and adversarial fixtures for both routes without requiring
  provider calls or real sealed holdout material.

Non-goals

• Creating real holdout tasks/oracles, freezing the evaluator bundle, running
  provider arms, changing budgets, or authorizing paid observations.
• Treating the current one-repetition public campaign as powered calibration or
  reclassifying the already-published -holdout tasks as independent evidence.
• Allowing the corpus preflight to invent a sample size, allowing a runner to
  adapt repetitions after holdout results, or recomputing the power plan from
  holdout observations.
• Changing the economic baseline/treatment semantics, inventing a new route, or
  adding a general-purpose statistics dependency when the existing
  deterministic implementation is sufficient.

Acceptance criteria

• A canonical PowerPlan is emitted only from complete validated
  calibration pairs and contains corpus/campaign/calibration digests,
  per-route required pairs, estimator revision, alpha, target power,
  observed log-ratio statistics, and its own digest.
• Planning fails closed for fewer than two valid positive pairs per route,
  missing/duplicate arms, mismatched pair seeds, non-positive or absent
  costs, infrastructure or quality failure, route/split contamination,
  non-beneficial observed effect, zero/invalid variance, or synthetic data
  presented as real calibration evidence.
• Identical canonical calibration inputs produce an identical PowerPlan;
  any changed observation, route, pair seed, estimator parameter, corpus, or
  campaign identity changes the digest.
• The final schedule references exactly one accepted PowerPlan and exactly
  matches its required pair count per route. Missing, stale, mismatched,
  synthetic-only, under-sized, or over-sized schedules fail before ratio or
  interval calculation.
• Every scheduled holdout identity has exactly one observation or explicit
  failure record, including every FinalArm::ALL arm. Missing, duplicate,
  extra, unmatched, cross-route, or cross-split records fail rather than
  being filtered out.
• Calibration cannot contribute to the holdout ratio, BCa interval, quality
  gate, or staleness gate; holdout observations cannot change the frozen
  power plan or schedule.
• Infrastructure failure, quality failure, stale hit, invalidated-node
  mismatch, worker-count violation, covered-scope discovery, and missing or
  non-positive cost are explicit route failures and remain visible in the
  report.
• The report records plan and schedule digests, valid/required pair counts,
  calibration diagnostics, holdout ratio, BCa seed/resamples and interval,
  plus all validation failures.
• Deterministic fixtures for both built-in routes pass only when the frozen
  plan and complete schedule are satisfied and the current two-sided 95%
  BCa interval has an upper bound below 1.0 for every claimed route.
• Adversarial tests cover missing counterpart, duplicate arm, duplicate
  identity, mismatched seed, route/split contamination, calibration
  leakage, holdout-to-plan leakage, extra repetition, stale plan, schedule
  mismatch, non-positive cost, quality failure, and infrastructure failure.

Validation

cd crates/needle-app/web
npm ci
npm run build
cd ../../..
cargo fmt --all -- --check
cargo test --locked -p needle-bench final_gate
cargo test --locked -p needle-bench --test publication_surface
cargo run --locked -p needle-app -- experiment final-report target/final-observations.jsonl --corpus benchmarks/corpus/router-cache/manifest.json --bootstrap-resamples 10000 --seed 42 --output target/final-gate-report.json

The final-report command is an offline fixture/report check. It must not launch
a provider. The checked-in public corpus and synthetic plans remain ineligible
for an economic claim.

Starting points

• crates/needle-bench/src/final_gate.rs - FinalObservation,
  estimate_required_pairs, validation, pairing, BCa calculation, and
  evaluate_final_gate.
• crates/needle-bench/src/corpus.rs and Build the sealed holdout corpus protocol and evaluator boundary #6's answer-free schema - corpus,
  evaluator, schedule, and preflight boundaries.
• benchmarks/corpus/router-cache/manifest.json and campaign.json - current
  public development fixtures and one-repetition non-statistical campaign.
• docs/BENCHMARKING.md - paired-task, evidence-level, paid-run, power, and
  final statistical gate contracts.

Dependencies

#6 owns the answer-free
corpus/evaluator protocol and the final schedule that references the
PowerPlan. This issue owns the plan artifact and final paired-analysis gate.
Both can be implemented and tested in parallel using synthetic fixtures: #7
does not require a real #6 bundle to produce or validate a synthetic plan, and
#6 does not derive a real sample size itself.

The reproducible FinalObservation runner consumes the accepted #6 schedule
and this issue's gate. A future maintainer-only campaign supplies fresh private
holdouts, real calibration observations, an approved provider budget, and any
publishable interpretation.

Ownership and risk

The benchmark/statistics owner owns planning, pair identity, validation,
calculation, report schema, and adversarial fixtures. The principal risk is a
false positive caused by optional stopping, silently dropping failures, pairing
unlike observations, or accepting a schedule not bound to the frozen plan.
Fail closed and preserve every offending record for diagnosis.