Skip to content

calibration: pure per-repo corpus slicing + density stats #8215

Description

@JSONbored

Problem

Epic #8211 track B. Every BacktestCase carries its repo inside targetKey (owner/repo#N), but the calibration primitives only ever evaluate globally. Per-repo autonomy (track B's goal) needs one pure building block first: slicing a corpus by repo and knowing which repos have enough labeled density to evaluate at all.

Requirements

⚠️ Required pattern. Pure functions in the engine's calibration package beside buildBacktestCorpus (packages/loopover-engine/src/calibration/backtest-corpus.ts) — the same file-per-concern, 100%-covered, storage-agnostic discipline as #8083-#8087. Root vitest mirror suite per the engine blind-spot rule.

  • sliceCorpusByRepo(cases) → Map<repoFullName, BacktestCase[]>: deterministic, parses the repo from targetKey's last '#', drops unparseable keys (never guesses), preserves case order within each slice.
  • computeRepoCorpusDensity(cases, minVisible, minHeldOut, heldOutFraction, splitSeed) → per-repo {cases, confirmed, reversed, eligible: boolean} where eligible applies the SAME split + sample-minimum discipline the knob evaluators use — a repo is evaluable only if ITS OWN slice clears the floors after splitting.
  • Aggregates only in every returned shape (repo names + numbers — no target keys, no metadata).

Deliverables

  • Both functions exported from the engine package barrel
  • 100% line+branch root mirror suite: multi-repo slicing, unparseable keys, density above/below floors, split determinism per slice

Links & Resources

#8211 (epic), packages/loopover-engine/src/calibration/backtest-corpus.ts, backtest-split.ts

Boundaries

Engine-pure only — no store reads, no registry changes (those are the maintainer follow-ons).

Metadata

Metadata

Assignees

No one assigned

    Labels

    gittensor:featureGittensor-scored feature linked to a feature issue — scores a 0.25x multiplier.help wantedExtra attention is needed

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions