orb(public-stats): fleetAccuracy has no minimum instanceCount — at N=1 it publishes one operator's self-report as a fleet aggregate, and the anti-gaming detector it cites cannot fire

[JSONbored]

Problem

fleetAccuracy has a per-instance volume floor but no minimum instance count. With one registered
instance it publishes a single operator's own decision volume under fleet framing — and that operator is
the same party whose guarantee the number substantiates.

computeFleetAnalytics (src/orb/analytics.ts:258) filters:

const eligible = instances.filter((i) => i.decided >= MIN_DECIDED && registered.has(i.instanceId));

MIN_DECIDED bounds each instance's volume. Nothing bounds eligible.length. instanceCount is then
eligible.length (:313), and pooled is a plain sum over eligible (:299-306), so at N=1 the pooled
counts are that one instance's counts. The code says so itself
(src/review/public-stats.ts:471-472):

with one registered instance — the fleet today — pooled and median views coincide exactly.

Live state (not hypothetical)

GET https://api.loopover.ai/v1/public/stats returns 200 today — the flag is on in production:

"fleetAccuracy": {
  "accuracyPct": 94.6, "accuracyCiPct": {"lo": 93.9, "hi": 95.2},
  "mergePrecisionPct": 93.6, "closePrecisionPct": 96.7,
  "coveragePct": 64.7, "decidedCount": 4804,
  "guaranteed": {"close": {"alpha": 0.05, "lambda": 0.62, "coveragePct": 57.1, "n": 129}, "merge": null},
  "instanceCount": 1, "windowDays": 90, "gamingFlagsCaught": 0
}

The numbers are real and instanceCount: 1 is disclosed in the same payload — this is not dishonesty.
The problem is that three separate claims built on top of it do not survive N=1.

What breaks at N=1

1. The median's stated robustness property is absent. analytics.ts:256-257 justifies the design as
"median across REGISTERED instances … robust to a single bad contributor and to unregistered/untrusted
senders." A median of one is that one value. The robustness the comment claims is doing no work, and at
N=2 a median is just the mean of two — still no robustness.

2. gamingFlagsCaught is mathematically unsatisfiable, and is published as evidence. The detector
(analytics.ts:281-296) compares each instance against the fleet median. At N=1 the instance is the
median, so:

• highVolume = i.decided > fleetMedianDecided * GAMING_VOLUME_MULTIPLIER → d > d * m, false for m ≥ 1
• highPrecision = i.mergePrecision - fleetMergeP > GAMING_PRECISION_BAND → 0 > band, false

Both conjuncts are false by construction, so the flag count is structurally 0. Its doc comment
(public-stats.ts:265-267) presents that zero as "proof the fleet actively polices for gaming, not just
a claim of it." A zero that cannot be non-zero is not proof of policing. This is the mechanism behind
#9068's "structurally always 0" observation — that issue reports the symptom; the root cause is that
the detector needs N ≥ 3 to mean anything.

3. It is circular as guarantee evidence. guaranteed.close is published by a registered instance's
own risk-control calibration (#8835), and fleetAccuracy is that same instance's outcomes. The
"fleet-wide" framing invites a reader to treat the two as independent corroboration. They are one
operator's self-report.

What breaks when hosted goes live

This is the reason to fix it now rather than when it becomes urgent.

• Per-tenant volume becomes derivable by subtraction. At N=2, a reader who knows one instance's
  volume (ours is public via byProject/own-ledger totals) gets the other tenant's decision volume
  exactly. At N=3 it is still a tight bound. Publishing pooled counts alongside instanceCount leaks
  more than either alone.
• A hosted tenant's decision volume is their business metric — how many PRs they ship, how many get
  closed. Folding that into a public aggregate needs an explicit tenant-facing decision (opt-in, opt-out,
  or never) made before the first tenant is onboarded, not retrofitted after.
• The anti-farming detector only starts working at N ≥ 3, which is exactly when the fleet contains
  parties we do not control — i.e. when it first matters. Its thresholds have never been exercised
  against real multi-instance data.

Requirements

• Add a minimum instanceCount before fleetAccuracy publishes fleet-framed figures. Below it, either
  suppress the block or relabel it explicitly as single-instance self-report — the honest fallback the
  caller already has for the null case.
• Gate gamingFlagsCaught on N ≥ 3 and publish it as null/absent below that, rather than 0. Correct
  its doc comment: a zero must be distinguishable from "the detector cannot fire".
• Correct the median-robustness comment in analytics.ts to state the N it actually requires.
• Decide the hosted-tenant policy before the first tenant lands and record it here: does a hosted
  tenant's outcome data enter the public fleet aggregate at all, and if so, opt-in or opt-out? If yes,
  suppress pooled counts at small N so per-tenant volume is not derivable by subtraction — publish the
  median view only, or add k-anonymity on decidedCount.
• Exercise the outlier and gaming thresholds against synthetic multi-instance fixtures before N ≥ 3
  happens in production; they are currently untested against the shape they were written for.

Test Coverage Requirements

99%+ patch coverage, branch-counted. Both arms of the instance-count floor; a fixture asserting
gamingFlagsCaught is null (not 0) below the floor; a multi-instance fixture exercising the outlier
and gaming detectors for the first time.

Links & Resources

• src/orb/analytics.ts ~244, ~256-258, ~281-296, ~299-313; src/review/public-stats.ts ~235-270,
  ~448-520 (esp. the N=1 comment at ~471-472)
• orb(data): fleet guarantee is a single last-writer-wins key with no validation, and gamingFlagsCaught is structurally always 0 #9068 (gamingFlagsCaught structurally 0 — this is the root cause and the hosted-transition half),
  gate: distribution-free risk-control act/hold thresholds, separate Neyman-Pearson bounds per arm #8835 (the guarantee this corroborates), metrics: publish per-arm precision, coverage, and Wilson intervals — retire the bare accuracy scalar #8829 (the per-arm/CI/coverage discipline this block follows),
  Epic: ORB Cloud Readiness #4877 (ORB Cloud Readiness — the hosted tenancy that makes the confidentiality half urgent)

Boundaries

Public-stats fleet framing and the hosted-tenant data policy. No change to how any instance computes its
own accuracy, and no change to the risk-control guarantee itself.

maintainer-only — public claims accuracy and tenant confidentiality.

[linear-code]

JSO-6340

[JSONbored]

Hosted-tenant data policy — decided

#9168 asks for this to be recorded before the first tenant lands. Deciding it now rather than leaving it for the onboarding PR to assume.

A hosted tenant's outcomes do NOT enter the public fleet aggregate by default. Entry is opt-in, per tenant, and revocable.

Reasoning, in the order that decided it:

1. A tenant's decision volume is their business metric, not ours. How many PRs they ship and how many get closed is competitive information. We collect it to run their instance; that is not the same as consent to publish it, even folded into an aggregate.
2. Opt-out fails the one test that matters. Opt-out means the first publication happens before the tenant has considered the question — and for a figure that is derivable by subtraction at small N, the first publication is the whole harm. There is no un-publishing.
3. Opt-in costs us very little. The fleet aggregate exists to substantiate a public accuracy claim. Our own instance already supplies that claim honestly, now labelled single_instance_self_report rather than overclaimed as a fleet. Tenants who want to be counted (and some will — being part of a credible fleet number is a reason to opt in) can be, and the number gets stronger for it.
4. It keeps the anti-farming detector honest. A detector running over parties who opted into being compared is defensible. One running over parties who were enrolled silently is not.

What this means concretely, for whoever implements hosted onboarding:

• Registration (orb_instances.registered = 1) is already the fleet trust gate, and it is already a deliberate human action for self-hosted instances (Orb: fleet calibration collector + analytics (retire per-instance App; feed from review_audit) #1255). A hosted tenant must be a separate, explicit consent, not a side effect of provisioning — do not auto-register hosted tenants.
• The consent must be revocable, and revocation must take effect on the next publish (the aggregate is recomputed per request over a rolling window, so this falls out naturally — nothing to backfill or purge).
• The k-anonymity floor shipped here stays regardless of consent. Opt-in makes publication permissible; it does not make a count that isolates one participant safe. Both gates apply.

This is deliberately the conservative branch. If it later proves too strict, relaxing it is a one-line change to a default and a conversation with tenants who already said yes — whereas the opposite mistake cannot be undone.