fix(orb): live fleet reversal + reuse telemetry for the homepage hero metrics - #8821
Merged
Conversation
…calibration The fleet pipeline silently dropped reversal_superseded (#8166) at every hop: the self-host exporter's rev CTE only matched reverted/reopened, the collector's ingest whitelist downgraded an unknown flag to 'none', and orb_signals' CHECK constraint would have rejected the value anyway (swallowed by the best-effort insert). Since supersession is the one-shot culture's dominant real reversal shape, the fleet's published reversalRate stayed pinned at 0 and the homepage's reversal-grounded decision accuracy read a degenerate 100%. Advances #8820 (the accuracy-number half; the reuse-rate tile is a separate change). - orb-collector: rev CTE + flag mapping carry 'superseded' (priority reverted > reopened > superseded), regression-tested for the reversal-recorded-after-first-export re-export path - ingest: whitelist 'superseded' - analytics: a superseded close disconfirms closePrecision and counts toward reversalRate exactly like a reopen - migration 0176: rebuild orb_signals with the widened CHECK
…rk-reused trend The homepage reuse-rate trend reads cache hit/miss audit events from the cloud worker's own ledger, which froze at the self-host cutover (last event 2026-06-29) — recent weekly buckets fall under the publish floor, so the hero tile renders a dash beside a decaying sparkline while the live signal (133k+ cache events) accrues unexported on the self-hosted instances. Advances #8820 (the reuse-rate half; the accuracy half is the superseded-reversal export on this same branch's sibling commit). - orb-collector: export day-bucketed hit/miss aggregates (counts only, no repos/PRs) over a 70-day rolling window on the same hourly POST; fail-safe when the ledger lacks the table - ingest: validate (strict day format, clamped non-negative counts) and upsert per (instance, day); malformed rows skipped row-by-row - public-reuse-rate-trend: fold counters from REGISTERED instances into the same weekly buckets, unconditional on the own-ledger repo allowlist (parity with the fleet-accuracy fold) - migration 0177: orb_reuse_counters
Contributor
|
Superagent didn't find any vulnerabilities or security issues in this PR. |
❌ 3 Tests Failed:
View the top 3 failed test(s) by shortest run time
To view more test analytics, go to the Test Analytics Dashboard |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Both engineering halves of #8820 (the homepage hero's broken metrics). Advances #8820 — post-deploy ops verification (cursor reset + live numbers) remains before it closes.
Half 1 — decision accuracy pegged at a degenerate 100%
Production evidence (2026-07-26): the self-host instance recorded 66
reversal_supersededevents (the one-shot culture's dominant real "bot was wrong" shape, #8166), yet all 6,127 exported fleet signals carryreversal_flag='none'. Three stacked drops: the exporter's rev CTE only matched reverted/reopened; the ingest whitelist downgraded the unknown flag to'none'; andorb_signals' CHECK constraint would have rejected the value anyway — swallowed silently by the best-effort insert. FleetreversalRatestayed pinned at 0 → published accuracy exactly 100%.orb-collector: rev CTE + flag mapping carrysuperseded(priorityreverted>reopened>superseded); regression test for the reversal-lands-after-first-export re-export pathingest: whitelistsupersededanalytics: a superseded close disconfirmsclosePrecisionand counts towardreversalRateexactly like a reopen (parity withpublic-stats.ts, which already counts it)0176: rebuildorb_signalswith the widened CHECK (SQLite can't alter one), preserving rowsHalf 2 — "AI work reused" renders a dash beside a live-looking sparkline
The trend reads cache hit/miss audit events from the cloud worker's own ledger, frozen since the self-host cutover (last event 2026-06-29) — recent weeks fall under the 5-sample publish floor → null → dash, while the live signal (133k+ cache events) accrues unexported on the self-host.
orb-collector: export day-bucketed hit/miss aggregates (counts only — no repos, no PRs, no content) over a 70-day rolling window, riding the same hourly POST; fail-safe when the ledger lacks the table; same event population as the trend's own query (sharedAI_REVIEW_REUSE_EVENT_TYPES)ingest: strict validation (YYYY-MM-DD day, clamped non-negative counts, bounded row count) + idempotent upsert per (instance, day)public-reuse-rate-trend: fold counters from REGISTERED instances (the same trust anchor ascomputeFleetAnalytics) into the same weekly buckets, unconditional on the own-ledger repo allowlist — parity with the fleet-accuracy fold0177:orb_reuse_countersNo UI changes needed — both tiles light up from the same fields once live data flows.
Post-merge ops (tracked on #8820)
orb_export_cursorrow so history re-exports with correct reversal flags (INSERT OR REPLACEdedup makes this safe and idempotent).Verification