Skip to content

feat(ui): add verify-this-review walkthrough and measured per-rule accuracy on the fairness report - #8250

Closed
RealDiligent wants to merge 1 commit into
JSONbored:mainfrom
RealDiligent:fix/critical-issue-verify-review-8231
Closed

feat(ui): add verify-this-review walkthrough and measured per-rule accuracy on the fairness report#8250
RealDiligent wants to merge 1 commit into
JSONbored:mainfrom
RealDiligent:fix/critical-issue-verify-review-8231

Conversation

@RealDiligent

Copy link
Copy Markdown
Contributor

Summary

  • Adds the public reproducibility walkthrough doc content/docs/verify-this-review.mdx: export the checksummed backtest corpus snapshot (scripts/backtest-corpus-export.ts, --remote and --pg variants), verify the checksum via buildBacktestCorpusManifest, replay the scorer with scoreBacktest + buildConfidenceThresholdClassifier, and compare against the published numbers — with an honest closing callout that replay proves the numbers are real computations over a real corpus but does NOT prove the live gate ran that exact code (the attestation boundary, deliberately left to Research: is TEE hardware attestation necessary, or does a reproducible-backtest checksum approach cover the real threat? #8136/Research: survey viable TEE/confidential-compute options on Cloudflare Workers/Containers #8137).
  • Renders the Measured accuracy per rule section on the fairness report from the /v1/public/stats rulePrecision block (public-stats: measured per-rule precision endpoint (aggregates only, sanitizer-invariant) #8230): per-rule decided-case counts and precision, reversal-derived numbers, the insufficient data null state (a below-floor precision: null is rendered as the words, never as 0%), and the reproducibility freeze point showing the latest persisted backtest run's corpus checksum, linking to the new walkthrough.
  • rulePrecision is typed as optional on PublicStats and the section hides entirely when the field is absent (older deployed API) or has no rows — deployment skew can never throw.
  • Wires the doc into the docs nav, docs index, and command palette next to the backtest & calibration entry.

Closes #8231

Scope

  • The PR title follows type(scope): short summary Conventional Commit format, for example fix(api): restore profile access checks.
  • This PR is focused and does not mix unrelated backend, UI, MCP, docs, dependency, and deploy changes.
  • This follows CONTRIBUTING.md and does not reintroduce GitHub Pages, VitePress, site/, or CNAME.
  • I linked a currently open issue this PR resolves (e.g. Closes #123) — a linked open issue is required for every contributor PR.

Validation

  • git diff --check
  • npm run actionlint
  • npm run typecheck
  • npm run test:coverage locally; codecov/patch requires ≥99% coverage of the lines AND branches you changed (aim for 100% on your diff so CI variance does not fail near the threshold). Global coverage is a non-blocking trend with a loose 90% backstop, not the gate.
  • npm run test:workers
  • npm run build:mcp
  • npm run test:mcp-pack
  • npm run ui:openapi:check
  • npm run ui:lint
  • npm run ui:typecheck
  • npm run ui:build
  • npm audit --audit-level=moderate
  • New or changed behavior has unit/integration tests for new branches, fallback paths, and sanitizer boundaries

If any required check was skipped, explain why:

  • This diff touches only apps/loopover-ui/** (docs MDX + fairness-report UI + its vitest suite). actionlint (no workflow changes), test:coverage/test:workers (no backend/root source or test changes), build:mcp/test:mcp-pack (no MCP changes), ui:openapi:check (no OpenAPI/route changes), and npm audit (no dependency changes) are not exercised by this change. ui:lint, ui:typecheck, ui:build, and the UI vitest suite (fairness-report-page.test.tsx, 8/8 passing, including the new rulePrecision render/null-state/absent-field cases) all pass locally.

Safety

  • No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed.
  • Public GitHub text stays sanitized, low-noise, and does not imply compensation guarantees or optimization tactics.
  • Auth, cookie, CORS, GitHub App, Cloudflare, or session changes include negative-path tests.
  • API/OpenAPI/MCP behavior is updated and tested where needed.
  • UI changes use live API data or real empty/error/loading states, not production mock/demo fallbacks.
  • Visible UI changes include a UI Evidence section below with JPG/JPEG or PNG screenshots arranged as organized, captioned, clickable thumbnails. SVG screenshots are not used as review evidence. Review-only screenshots or recordings are not committed to the repository.
  • Public docs/changelogs are updated where needed; changelogs are only edited for release-prep PRs.

UI Evidence

State / title JPG/PNG evidence
Fairness report — measured accuracy per rule (desktop, incl. insufficient data null state + freeze point) Fairness report desktop
Fairness report — mobile layout (table scrolls horizontally like the sibling tables) Fairness report mobile
Docs — Verify this review walkthrough (desktop, TOC + steps) Docs page desktop
Docs — mobile layout Docs page mobile

Notes

…ccuracy on the fairness report

Adds the public reproducibility walkthrough doc (export corpus snapshot,
verify checksum, replay scorer, compare against published numbers, with an
honest attestation-boundary callout) and renders the measured accuracy per
rule section on the fairness report from the public stats rulePrecision
block: decided cases, precision with the insufficient-data null state
(never 0%), reversal counts, and the corpus-checksum freeze point linking
to the walkthrough.

Closes JSONbored#8231
@RealDiligent
RealDiligent requested a review from JSONbored as a code owner July 23, 2026 14:47
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@loopover-orb loopover-orb Bot added the gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. label Jul 23, 2026
@loopover-orb

loopover-orb Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Tip

✅ LoopOver review result - approve/merge recommended

Review updated: 2026-07-23 14:55:30 UTC

7 files · 1 AI reviewer · no blockers · readiness 98/100 · CI green · clean

✅ Suggested Action - Approve/Merge

  • safe to merge

Review summary
This PR adds a fairness-report section rendering per-rule precision from the new optional `rulePrecision` field on `PublicStats`, with a proper null-vs-0% distinction and full hide-on-absence/empty-rows deployment-skew handling, plus a new `verify-this-review.mdx` walkthrough doc wired into nav/index/palette. The implementation is correct and tested end-to-end (both the populated and skew/hidden cases), the doc is honest about its reproducibility-vs-attestation boundary, and CI is fully green on this commit. The PR closes #8231 as required and stays scoped to this one feature; no correctness or wiring defects are visible in the diff.

Nits — 5 non-blocking
  • The external brief flags `console.log` in the .mdx code-fence examples (verify-this-review.mdx:49,66) as 'debug leftovers,' but these are illustrative one-off shell commands inside doc prose, not source code shipped by the PR — non-issue, safe to ignore.
  • fairness-report-page.tsx:312 slices the corpus checksum with the bare literal `16`; consider a named constant like `CHECKSUM_PREVIEW_LEN` for clarity, though it's a one-off display concern.
  • The command-palette, docs-nav, and docs-index entries for 'Verify this review' are manually duplicated in three files with no shared source of truth — pre-existing pattern in this file, so not this PR's problem to fix, but worth noting if a fourth surface is ever added.
  • Consider extracting the checksum-preview length in fairness-report-page.tsx into a small named constant for readability.
  • No functional changes needed — this is ready as-is pending confirmation the linked issue docs+ui: 'verify this review' — public reproducibility walkthrough + measured-accuracy display #8231 fully matches the scope delivered.

Decision drivers

  • ✅ Code review — No blockers (1 reviewer)
  • ✅ Gate result — Passing (No configured blocker found.)
Context & advisory signals — never blocks the verdict
Signal Result Evidence
Linked issue ✅ Linked #8231
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 342 registered-repo PR(s), 134 merged, 36 issue(s).
Contributor context ✅ Confirmed Gittensor contributor RealDiligent; Gittensor profile; 342 PR(s), 36 issue(s).
Improvement ✅ Minor risk: clean · value: minor · LLM: moderate
Linked issue satisfaction

Addressed
The PR adds the verify-this-review.mdx walkthrough with checksum export/verify/replay/compare steps and an explicit 'what this proves/does not prove' section citing #8136/#8137, wired into docs nav, docs index, and command palette, plus a fairness-report UI section pulling rulePrecision from the public stats endpoint with a deliberate insufficient-data null state (never 0%) and ui-kit tokens/borde

Review context
  • Author: RealDiligent
  • Role context: outside_contributor
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: Python, Ruby, JavaScript, Svelte, TypeScript, Markdown, MDX, Rust
  • Official Gittensor activity: 342 PR(s), 36 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Keep the PR focused and include validation evidence before maintainer review.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
🧪 Chat with LoopOver

Ask LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @loopover ask <question> answers contribution-quality Q&A with source citations and freshness.
  • @loopover chat <question> answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @loopover mention with a real question is routed to the closest matching read-only command automatically — no exact syntax required.

Full command reference: https://loopover.ai/docs/loopover-commands

🧪 Experimental — new and may change.

Visual preview
Route Viewport Before (production) After (this PR's preview) Diff
/docs desktop before /docs
before /docs
after /docs
after /docs
/docs mobile before /docs (mobile)
before /docs (mobile)
after /docs (mobile)
after /docs (mobile)

Click any thumbnail to open the full-size screenshot. Before = production · After = this PR's preview deploy.

Scroll preview
Route Before (production) After (this PR's preview)
/docs before /docs (scroll)
before /docs (scroll)
after /docs (scroll)
after /docs (scroll)

A short scroll-through clip (desktop) — click either thumbnail to open the full animation. Evidence for scroll-linked behavior a single screenshot can't show.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.

  • Re-run LoopOver review

@loopover-orb

loopover-orb Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

This pull request changes UI/visual code but its screenshot evidence is incomplete. Every required viewport × theme combination needs its own before/after image pair in a labeled table row (e.g. "Desktop · Light | before | after"). Still missing: Desktop · Dark, Tablet · Dark, Mobile · Dark.

Please resubmit with the remaining rows filled in.

See https://github.com/JSONbored/loopover/blob/main/.claude/skills/contributing-to-loopover/SKILL.md for the exact format and examples. This is an automated maintenance action.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

docs+ui: 'verify this review' — public reproducibility walkthrough + measured-accuracy display

1 participant