Skip to content

docs(selfhost): design doc for the scheduled docs-drift audit sweep - #5794

Merged
loopover-orb[bot] merged 1 commit into
JSONbored:mainfrom
galuis116:docs/scheduled-docs-drift-audit-sweep-rfc
Jul 14, 2026
Merged

docs(selfhost): design doc for the scheduled docs-drift audit sweep#5794
loopover-orb[bot] merged 1 commit into
JSONbored:mainfrom
galuis116:docs/scheduled-docs-drift-audit-sweep-rfc

Conversation

@galuis116

Copy link
Copy Markdown
Contributor

Summary

  • Design doc for feat(selfhost): scheduled docs-drift audit sweep for claims the mechanical lint can't check #3048's Phase 1 deliverable ("A design doc/RFC as the first step — this issue can stay open through that phase, then the actual scheduled job once the design is agreed").
  • Covers the gap the two mechanical siblings (scripts/check-docs-drift.mjs presence/shape lint, and the schema-anchored-examples phase) can't catch: behavioral/prose drift, where a docs claim was true when written and quietly stopped being true.
  • Proposes reusing src/upstream/ruleset.ts's fileUpstreamDriftIssues drift-report → consolidated-GitHub-issue pipeline (fingerprinted, update-not-duplicate, kill-switch respecting, already cron-wired at a slow cadence) for the filing half — genuinely new work is scoped to just the detection step: a bounded, rotating per-run slice of docs pages, re-verified against their mapped source through the existing self-host AI provider abstraction (src/selfhost/ai.ts), producing structured findings only with a real file:line citation each.
  • Cites concrete existing precedents throughout by file:line (per house style): fileUpstreamDriftIssues, driftIssueUnchanged, findGitHubIssueForFingerprint, the repo-doc-refresh-sweep/maintainer-recap cron cadence patterns in src/index.ts, and the existing docs.self-hosting-docs-audit.tsx page→source mapping table this sweep's own page list should extend rather than duplicate.
  • Ends with the explicit maintainer sign-off questions the issue's own two-phase structure implies (page-list scope, N-per-run/cadence, new table + flag naming) before Phase 3 (the actual scheduled job) begins.

Scope

  • The PR title follows type(scope): short summary Conventional Commit format, for example fix(api): restore profile access checks.
  • This PR is focused and does not mix unrelated backend, UI, MCP, docs, dependency, and deploy changes.
  • This follows CONTRIBUTING.md and does not reintroduce GitHub Pages, VitePress, site/, or CNAME.
  • I linked a currently open issue this PR resolves (e.g. Closes #123) — a linked open issue is required for every contributor PR.

Closes #3048

Validation

  • git diff --check
  • npm run actionlint
  • npm run typecheck
  • npm run test:coverage locally; codecov/patch requires ≥99% coverage of the lines AND branches you changed (aim for 100% on your diff so CI variance does not fail near the threshold). Global coverage is a non-blocking trend with a loose 90% backstop, not the gate.
  • npm run test:workers
  • npm run build:mcp
  • npm run test:mcp-pack
  • npm run ui:openapi:check
  • npm run ui:lint
  • npm run ui:typecheck
  • npm run ui:build
  • npm audit --audit-level=moderate
  • New or changed behavior has unit/integration tests for new branches, fallback paths, and sanitizer boundaries

All ran via npm run test:ci (the full local gate) plus npm audit --audit-level=moderate, both clean. This PR is a pure documentation addition (src/selfhost/docs-drift-audit-sweep-rfc.md, a single new .md file) — no .ts/.js source changed, so there is no new Codecov patch-coverage surface; docs:drift-check and manifest:drift-check (both part of test:ci) also pass unaffected.

If any required check was skipped, explain why:

  • N/A — the full npm run test:ci chain ran end to end.

Safety

  • No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed.
  • Public GitHub text stays sanitized, low-noise, and does not imply compensation guarantees or optimization tactics.
  • Auth, cookie, CORS, GitHub App, Cloudflare, or session changes include negative-path tests.
  • API/OpenAPI/MCP behavior is updated and tested where needed.
  • UI changes use live API data or real empty/error/loading states, not production mock/demo fallbacks.
  • Visible UI changes include a UI Evidence section below with JPG/JPEG or PNG screenshots arranged as organized, captioned, clickable thumbnails. SVG screenshots are not used as review evidence. Review-only screenshots or recordings are not committed to the repository.
  • Public docs/changelogs are updated where needed; changelogs are only edited for release-prep PRs.

Not applicable: this PR adds a single internal design doc, not a UI/API/auth/session change.

Notes

  • The actual scheduled job (new docs_drift_reports table, the detection job, the filing job, cron wiring) is intentionally NOT implemented here, per the issue's own explicit two-phase structure — a follow-up implementation issue is the natural next step once the design questions in the doc's final section are settled by a maintainer.

…3048)

Phase-3 catch-all for behavioral/prose docs drift that the two
mechanical siblings (presence/shape lint, schema-anchored examples)
can't catch mechanically. Proposes reusing the existing
fileUpstreamDriftIssues drift-report -> consolidated-GitHub-issue
pipeline (fingerprinted, update-not-duplicate, kill-switch respecting,
already cron-wired) for the filing half, with new work scoped to just
the detection step: a bounded, rotating per-run slice of docs pages
re-verified against their mapped source through the existing
self-host AI provider abstraction, structured findings only with a
real file:line citation each.

This is the design-doc phase the issue's own text says can stay open
through -- the actual scheduled job lands once this is agreed.
@galuis116
galuis116 requested a review from JSONbored as a code owner July 14, 2026 16:39
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 14, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.07%. Comparing base (f3d1add) to head (62112da).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #5794   +/-   ##
=======================================
  Coverage   95.07%   95.07%           
=======================================
  Files         581      581           
  Lines       46178    46178           
  Branches    14809    14809           
=======================================
  Hits        43906    43906           
  Misses       1516     1516           
  Partials      756      756           
Flag Coverage Δ
shard-1 43.52% <ø> (-0.43%) ⬇️
shard-2 35.91% <ø> (+0.15%) ⬆️
shard-3 32.19% <ø> (-0.07%) ⬇️
shard-4 32.88% <ø> (-0.13%) ⬇️
shard-5 31.35% <ø> (-0.44%) ⬇️
shard-6 44.99% <ø> (+0.37%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@loopover-orb loopover-orb Bot added the gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. label Jul 14, 2026
@loopover-orb

loopover-orb Bot commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Tip

🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩

✅ LoopOver review result - approve/merge recommended

Review updated: 2026-07-14 16:49:13 UTC

1 file · 1 AI reviewer · no blockers · readiness 100/100 · CI green · clean

✅ Suggested Action - Approve/Merge

  • safe to merge

Review summary
This is a pure design-doc addition (a new .md file under src/selfhost/), proposing a Phase 3 scheduled docs-drift audit sweep that reuses the existing fileUpstreamDriftIssues pipeline for filing and adds a new AI-driven detection step. It correctly stays as documentation-only with no code/schema/migration changes, matches its stated intent as the Phase 1 design-doc deliverable for #3048, and closes with explicit open questions for maintainer sign-off rather than presupposing implementation decisions. Citations to existing code (ruleset.ts, src/index.ts, selfhost-docs-audit.ts) are plausible and consistent with the described precedents, though I cannot verify their exact line numbers from the provided context.

Nits — 6 non-blocking
  • The doc contains garbled unicode ('→' at docs-drift-audit-sweep-rfc.md around the 'One consolidated issue per finding-run' bullet) that should be fixed to a proper en-dash/arrow.
  • Several file:line citations (e.g. ruleset.ts:275-331, src/index.ts:225, src/db/repositories.ts:1665) are asserted precisely but unverifiable from this diff alone — worth a final pass to confirm they still match head before merge, since design docs citing stale line numbers age poorly.
  • The doc doesn't specify where docs-drift-audit-sweep-rfc.md is discoverable from (e.g. no link from a README or existing RFC index), so a nit: consider cross-linking it from wherever other design docs / the feat(selfhost): scheduled docs-drift audit sweep for claims the mechanical lint can't check #3048 issue are indexed.
  • Consider adding a short 'Status' header (e.g. Draft/Proposed) and a merge/close-out note for when maintainer sign-off lands, so the doc doesn't silently go stale once Phase 3 begins.
  • Fix the mojibake character noted above before merge for polish.
  • Diff looks like trivial or whitespace-only churn — Reduce whitespace-only or formatting-only churn and keep the diff focused on substantive changes.
Signal Result Evidence
Code review ✅ No blockers 1 reviewer
Linked issue ✅ Linked #3048
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 1873 registered-repo PR(s), 1236 merged, 51 issue(s).
Contributor context ✅ Confirmed Gittensor contributor galuis116; Gittensor profile; 1873 PR(s), 51 issue(s).
Gate result ✅ Passing No configured blocker found.
Improvement ⚠️ ℹ️ Insufficient signal risk: low · value: insufficient-signal — Nothing measurable for the structural-improvement analyzers on this PR (e.g. no code files changed). LLM value judgment: moderate — The doc is a well-scoped, precedent-citing Phase 1 deliverable for an already-open issue that concretely advances the multi-phase docs-drift initiative without shipping any premature or speculative code.
Linked issue satisfaction

Partially addressed
The PR delivers exactly the issue's explicitly-named first deliverable — a design doc/RFC citing concrete file:line precedents, proposing reuse of the existing drift-report/consolidated-issue pipeline, bounding scope with rotating N-pages-per-run and monthly cadence, and cross-referencing the sibling Phase 1/2 issues as required — but per the issue's own two-phase structure the actual scheduled jo

Review context
  • Author: galuis116
  • Role context: outside_contributor
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: JavaScript, Python, Dart, TypeScript, HTML, MDX, Rust, C++
  • Official Gittensor activity: 1873 PR(s), 51 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Keep the PR focused and include validation evidence before maintainer review.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
[BETA] Chat with Gittensory

Ask Gittensory a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @gittensory ask &lt;question&gt; answers contribution-quality Q&A with source citations and freshness.
  • @gittensory chat &lt;question&gt; answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @gittensory mention with a real question is routed to the closest matching read-only command automatically -- no exact syntax required.

Full command reference: https://gittensory.aethereal.dev/docs/gittensory-commands

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.

  • Re-run LoopOver review

@loopover-orb loopover-orb Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gittensory approves — the gate is satisfied and CI is green.

@loopover-orb
loopover-orb Bot merged commit b83d060 into JSONbored:main Jul 14, 2026
16 checks passed
This was referenced Jul 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(selfhost): scheduled docs-drift audit sweep for claims the mechanical lint can't check

1 participant