Skip to content

feat(miner): persist ranked-candidates snapshots and serve them locally - #5619

Merged
JSONbored merged 1 commit into
mainfrom
miner-ranked-candidates-store
Jul 13, 2026
Merged

feat(miner): persist ranked-candidates snapshots and serve them locally#5619
JSONbored merged 1 commit into
mainfrom
miner-ranked-candidates-store

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

  • Adds packages/gittensory-miner/lib/ranked-candidates.js: a new local SQLite snapshot store. discover-cli.js's runDiscover now persists the FULL per-issue ranking breakdown (rankScore/laneFit/freshness/potential/feasibility/dupRisk, from opportunity-ranker.js) on every real (non-dry-run) invocation, replacing the whole snapshot atomically each time — a disposable "the miner's current opinion" cache, not a ledger.
  • Adds apps/gittensory-miner-ui/vite-ranked-candidates-api.ts: a read-only GET /api/ranked-candidates endpoint over that store, authenticated the same way as every other /api/* route via Add auth to the local miner-ui API #4858's authPlugin (registered first in the plugin chain).
  • Wired non-fatally into discover-cli.js: same "own try/catch, degrade gracefully" discipline as the existing policy-doc/policy-verdict caches — an unopenable or unwritable snapshot store never aborts discovery's actual job (fan out, rank, enqueue to the portfolio queue).

Why

#4859 ("Replace the manual copy/paste workflow with a live fetch") wants the browser extension to fetch ranked candidates from the local miner-ui instead of the operator hand-pasting discover --json's output. But nothing durable held that output anywhere — discover --json prints the full breakdown to stdout and it's gone; the portfolio queue only ever stored a single derived priority number, not the per-dimension detail opportunity-badge.js's "why" reasoning needs (laneFit/freshness/potential/feasibility/dupRisk). This PR builds that missing persistence + read API as the prerequisite; the extension's own fetch/fallback logic is a separate follow-up PR.

Advances #4859 (not closing it — the extension-side consumer is still open).

Test plan

  • npm run build:miner, npm run typecheck
  • npm run miner:env-reference:check (new GITTENSORY_MINER_RANKED_CANDIDATES_DB var documented)
  • npm run docs:drift-check, npm run test:miner-pack
  • npm run ui:typecheck, npm run ui:lint (0 errors), npm run ui:build
  • New store tests (test/unit/miner-ranked-candidates.test.ts, 11 cases): round-trip, atomic wholesale replace (a second save wipes the first, never accumulates), neutral-default dimension fallback, invalid-candidate rejection (aborts the whole save, no partial write), and a genuine SQL-level rollback test (a duplicate repo+issue within one save forces a real PRIMARY KEY constraint failure, proving the transaction wrapper actually rolls back)
  • discover-cli.js integration tests (4 new cases in test/unit/miner-discover-cli.test.ts): persists the real snapshot after a run, opens/closes the default on-disk store, and two REGRESSION tests proving an unopenable store or a save failure never fails discovery itself
  • New miner-ui endpoint tests (ranked-candidates-api.test.ts, 8 cases): handler + fresh-install-safe GET path + the actual Vite middleware wiring (configureServer/configurePreviewServer)
  • packages/gittensory-miner/lib/ranked-candidates.js and discover-cli.js: 100% line/branch/function coverage confirmed via lcov on my diff's changed lines specifically (one pre-existing branch gap in discover-cli.js's parseRepoTarget, outside this diff's hunks, is unaffected)
  • Manually started the real miner-ui dev server: populated the store directly, curled GET /api/ranked-candidates authenticated (200, correct shape matching what opportunity-badge.js needs) and unauthenticated (401)

Note: while writing the transactional replace, hit a real bug — node:sqlite's DatabaseSync has no .transaction() helper (unlike better-sqlite3); fixed by mirroring portfolio-queue.js's explicit BEGIN IMMEDIATE/COMMIT/ROLLBACK pattern. Also hit and worked around a v8 coverage-instrumentation quirk where an if (x) { try {} catch {} } shape undercounted a genuinely-exercised branch (verified via direct runtime instrumentation) — restructured to x?.method() inside the try/catch, which both reads cleaner and reports correctly.

Adds packages/gittensory-miner/lib/ranked-candidates.js: a new
snapshot store that discover-cli.js now populates on every real run
with the full per-issue ranking breakdown (rankScore/laneFit/
freshness/potential/feasibility/dupRisk), replaced wholesale each run.
Nothing durable held this before -- discover --json printed it but
never persisted it.

Adds apps/gittensory-miner-ui/vite-ranked-candidates-api.ts: a
read-only GET /api/ranked-candidates endpoint over that store,
authenticated the same way as every other /api/* route via #4858's
authPlugin.

This is the prerequisite for #4859 (extension live-fetch) -- the
extension's opportunity badge needs exactly this per-issue breakdown
to replace its manual copy/paste workflow, and no data source existed
for it until now.
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
loopover-ui 9e15246 Commit Preview URL

Branch Preview URL
Jul 13 2026, 10:38 AM

@codecov

codecov Bot commented Jul 13, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.01%. Comparing base (2e8fdd7) to head (9e15246).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #5619   +/-   ##
=======================================
  Coverage   95.01%   95.01%           
=======================================
  Files         574      575    +1     
  Lines       45684    45738   +54     
  Branches    14661    14661           
=======================================
+ Hits        43405    43459   +54     
  Misses       1528     1528           
  Partials      751      751           
Flag Coverage Δ
shard-1 43.90% <0.00%> (-0.10%) ⬇️
shard-2 35.72% <0.00%> (+0.07%) ⬆️
shard-3 32.13% <83.33%> (-0.09%) ⬇️
shard-4 30.96% <0.00%> (-1.72%) ⬇️
shard-5 32.65% <81.48%> (+0.93%) ⬆️
shard-6 44.42% <3.70%> (+0.31%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
packages/gittensory-miner/lib/discover-cli.js 100.00% <100.00%> (ø)
packages/gittensory-miner/lib/ranked-candidates.js 100.00% <100.00%> (ø)
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 13, 2026
@loopover-orb

loopover-orb Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Warning

🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨

⏸️ LoopOver review result - manual review recommended

Review updated: 2026-07-13 10:43:37 UTC

11 files · 1 AI reviewer · 2 blockers · readiness 93/100 · CI green · clean

⏸️ Suggested Action - Manual Review

  • AI review could not be completed: The dual-model AI review did not return a usable verdict for this change.

Review summary
This PR adds a new local SQLite snapshot store for ranked candidates (packages/gittensory-miner/lib/ranked-candidates.js), wires it non-fatally into runDiscover with the same own-try/catch-degrade discipline as the existing policy caches, and exposes a read-only authenticated GET /api/ranked-candidates endpoint over it. The implementation is careful and consistent with existing conventions (atomic replace via BEGIN IMMEDIATE, fresh-install no-write fast path in the API handler, neutral defaults matching opportunity-ranker.js's own normalization), and the test suite is thorough including a genuine rollback-on-PK-violation test. Per repository convention this is a prerequisite/enabling PR for #4859 rather than a standalone fix — it explicitly does not close #4859, which is a scope concern worth flagging even though the description frames it as authorized prerequisite work.

Nits — 6 non-blocking
  • This PR states it 'advances' but does not close Replace the manual copy/paste workflow with a live fetch #4859 — confirm this prerequisite-PR pattern is acceptable under the repo's issue-linkage policy for external contributions.
  • apps/gittensory-miner-ui/vite-ranked-candidates-api.ts:67 returns the raw error.message in the 500 body (e.g. 'sqlite locked'), which could leak internal path/state details to any authenticated caller — consider a generic message with the detail only logged server-side.
  • packages/gittensory-miner/lib/ranked-candidates.js has no forge/api_base_url scoping unlike portfolio-queue/claim-ledger, which is explained in the comment as intentional but is a divergence from sibling stores worth double-checking against future multi-forge usage.
  • test/unit/miner-ranked-candidates.test.ts:68 uses 'master' terminology in a SQL comment about sqlite_master — this is SQLite's own table name, not a naming choice, so it's a non-issue in context.
  • Consider redacting the raw error message in vite-ranked-candidates-api.ts's 500 response and logging the detail server-side instead, matching typical safe-error-surface practice for authenticated local endpoints.
  • AI review could not be completed — The gate is held for a human reviewer rather than passed automatically; it re-evaluates on the next update.

Concerns raised — review before merging

  • No linked issue detected — If this PR is intended to solve an issue, link it explicitly in the PR body.
  • Maintainer requires a linked issue — Link the relevant issue (for example Closes #123) before opening the PR.
📋 Copy for AI agents — paste into your coding agent
Fix the following blocker(s) from this PR review:

1. No linked issue detected — If this PR is intended to solve an issue, link it explicitly in the PR body.

2. Maintainer requires a linked issue — Link the relevant issue (for example `Closes #123`) before opening the PR.
Signal Result Evidence
Code review ❌ 2 blockers 1 reviewer
Linked issue ⚠️ Missing No linked issue or no-issue rationale found.
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (no linked issue context).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 44 registered-repo PR(s), 36 merged, 330 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 44 PR(s), 330 issue(s).
Gate result ❌ Blocking Repo-configured hard blocker found.
Improvement ✅ Minor risk: clean · value: minor — Code changes are accompanied by test evidence. LLM value judgment: moderate — This is well-scoped enabling infrastructure (persistence + read API) that directly unblocks the stated follow-up (#4859's live-fetch), implemented with strong test coverage and consistent with existing store conventions in the codebase.
Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: not available
  • Official Gittensor activity: 44 PR(s), 330 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Treat this as maintainer-lane context rather than normal contributor-lane activity.
  • Explain no-issue PR.
  • Link the issue being solved, or explicitly explain why this is a no-issue PR.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
[BETA] Chat with Gittensory

Ask Gittensory a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @gittensory ask &lt;question&gt; answers contribution-quality Q&A with source citations and freshness.
  • @gittensory chat &lt;question&gt; answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @gittensory mention with a real question is routed to the closest matching read-only command automatically -- no exact syntax required.

Full command reference: https://gittensory.aethereal.dev/docs/gittensory-commands

Visual preview
Route Viewport Before (production) After (this PR's preview) Diff
/ desktop before / after /
/ mobile before / (mobile) after / (mobile)

Click any thumbnail to open the full-size screenshot. Before = production · After = this PR's preview deploy.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.

  • Re-run LoopOver review

@loopover-orb loopover-orb Bot added the manual-review Gittensor contributor context label Jul 13, 2026
@JSONbored
JSONbored merged commit 6553b54 into main Jul 13, 2026
19 checks passed
@JSONbored
JSONbored deleted the miner-ranked-candidates-store branch July 13, 2026 10:47
JSONbored added a commit that referenced this pull request Jul 13, 2026
…ner UI (#5622)

Adds syncRankedCandidatesFromMinerUi() to background.js, pulling from
#5619's GET /api/ranked-candidates and writing into the same
chrome.storage.local keys the manual-paste flow already writes -- so
content.js/opportunity-badge.js/toolbar-badge.js need zero changes.
Triggers on startup/install and every 10 minutes via chrome.alarms,
plus a manual "Sync now" button. A failed sync leaves storage
untouched, naturally falling back to whatever's already there (a
stale fetch or a manual paste) with no merge logic needed.

Adds a "Local miner UI URL" field to the options page (default
http://localhost:5174, persisted to chrome.storage.sync alongside
watchedRepos).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. manual-review Gittensor contributor context

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant