Skip to content

docs(miner): publish a measured CPU/RAM/disk sizing table for laptop vs fleet mode - #5645

Merged
JSONbored merged 1 commit into
JSONbored:mainfrom
joaovictor712:feat/miner-sizing-docs
Jul 13, 2026
Merged

docs(miner): publish a measured CPU/RAM/disk sizing table for laptop vs fleet mode#5645
JSONbored merged 1 commit into
JSONbored:mainfrom
joaovictor712:feat/miner-sizing-docs

Conversation

@joaovictor712

Copy link
Copy Markdown
Contributor

Summary

  • Neither the operational-runbook issue (Write an operational runbook #4875) nor the local-stores documentation issue (Update DEPLOYMENT.md to document all local stores #4870) commits to
    publishing real, measured resource numbers — both are scoped to narrative/reference content, not a sizing
    table. Operators had no data-backed answer to "how much CPU/RAM/disk does one laptop-mode run need, and how
    does that scale in fleet mode?"
  • Added packages/gittensory-miner/docs/sizing.md with real, measured numbers for both modes, the exact
    reproduction commands, and the measurement environment, so the numbers are reproducible and falsifiable by a
    reviewer — not asserted from memory.
  • Workload measured: gittensory-miner init (laptop mode only) followed by gittensory-miner discover <owner/repo>... --json against real, small public repos (octocat/Hello-World, octocat/Spoon-Knife) with
    no GITHUB_TOKEN — real unauthenticated GitHub GETs, metadata fan-out + ranking, zero writes.
  • Deliberately excluded a live attempt cycle: that needs operator-supplied GITHUB_TOKEN + coding-agent
    CLI credentials and writes a real branch/PR to a real target repo — not something to spend for a
    resource-sizing exercise, and not reproducible by a reviewer without their own credentials. discover is
    measured instead as the dominant, always-run, network-bound phase every mode exercises identically.
  • Laptop mode: measured directly with GNU /usr/bin/time -v around the CLI process (no container).
  • Fleet mode: built the real packages/gittensory-miner/Dockerfile image and measured with docker stats --no-stream polled once per second per container, at N=1 and N=4 workers. Per docker-compose.miner.yml's
    own documented warning that N replicas sharing one volume corrupt/contend on the SQLite ledgers, N=4 here
    means four separate named volumes (docker run -d -v miner-n<i>:/data/miner ... per worker) — the same
    isolation pattern the compose file itself recommends, not a single shared-volume --scale.
  • Disk: du -sh/du -ah on the resolved state dir (laptop mode) or each worker's mounted volume via a
    disposable alpine container (fleet mode), after the run.

Measured results

Mode Workers Peak CPU Peak RAM Disk (per worker)
Laptop (init) 1 125% (short Node-startup burst) 74 MB
Laptop (discover, 1 repo, 90 issues) 1 16% (network-bound) 98 MB 100 KB
Fleet/Docker (discover, 2 repos) 1 54% (short burst) 46 MB 92 KB
Fleet/Docker (discover, 2 repos) 4 (isolated volumes) 5–38% per worker 34–45 MB per worker 92 KB per worker

Measured on a 4 vCPU / 9.5 GiB RAM Linux sandbox — see docs/sizing.md for the full methodology, exact
commands, environment caveat, and takeaways (per-worker memory does not measurably change between N=1 and N=4
since each worker is an independent Node process with no shared heap; CPU is dominated by waiting on GitHub's
API, not local computation).

Fixes #5182

Scope

  • The PR title follows type(scope): short summary Conventional Commit format, for example fix(api): restore profile access checks.
  • This PR is focused and does not mix unrelated backend, UI, MCP, docs, dependency, and deploy changes.
  • This follows CONTRIBUTING.md and does not reintroduce GitHub Pages, VitePress, site/, or CNAME.
  • I linked a currently open issue this PR resolves (e.g. Closes #123) — a linked open issue is required for every contributor PR.

Validation

  • git diff --check
  • npm run actionlint
  • npm run typecheck
  • npm run test:coverage — N/A: this is a docs-only deliverable with no src/** logic change and no new measurement script/helper added (the issue's own Test Coverage Requirements only apply "if any measurement script or helper is added to produce the numbers" — none was; the numbers were captured with ad-hoc, documented shell commands, all reproduced exactly in the doc itself).
  • npm run test:workers
  • npm run build:mcp
  • npm run test:mcp-pack
  • npm run build:miner
  • npm run test:miner-pack — confirmed docs/sizing.md packs correctly into the published @loopover/miner tarball.
  • npm run ui:openapi:check
  • npm run ui:lint
  • npm run ui:typecheck
  • npm run ui:build
  • npm audit --audit-level=moderate
  • New or changed behavior has unit/integration tests for new branches, fallback paths, and sanitizer boundaries — N/A, docs-only.

Ran the full local gate: npm run test:ci (0 failures) and npm audit --audit-level=moderate (0 vulnerabilities), both clean on the final commit. Also directly ran the exact FORBIDDEN_PATH/FORBIDDEN_CONTENT regexes from scripts/check-miner-package.mjs/scripts/forbidden-content.mjs against every new/touched file — zero matches.

Safety

  • No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed.
  • Public GitHub text stays sanitized, low-noise, and does not imply compensation guarantees or optimization tactics.
  • Auth, cookie, CORS, GitHub App, Cloudflare, or session changes include negative-path tests. — N/A, docs-only, no auth/session/CORS surface touched.
  • API/OpenAPI/MCP behavior is updated and tested where needed. — no public API/OpenAPI/MCP surface touched.
  • UI changes use live API data or real empty/error/loading states, not production mock/demo fallbacks. — N/A, no UI surface.
  • Visible UI changes include a UI Evidence section below. — N/A, see below.
  • Public docs/changelogs are updated where needed; changelogs are only edited for release-prep PRs. — added docs/sizing.md, cross-linked from README.md and DEPLOYMENT.md.

UI Evidence

N/A — this is a docs-only change with no visible UI surface. No screenshots apply.

Notes

  • Deliberately did not fold broader operational-runbook content into this PR, per the issue's own scope note —
    strictly the sizing table, methodology, and reproduction commands.
  • All numbers in this PR were captured by directly running the commands shown in docs/sizing.md's
    "Methodology" section against this repo's own Dockerfile/CLI on the environment described — none were
    estimated, interpolated, or asserted from memory.

…vs fleet mode

Neither the operational-runbook issue nor the local-stores documentation
commits to publishing real, measured resource numbers, so operators had no
data-backed answer to how much CPU/RAM/disk a laptop-mode run needs, or how
fleet mode scales with worker count.

Adds packages/gittensory-miner/docs/sizing.md: real numbers for `init` +
`discover` measured with /usr/bin/time -v (laptop mode) and docker stats
(fleet mode, N=1 and N=4 isolated Docker volumes, matching
docker-compose.miner.yml's own documented isolation requirement), plus the
exact reproduction commands and measurement environment. Deliberately
excludes a live coding-agent attempt cycle, which would need real operator
credentials and write to a real target repo -- discover is measured instead
as the dominant, always-run, network-bound phase every mode shares.
@joaovictor712
joaovictor712 requested a review from JSONbored as a code owner July 13, 2026 19:03
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@loopover-orb loopover-orb Bot added the gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. label Jul 13, 2026
@loopover-orb

loopover-orb Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Tip

🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩

✅ LoopOver review result - approve/merge recommended

Review updated: 2026-07-13 19:15:50 UTC

3 files · 1 AI reviewer · no blockers · readiness 100/100 · CI green · clean

✅ Suggested Action - Approve/Merge

  • safe to merge

Review summary
Pure documentation addition: a new sizing.md with measured CPU/RAM/disk numbers for laptop vs fleet mode, plus two one-line cross-references from DEPLOYMENT.md and README.md. The doc is well-structured, includes exact reproduction commands, and explicitly states its measurement environment and methodology, which is the substantive detail — it's falsifiable rather than asserted from memory. No code, schema, or config changes are involved, so there's nothing here that can break a build or runtime path.

Nits — 6 non-blocking
  • docs/sizing.md's laptop-mode example command uses `discover octocat/Hello-World --dry-run --json` (single repo, `--dry-run`) but the prose and Results table describe measuring two repos with no `--dry-run` flag — the exact command shown doesn't match what was actually measured, so a reviewer can't reproduce the reported numbers verbatim.
  • The Results table row "Laptop (`discover`, 1 repo, 90 issues)" introduces an unexplained "90 issues" figure with no methodology note on how issue count was determined or why it's relevant to sizing.
  • Fleet mode's `docker stats` polling loop (`for i in $(seq 1 20)`) only prints the last-seen sample per iteration rather than averaging/aggregating, so "peak observed sample" in the methodology is manually eyeballed from the loop output — worth a one-line note on how the peak was extracted from the 20 samples.
  • Fix the laptop-mode command block in sizing.md to match the two-repo, non-dry-run workload actually described in the prose and Results table.
  • Add a short note on how "peak" was derived from the 20 polled `docker stats` samples (max column value, manual read, etc.) so the methodology section is fully mechanical/reproducible.
  • Diff looks like trivial or whitespace-only churn — Reduce whitespace-only or formatting-only churn and keep the diff focused on substantive changes.
Signal Result Evidence
Code review ✅ No blockers 1 reviewer
Linked issue ✅ Linked #5182
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 1874 registered-repo PR(s), 1240 merged, 49 issue(s).
Contributor context ✅ Confirmed Gittensor contributor galuis116; Gittensor profile; 1874 PR(s), 49 issue(s).
Gate result ✅ Passing No configured blocker found.
Improvement ⚠️ ℹ️ Insufficient signal risk: low · value: insufficient-signal — Nothing measurable for the structural-improvement analyzers on this PR (e.g. no code files changed).
Review context
  • Author: galuis116
  • Role context: outside_contributor
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: not available
  • Official Gittensor activity: 1874 PR(s), 49 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Keep the PR focused and include validation evidence before maintainer review.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
[BETA] Chat with Gittensory

Ask Gittensory a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @gittensory ask &lt;question&gt; answers contribution-quality Q&A with source citations and freshness.
  • @gittensory chat &lt;question&gt; answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @gittensory mention with a real question is routed to the closest matching read-only command automatically -- no exact syntax required.

Full command reference: https://gittensory.aethereal.dev/docs/gittensory-commands

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.

  • Re-run LoopOver review

@loopover-orb loopover-orb Bot added the manual-review Gittensor contributor context label Jul 13, 2026
@JSONbored
JSONbored merged commit db0a34c into JSONbored:main Jul 13, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. manual-review Gittensor contributor context

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Publish a measured CPU/RAM/disk sizing table for gittensory-miner laptop-mode vs fleet-mode

2 participants