Skip to content

fix(miner): isolate replay scoring key - #3235

Merged
loopover-orb[bot] merged 2 commits into
mainfrom
codex/fix-information-leakage-in-replay-task
Jul 5, 2026
Merged

fix(miner): isolate replay scoring key#3235
loopover-orb[bot] merged 2 commits into
mainfrom
codex/fix-information-leakage-in-replay-task

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Motivation

  • The replay task generator previously returned both the scrubbed frozen context and the revealed post‑T ground truth together, creating an API that can leak answer keys to replay workers, serializers, log output, or model/planner callers.
  • The intent is to preserve leakage isolation by ensuring replay execution only receives the frozen bundle while scoring receives revealed data via a separate, scoped accessor.

Description

  • Change generateReplayTask to return only the scrubbed frozen replay bundle (frozen) for eligible tasks and remove any revealed field from the returned object.
  • Add generateReplayScoringKey(candidate, options) as a scoring-only accessor that returns commitCount and groundTruth (or an eligible rejection) and never contains frozen context.
  • Update public types in replay-task-generation.d.ts to remove revealed from ReplayTask and add a new ReplayScoringKey type and function signature for generateReplayScoringKey.
  • Update unit tests in test/unit/miner-replay-task-generation.test.ts to assert the replay task contains no revealed bundle and to exercise the new scoring-only accessor, plus keep existing rejection/determinism assertions.

Testing

  • Ran the focused unit suite: npx vitest run test/unit/miner-replay-task-generation.test.ts, and the tests covering the miner replay task file passed (19 tests all green).
  • Ran package build checks for the miner workspace: npm --workspace @jsonbored/gittensory-miner run build, which completed (node --check validation passed for the changed files).
  • Ran type checking: npm run typecheck, which completed without errors.
  • Attempted full local gate (npm run test:ci) and coverage runs in this environment but they did not complete to a clean green due to unrelated environment/network limitations (DNS/setup for actionlint, selfhost:env-reference:check drift, long-running unrelated full-suite tests) and registry audit access (npm audit) returning a 403; these are environmental and unrelated to the isolated behavioral change in the miner module.

Codex Task

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jul 4, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
gittensory-ui 2442a0a Commit Preview URL

Branch Preview URL
Jul 04 2026, 11:58 PM

@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 4, 2026
@loopover-orb

loopover-orb Bot commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

Tip

🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩

✅ Gittensory review result - approve/merge recommended

Review updated: 2026-07-05 00:01:26 UTC

3 files · 1 AI reviewer · no blockers · readiness 93/100 · CI green · clean

✅ Suggested Action - Approve/Merge

  • safe to merge

Review summary
The change cleanly separates replay execution data from scoring data: `generateReplayTask` now returns only the frozen bundle, while `generateReplayScoringKey` exposes the revealed commit count and ground truth behind a separate accessor. The implementation traces to the same `selectFreezePoint` source for revealed commit count and does not carry frozen context into the scoring key, so the stated leak-isolation goal is met in the changed code. I do not see a reachable correctness blocker in the provided diff.

Nits — 4 non-blocking
  • nit: packages/gittensory-miner/lib/replay-task-generation.d.ts:114 types `generateReplayScoringKey` as `ReplayScoringKey | ReplayTaskRejected`, which advertises the `unscrubbable_forward_reference` rejection even though the scoring accessor never lints frozen context and can only reject selection.
  • nit: packages/gittensory-miner/lib/replay-task-generation.js:216 documents the intentional divergence between task eligibility and scoring-key eligibility, but the tests do not lock that behavior with an unscrubbable frozen-context candidate.
  • packages/gittensory-miner/lib/replay-task-generation.d.ts:114: narrow the scoring rejection type so callers do not have to handle an impossible branch, for example `export type ReplayScoringKeyRejected = { eligible: false; rejected: "selection"; reasons: string[]; };` and return `ReplayScoringKey | ReplayScoringKeyRejected`.
  • test/unit/miner-replay-task-generation.test.ts:218: add a focused test showing `generateReplayTask` rejects an unscrubbable frozen context while `generateReplayScoringKey` for the same candidate still returns the scoring key, matching the comment in `generateReplayScoringKey`.
Signal Result Evidence
Code review ✅ No blockers 1 reviewer
Linked issue ⚠️ Missing No linked issue or no-issue rationale found.
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (no linked issue context).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 56 registered-repo PR(s), 46 merged, 429 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 56 PR(s), 429 issue(s).
Gate result ✅ Passing No configured blocker found.
Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository registration is not available in the local Gittensory cache.
  • Public profile languages: Python, TypeScript, JavaScript, Ruby, Go, Kotlin, MDX, Shell
  • Official Gittensor activity: 56 PR(s), 429 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Treat this as maintainer-lane context rather than normal contributor-lane activity.
  • Explain no-issue PR.
  • No action.
  • Link the issue being solved, or explicitly explain why this is a no-issue PR.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@codecov

codecov Bot commented Jul 4, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.12%. Comparing base (295c213) to head (2442a0a).
⚠️ Report is 2 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #3235   +/-   ##
=======================================
  Coverage   94.12%   94.12%           
=======================================
  Files         276      276           
  Lines       30202    30202           
  Branches    11011    11011           
=======================================
  Hits        28429    28429           
  Misses       1127     1127           
  Partials      646      646           
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@loopover-orb loopover-orb Bot added the manual-review Gittensor contributor context label Jul 4, 2026
JSONbored added 2 commits July 4, 2026 16:51
The review flagged that eligible: true here only mirrors
selectFreezePoint, not generateReplayTask's full scrub/lint pipeline --
a candidate can pass selection but still fail generateReplayTask's
lintFrozenContext check (unscrubbable forward reference), producing a
scoring key with no matching frozen task. Document this explicitly so
a caller never assumes the two are a matched pair without checking
generateReplayTask's own result.
@JSONbored
JSONbored force-pushed the codex/fix-information-leakage-in-replay-task branch from a5c9b8a to 2442a0a Compare July 4, 2026 23:56

@loopover-orb loopover-orb Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gittensory approves — the gate is satisfied and CI is green.

@loopover-orb
loopover-orb Bot merged commit 5f5c584 into main Jul 5, 2026
12 checks passed
@loopover-orb
loopover-orb Bot deleted the codex/fix-information-leakage-in-replay-task branch July 5, 2026 00:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. manual-review Gittensor contributor context

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant