Skip to content

[Bug]: Recommendation-quality trends double-count outcomes exactly on bucket boundaries, while rollups count them once #519

Description

@joaovictor712

Summary

buildRecommendationQualityReportFromOutcomes builds two time-bucketed views over the same recommendation outcomes:

  • trends via trendBuckets
  • rollups via qualityRollups

They use the same trendPeriods(...) boundaries, but disagree on interval semantics. trendBuckets treats every bucket as closed on both ends:

// src/services/recommendation-quality-report.ts:224-228
const bucketOutcomes = outcomes.filter((outcome) => {
  const timestamp = Date.parse(outcomeTimestamp(outcome));
  return Number.isFinite(timestamp) && timestamp >= period.startMs && timestamp <= period.endMs;
});

qualityRollups uses the usual non-overlapping rule: [start, end) for every bucket except the final bucket, which includes end:

// src/services/recommendation-quality-report.ts:245-248
const timestamp = Date.parse(outcomeTimestamp(outcome));
if (!Number.isFinite(timestamp)) continue;
const period = periods.find((candidate) => timestamp >= candidate.startMs && (candidate.last ? timestamp <= candidate.endMs : timestamp < candidate.endMs));

So an outcome whose timestamp is exactly equal to an internal bucket boundary is counted in two adjacent trends buckets but only one rollups bucket. The operator dashboard can therefore show trend totals whose bucket sum exceeds totals.total, and the trend chart disagrees with the rollup table for the same report window.

Concrete trace

Use generatedAt = "2026-06-01T00:00:00.000Z" and windowDays = 14.

trendPeriods creates two 7-day buckets:

  • bucket 1: 2026-05-18T00:00:00.000Z through 2026-05-25T00:00:00.000Z
  • bucket 2: 2026-05-25T00:00:00.000Z through 2026-06-01T00:00:00.000Z

Now evaluate one accepted recommendation outcome with updatedAt = "2026-05-25T00:00:00.000Z".

Current trends logic:

  • bucket 1 includes it because timestamp >= 2026-05-18 and timestamp <= 2026-05-25
  • bucket 2 also includes it because timestamp >= 2026-05-25 and timestamp <= 2026-06-01
  • result: trends[0].total = 1, trends[1].total = 1

But totals.total = 1, and rollups puts the same outcome only in bucket 2 because the first bucket is treated as < period.endMs.

That means a single outcome can produce a trend-bucket sum of 2 while the report's actual total is 1.

Reachability

This is reachable through the operator dashboard path:

// src/services/operator-dashboard.ts:97
buildRecommendationQualityReport(env, { windowDays: 90 })

buildRecommendationQualityReport loads persisted outcomes and then calls buildRecommendationQualityReportFromOutcomes (recommendation-quality-report.ts:77-88). Persisted outcomes use ISO timestamps (updatedAt, falling back through detectedAt / createdAt in outcomeTimestamp), so exact midnight boundaries are realistic:

  • scheduled/automated jobs commonly write ...T00:00:00.000Z
  • tests already use exact boundary-style timestamps (recommendation-quality-report.test.ts:80-90 expects the 14-day periods to split at 2026-05-25T00:00:00.000Z)
  • another fixture uses detectedAt: "2026-05-25T00:00:00.000Z" (recommendation-quality-report.test.ts:215), but it only checks that some trend bucket has data, not that the bucket sum is non-duplicated

Test status

Not locked in. Existing coverage asserts:

  • report.trends has the expected number of buckets (recommendation-quality-report.test.ts:42)
  • at least one trend bucket contains a negative outcome (:130)
  • at least one trend bucket has data when timestamp fallbacks are used (:248)
  • rollups put 2026-05-25-to-2026-06-01 outcomes in the second 14-day bucket (:74-102)

No test asserts that sum(report.trends.map((bucket) => bucket.total)) <= report.totals.total, or that a boundary timestamp is assigned to exactly one trend bucket.

Expected behavior

Each outcome belongs to exactly one trend bucket, matching qualityRollups:

  • internal buckets are [periodStart, periodEnd)
  • the final bucket is [periodStart, periodEnd]

For the single boundary outcome above, trends should be:

  • bucket 1 total: 0
  • bucket 2 total: 1
  • sum of trend totals: 1

Actual behavior

trendBuckets uses timestamp <= period.endMs for every bucket. Internal boundary timestamps are included in both the ending bucket and the starting bucket, inflating trends and making it disagree with rollups and totals.

Suggested fix

Mirror the rollup bucket predicate inside trendBuckets:

const bucketOutcomes = outcomes.filter((outcome) => {
  const timestamp = Date.parse(outcomeTimestamp(outcome));
  return Number.isFinite(timestamp)
    && timestamp >= period.startMs
    && (period.last ? timestamp <= period.endMs : timestamp < period.endMs);
});

Add a fail-on-revert test with generatedAt = "2026-06-01T00:00:00.000Z", windowDays = 14, and a single outcome at updatedAt = "2026-05-25T00:00:00.000Z". Assert:

  • report.totals.total === 1
  • report.trends.map((bucket) => bucket.total) is [0, 1]
  • the sum of trend totals is 1
  • report.rollups places the outcome in the 2026-05-25 to 2026-06-01 bucket

Distinct from prior reports

This is not a duplicate of the earlier dashboard/counting reports:

  • issue 11 covered a maintainer-dashboard metric capped to the first 12 repos
  • issue 12 covered duplicate-risk count aggregation in public PR comments
  • issue 17 covered burden-forecast tier classification
  • issue 18 covered install-preview permission under-reporting

This bug is isolated to the newly-added operator-only recommendation-quality report's time-bucket math, and specifically to inconsistent boundary handling between trends and rollups.

Metadata

Metadata

Assignees

No one assigned

    Labels

    gittensor:bugGittensor-scored bug fix — scores a 0.05x multiplier.slopAI slop and/or attempts to game additional points via manipulation or alt profiles.

    Projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions