Skip to content

feat(analytics): persist recommendation outcome events - #334

Merged
JSONbored merged 2 commits into
JSONbored:mainfrom
Lobster-0429:feat/recommendation-outcome-events
Jun 3, 2026
Merged

feat(analytics): persist recommendation outcome events#334
JSONbored merged 2 commits into
JSONbored:mainfrom
Lobster-0429:feat/recommendation-outcome-events

Conversation

@Lobster-0429

Copy link
Copy Markdown
Contributor

Summary

  • Implements feat(analytics): persist recommendation outcome events #301: persists recommendation outcome events as the private data spine for later recommendation quality analytics
  • Adds "rejected" as the seventh outcome state — triggered when a targeted PR receives a CHANGES_REQUESTED review decision after the recommendation snapshot
  • Adds surface and snapshotId fields to every persisted outcome record so analytics can slice by product surface (api/mcp/github_app/…) and context snapshot
  • New D1 migration (0018) adds surface and snapshot_id columns with ALTER TABLE (nullable, backwards-compatible)

Closes: #301

Scope

  • This PR is focused and does not mix unrelated backend, UI, MCP, docs, dependency, and deploy changes.
  • This follows CONTRIBUTING.md and does not reintroduce GitHub Pages, VitePress, site/, or CNAME.
  • I linked an issue, or this is small enough that the summary explains why an issue is not needed.

Validation

  • git diff --check
  • npm run actionlint
  • npm run typecheck — clean
  • npm run test:coverage locally — 775 pass (1 skipped); pre-existing Windows failures in mcp-cli.test.ts and two syntax-error files confirmed on main before this branch; coverage stays above 97%
  • npm run test:workers
  • npm run build:mcp
  • npm run test:mcp-pack
  • npm run ui:openapi:check
  • npm run ui:lint
  • npm run ui:typecheck
  • npm run ui:build
  • npm audit --audit-level=moderate
  • New or changed behavior has unit/integration tests for new branches, fallback paths, and sanitizer boundaries

If any required check was skipped, explain why:

  • UI/MCP/worker checks not applicable — this PR is backend-only (types, DB, service, tests)

Safety

  • No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed.
  • Public GitHub text stays sanitized, low-noise, and does not imply compensation guarantees or optimization tactics.
  • Auth, cookie, CORS, GitHub App, Cloudflare, or session changes include negative-path tests. (not applicable — no auth changes)
  • API/OpenAPI/MCP behavior is updated and tested where needed. (not applicable — no public API surface changed)
  • UI changes use live API data or real empty/error/loading states, not production mock/demo fallbacks. (not applicable)
  • Visible UI changes include screenshots or a short recording. (not applicable)
  • Public docs/changelogs are updated where needed. (not applicable)

Notes

  • "rejected" is classified only on exact targetPullNumber matches with reviewDecision === "CHANGES_REQUESTED" after the action timestamp, consistent with the existing "improved" mirror condition
  • rejected is counted as negative in totals (alongside closed, stale, ignored)
  • Legacy rows without surface/snapshot_id map cleanly: surface is null (column is nullable), snapshotId is null
  • Outcome records remain entirely private — the three new test cases verify this explicitly (serialized event JSON, privateSummary, and public-safe action copy)

Adds 'rejected' as a seventh outcome state (PR with CHANGES_REQUESTED
review decision after action timestamp), and separates surface and
snapshotId on every persisted outcome record so later analytics can
slice by product surface and context snapshot.

- src/types.ts: add "rejected" to AgentRecommendationOutcomeState;
  add surface/snapshotId fields to AgentRecommendationOutcomeRecord;
  add rejected count to AgentRecommendationOutcomeSummary totals and
  AgentRecommendationOutcomeRepoSummary.
- migrations/0018: ALTER TABLE adds surface and snapshot_id columns.
- src/db/schema.ts: matching Drizzle column definitions.
- src/db/repositories.ts: upsert/mapper/parse/bucket/totals/repo-
  summary all handle rejected and the new columns.
- src/services/recommendation-outcomes.ts: fetch context snapshot per
  run in evaluateRecommendationOutcomes; thread surface/snapshotId
  through classifyRecommendationOutcome and baseOutcome; add rejected
  branch in pullRequestOutcomeState.
- tests: fixture for all seven outcome states, surface/snapshotId
  capture round-trip, and privacy serialization assertion.

@JSONbored JSONbored left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@web-dev0521 this is the analytics persistence baseline I would land first.

A few notes:

  • The migration and repository changes establish the outcome-event surface that later rollups can build on.
  • The test coverage is tied to the persistence behavior rather than just the API shape.
  • Follow-on analytics PRs should rebase and renumber migrations after this lands.

No code changes requested.

Validation expected:

  • Keep the database/unit suite green through merge.

@dosubot dosubot Bot added the lgtm label Jun 3, 2026
@JSONbored
JSONbored merged commit dadf052 into JSONbored:main Jun 3, 2026
4 checks passed
@github-project-automation github-project-automation Bot moved this from Todo to Done in gittensory - v1 roadmap Jun 3, 2026
@github-actions github-actions Bot mentioned this pull request Jun 3, 2026
12 tasks
@JSONbored JSONbored added gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. and removed feature labels Jun 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier.

Projects

No open projects
Status: Done

Development

Successfully merging this pull request may close these issues.

feat(analytics): persist recommendation outcome events

2 participants