[Bug]: Recommendation-outcome classifier reports a changes-requested PR as `improved` when the branch is also merge-clean

[joaovictor712]

Summary

classifyRecommendationOutcome can turn an explicit negative GitHub review signal into a positive recommendation outcome. In pullRequestOutcomeState, the classifier treats either approval or clean mergeability as a positive open-PR signal, then returns "improved" before it checks for "CHANGES_REQUESTED":

// src/services/recommendation-outcomes.ts:281-285
if ((Number.isFinite(mergedAt) && mergedAt >= actionAt) || (!pr.mergedAt && pr.state === "merged" && updatedAt >= actionAt)) return "merged";
if (pr.state === "closed" && updatedAt >= actionAt) return "closed";
const positiveOpenSignal = pr.reviewDecision === "APPROVED" || pr.mergeableState === "clean";
if (action.targetPullNumber && positiveOpenSignal && updatedAt >= actionAt) return "improved";
if (action.targetPullNumber && pr.reviewDecision === "CHANGES_REQUESTED" && updatedAt >= actionAt) return "rejected";

That means a targeted PR with:

reviewDecision: "CHANGES_REQUESTED",
mergeableState: "clean",
updatedAt: <after recommendation>

is classified as "improved", not "rejected".

mergeableState: "clean" only says the branch can merge technically. It does not override a reviewer requesting changes. The explicit review decision should win.

Failure mode

A miner receives a recommendation to work on owner/repo#123. After the recommendation, the PR is still open and GitHub reports:

• reviewDecision = "CHANGES_REQUESTED"
• mergeableState = "clean"
• updatedAt >= action.createdAt

Current classifier path:

1. PR is not merged.
2. PR is not closed.
3. positiveOpenSignal = false || true, because mergeableState === "clean".
4. The function returns "improved".
5. The later CHANGES_REQUESTED branch is never reached.

This corrupts the recommendation-quality feedback loop:

• improved is counted as positive in recommendation-quality-report.ts (POSITIVE_STATES)
• rejected is counted as negative (NEGATIVE_STATES)
• failure categories miss the "Changes requested" row
• per-repo/top-repo signals can flip from negative to positive

Concrete reproduction

Call classifyRecommendationOutcome with:

• action: targetRepoFullName: "owner/rejected-clean", targetPullNumber: 200, createdAt: "2026-05-01T00:00:00.000Z"
• PR record: same repo/number, state: "open", updatedAt: "2026-05-10T00:00:00.000Z", reviewDecision: "CHANGES_REQUESTED", mergeableState: "clean"
• evaluation: evaluatedAt: "2026-06-01T00:00:00.000Z"

Expected outcome:

{ outcomeState: "rejected" }

Actual outcome:

{ outcomeState: "improved" }

Test status

Partially covered, but the conflicting-signal case is missing.

Existing tests prove each individual branch works in isolation:

// test/unit/recommendation-outcomes.test.ts:414-421
reviewDecision: "CHANGES_REQUESTED"  // no clean mergeability -> rejected

// test/unit/recommendation-outcomes.test.ts:441-448
reviewDecision: "APPROVED"           // approved -> improved

// test/unit/recommendation-outcomes.test.ts:345-348
mergeableState: "clean"              // no changes-requested review -> improved

No fixture combines reviewDecision: "CHANGES_REQUESTED" with mergeableState: "clean", so the precedence bug is not locked in.

Expected behavior

For targeted PR outcomes, negative review decisions should take precedence over mergeability-only positive signals:

• merged after recommendation -> "merged"
• closed after recommendation -> "closed"
• CHANGES_REQUESTED after recommendation -> "rejected"
• APPROVED after recommendation -> "improved"
• clean mergeability with no changes-requested review -> "improved"

Actual behavior

mergeableState === "clean" is folded into positiveOpenSignal and checked before CHANGES_REQUESTED, so a technically merge-clean PR with requested changes is reported as improved.

Suggested fix

Prioritize explicit negative review decisions before mergeability-based improvement:

const hasTargetPullNumber = Boolean(action.targetPullNumber);
const hasRecentUpdate = updatedAt >= actionAt;
const reviewDecision = pr.reviewDecision?.toUpperCase();
const mergeableState = pr.mergeableState?.toLowerCase();

if (hasTargetPullNumber && reviewDecision === "CHANGES_REQUESTED" && hasRecentUpdate) return "rejected";

const positiveOpenSignal = reviewDecision === "APPROVED" || mergeableState === "clean";
if (hasTargetPullNumber && positiveOpenSignal && hasRecentUpdate) return "improved";

Add fail-on-revert coverage to test/unit/recommendation-outcomes.test.ts: a targeted open PR with both reviewDecision: "CHANGES_REQUESTED" and mergeableState: "clean" must classify as "rejected", not "improved".

Distinct from prior reports

This is not a duplicate of the earlier changes-requested bug in gittensory-philluiz-issue3.md.

• issue 3 is about src/signals/local-branch.ts, where stale PR classification runs before blocked PR classification and inflates score-preview scenarios.
• this issue is about src/services/recommendation-outcomes.ts, where a targeted PR's feedback outcome is marked positive (improved) before the explicit negative review state (rejected) can be observed.

The blast radius is different too: issue 3 affects scenario/open-PR pressure projections; this one affects recommendation-quality history, private summaries, role/report rollups, and the agent feedback loop.