Skip to content

ci(e2e): Make E2E/Override actually unblock the PR - #3970

Merged
esarafianou merged 3 commits into
masterfrom
fix/e2e-override-marks-checks-passing
Aug 27, 2026
Merged

ci(e2e): Make E2E/Override actually unblock the PR#3970
esarafianou merged 3 commits into
masterfrom
fix/e2e-override-marks-checks-passing

Conversation

@esarafianou

@esarafianou esarafianou commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Summary

The E2E/Override label is documented as "Cancels/skips E2E runs for this PR and marks the check as skipped", but applying it currently makes the PR permanently unmergeable.

All five e2e/* contexts are required status checks on master:

e2e/linux, e2e/macos, e2e/macos-policy, e2e/windows, e2e/windows-policy

A required status passes only on success — unlike check runs, commit statuses have no skipped state. markE2EStatusesCancelled wrote error to all five, so the override converted "E2E pending" into "E2E permanently blocked", exactly the state the label exists to clear.

#3955 is the live case

Changes

  • markE2EStatusesCancelled takes a state parameter, so the two callers can express different intent:
    • E2E/Override appliedsuccess, described as E2E waived (E2E/Override label applied) — no tests ran. A maintainer explicitly waived E2E, so the gate should open, and the description keeps the check list honest about the fact that nothing ran.
    • E2E/Run removed mid-flight → stays error. That withdraws coverage without waiving it, so the gate should stay closed. Unchanged behaviour.
  • Removing E2E/Override now re-arms a real run. There was no unlabel handler for E2E/Override at all before — harmless while the statuses were red, but with a green waiver it would leave a passing required check that never ran tests and no way to reset it short of pushing a new commit.

Test plan

  • npx eslint e2e/utils/github-actions.js — clean
  • Existing e2e/utils suites still pass (node --test e2e/utils/*.test.js)
  • Workflow YAML parses and the new step lands in the expected order
  • This PR exercises the happy path itself: the trigger job ran on open and applied E2E/Run, so E2E should go green normally
  • After merge, on Attribute Sentry events to a persisted install ID #3955: remove E2E/Override, then re-add it. The labeled event re-fires the job, which reads the helper from master at run time (the step checks out base.ref), so Attribute Sentry events to a persisted install ID #3955 gets the fix without rebasing. The five e2e/* statuses should flip to green and auto-merge should proceed.

Note on the alternative

The cleaner long-term shape is migrating e2e/* from commit statuses to check runs, since branch protection accepts a skipped check run conclusion as satisfying a required check — no need to write success for something that didn't run. That touches e2e-functional.yml and needs checks: write, so it's left out of this fix. Happy to file a follow-up if there's appetite.

Release Note

NONE

Summary by CodeRabbit

  • Bug Fixes
    • Improved end-to-end test status handling when coverage is waived or overrides are removed.
    • Required checks now accurately reflect waived coverage and manually cancelled runs.
    • Restored end-to-end test execution indicators for applicable non-draft pull requests.
    • Added clearer status reasons when test coverage is explicitly waived.
    • Improved consistency of status reporting across end-to-end test and policy checks.

The e2e/* contexts are required status checks on master, so a required
status only passes on `success`. E2E/Override wrote `error` to all five,
which turned "E2E pending" into "E2E permanently blocked" — the opposite
of what the label exists to do. PR #3955 sat unmergeable with a green
approved review and an E2E suite that had already passed on that SHA
before the override overwrote the results.

Give markE2EStatusesCancelled a state parameter so the two callers can
express different intent. Applying E2E/Override waives E2E and writes
`success`, with a description recording that no tests ran. Pulling
E2E/Run mid-flight withdraws coverage without waiving it, so it keeps
`error` and the gate stays closed.

Removing E2E/Override now re-arms a real run. Previously there was no
unlabel handler at all, which was harmless when the statuses were red
but would otherwise leave a waived-green required check standing with
no way to reset it short of pushing a new commit.

Co-authored-by: Cursor <cursoragent@cursor.com>
@mm-cloud-bot mm-cloud-bot added the kind/bug Categorizes issue or PR as related to a bug. label Aug 27, 2026
@mm-cloud-bot

Copy link
Copy Markdown

@esarafianou: Adding the "do-not-merge/release-note-label-needed" label because no release-note block was detected, please follow our release note process to remove it.

Details

I understand the commands that are listed here

@coderabbitai

coderabbitai Bot commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 2ba5b453-b00f-4136-a505-4276e556777e

📥 Commits

Reviewing files that changed from the base of the PR and between c079ec8 and 06d263f.

📒 Files selected for processing (1)
  • .github/workflows/e2e-pr-trigger.yml
🚧 Files skipped from review as they are similar to previous changes (1)
  • .github/workflows/e2e-pr-trigger.yml

Included review availability: 2 reviews are currently available. Your included PR review attempts over the past 7 days set your current allowance at 5 reviews per hour.


📝 Walkthrough

Walkthrough

The PR updates E2E override handling. It restores E2E/Run after override removal on non-draft pull requests, reports waivers as successful statuses, and supports configurable commit-status states.

Changes

E2E override waivers

Layer / File(s) Summary
Configurable E2E status states
e2e/utils/github-actions.js
markE2EStatusesCancelled accepts an optional status state and applies it to E2E and policy commit statuses.
Override lifecycle workflow
.github/workflows/e2e-pr-trigger.yml
The workflow detects override removal, restores E2E/Run for non-draft pull requests, and reports active or removed overrides as successful waivers.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: ⚪ Minimal · up to 06d26

This change makes the E2E override unblock required checks while preserving the blocked state when E2E coverage is withdrawn; no actionable merge-blocking risk remains after normal checks and review.

Suggested reviewers: yasserfaraazkhan

Sequence Diagram(s)

sequenceDiagram
  participant PullRequest
  participant E2ETriggerWorkflow
  participant GitHubStatuses
  PullRequest->>E2ETriggerWorkflow: remove E2E/Override on non-draft pull request
  E2ETriggerWorkflow->>GitHubStatuses: remove and re-add E2E/Run
  E2ETriggerWorkflow->>GitHubStatuses: report successful waiver status
Loading
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 2 functions across 2 files. (1 skipped: 1… Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly identifies the main change: making E2E/Override unblock pull requests. It is concise and matches the workflow and status changes.
Full details: Docstring Coverage

Explanation

Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 2 functions across 2 files. (1 skipped: 1 unsupported.)

  • Fix all pre-merge checks with AI
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/e2e-override-marks-checks-passing

Comment @coderabbitai help to get the list of available commands.

Co-authored-by: Cursor <cursoragent@cursor.com>
@github-actions github-actions Bot added E2E/Run Run Desktop E2E Tests and removed E2E/Run Run Desktop E2E Tests labels Aug 27, 2026
Co-authored-by: Cursor <cursoragent@cursor.com>
@github-actions github-actions Bot added E2E/Run Run Desktop E2E Tests and removed E2E/Run Run Desktop E2E Tests labels Aug 27, 2026
@esarafianou
esarafianou merged commit 04e4dbf into master Aug 27, 2026
45 checks passed
@esarafianou
esarafianou deleted the fix/e2e-override-marks-checks-passing branch August 27, 2026 20:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind/bug Categorizes issue or PR as related to a bug. release-note-none

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants