Skip to content

[py] run grid specific tests on windows during scheduled tests - #18054

Merged
titusfortner merged 2 commits into
SeleniumHQ:trunkfrom
titusfortner:py-grid-nightly
Sep 19, 2026
Merged

titusfortner merged 2 commits into
SeleniumHQ:trunkfrom
titusfortner:py-grid-nightly

Conversation

@titusfortner

Copy link
Copy Markdown
Member

🔗 Related Issues

  • Python counterpart to the Ruby Grid nightly coverage from 6bf7598

💥 What does this PR do?

  • Runs the Python Grid-specific tests (test/selenium/webdriver/remote/) on the scheduled Windows job for chrome and firefox; until now they ran only on RBE.

🔧 Implementation Notes

  • The remote/ directory moves out of the remote-common glob into its own test-<browser>-remote-grid suite with the same -<browser>-remote target suffix, so the per-file targets, the test index, and test-<browser>-remote (still 63 tests for chrome) are unchanged.
  • A test-<browser>-grid aggregate wraps that suite for the nightly matrix; it also includes test-<browser>-remote-bidi when REMOTE_BIDI_TESTS is non-empty, which it currently is not, so a BiDi file opted in later reaches the Windows job without a CI edit.
  • One Grid starts per file, 16 for 19 tests across both browsers; merging them into one target per browser would change the per-file target names, so that is left for later.

🤖 AI assistance

  • AI assisted (complete below)
    • Tool(s): Claude Code (Fable 5.1)
    • What was generated: the CI coverage analysis, the suite wiring, this description
    • I reviewed all AI output and can explain the change

💡 Additional Considerations

🔄 Types of changes

  • Cleanup (formatting, renaming)

@selenium-ci selenium-ci added C-py Python Bindings B-build Includes scripting, bazel and CI integrations labels Sep 18, 2026
@qodo-code-review

Copy link
Copy Markdown
Contributor

PR Summary by Qodo

Run Python Grid-specific suites in Windows nightly CI

🧪 Tests ⚙️ Configuration changes 📝 Documentation 🕐 20-40 Minutes

Grey Divider

AI Description

• Isolates Grid-specific Python tests into Chrome and Firefox Bazel suites.
• Preserves existing remote aggregates and per-file target naming.
• Adds Grid suites to Windows CI and documents local execution.
Diagram

graph TD
  Nightly["Windows Browser CI"] --> Matrix["Grid Matrix"] --> Aggregate["Grid Aggregate"] --> Classic["Remote Grid Suite"] --> Tests["Grid Test Files"] -->|"Remote WebDriver"| Server["Standalone Grid"]
  Aggregate -.->|when configured| Bidi["Remote BiDi Suite"] -->|"Remote WebDriver"| Server
Loading
High-Level Assessment

The following are alternative approaches to this PR:

1. Batch tests per browser
  • ➕ Reduces repeated Selenium Grid startup overhead.
  • ➕ May shorten nightly execution time.
  • ➖ Changes established per-file Bazel target names.
  • ➖ Reduces test isolation and failure granularity.
2. Run full remote suites on Windows
  • ➕ Provides broader Windows remote coverage.
  • ➕ Avoids introducing dedicated Grid aggregates.
  • ➖ Substantially increases nightly execution cost.
  • ➖ Duplicates coverage already provided by Linux RBE.

Recommendation: Keep the scoped Grid aggregates because they add targeted Windows coverage while preserving existing remote targets, per-file naming, and test indexing. Consider browser-level batching later only if repeated Grid startup becomes a material runtime problem.

Files changed (3) +53 / -3

Tests (1) +51 / -2
BUILD.bazelSplit Grid tests into dedicated browser suites +51/-2

Split Grid tests into dedicated browser suites

• Removes webdriver/remote tests from remote-common and generates dedicated Chrome and Firefox remote-grid suites. Preserves the full remote aggregates, adds lightweight Grid aggregates, and conditionally includes opted-in remote BiDi tests.

py/BUILD.bazel

Documentation (1) +1 / -0
TESTING.mdDocument the Grid-only Bazel target +1/-0

Document the Grid-only Bazel target

• Adds an example for running only Chrome's Grid-specific Python tests.

py/TESTING.md

Other (1) +1 / -1
ci-python.ymlAdd Chrome and Firefox Grid jobs to Windows CI +1/-1

Add Chrome and Firefox Grid jobs to Windows CI

• Extends the Windows browser matrix with chrome-grid and firefox-grid entries, invoking the new browser-specific Grid aggregate targets.

.github/workflows/ci-python.yml

@qodo-code-review

qodo-code-review Bot commented Sep 18, 2026 •

Copy link
Copy Markdown
Contributor

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📜 Skill insights (0)

Grey Divider


Action required

1. Two suite comments restate build logic ✓ Resolved 📘 Rule violation ⚙ Maintainability
Description
The comments above the new Grid suite and modified remote aggregate repeat the target names,
supported browsers, and suite composition expressed directly below them. A later change to either
target list must update redundant prose separately, allowing the comments to become stale and
mislead maintainers.
Code

py/BUILD.bazel[1241]

+# Generate test-<browser>-remote-grid targets (chrome and firefox only)
Evidence
Compliance rule 7 reserves comments for rationale or non-obvious context. The cited comments
directly summarize the adjacent generated target and aggregate definitions rather than explaining
why those definitions are necessary.

AGENTS.md: Write Comments That Explain Why Rather Than What
py/BUILD.bazel[1241-1241]
py/BUILD.bazel[1351-1351]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Two changed comments merely restate the target names, supported browsers, and suite composition already expressed by the adjacent build definitions.

## Fix Focus Areas
- py/BUILD.bazel[1241-1241]
- py/BUILD.bazel[1351-1351]

## Recommended Fix
Remove these two comments, or replace them only if there is non-obvious rationale or a constraint that the build definitions themselves cannot communicate.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Context sources
Review mode: ⏭️ Skipped: The latest push only removes comments in a BUILD file and has no behavioral or semantic effect.

Grey Divider

Tip of the day
💡 Did you know, you can describe a rule in plain language on the Rules page and Qodo drafts it for you

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

Comment thread py/BUILD.bazel Outdated
@titusfortner titusfortner changed the title [py] split the Grid-specific tests into test-<browser>-grid suites and run them on the Windows nightly [py] run grid specific tests on windows during scheduled tests Sep 18, 2026
@qodo-code-review

Copy link
Copy Markdown
Contributor

Code review by qodo was updated up to the latest commit 59f0905

@titusfortner
titusfortner merged commit 3e8b086 into SeleniumHQ:trunk Sep 19, 2026
30 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

B-build Includes scripting, bazel and CI integrations C-py Python Bindings

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants