Skip to content

feat(evaluate): add "world" option to evaluate methods - #42635

Merged
Dmitry Gozman (dgozman) merged 1 commit into
microsoft:mainfrom
dgozman:evaluate-world
Sep 9, 2026
Merged

Dmitry Gozman (dgozman) merged 1 commit into
microsoft:mainfrom
dgozman:evaluate-world

Conversation

@dgozman

Copy link
Copy Markdown
Collaborator

Summary

  • Adds world: 'main' | 'utility' to Page/Frame/Locator.evaluate, Locator.evaluateAll and Page/Frame/ElementHandle.$eval/$$eval. The utility world shares the DOM with the page, but has an isolated JavaScript environment.
  • Methods returning a handle are intentionally left out — utility world handles are harder to reason about.
  • $$eval resolves the elements in the requested world; when a custom selector engine forces main world resolution, the elements are moved over so the page function still runs in the requested world.

Allows running the page function in the isolated utility world, which
shares the DOM with the page but has a separate JavaScript environment.

Supported on the methods that return a serializable value:
Page/Frame/Locator.evaluate, Locator.evaluateAll and
Page/Frame/ElementHandle.$eval/$$eval. Methods returning a handle are
not included, since utility world handles are hard to reason about.
@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Test results for "MCP"

3 failed
❌ [firefox] › mcp/annotate.spec.ts:446 › should switch screencast to -s session on show --annotate @mcp-windows-latest-firefox
❌ [firefox] › mcp/cli-core.spec.ts:97 › fill @mcp-windows-latest-firefox
❌ [msedge] › mcp/http.spec.ts:105 › http transport browser lifecycle (isolated) @mcp-windows-latest-msedge

8345 passed, 1376 skipped


Merge workflow run.

@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Test results for "tests 1"

10 flaky ⚠️ [chromium-library] › library/browsercontext-page-event.spec.ts:173 › should work with Ctrl-clicking `@chromium-ubuntu-22.04-arm-node20`
⚠️ [chromium-library] › library/video.spec.ts:664 › screencast › should capture full viewport `@chromium-ubuntu-22.04-arm-node20`
⚠️ [chromium-library] › library/video.spec.ts:699 › screencast › should capture full viewport on hidpi `@realtime-time-library-chromium-linux`
⚠️ [chromium-library] › library/chromium/chromium.spec.ts:436 › should produce network events, routing, and annotations for Service Worker (advanced) `@chromium-ubuntu-22.04-node20`
⚠️ [chromium-library] › library/chromium/connect-over-cdp.spec.ts:64 › should write traces to provided artifactsDir on connectOverCDP `@chromium-ubuntu-22.04-node22`
⚠️ [chromium-library] › library/video.spec.ts:664 › screencast › should capture full viewport `@chromium-ubuntu-22.04-node22`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:257 › third party 'Partitioned;' cookies `@firefox-ubuntu-22.04-node20`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:470 › top level 'Partitioned;' cookie and same origin iframe `@firefox-ubuntu-22.04-node20`
⚠️ [firefox-page] › page/page-goto.spec.ts:90 › should work with Cross-Origin-Opener-Policy `@firefox-ubuntu-22.04-node20`
⚠️ [playwright-test] › ui-mode-test-update.spec.ts:202 › should update test locations `@ubuntu-latest-node22`

51479 passed, 1240 skipped


Merge workflow run.

@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Hi, I'm the Playwright bot and I triaged this PR's CI failures.

🟢 CI is clear — all three failures are pre-existing flakes

The only failing job is MCP, and all three failures are known flakes on Windows that flip verdict across many unrelated SHAs. This PR only adds the world option to evaluate/$eval in playwright-core and doesn't touch the MCP package or the flows these tests exercise.

Details

Overall: No failure reaches the PR's diff. The change is scoped to evaluate/$eval/evaluateAll (client, dispatchers, dom.ts, frames.ts, protocol) plus docs and one new test, tests/page/page-evaluate-world.spec.ts. None of the failing tests touch evaluate-world; each has a long flake history on SHAs the PR can't be responsible for.

Pre-existing flake / infra

  • [firefox] › mcp/annotate.spec.ts:446 › should switch screencast to -s session on show --annotate — pre-existing flake. On the firefox project it failed 46 of 710 runs (~6.5%) across 46 distinct SHAs, passing the other 664; it also flakes on chromium and msedge. Unrelated to evaluate-world.
  • [firefox] › mcp/cli-core.spec.ts:97 › fill — pre-existing flake. On firefox it failed 3 of 713 runs across 3 distinct SHAs, passing 710. The fill MCP flow doesn't exercise the evaluate world option.
  • [msedge] › mcp/http.spec.ts:105 › http transport browser lifecycle (isolated) — pre-existing flake. On msedge it failed 3 of 707 runs across 3 distinct SHAs, and it flakes on all four projects (chromium 4/721, firefox 2/713, webkit 1/723). A browser-lifecycle/transport test, untouched by this PR.

Triaged by the Playwright bot - agent run

type: enum?
literals:
- main
- utility

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we use a separate world for user evals to avoid interference with playwright internals in the utility world?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's too big of a change for my taste. But I will make a not for the API review.

@dgozman
Dmitry Gozman (dgozman) merged commit aeebee1 into microsoft:main Sep 9, 2026
43 of 45 checks passed
@kblok

Copy link
Copy Markdown
Contributor

Dmitry Gozman (@dgozman) why do you say that handles are harder to reason about?
Can't we adopt them to the main world before returning them?

@dgozman

Copy link
Copy Markdown
Collaborator Author

Dmitry Gozman (Dmitry Gozman (@dgozman)) why do you say that handles are harder to reason about? Can't we adopt them to the main world before returning them?

Only the element handles. And they would not preserve any JS properties. I think that's not very obvious from the API standpoint, so I decided to not rush with utility-world handles 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants