Skip to content

docs(cli): do not suggest generating test scenarios in parallel - #42833

Merged
Yury Semikhatsky (yury-s) merged 1 commit into
microsoft:mainfrom
yury-s:skill-sequential-generation
Sep 21, 2026
Merged

Yury Semikhatsky (yury-s) merged 1 commit into
microsoft:mainfrom
yury-s:skill-sequential-generation

Conversation

@yury-s

Copy link
Copy Markdown
Member

Summary

  • Section 2.2 of the test generation skill says scenarios must be generated in sequence, while 2.3 claimed the loop is safe to parallelise. Concurrent npx playwright test runs in the same directory are not safe, so drop that claim.

@yury-s
Yury Semikhatsky (yury-s) merged commit 2a095f6 into microsoft:main Sep 21, 2026
15 of 17 checks passed
@yury-s
Yury Semikhatsky (yury-s) deleted the skill-sequential-generation branch September 21, 2026 18:09
@github-actions

Copy link
Copy Markdown
Contributor

Test results for "MCP"

4 failed
❌ [chrome] › mcp/cli-webmcp.spec.ts:188 › webmcp-call disambiguates same-name tools in identical same-origin frames @mcp-macos-latest-chrome
❌ [firefox] › mcp/cli-keyboard.spec.ts:19 › press @mcp-windows-latest-firefox
❌ [firefox] › mcp/cli-session.spec.ts:54 › idle timeout shuts the session down @mcp-windows-latest-firefox
❌ [firefox] › mcp/http.spec.ts:145 › http transport browser sigint @mcp-windows-latest-firefox

8688 passed, 1474 skipped


Merge workflow run.

@github-actions

Copy link
Copy Markdown
Contributor

🟢 All four failures are pre-existing flakes — this PR is clear

Hi, I'm the Playwright bot and I triaged the failing MCP jobs.

This PR changes one line of markdown in a CLI skill reference, so it can't reach any product code path. All four failing tests also fail on other SHAs in the results DB, three of them on main pushes.

Details

Pre-existing flake / infra

  • [firefox] › mcp/cli-session.spec.ts:54 › idle timeout shuts the session down — the noisy one. On mcp-windows-latest-firefox it failed 21 of 53 runs (40%), across 21 distinct SHAs, and passes on every run on ubuntu (44/44) and macOS (43/43). Recent failures on unrelated SHAs: b8627f95, f14e9f5c, dc82cd77 — all main pushes. A windows-only timing race in the idle-shutdown path.

    🟩🟥🟩🟥🟥🟥🟩🟩🟩🟩🟩🟩🟩🟩🟩🟥🟩🟥🟩🟩🟥🟩🟩🟥🟩🟥

  • [chrome] › mcp/cli-webmcp.spec.ts:188 › webmcp-call disambiguates same-name tools in identical same-origin frames — also failed on #42792, #42789 and #42785, and passes 14/14 on windows.

  • [firefox] › mcp/cli-keyboard.spec.ts:19 › press — 6 of 427 runs fail, windows-firefox only, 0/124 on ubuntu and 0/169 on macOS. Most recently on dc82cd77, a main push.

  • [firefox] › mcp/http.spec.ts:145 › http transport browser sigint — 5 of 466 runs, again windows-firefox only, e.g. 96af9544 (push) and #42663.

Every one of these is windows-firefox or a known-bimodal chrome test, and this PR touches references/test-generation.md only — no runtime code ships from it.

Triaged by the Playwright bot - agent run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants