Skip to content

test(agents): check model lookup Cancel by visible text - #259

Merged
loganj merged 1 commit into
mainfrom
larry/flake-agent-model-picker
Sep 25, 2026
Merged

loganj merged 1 commit into
mainfrom
larry/flake-agent-model-picker

Conversation

@loganj

@loganj loganj commented Sep 25, 2026

Copy link
Copy Markdown
Contributor

🤖

Summary

  • Fixes an intermittent failure in the model picker test for Pi agents. On main and several branches it failed about one run in three.
  • The test checks that a Cancel model lookup button is visible while the model list loads. While the model menu is open, the menu hides the rest of the screen from assistive technology. It does this on its own timing, so the button sometimes had no accessible name when the test looked for it by name.
  • The test now waits for the "Loading Pi models…" status. Then it finds the button by its visible text and checks that the button is visible. What the test checks stays the same: Cancel is shown during the lookup.

Details

  • Test-only change in src/bundled/agents/AgentModelPicker.test.tsx.
  • Local evidence: 11 of 32 runs failed before the change, and 48 of 48 runs passed after it.

The open model Combobox marks outside content aria-hidden on its own
schedule. That removes the Cancel button's accessible name, so the
role query failed about 1 in 3 local runs. Check Cancel after the
lookup status renders, and find it by its visible text.

Signed-off-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz>
@loganj
loganj marked this pull request as ready for review September 25, 2026 15:23
@loganj
loganj requested review from a team, comp615 and wesbillman as code owners September 25, 2026 15:23

@wesbillman wesbillman left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Carl, an automated reviewer, commenting via Wes’s GitHub account.

No actionable findings at head 2fcc2e9edb5a2fe43c398524f9601f72fdb5a07b against base 01523e2c1781da1cd84620e61188df6c3d365938. Comment only, not approval.

The replacement still requires the actual Cancel button to be visible while a deferred lookup is held pending. It retains the named loading-status and busy-state assertions, then verifies successful model selection. The neighboring cancellation test still finds Cancel by accessible role/name after closing the popup and exercises cancellation, late-result rejection and retry. No production change, sleeps, retries or weakened completion checks.

Hosted CI is green, including CI required. The JavaScript log confirms 9/9 picker tests and 3,788 tests across 347 files passed on synthetic merge 0f1915350787ad5f8da72656a4b24c327f272df2. Current main has no changes to the picker, Combobox, Button or dependency lockfile since this base. I did not rerun tests locally or independently reproduce the author's repeated-run flake statistics; a passing run is not proof that every possible flake is gone.

@loganj
loganj merged commit a7a5727 into main Sep 25, 2026
12 checks passed
@loganj
loganj deleted the larry/flake-agent-model-picker branch September 25, 2026 17:56
johnmatthewtennant pushed a commit that referenced this pull request Sep 25, 2026
* origin/main:
  Explain missing Pi provider models (#263)
  Browse Goose models and enter provider API keys (#230)
  test(agents): check model lookup Cancel by visible text (#259)
  Ask before mentioning people outside the channel (#257)
  Refine direct message opening (#107)
  feat(messages): report messages to community moderators (#255)

Signed-off-by: Sol <49aa1f65411fd096d2e2ec144f1e7aa36fdc76d1b907cfdf7be000c66f9d3b8e@buzz.block.builderlab.xyz>
johnmatthewtennant pushed a commit that referenced this pull request Sep 25, 2026
* origin/main:
  ci: publish scheduled macOS test prereleases (#262)
  feat: add private text feedback plugin (#242)
  🤖 docs: add pre-PR checklist and review guidance to AGENTS.md (#268)
  perf(sidebar): stop rerendering every row's menu on channel switch (#265)
  Explain missing Pi provider models (#263)
  Browse Goose models and enter provider API keys (#230)
  test(agents): check model lookup Cancel by visible text (#259)

Signed-off-by: Sol <49aa1f65411fd096d2e2ec144f1e7aa36fdc76d1b907cfdf7be000c66f9d3b8e@buzz.block.builderlab.xyz>

# Conflicts:
#	src/bundled/profiles/ProfileAgentIdentity.test.tsx
zrmarley added a commit that referenced this pull request Sep 25, 2026
…-image

* origin/main: (23 commits)
  fix(agents): recover status polling and scope failure diagnostics (#283)
  Share avatar editing across community profiles and managed agents (#271)
  feat(profiles): archive, unarchive and delete agents from the profile pane (#256)
  ci: run browser journeys on three shards per engine (#280)
  ci: publish scheduled macOS test prereleases (#262)
  feat: add private text feedback plugin (#242)
  🤖 docs: add pre-PR checklist and review guidance to AGENTS.md (#268)
  perf(sidebar): stop rerendering every row's menu on channel switch (#265)
  Explain missing Pi provider models (#263)
  Browse Goose models and enter provider API keys (#230)
  test(agents): check model lookup Cancel by visible text (#259)
  Ask before mentioning people outside the channel (#257)
  Refine direct message opening (#107)
  feat(messages): report messages to community moderators (#255)
  perf(channels): stop rerendering message rows after each channel switch (#269)
  feat(profiles): open targeted agent editor from owner profile (#254)
  Let plugins declare local commands and HTTPS origins (#169)
  feat(profiles): show agent metadata and copyable nip05 (#253)
  Organize app and community settings (#173)
  Add status badge cutouts to avatars (#211)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants