Skip to content

Smoke Test: Claude - 22487258068 #18683

Description

@github-actions

Test Results Summary

Overall Status: PARTIAL — Most tests passed, with 1 failure, 1 partial failure, and 2 skips.

PR Used for Tests: #18669 (chore: bump CLI tool versions)
Run: §22487258068
Timestamp: 2026-02-27T13:05:00Z


Core Tests (#1–10)

# Test Result Notes
1 GitHub MCP Testing Response exceeded 25k token limit (73k chars)
2 Safe Inputs GH CLI Testing Got 2 PRs: #18671, #18669
3 Serena MCP Testing ⚠️ activate_project succeeded; find_symbol failed with EOF
4 Make Build Testing make build succeeded
5 Playwright Testing Page title contains "GitHub"
6 Tavily Web Search Testing 3 results returned for "GitHub Agentic Workflows"
7 File Writing Testing /tmp/gh-aw/agent/smoke-test-claude-22487258068.txt created
8 Bash Tool Testing File contents verified via cat
9 Discussion Interaction Testing Discussion #18673 found; comic-book comment added
10 Agentic Workflows MCP Testing smoke-claude: active, compiled, engine=claude

PR Review Tests (#11–17)

# Test Result Notes
11 Update PR Testing PR #18669 body appended with test message
12 PR Review Comment Testing 2 inline comments added on different files
13 Submit PR Review Testing Review submitted (COMMENT)
14 Resolve Review Thread Testing ⚠️ Skipped: GraphQL thread node IDs not retrievable via safe inputs
15 Add Reviewer Testing copilot added as reviewer to PR #18669
16 Push to PR Branch Testing Failed: "context is not defined" error from safeoutputs
17 Close PR Testing ⚠️ Skipped: no safe test PR to close

Failure Details

Test #1 — GitHub MCP list_pull_requests token overflow

mcp__github__list_pull_requests response exceeded the 25,000 token limit (73,588 characters). Used safeinputs-gh as fallback for PR data retrieval.

Test #3 — Serena find_symbol EOF errors

activate_project succeeded and created project gh-aw. Subsequent find_symbol calls failed with MCP error 0: connection closed: calling "tools/call": client is closing: EOF. LSP server may need more initialization time.

Test #16 — push_to_pull_request_branch context error

File was committed locally (tmp-smoke-test-22487258068.txt on branch copilot/update-cli-tool-versions-another-one), but push failed with backend error: code=-32603, message=context is not defined.

References:

💥 [THE END] — Illustrated by Smoke Claude

  • expires on Feb 27, 2026, 3:10 PM UTC

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions