-
Notifications
You must be signed in to change notification settings - Fork 278
Closed as not planned
Labels
Description
Smoke test results for Claude engine validation run §22656026286.
PR used for review testing: #19493 (copilot/fix-safe-inputs-script-error)
Overall Status: PARTIAL (1 failure, 2 skipped)
Core Tests (#1–10)
| # | Test | Result | Notes |
|---|---|---|---|
| 1 | GitHub MCP - Last 2 merged PRs | Response exceeded token limit (71k chars); data retrieved via gh CLI fallback. PRs: #19502, #19501 | |
| 2 | Safe Inputs GH CLI | ✅ | PRs returned: #19509, #19507 |
| 3 | Serena MCP | ❌ | Project activated OK; find_symbol failed with EOF (MCP server closed connection repeatedly) |
| 4 | Make Build | ✅ | make build succeeded, binary compiled |
| 5 | Playwright | ✅ | Title: "GitHub · Change is constant. GitHub keeps you ahead. · GitHub" |
| 6 | Tavily Web Search | ✅ | 3 results for "GitHub Agentic Workflows" |
| 7 | File Writing | ✅ | /tmp/gh-aw/agent/smoke-test-claude-22656026286.txt created |
| 8 | Bash Tool | ✅ | File read back successfully |
| 9 | Discussion Interaction | ✅ | Discussion #19508 found; comic-book comment added |
| 10 | Agentic Workflows MCP | ✅ | smoke-claude: active, compiled, triggers: PR/label, reaction:heart, every 12h |
PR Review Tests (#11–17)
| # | Test | Result | Notes |
|---|---|---|---|
| 11 | Update PR Body | ✅ | Appended test message to PR #19493 |
| 12 | PR Review Comments | ✅ | 2 comments added on safe-inputs-runner.cjs:27 and safe_inputs_generator.go:232 |
| 13 | Submit PR Review | ✅ | COMMENT review submitted |
| 14 | Resolve Review Thread | Skipped — no existing review threads on PR | |
| 15 | Add Reviewer | ✅ | copilot added as reviewer |
| 16 | Push to PR Branch | ✅ | Committed and pushed smoke-test-push-22656026286.json to branch |
| 17 | Close PR | Skipped — no safe test PR to close |
Failure Details
Test 3 — Serena find_symbol (EOF)
activate_projectsucceeded for/home/runner/work/gh-aw/gh-aw- Subsequent
find_symbolcalls failed withMCP error 0: connection closed: calling "tools/call": client is closing: EOF - Tried patterns:
main,run(substring),runSafeInput— all failed
References:
- §22656026286
- Timestamp: 2026-03-04T05:12:00Z
💥 [THE END] — Illustrated by Smoke Claude · ◷
- expires on Mar 4, 2026, 7:19 AM UTC
Reactions are currently unavailable