diff --git a/plugins/issue-driven-dev/skills/idd-verify/SKILL.md b/plugins/issue-driven-dev/skills/idd-verify/SKILL.md index c1ce8a5..87ccecc 100644 --- a/plugins/issue-driven-dev/skills/idd-verify/SKILL.md +++ b/plugins/issue-driven-dev/skills/idd-verify/SKILL.md @@ -2,7 +2,7 @@ name: idd-verify description: | 驗證 uncommitted/committed/PR code 是否滿足 Issue 的所有要求。 - 預設用 Agent Team（5 Claude reviewers 互相挑戰）+ Codex CLI（gpt-5.5）平行驗證。 + 預設用 5 個 general-purpose Agents（Claude reviewers 互相挑戰）+ Codex CLI（gpt-5.5）平行驗證。 6 個獨立 AI、兩個模型家族、互相看不到對方的結果。 支援 cluster verify（v2.34.0+）：多個 #N 共用 1 PR 時（如 `#34 #36 #38`），report 按 issue 分區段。 支援 external-agent / PR mode（v2.37.0+）：`--pr ` 驗證外部 agent（Codex/Copilot）開的 PR，PR 是 master comment、ref'd issue 拿 pointer；`--commits N` / `--since ` / `--branch ` 為其他輸入來源；缺 flag 時 auto-detect 本地 commits 與 open PR 並 AskUserQuestion。 @@ -23,7 +23,6 @@ allowed-tools: - Glob - Grep - Agent - - TeamCreate - SendMessage - AskUserQuestion --- @@ -75,9 +74,9 @@ PR mode 下： ## 參數 ``` -/idd-verify #42 → Agent Team (5) + Codex 平行（預設）；auto-detect input source +/idd-verify #42 → 5 general-purpose Agents + Codex 平行（預設）；auto-detect input source /idd-verify #42 codex → 只用 Codex CLI -/idd-verify #42 team → 只用 Agent Team（不跑 Codex） +/idd-verify #42 team → 只用 5 general-purpose Agents（不跑 Codex；legacy alias `team` 保留 backward compat） /idd-verify #42 --loop → 驗證 + ralph-loop 自動修復迴圈 /idd-verify #42 --pr 123 → PR mode（master 落在 PR、issue 拿 pointer） /idd-verify --pr 123 → PR mode 不帶 issue：從 PR body Refs #N 自動 discover @@ -104,7 +103,7 @@ PR mode 下： ``` idd-verify #NNN │ -├── Agent Team（5 Claude teammates，互相挑戰） +├── 5 general-purpose Agents（Claude reviewers，互相挑戰；file-based output） │ ├── Requirements — issue 要求覆蓋率 │ ├── Logic — 邏輯正確性、edge cases、null handling │ ├── Security — injection、權限、hardcoded secrets @@ -112,13 +111,13 @@ idd-verify #NNN │ └── Devil's Advocate — 反駁前四個的「通過」判斷 │ └── Codex CLI（gpt-5.5 xhigh，獨立 process） - └── 完全獨立，看不到 team 的討論 + └── 完全獨立，看不到其他 reviewer Agents 的 findings 檔 → 6 個 findings 合併去重 → 呈現結果 ``` **為什麼 6 個？** -- 5 個 Claude teammates 在同一個 team 裡**互相挑戰**（不是各自獨立報告） +- 5 個 Claude reviewers 各為獨立 `Agent(subagent_type=general-purpose)` call，平行 spawn 並寫 file-based output；Devil's Advocate 透過 polling 等其他 4 個 findings 檔產生後讀取 + 反駁（v2.59.0+, #52 重構自 pre-2.59 TeamCreate model — 詳見 Step 2 Engine note） - Devil's Advocate 的工作是**試著證明其他 4 個的通過判斷是錯的** - Codex 是完全不同的模型家族（gpt-5.5），提供**跨模型盲驗** @@ -133,8 +132,9 @@ TaskCreate(name="resolve_input_source", description="Step 0.5: 解析 --pr / --c TaskCreate(name="gate_pr_correspondence", description="Step 0.7: PR mode 下強制檢查 issue↔PR 對應 — gh pr view --json body 抓 Refs #N，跟 user 指定的 issue 比對；PR 沒任何 Refs 或 user issue 不在 set 內 → abort 並告訴使用者怎麼修") TaskCreate(name="get_diff_and_issue", description="依 input source 取 diff（gh pr diff / git diff HEAD~N / git diff origin/...） + gh issue view,存 diff 到 /tmp 供 agents 讀取；PR mode 額外做 gh pr checkout 並記住原 branch") TaskCreate(name="check_attachments", description="確認 .claude/.idd/attachments/issue-NNN/ 存在,把 attachment 路徑塞進 reviewer agent prompt 作為 source-of-truth context。manifest 缺漏 → 警告繼續(reviewer 仍跑,但 verification 完整度受限)。依 rules/process-attachments.md。") -TaskCreate(name="launch_parallel_reviewers", description="6 個 tool calls 同一 message: TeamCreate + 5 Agent(requirements/logic/security/regression/devils-advocate) + 1 Bash codex,prompt 中引用 attachment 路徑") -TaskCreate(name="wait_for_claude_agents", description="等 5 Claude teammates 全部 idle,讀 /tmp/verify_${NUMBER}_findings_*.md") +TaskCreate(name="launch_parallel_reviewers", description="6 個 tool calls 同一 message: 5 Agent(subagent_type=general-purpose) for requirements/logic/security/regression/devils-advocate + 1 Bash codex(run_in_background:true),prompt 中引用 attachment 路徑 + 強制 file-output rule (per #52 v2.59.0+,replaces TeamCreate model from #47 incident)") +TaskCreate(name="wait_for_claude_agents", description="5 Agent calls 是 blocking,return 後立刻 ls /tmp/verify_${NUMBER}_findings_*.md 確認 5 個 findings 檔都 non-empty;缺者進 Step 2.5 Recovery Protocol") +TaskCreate(name="recovery_protocol", description="Step 2.5 (NEW per #52): 缺 findings 檔者 SendMessage retry with FULL context re-paste(不假設 context 倖存 idle/wake);二次 idle → coordinator self-review for that role + 在 master report 標 process gap") TaskCreate(name="wait_for_codex", description="等 Codex 背景任務完成,讀 /tmp/codex-verify-${NUMBER}.md") TaskCreate(name="merge_findings", description="合併 6 個來源 findings 去重,severity 取最高") TaskCreate(name="post_master_and_pointers", description="PR mode: master 貼到 PR + capture URL → 為每個 ref'd issue 貼 pointer comment；本地 mode: 貼到 issue（單 issue 直接貼／多 issue 用 SOP master+pointer）") @@ -148,9 +148,10 @@ TaskCreate(name="triage_followup_issues", description="Step 5b: 分類 non-block **v2.32.0+ tagging 規則**：若 Verify findings comment 要 @-tag 寫 code 的人或要求審閱者，**必須**遵循 [`rules/tagging-collaborators.md`](../../rules/tagging-collaborators.md) 5 步協定（gh api → fuzzy match → AskUserQuestion fallback → @login 不用 display name → post 前 verify）。違反 = 通知錯人，不可逆。 **鐵律**: -- `wait_for_claude_agents` 和 `wait_for_codex` 都要跑到真的有 findings 內容,不能只看到 idle notification 就 completed -- 如果某個 reviewer 沒寫 findings 檔,用 `SendMessage` 請它補寫,或 fallback 自己做 coordinator backup 檢查 +- `wait_for_claude_agents` 和 `wait_for_codex` 都要跑到真的有 findings 內容,不能只看到 Agent return / idle notification 就 completed — 必須 `ls /tmp/verify_${NUMBER}_findings_*.md` 確認 5 個檔案 + non-empty +- 如果某個 reviewer 沒寫 findings 檔 → 進 Step 2.5 Recovery Protocol(SendMessage retry with FULL context re-paste);二次 idle → coordinator self-review fallback + master report 標 process gap - `comment_to_issue` 一定要實際 post 到 GitHub,不是只在對話中顯示 +- **絕對禁用** `subagent_type=Explore` for reviewer agents — Explore 是 read-only,**沒有 Write tool**,無法寫 findings 檔(#47 incident proved this; per #52 v2.59.0+ 強制 general-purpose) --- @@ -247,95 +248,173 @@ Exit code: 把 attachment 路徑列入 Step 2 的 reviewer prompt 作為 source-of-truth context(尤其 requirements reviewer 需要原始需求文件)。**禁止**只在 prompt 寫「issue 有附件」而不給具體 path — reviewer agents 看不到 path 等於沒附件。 -### Step 2: 平行啟動 Agent Team + Codex +### Step 2: 平行啟動 5 Reviewer Agents + Codex (v2.59.0+, #52) **CRITICAL: 6 個 tool calls（5 Agent + 1 Bash codex）必須在同一個 message 送出。不可分步驟。** -#### 2a. Agent Team（5 reviewers） - -用 TeamCreate 建立 team： - -``` -TeamCreate: - name: "verify-{NUMBER}" - teammates: - - name: "requirements" - prompt: | - 你是 Requirements Reviewer。 - Review code changes for Issue #{NUMBER}: {title} - - Issue body: - {body} - - 你的任務：逐一檢查 issue 的每個要求是否在 code 中被實現。 - 對每個要求標記：FULLY / PARTIALLY / NOT addressed。 - 用 Read/Grep 工具實際去看相關檔案確認。 - tools: Read, Grep, Glob, Bash - # model 省略 → 繼承主對話模型 - - - name: "logic" - prompt: | - 你是 Logic Reviewer。 - Review code changes for Issue #{NUMBER}: {title} - - Changes: - {diff} - - 你的任務：檢查邏輯正確性。 - - Edge cases（null、empty、boundary values） - - 型別安全（numeric vs character、NA handling） - - 控制流程（if/else 覆蓋、switch fall-through） - 用 Read 工具查看完整函數上下文。 - tools: Read, Grep, Glob, Bash - # model 省略 → 繼承主對話模型 - - - name: "security" - prompt: | - 你是 Security Reviewer。 - Review code changes for Issue #{NUMBER}: {title} - - Changes: - {diff} - - 你的任務：檢查安全問題。 - - SQL injection（字串拼接 vs parameterized） - - Hardcoded secrets - - 權限檢查 - - 輸入驗證 - tools: Read, Grep, Glob, Bash - # model 省略 → 繼承主對話模型 - - - name: "regression" - prompt: | - 你是 Regression Reviewer。 - Review code changes for Issue #{NUMBER}: {title} - - Changes: - {diff} - - 你的任務： - 1. 有沒有改到 issue 範圍外的東西（scope creep）？ - 2. 改動有沒有破壞既有功能？ - 3. 有沒有引入新的 dependency 但沒處理？ - 用 Grep 搜尋被改動的函數在哪裡被呼叫。 - tools: Read, Grep, Glob, Bash - # model 省略 → 繼承主對話模型 - - - name: "devils-advocate" - prompt: | - 你是 Devil's Advocate。 - Review code changes for Issue #{NUMBER}: {title} - - 你的任務：等其他 4 個 reviewer 完成後， - 讀取他們的結論，然後**試著反駁每一個「通過」的判斷**。 - - 如果他們說「FULLY addressed」，你要找理由說它其實沒有。 - 如果他們說「no security issues」，你要找他們漏掉的攻擊向量。 - 如果你找不到反駁的理由，才承認確實通過。 - - 這是對抗性驗證 — 你的存在是為了防止群體盲點。 - tools: Read, Grep, Glob, Bash - # model 省略 → 繼承主對話模型 +> **Engine note (#47 incident lesson, #52 v2.59.0+ fix)**： +> +> **絕對禁用** `subagent_type=Explore` for reviewer agents。Explore agent 是 read-only（per Agent tool docs：「All tools except Agent, ExitPlanMode, Edit, Write, NotebookEdit」），**沒有 Write tool**，無法寫 findings 檔。`#47` verify 真實發生過：spawn 5 個 Explore agents，5 個全部 idle without output，verify 退化成 1-AI (Codex only)。 +> +> **正確選擇** `subagent_type=general-purpose`：含完整 tool set (Read/Grep/Glob/Bash/**Write**/Edit)，可寫 `/tmp/verify__findings_.md`。 +> +> **不用 TeamCreate**（pre-v2.59.0 model）的原因： +> - TeamCreate teammates 必須在 `tools` field 顯式列出 Write，現有 prompt template 配置只給 Read/Grep/Glob/Bash —— 同 Explore 一樣 Write-missing failure mode +> - TeamCreate's `wait_for_idle` 在 idle/wake cycle 後 context 流失（#47 觀察到） +> - **Side benefit**：no team → no `TeamDelete` cleanup gap (#70 dissolves structurally) +> +> 完整 #47 process gap 詳細 root cause 三層見 issue body。 + +#### 2a. 5 Reviewer Agents（parallel, file-based output） + +每個 reviewer 用 single `Agent` tool call（**not** TeamCreate teammate）。所有 5 個 + 1 個 Bash codex **必須在同一個 message** 一起發出（單 message 多 tool calls = parallel）。 + +> **Pre-spawn prompt persistence (per /idd-verify --pr 73 round 1 P1.2)**: BEFORE invoking the 5 Agent calls, coordinator MUST save each reviewer's full prompt to `/tmp/verify_${NUMBER}_prompt_.md`. Step 2.5b Recovery Protocol re-paste step reads these files; if they don't exist, retry fails. Save via: +> +> ```bash +> # Coordinator runs BEFORE Agent invocations +> cat > /tmp/verify_${NUMBER}_prompt_requirements.md <<'EOF' +> 你是 Requirements Reviewer for Issue #... +> (full prompt body here, exactly as passed to Agent below) +> EOF +> # ... same for logic, security, regression, devils-advocate +> ``` +> +> Do this once per verify invocation (paths include `${NUMBER}` so different issues don't collide). The 5 prompt files + 5 findings files share the same `verify__*` naming convention. + +**Prompt template 強制要素**（每個 reviewer 都必含這 3 條，違反 = process gap）： + +1. **明示 file output path**：`Write your findings to /tmp/verify_${NUMBER}_findings_.md when done.` +2. **明示 DO NOT idle**：`Your task is NOT complete until the file is written. Do NOT idle without producing the output file.` +3. **明示 retry context expectation**：`If you receive a later SendMessage with the same prompt re-pasted, treat that as a retry signal; the original context may have been lost across an idle/wake cycle.` + +``` +Agent({ + description: "Requirements review for #${NUMBER}", + subagent_type: "general-purpose", + prompt: `你是 Requirements Reviewer for Issue #${NUMBER}: ${TITLE}. + +Issue body: +${BODY} + +Diff path: /tmp/diff_${NUMBER}.patch +Attachment paths (if any): .claude/.idd/attachments/issue-${NUMBER}/... + +你的任務：逐一檢查 issue 的每個要求是否在 code 中被實現。 +對每個要求標記：FULLY / PARTIALLY / NOT addressed。 +用 Read/Grep 工具實際去看相關檔案確認。 + +OUTPUT (mandatory): Write your findings to /tmp/verify_${NUMBER}_findings_requirements.md when done. +Your task is NOT complete until the file is written. Do NOT idle without producing the output file. +If you receive a later SendMessage with the same prompt re-pasted, treat that as a retry signal — the original context may have been lost across an idle/wake cycle.` +}) + +Agent({ + description: "Logic review for #${NUMBER}", + subagent_type: "general-purpose", + prompt: `你是 Logic Reviewer for Issue #${NUMBER}: ${TITLE}. + +Diff path: /tmp/diff_${NUMBER}.patch + +你的任務：檢查邏輯正確性。 +- Edge cases（null、empty、boundary values） +- 型別安全（numeric vs character、NA handling） +- 控制流程（if/else 覆蓋、switch fall-through） +用 Read 工具查看完整函數上下文。 + +OUTPUT (mandatory): Write findings to /tmp/verify_${NUMBER}_findings_logic.md. +Your task is NOT complete until the file is written. Do NOT idle without producing output. +If you receive a later SendMessage with the same prompt re-pasted, treat as retry signal.` +}) + +Agent({ + description: "Security review for #${NUMBER}", + subagent_type: "general-purpose", + prompt: `你是 Security Reviewer for Issue #${NUMBER}: ${TITLE}. + +Diff path: /tmp/diff_${NUMBER}.patch + +你的任務：檢查安全問題。 +- SQL injection（字串拼接 vs parameterized） +- Hardcoded secrets +- 權限檢查 +- 輸入驗證 + +OUTPUT (mandatory): Write findings to /tmp/verify_${NUMBER}_findings_security.md. +Your task is NOT complete until the file is written. Do NOT idle without producing output. +If you receive a later SendMessage with the same prompt re-pasted, treat as retry signal.` +}) + +Agent({ + description: "Regression review for #${NUMBER}", + subagent_type: "general-purpose", + prompt: `你是 Regression Reviewer for Issue #${NUMBER}: ${TITLE}. + +Diff path: /tmp/diff_${NUMBER}.patch + +你的任務： +1. 有沒有改到 issue 範圍外的東西（scope creep）？ +2. 改動有沒有破壞既有功能？ +3. 有沒有引入新的 dependency 但沒處理？ +用 Grep 搜尋被改動的函數在哪裡被呼叫。 + +OUTPUT (mandatory): Write findings to /tmp/verify_${NUMBER}_findings_regression.md. +Your task is NOT complete until the file is written. Do NOT idle without producing output. +If you receive a later SendMessage with the same prompt re-pasted, treat as retry signal.` +}) + +Agent({ + description: "Devil's Advocate review for #${NUMBER}", + subagent_type: "general-purpose", + prompt: `你是 Devil's Advocate for Issue #${NUMBER}: ${TITLE}. + +Diff path: /tmp/diff_${NUMBER}.patch + +你的任務：**等其他 4 個 reviewer 完成後**，讀取他們的結論，然後試著反駁每一個「通過」的判斷。 + +**Sequencing protocol** (因為 5 個 Agent 是 parallel spawn，沒有 TeamCreate 的 wait_for_idle): +先用 bash polling loop 等其他 4 個 findings 檔案產生，再開始你的 review: + +\`\`\`bash +# Poll for sibling findings files (max 30 iterations × 5s = 2.5 min timeout) +for i in $(seq 1 30); do + ready=0 + for role in requirements logic security regression; do + [ -s /tmp/verify_${NUMBER}_findings_$role.md ] && ready=$((ready+1)) + done + [ "$ready" = "4" ] && break + sleep 5 +done + +if [ "$ready" != "4" ]; then + # Timeout fallback: write SENTINEL marker so Step 2.5 recovery scan detects + # the timeout case (rather than treating non-empty file as valid review). + # Sentinel = literal first line "[STAGE 2.5 RECOVERY: DEVILS_ADVOCATE_TIMEOUT_/4]" + # Step 2.5a file existence check looks for this sentinel and DELETES the file + # so retry/fallback's -s test correctly sees it as missing (per round 2 P1.1 fix). + # + # NOTE on quoting (per /idd-verify --pr 73 round 2 P1.2): bash single quotes + # CANNOT escape apostrophes via backslash. Use printf '%s\n%s\n' "header" "body" + # with double-quoted args (where backslash-apostrophe is unnecessary). + printf '%s\n\n%s\n' \ + "[STAGE 2.5 RECOVERY: DEVILS_ADVOCATE_TIMEOUT_${ready}/4]" \ + "Devil's Advocate skipped: timeout waiting for sibling findings (${ready}/4 ready after 2.5min). Coordinator detects this sentinel and routes to retry once siblings arrive, or coordinator self-review fallback." \ + > /tmp/verify_${NUMBER}_findings_devils-advocate.md + exit 0 +fi +\`\`\` + +After polling succeeds, read the 4 sibling findings files, then: +- 如果他們說「FULLY addressed」，找理由說它其實沒有 +- 如果他們說「no security issues」，找他們漏掉的攻擊向量 +- 如果找不到反駁的理由，才承認確實通過 + +這是對抗性驗證 — 你的存在是為了防止群體盲點。 + +OUTPUT (mandatory): Write findings to /tmp/verify_${NUMBER}_findings_devils-advocate.md. +Your task is NOT complete until the file is written. Do NOT idle without producing output. +If you receive a later SendMessage with the same prompt re-pasted, treat as retry signal.` +}) ``` #### 2b. Codex CLI（背景執行，via companion script） @@ -352,13 +431,122 @@ Bash({ 完成後用 Read 讀取 `/tmp/codex-verify-$NUMBER.md`。 +### Step 2.5: Recovery Protocol（NEW v2.59.0+, #52） + +5 個 reviewer Agent calls return 後，**先做 file existence check**（不是直接進 Step 3 merge）。Coordinator 負責偵測哪些 reviewer 沒寫 findings 檔，並執行兩階段 recovery。 + +**Rule**: Step 3 merge logic 假設所有 5 個 findings 檔都存在 + non-empty。任何缺漏 = process gap，必須在 master report 顯式標示，**不可靜默繼續**。 + +#### 2.5a — File existence check + +```bash +EXPECTED_FILES=( + "/tmp/verify_${NUMBER}_findings_requirements.md" + "/tmp/verify_${NUMBER}_findings_logic.md" + "/tmp/verify_${NUMBER}_findings_security.md" + "/tmp/verify_${NUMBER}_findings_regression.md" + "/tmp/verify_${NUMBER}_findings_devils-advocate.md" +) + +MISSING_ROLES=() +for f in "${EXPECTED_FILES[@]}"; do + role=$(basename "$f" | sed -E 's/^verify_[0-9]+_findings_(.*)\.md$/\1/') + if [ ! -s "$f" ]; then + # File missing or empty + MISSING_ROLES+=("$role") + elif head -1 "$f" | grep -q '^\[STAGE 2.5 RECOVERY: DEVILS_ADVOCATE_TIMEOUT_'; then + # Devil's Advocate sentinel — file exists but reviewer didn't actually run + # (timed out waiting for siblings, per Step 2 DA polling loop fallback). + # DELETE the sentinel file so downstream 2.5b retry polling (`-s` check) + # and 2.5c fallback (`! -s` check) correctly see it as missing + # (per /idd-verify --pr 73 round 2 P1.1 fix — sentinel file IS non-empty, + # so without `rm` it would silently pass downstream -s checks). + echo "→ Detected DEVILS_ADVOCATE_TIMEOUT sentinel for $role — deleting + routing to retry/fallback" + rm -f "$f" + MISSING_ROLES+=("$role") + fi +done + +if [ ${#MISSING_ROLES[@]} -eq 0 ]; then + echo "→ All 5 reviewer findings files present + valid. Proceeding to Step 3 merge." +else + echo "→ Recovery Protocol fires for: ${MISSING_ROLES[*]}" + # 進 2.5b +fi +``` + +#### 2.5b — Retry with FULL context re-paste + +對每個缺漏的 role： + +1. **Send retry message** to that Agent's running instance via `SendMessage` (if Agent name is addressable; else this step is moot — Agent already returned to coordinator). +2. **CRITICAL**: re-paste the **FULL original prompt** including issue title / body / diff path / attachment paths / file output path. **不要假設 context 倖存 idle/wake cycle** — 這是 #47 incident 的核心 root cause。 +3. Wait up to 90s (18 × 5s polling) for the file to appear. + +```bash +for role in "${MISSING_ROLES[@]}"; do + # Build retry prompt with full context re-paste + RETRY_PROMPT="[RETRY] Original prompt re-pasted because the previous instance idled without producing output. Treat this as the canonical task instruction; do not assume any prior context. + +$(cat /tmp/verify_${NUMBER}_prompt_${role}.md)" # Coordinator saved prompts before spawn + + # If Agent instance is addressable via SendMessage (named team member or running agent): + SendMessage(to="verify-${NUMBER}-${role}", body="$RETRY_PROMPT") # OR spawn a fresh Agent if previous returned + + # Poll for file (90s max) + for i in $(seq 1 18); do + [ -s "/tmp/verify_${NUMBER}_findings_${role}.md" ] && break + sleep 5 + done +done +``` + +> **Note on `SendMessage` applicability**: standalone `Agent` calls return to coordinator after completion (no persistent addressable instance). The retry path therefore typically means **spawn a fresh `Agent(subagent_type=general-purpose, ...)` with the retry prompt** rather than literal `SendMessage`. The retry distinction matters most for context-re-paste discipline — always re-paste the FULL original prompt. + +#### 2.5c — Second-idle fallback: coordinator self-review + +如果 retry 後仍缺檔（90s timeout），coordinator 自己做 self-review for that role： + +```bash +for role in "${MISSING_ROLES[@]}"; do + if [ ! -s "/tmp/verify_${NUMBER}_findings_${role}.md" ]; then + # Coordinator self-review + echo "## ${role} review (coordinator self-review — process gap)" \ + > "/tmp/verify_${NUMBER}_findings_${role}.md" + echo "" >> "/tmp/verify_${NUMBER}_findings_${role}.md" + echo "(${role} Agent failed to produce output after retry. Coordinator self-reviewed:)" \ + >> "/tmp/verify_${NUMBER}_findings_${role}.md" + + # Coordinator reads diff + issue + does role-specific review inline + # (Quality lower than independent reviewer; flagged as process gap.) + PROCESS_GAPS+=("${role}: Agent failed → coordinator fallback") + fi +done +``` + +#### 2.5d — Process gap noting in master report + +Step 4 master report **必須** 標示 process gap： + +```markdown +### Process Gaps (if any) +- requirements: Agent failed → coordinator fallback (lower-quality review) +- (other roles as applicable) +``` + +無 gap 時不顯示此 section。 + +> **Why explicit process gap section?** Hiding "1 of 5 reviewers failed" inside aggregate metrics looks like clean PASS but is actually 4-AI not 5-AI ensemble. Verify discipline = explicitly mark engine degradation. Future maintainers reading old verify comments can spot when an Agent class was systematically failing (e.g. infrastructure issue, prompt bug) by grepping `Process Gaps` sections. + +--- + ### Step 3: 合併 Findings -等 Agent Team 和 Codex 都完成後： +等 5 reviewer Agents（Step 2.5 Recovery Protocol 已 satisfy: 所有 findings 檔 present + non-empty）和 Codex 都完成後： -1. 收集 5 個 teammates 的 findings -2. 收集 Codex 的 findings -3. **去重**：相同檔案 + 相似描述 → 合併，標註來源 `[team:logic+codex]` +1. 收集 5 個 reviewer Agents 的 findings（從 `/tmp/verify_${NUMBER}_findings_*.md`） +2. 收集 Codex 的 findings（從 `/tmp/codex-verify-${NUMBER}.md`） +3. **去重**：相同檔案 + 相似描述 → 合併，標註來源 `[agents:logic+codex]` 4. **severity 以最高為準**：如果 logic 說 P2 但 codex 說 P1 → P1 5. Devil's Advocate 的反駁如果成立 → 升級 severity @@ -440,7 +628,7 @@ git checkout $ORIGINAL_BRANCH # Step 0.5 記住的 ## Verify: #NNN ### Engine -Agent Team (5 Claude reviewers) + Codex (gpt-5.5) +5 general-purpose Agents (Claude reviewers, file-based output) + Codex (gpt-5.5, run_in_background) ### 要求覆蓋率 X / Y requirements addressed @@ -448,9 +636,9 @@ X / Y requirements addressed ### Findings（合併後） | # | Severity | Finding | Source | |---|----------|---------|--------| -| 1 | P1 | ... | team:logic+codex | -| 2 | P2 | ... | team:security | -| 3 | — | (devil's advocate 未能反駁) | team:devils-advocate | +| 1 | P1 | ... | agents:logic+codex | +| 2 | P2 | ... | agents:security | +| 3 | — | (devil's advocate 未能反駁) | agents:devils-advocate | ### Scope Check {有沒有超出 issue 範圍的改動} @@ -461,7 +649,7 @@ X / Y requirements addressed ## Verify Report — PR #PPP ### Engine -Agent Team (5 Claude reviewers) + Codex (gpt-5.5) +5 general-purpose Agents (Claude reviewers, file-based output) + Codex (gpt-5.5, run_in_background) ### Aggregate **PASS / FAIL** — N blocking, M follow-up @@ -478,7 +666,7 @@ Verified scope: #98, #105 | # | Severity | Finding | Source | Action | |---|----------|---------|--------|--------| -| 1 | P1 | ... | team:logic+codex | Blocking | +| 1 | P1 | ... | agents:logic+codex | Blocking | --- @@ -488,7 +676,7 @@ Verified scope: #98, #105 | # | Severity | Finding | Source | Action | |---|----------|---------|--------|--------| -| 2 | P3 | ... | team:security | Follow-up | +| 2 | P3 | ... | agents:security | Follow-up | ``` ### Step 5: 後續動作 @@ -508,9 +696,9 @@ Verified scope: #98, #105 ```markdown | # | Severity | Finding | Source | Action | |---|----------|---------|--------|--------| -| 1 | MEDIUM | ... | team:logic+codex | Follow-up | -| 2 | MEDIUM | ... | team:regression | In-scope fix | -| 3 | LOW | ... | team:security | Follow-up | +| 1 | MEDIUM | ... | agents:logic+codex | Follow-up | +| 2 | MEDIUM | ... | agents:regression | In-scope fix | +| 3 | LOW | ... | agents:security | Follow-up | ``` #### Step 5b: Follow-up Issue Triage（強制，不可省略） @@ -634,9 +822,9 @@ codex exec --full-auto \ > **Fast mode note**: `service_tier="fast"` 加速 GPT-5.5 回應（需較多 credits,換取 2-5x 速度）。驗證場景對速度敏感（user 在等 findings），預設開啟;若要省 credit 可移除此 flag。 -## Engine: team（只用 Agent Team） +## Engine: team（只用 5 Reviewer Agents，alias `team` 保留為 backward-compat） -只開 5 人 team，不跑 Codex。適合不需要跨模型驗證的場景。 +只 spawn 5 個 `Agent(subagent_type=general-purpose)` reviewer，不跑 Codex。適合不需要跨模型驗證、或 Codex 不可用的場景。CLI alias `team` 保留為 backward compat（pre-v2.59.0 model name），實際底層為 standalone Agent calls。 ## Loop 模式 @@ -661,7 +849,7 @@ helper 行為見 `scripts/check-ralph-loop.sh`:exit 0 if installed, exit 1 with - **不跳過驗證**。「看起來對了」不算。 - **有 findings 就不 close**。先修，再 verify。 - **Devil's Advocate 是必要的**。防止 4 個 reviewer 的群體盲點。 -- **Codex 是獨立的**。它看不到 team 的討論，提供真正的盲驗。 +- **Codex 是獨立的**。它看不到 5 reviewer Agents 的 findings 檔，提供真正的盲驗。 ## Auto-Update