Workflow Health — 2026-06-13
Executive read: Acceptance rate declined to 55.6% from 100% (regression >5pp), with 30 of 45 items pending evaluation. Smoke Claude and Issue Monster are stuck with high pending volume. Three workflows are underdefined (all outputs pending, no clear acceptance signal). Priority: investigate stalled Smoke Claude items and clarify acceptance criteria for underdefined workflows.
Legend:
- Status: 🟩 accepted · 🟥 rejected · 🟨 pending · ⬜ unknown
- Lifecycle health: 🟢 resolving · 🟡 in flight · 🟠 aging · 🔴 stuck · ⚪ underdefined
🔴 Action Items
-
Smoke Claude: Stuck workflow — 15 of 24 items (62%) pending with no resolution in recent runs. Multiple items lack URLs (unverifiable state). Recommend reviewing workflow prompt, acceptance criteria, and whether async operations need timeout handling.
-
Issue Monster & Smoke Codex: Underdefined workflows — All outputs pending with no accepted or rejected signals. These workflows lack clear acceptance/rejection criteria or a dedicated evaluator. Define what "done" means (e.g., comment acted upon by assignee, issue closed within X hours).
-
Regression in quality metrics — Acceptance rate dropped from 100% (Jun 10) to 55.6% (-44.4pp). Waste rate increased from 0% to 8.9%. This indicates either: (a) new workflows lack mature evaluation signals, or (b) a recent prompt/safe-output change degraded quality. Investigate recent changes to Smoke Claude.
-
Zero-touch rate remains 0% — All accepted items required human follow-up. Agent outputs consistently need editing before acceptance. Recommend reviewing agent quality, prompt specificity, or safe-output configuration.
-
Data quality note — 1 of 45 items (2%) were evaluated with fallback existence check only. Most outcomes have medium/strong signals, so numbers are reasonably reliable.
Detailed metrics, evidence quality, workflow counts, and trends
Outcome Scorecard — 2026-06-13
| Metric |
Value |
Status |
| Acceptance rate |
55.6% |
🔴 <60% (declining) |
| Zero-touch rate |
0% |
🔴 <25% |
| Waste rate |
8.9% |
🟡 10-25% |
| Median time to resolution |
12h |
— |
| Accepted |
5 / 45 |
— |
| — strong evidence |
0 |
merged, completed, approved |
| — medium evidence |
5 |
engaged, retained |
| — weak evidence |
0 |
existence only |
| Rejected |
4 |
— |
| Ignored |
0 |
no observable follow-up |
| Zero-touch |
0 / 5 |
— |
| Pending |
30 |
— |
| Unknown |
2 |
— |
| Runs checked |
14 |
— |
Per-Workflow Breakdown
| Workflow |
Accepted |
Rejected |
Ignored |
Pending |
Unknown |
Waste % |
Acceptance % |
| Changeset Generator |
0 |
1 |
0 |
1 |
0 |
50.0% |
0% |
| Smoke Claude |
2 |
3 |
0 |
15 |
4 |
12.5% |
40% |
| Issue Monster |
0 |
0 |
0 |
6 |
0 |
0% |
— |
| Smoke Codex |
0 |
0 |
0 |
2 |
0 |
0% |
— |
| Agent Container Smoke Test |
1 |
0 |
0 |
1 |
0 |
0% |
100% |
| Semantic Function Refactoring |
1 |
0 |
0 |
1 |
0 |
0% |
100% |
| Smoke Copilot - AOAI (apikey) |
1 |
0 |
0 |
1 |
0 |
0% |
100% |
| AI Moderator |
0 |
0 |
0 |
1 |
0 |
0% |
— |
| Auto-Triage Issues |
0 |
0 |
0 |
1 |
0 |
0% |
— |
| Daily Reliability Review |
0 |
0 |
0 |
1 |
0 |
0% |
— |
| Daily Documentation Healer |
0 |
0 |
0 |
0 |
1 |
0% |
— |
| Daily Sentrux Report |
0 |
0 |
0 |
0 |
1 |
0% |
— |
Evidence Quality
⚠️ 1 item was evaluated using only a generic existence check (signal: target_exists_only). This contributes to accepted_weak and may overstate acceptance.
Trend Signal
Comparing 2026-06-13 vs 2026-06-10 (previous report):
- Acceptance rate: ⬇️ Regressing — 100% → 55.6% (-44.4pp). Decline exceeds acceptable 5pp threshold.
- Zero-touch rate: ➡️ Stable — 0% → 0% (no change).
- Waste rate: ⬆️ Regressing — 0% → 8.9% (+8.9pp). All rejections new in this cycle.
Interpretation: The system has shifted from an accept-all pattern (Jun 10) to one where workflows are systematically pending evaluation. This could indicate: (1) new workflows added with different evaluation patterns, (2) more complex safe-output types now tracked, or (3) recent evaluation rule changes. The pending volume requires investigation.
📊 Measured by Outcome Collector · 67.7 AIC · ⌖ 4.72 AIC · ⊞ 23.6K
Workflow Health — 2026-06-13
Executive read: Acceptance rate declined to 55.6% from 100% (regression >5pp), with 30 of 45 items pending evaluation. Smoke Claude and Issue Monster are stuck with high pending volume. Three workflows are underdefined (all outputs pending, no clear acceptance signal). Priority: investigate stalled Smoke Claude items and clarify acceptance criteria for underdefined workflows.
Legend:
🔴 Action Items
Smoke Claude: Stuck workflow — 15 of 24 items (62%) pending with no resolution in recent runs. Multiple items lack URLs (unverifiable state). Recommend reviewing workflow prompt, acceptance criteria, and whether async operations need timeout handling.
Issue Monster & Smoke Codex: Underdefined workflows — All outputs pending with no accepted or rejected signals. These workflows lack clear acceptance/rejection criteria or a dedicated evaluator. Define what "done" means (e.g., comment acted upon by assignee, issue closed within X hours).
Regression in quality metrics — Acceptance rate dropped from 100% (Jun 10) to 55.6% (-44.4pp). Waste rate increased from 0% to 8.9%. This indicates either: (a) new workflows lack mature evaluation signals, or (b) a recent prompt/safe-output change degraded quality. Investigate recent changes to Smoke Claude.
Zero-touch rate remains 0% — All accepted items required human follow-up. Agent outputs consistently need editing before acceptance. Recommend reviewing agent quality, prompt specificity, or safe-output configuration.
Data quality note — 1 of 45 items (2%) were evaluated with fallback existence check only. Most outcomes have medium/strong signals, so numbers are reasonably reliable.
Detailed metrics, evidence quality, workflow counts, and trends
Outcome Scorecard — 2026-06-13
Per-Workflow Breakdown
Evidence Quality
target_exists_only). This contributes toaccepted_weakand may overstate acceptance.Trend Signal
Comparing 2026-06-13 vs 2026-06-10 (previous report):
Interpretation: The system has shifted from an accept-all pattern (Jun 10) to one where workflows are systematically pending evaluation. This could indicate: (1) new workflows added with different evaluation patterns, (2) more complex safe-output types now tracked, or (3) recent evaluation rule changes. The pending volume requires investigation.