[cli-tools-test] [exploratory-test] audit: comparison.delta.turns.after always reports 0 regardless of actual turn count

[github-actions]

Problem Description

The audit command comparison delta always shows turns.after = 0 regardless of the actual number of turns the workflow run executed. This causes all runs to appear as regressions (turns_decrease) compared to their baselines even when the run completed successfully with many turns.

Command/Tool

• Tool: audit
• MCP command: agenticworkflows audit --run_id <run_id>

Steps to Reproduce

1. Run agenticworkflows audit --run_id 28731332447 --max_tokens 3000
2. Observe comparison.delta.turns
3. Compare against metrics.turns

Expected Behavior

comparison.delta.turns.after should equal the actual turn count from metrics.turns for the current run.

Actual Behavior

comparison.delta.turns.after always shows 0 even when metrics.turns shows the real count:

// Run 28731332447 (Static Analysis Report, claude engine, 34 turns)
"metrics": { "turns": 34 },
"comparison": {
  "delta": {
    "turns": { "before": 18, "after": 0, "changed": true }
  },
  "classification": { "label": "changed", "reason_codes": ["turns_decrease"] }
}

// Run 28731939775 (Duplicate Code Detector)
"comparison": { "delta": { "turns": { "before": 12, "after": 0, "changed": true } } }

// Run 28731355046 (Daily Windows Terminal, copilot engine)
"comparison": { "delta": { "turns": { "before": 7, "after": 0, "changed": true } } }

All three audited runs show after: 0, triggering false turns_decrease regression classification.

Root Cause Hypothesis

The token_usage.jsonl file is empty (0 bytes) for all recent runs. The audit comparison logic likely reads per-turn counts from this file. Since it is empty, turns always computes to 0. For comparison:

• token_usage.jsonl: 0 lines for all tested runs
• agent_usage.json: correct totals (input_tokens, output_tokens, cache data present)
• metrics.turns: correctly populated for claude runs (e.g., 34 for run 28731332447)

Impact

• Severity: Medium
• Frequency: Always (100% of audited runs across claude, copilot, and codex engines)
• Workaround: Use metrics.turns directly; ignore comparison.delta.turns.after

Affected Runs Tested

Run ID	Workflow	Engine	metrics.turns	delta.after
28731332447	Static Analysis Report	claude	34	0
28731939775	Duplicate Code Detector	codex	None	0
28731355046	Daily Windows Terminal	copilot	None	0

Environment

• Repository: github/gh-aw
• Run ID: 28731918594
• Date: 2026-07-05
• AWF version: v0.27.22 (agent version 1.0.68)

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

Generated by 🧪 Daily Cli Tools Tester · 145.1 AIC · ⌖ 11.3 AIC · ⊞ 9.1K · ◷

• expires on Jul 11, 2026, 10:41 PM UTC-08:00