⚡ Copilot Token Optimization2026-04-13 — Build Test Suite

[github-actions]

Target Workflow: build-test.md

Source report: #1959
Estimated cost per run: N/A (cost data unavailable; ranked by total token consumption)
Total tokens per run: ~581K avg (range 401K–759K; 8 successful runs)
Cache hit rate: ~88% of input tokens served from cache (good — context is stable between turns)
LLM requests/run: avg 12.4 (range 9–16)
Model: claude-sonnet-4.6

---

Current Configuration

Setting	Value
Tools loaded	bash: ["*"] + github: (no toolsets: restriction — loads ~22 MCP tools)
Tools actually used	Unknown — tool_usage not populated in logs
Network groups	12 groups: defaults, github, node, go, rust, crates.io, java, dotnet, bun.sh, deno.land, jsr.io, dl.deno.land
Pre-agent steps	None — agent executes all 8 ecosystem builds inline
Prompt size	8,592 bytes (~2,150 tokens)
Output/input ratio	~1.3% (agent outputs very little relative to input — classic symptom of over-loaded context)

The Core Problem

The agent executes 8 ecosystem build suites (Bun, C++, Deno, .NET, Go, Java, Node.js, Rust) sequentially via LLM tool calls. Each build command's full verbose output is appended to the context window, which then gets re-sent on every subsequent LLM request. By request 12, the agent is re-processing the accumulated stdout of all previous cargo build, mvn test, deno test, etc. — most of which is irrelevant at that point.

All 8 tasks are deterministic bash workflows with no branching logic — the agent doesn't need to make decisions between tasks. This is the ideal steps: migration candidate.

---

Recommendations

1. Move All Build Tasks to Pre-Agent steps:

Estimated savings: ~470K tokens/run (~81%)

All 8 ecosystem build tasks are pure bash pipelines. Move them to steps: in the workflow frontmatter, collect pass/fail status and truncated error output via $GITHUB_OUTPUT, and let the agent's single turn do only what only AI can do: format the markdown table and post the PR comment.

Before (current — agent runs all 8 builds via tool calls, accumulating context across 12+ requests):

tools:
  bash:
    - "*"
  github:
    github-token: "$\{\{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN }}"

After (add steps: block in frontmatter, before the closing ---):

steps:
  - name: Setup Maven proxy
    run: |
      mkdir -p ~/.m2
      cat > ~/.m2/settings.xml << 'SETTINGS'
      <settings><proxies>
        <proxy><id>awf-http</id><active>true</active><protocol>http</protocol><host>squid-proxy</host><port>3128</port></proxy>
        <proxy><id>awf-https</id><active>true</active><protocol>https</protocol><host>squid-proxy</host><port>3128</port></proxy>
      </proxies></settings>
      SETTINGS

  - name: Test Bun
    id: bun
    continue-on-error: true
    run: |
      curl -fsSL (bun.sh/redacted) | bash
      export BUN_INSTALL="$HOME/.bun" && export PATH="$BUN_INSTALL/bin:$PATH"
      STATUS="PASS"
      gh repo clone Mossaka/gh-aw-firewall-test-bun /tmp/test-bun || { echo "BUN_RESULT=CLONE_FAILED" >> $GITHUB_OUTPUT; exit 0; }
      ELYSIA=$(cd /tmp/test-bun/elysia && bun install && bun test 2>&1 | tail -30 && echo "PASS" || echo "FAIL")
      HONO=$(cd /tmp/test-bun/hono && bun install && bun test 2>&1 | tail -30 && echo "PASS" || echo "FAIL")
      echo "BUN_ELYSIA=$ELYSIA" >> $GITHUB_OUTPUT
      echo "BUN_HONO=$HONO" >> $GITHUB_OUTPUT

  - name: Test C++
    id: cpp
    continue-on-error: true
    run: |
      gh repo clone Mossaka/gh-aw-firewall-test-cpp /tmp/test-cpp || { echo "CPP_RESULT=CLONE_FAILED" >> $GITHUB_OUTPUT; exit 0; }
      FMT=$(cd /tmp/test-cpp/fmt && mkdir -p build && cd build && cmake .. && make 2>&1 | tail -20 && echo "PASS" || echo "FAIL")
      JSON=$(cd /tmp/test-cpp/json && mkdir -p build && cd build && cmake .. && make 2>&1 | tail -20 && echo "PASS" || echo "FAIL")
      echo "CPP_FMT=$FMT" >> $GITHUB_OUTPUT
      echo "CPP_JSON=$JSON" >> $GITHUB_OUTPUT

  # ... (similar steps for Deno, .NET, Go, Java, Node.js, Rust)

Then replace the entire prompt body with a minimal formatter prompt:

# Build Test Suite

Format the build results from the pre-computed steps into a PR comment table.

## Results to Format

Steps outputs are available in `$\{\{ steps.bun.outputs.* }}`, `$\{\{ steps.cpp.outputs.* }}`, etc.

Post a single comment on the current pull request:

### 🏗️ Build Test Suite Results

| Ecosystem | Project | Build/Install | Tests | Status |
|-----------|---------|---------------|-------|--------|
[populate from steps outputs]

**Overall: X/8 ecosystems passed**

If ALL tests pass AND triggered by a pull request (not `workflow_dispatch`), add the label `build-test`.

This reduces agent work to 1 LLM request (format table + post comment) instead of 12+.

Token math:

• Current: avg 581K tokens across 12.4 requests
• After: ~1 request × ~80K tokens (steps outputs + minimal prompt) = ~80K tokens
• Savings: ~501K tokens/run (86%)

---

2. Restrict GitHub Toolset

Estimated savings: ~144K tokens/run now; ~13K/run after Rec #1

The github: tools entry has no toolsets: restriction, which loads all ~22 GitHub MCP tools into every request's context. Looking at the workflow, all actual operations use:

• gh CLI via bash (cloning repos, posting comments)
• safe-outputs extension (PR comment, labeling)

The GitHub MCP tools appear unnecessary, or at minimum only pull_requests is needed.

Change:

# Before
tools:
  github:
    github-token: "$\{\{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN }}"

# After — restrict to only what's needed
tools:
  github:
    github-token: "$\{\{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN }}"
    toolsets: [pull_requests]

Each tool schema is ~600 tokens. Removing ~20 unnecessary tools saves ~12K tokens/turn.

• At current 12.4 requests/run: saves ~149K tokens/run
• After Rec Improve links in readme to AW project #1 (1 request/run): saves ~12K tokens/run

---

3. Cap Bash Command Output with tail

Estimated savings: ~30–50K tokens/run

Build commands like cargo build, mvn test, go mod download emit thousands of lines. Without output limits, everything feeds back into the LLM context.

If Rec #1 is implemented, this is handled naturally (steps outputs use tail -30 per command). If Rec #1 is not implemented, add explicit truncation to the prompt instructions:

## Important: Limit command output
For all build/test commands, pipe output through `2>&1 | tail -50` to limit LLM context growth.
Example: `cargo build 2>&1 | tail -50 && cargo test 2>&1 | tail -50`

This prevents a single verbose mvn test or cargo build output from adding 10K+ tokens to subsequent requests.

---

4. Prompt Compression

Estimated savings: ~3–5K tokens/run (minor)

The 18-row result table template at the bottom of the prompt (~300 tokens) is redundant — the agent can infer the format from the column headers alone. Remove the prefilled template rows.

Similarly, the Maven settings.xml block (~15 lines, ~200 tokens) is repeated in every context reload. After Rec #1, it moves to a steps: entry and disappears from the agent's context entirely.

---

Expected Impact

Metric	Current	After Rec #1+2	After All Recs	Savings
Total tokens/run	~581K	~80K	~68K	-88%
LLM requests/run	~12.4	~1	~1	-92%
Input tokens/run	~574K	~79K	~67K	-88%
Output tokens/run	~7K	~2K	~1.5K	-79%
Session time (est.)	~95s	~15s	~13s	-86%
10-run period total	4.7M	~0.8M	~0.7M	-85%

Cost per run is unavailable from current logs, but token reduction tracks linearly with cost.

---

Implementation Checklist

• Add steps: block to .github/workflows/build-test.md frontmatter for all 8 ecosystem builds (see Rec Improve links in readme to AW project #1 scaffold above)
• Replace prompt body with minimal formatter prompt (Rec Improve links in readme to AW project #1)
• Add toolsets: [pull_requests] to github: tools entry (Rec Secret proxying #2)
• Verify tail -30 output caps are included in each step's run command (Rec feat: add integration test for rostbuness #3)
• Remove the 18-row prefilled result table template from prompt (Rec fix: add missing Docker image pulls to robustness test workflow #4)
• Recompile: gh aw compile .github/workflows/build-test.md
• Post-process: npx tsx scripts/ci/postprocess-smoke-workflows.ts
• Open a test PR and verify CI passes with the new workflow
• Compare token usage on new run vs baseline (~581K) — target: under 100K tokens/run

Generated by Daily Copilot Token Optimization Advisor · ● 482.5K · ◷