Add guided `gh aw fix` diagnostic for restricted `tools.bash` on engines that ignore allow-listing by Copilot · Pull Request #51102 · github/gh-aw

Copilot · 2026-08-07T14:39:25Z

gh aw compile --strict rejects workflows that pair an engine without bash allow-list support (today: codex) with a restricted tools.bash, but gh aw fix reported "No fixes needed" — leaving users with a broken workflow and no remediation path.

Changes

New guided codemod (pkg/cli/codemod_bash_allowlist_unsupported_engine.go, ID bash-allowlist-unsupported-engine-guided-error): flags any explicit bash restriction (bash: [cmd, ...], bash: [], bash: false) on an engine whose capabilities lack BashCommandAllowlist. Capability is read from the global engine registry rather than hardcoding codex, so it stays correct as engines change. Unknown/custom engines are a no-op.
Guided, not auto-rewriting: both remediations change semantics — rewriting to bash: ["*"] makes unrestricted access explicit, and switching engines changes which agent runs — so the codemod never mutates the file and instead names the offending config plus both options. This matches the existing top-level-env-secrets-guided-error pattern.
Shared detection logic: exported hasBashExplicitRestriction → workflow.HasBashExplicitRestriction so the codemod and the strict-mode compiler check cannot drift.
Registration + tests: registered after the other bash codemods; registry ID/order tests updated; unit tests cover metadata, every restriction shape, and no-op paths (wildcard, bash: true, supported/default/unknown engines, no tools). Changeset included.

Behavior

---
engine:
  id: codex
tools:
  bash: ["git status", "npm ci"]
---

$ gh aw fix .github/workflows/demo.md
✗ Error processing demo.md: codemod bash-allowlist-unsupported-engine-guided-error failed:
  engine 'codex' does not support bash command allow-listing: 'bash: [git status, npm ci]' is
  silently ignored at runtime for this engine. Manual fix required: switch to an engine that
  enforces the allow-list (copilot, claude, or gemini), or replace the configuration with
  'bash: ["*"]' to make the unrestricted access explicit.

✗ 1 file needs a manual fix

The issue suggested the id codex-bash-allowlist-to-wildcard; I went with an engine-agnostic id since the check is capability-driven, but happy to rename.

Fixes [aw-compat] Missing codemod: engine:codex + restricted tools.bash allow-list fails --strict, gh aw fix does nothing #51045

Generated by 👨‍🍳 PR Sous Chef · gpt54 · 10.3 AIC · ⊞ 5.9K · ◷
_{Comment /souschef to run again}

Generated by 👨‍🍳 PR Sous Chef · gpt54 · 10.2 AIC · ⊞ 8.4K · ◷
_{Comment /souschef to run again}

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot

Pull request overview

Adds a guided gh aw fix diagnostic for unsupported restricted Bash configurations.

Changes:

Adds and registers a capability-driven guided codemod.
Shares Bash-restriction detection with strict compilation.
Adds tests, release notes, and regenerated workflow output.

Show a summary per file

File	Description
`pkg/workflow/agent_validation.go`	Exports shared restriction detection.
`pkg/cli/fix_codemods.go`	Registers the codemod.
`pkg/cli/fix_codemods_test.go`	Updates registry expectations.
`pkg/cli/codemod_bash_allowlist_unsupported_engine.go`	Implements the diagnostic.
`pkg/cli/codemod_bash_allowlist_unsupported_engine_test.go`	Tests codemod behavior.
`.github/workflows/smoke-cursor.lock.yml`	Refreshes generated workflow output.
`.changeset/bash-allowlist-unsupported-engine-guided-error.md`	Documents the patch.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Files reviewed: 7/7 changed files
Comments generated: 2
Review effort level: Balanced

github-actions · 2026-08-07T16:52:29Z

⚠️ Security scanning failed for Matt Pocock Skills Reviewer. Review the logs for details.

github-actions · 2026-08-07T16:52:29Z

⚠️ Security scanning failed for Test Quality Sentinel. Review the logs for details.

github-actions · 2026-08-07T16:52:34Z

⚠️ Security scanning failed for Design Decision Gate 🏗️. Review the logs for details.

github-actions · 2026-08-07T16:52:55Z

⚠️ PR Code Quality Reviewer failed during code quality review.

Warning

Threat Detection Engine Failure — The analysis engine could not complete. This is a tooling failure, not a security finding.

What happened

The threat detection engine failed to produce results.

Review the workflow run logs for details.

github-actions

Review

The implementation is well-structured, follows the guided-error pattern correctly, and the test coverage is comprehensive.

Two pre-existing inline comments already flag the main concerns:

Shallow frontmatter inspection (line 29): the codemod reads the raw top-level tools map from frontmatter, but the compiler counterpart operates on the merged tools after resolving imports and includes. A workflow that restricts bash only via an imported tool definition would be flagged by gh aw compile --strict but silently pass the codemod.
Unescaped user-controlled content in error message (line 73): describeBashRestriction formats YAML command strings directly into the error output via %v. Commands containing ANSI escape sequences or newlines can spoof diagnostic lines on terminals that render control characters.

No new blocking issues found beyond those two.

🧵 Reviewed using Impeccable skills by Impeccable Skills Reviewer · sonnet46 · 23.9 AIC · ⊞ 5.4K

github-actions · 2026-08-07T16:58:10Z

🧪 Test Quality Sentinel Report

✅ Test Quality Score: 100/100 — Excellent

Analyzed 2 test function(s): 2 design, 0 implementation, 0 violation(s).

📊 Metrics (2 tests)

Metric	Value
Analyzed	2 (Go: 2, JS: 0)
✅ Design	2 (100%)
⚠️ Implementation	0 (0%)
Edge/error coverage	2/2 (100%)
Duplicate clusters	0
Inflation	No (1.81:1 ratio, < 2:1 threshold)
🚨 Violations	0

Test	File	Classification	Details
`TestBashAllowlistUnsupportedEngineCodemod_Metadata`	`pkg/cli/codemod_bash_allowlist_unsupported_engine_test.go`	`design_test`	Validates metadata contract (ID, Name, Description, IntroducedIn, Guided flag, Apply func)
`TestBashAllowlistUnsupportedEngineCodemod_Apply`	`pkg/cli/codemod_bash_allowlist_unsupported_engine_test.go`	`design_test`	Table-driven with 11 scenarios covering error paths, edge cases, and no-op behavior; validates semantic invariants (content preserved, never applied)

Quality Highlights

✅ Comprehensive edge-case coverage — 11 distinct scenarios in table-driven Apply test:

4 error cases (restricted bash list, bool false, empty list, combined with codex engine)
7 no-op cases (supported engines, unknown engines, edge cases like wildcard bash and missing tools)

✅ Semantic invariant validation — Tests enforce that:

Guided codemod never modifies workflow content
Applied flag always remains false
Error messages contain actionable guidance (4 strings verified per primary error case)

✅ Design contract testing — Both tests focus on public behavior and contracts:

Metadata test validates the codemod registry interface
Apply test validates error detection logic and behavioral guarantees

✅ Code quality — Proper build tag, no mock violations, testify best practices, clear test names.

Verdict

✅ Passed. 0% implementation tests (threshold: 30%). Test Quality Score 100/100 — excellent coverage of design invariants and error paths with appropriate table-driven structure.

🧪 Test quality analysis by Test Quality Sentinel · haiku45 · 22 AIC · ⊞ 7.7K · ◷
_{Comment /review to run again}

github-actions

✅ Test Quality Sentinel: 100/100. 0% implementation tests (threshold: 30%).

…pported engines

github-actions · 2026-08-07T17:02:44Z

🏗️ Design Decision Gate — ADR Required

This PR makes significant changes to core business logic (226 new lines in pkg/) but does not yet have a linked Architecture Decision Record (ADR).

📄 Draft ADR committed: docs/adr/51102-guided-codemod-for-bash-allowlist-on-unsupported-engines.md — review and complete it before merging.

🔒 This PR cannot merge until an ADR is linked in the PR body.

📋 What to do next

Review the draft ADR committed to your branch — it was generated from the PR diff
Complete the missing sections — add context the AI couldn't infer, refine the decision rationale, and list real alternatives you considered
Commit the finalized ADR to docs/adr/ on your branch
Reference the ADR in this PR body by adding a line such as:

ADR: ADR-51102: Guided Codemod for Bash Allow-list on Unsupported Engines

Once an ADR is linked in the PR body, this gate will re-run and verify the implementation matches the decision.

❓ Why ADRs Matter

"AI made me procrastinate on key design decisions. Because refactoring was cheap, I could always say 'I'll deal with this later.' Deferring decisions corroded my ability to think clearly."

ADRs create a searchable, permanent record of why the codebase looks the way it does. Future contributors (and your future self) will thank you.

📋 Michael Nygard ADR Format Reference

An ADR must contain these four sections to be considered complete:

Context — What is the problem? What forces are at play?
Decision — What did you decide? Why?
Alternatives Considered — What else could have been done?
Consequences — What are the trade-offs (positive and negative)?

All ADRs are stored in docs/adr/ as Markdown files numbered by PR number (e.g., 51102-guided-codemod-for-bash-allowlist-on-unsupported-engines.md for PR #51102).

🏗️ ADR gate enforced by Design Decision Gate 🏗️ · sonnet46 · 54.3 AIC · ⊞ 9.6K · ◷
_{Comment /review to run again}

github-actions

Skills-Based Review 🧠

Applied /grill-with-docs, /tdd, and /codebase-design — requesting changes on three issues.

📋 Key Themes & Highlights

Key Themes

Hardcoded engine list in error message (line 56): "copilot, claude, or gemini" will drift as engines gain BashCommandAllowlist support. The capability is already queried from the registry for detection; it should drive the message too.
Terminal output injection (line 73, pre-existing comment): cmd from workflow YAML is inserted into stderr unescaped — control characters or newlines could spoof diagnostics.
Detection scope gap (line 29, pre-existing comment): only the raw top-level tools map is inspected; merged/imported tool definitions are not considered.
Implicit test assumption (test line 117): the "default engine" no-op relies on copilot being the default and supporting the capability — a comment explaining that chain would make the test robust to future change.

Positive Highlights

✅ Excellent pattern: guided (non-auto-rewriting) codemod that explains both options and their semantics trade-offs
✅ Shared detection logic (HasBashExplicitRestriction) exported and reused between compiler and codemod — no drift
✅ Comprehensive test table covering every bash restriction shape and all engine scenarios
✅ Follows the established top-level-env-secrets-guided-error pattern consistently

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · sonnet46 · 41.7 AIC · ⊞ 7.1K
_{Comment /matt to run again}

gh-aw-bot · 2026-08-07T17:07:39Z

@copilot Please address the current review feedback, refresh the branch if GitHub can update it cleanly, and run the pr-finisher skill before handing back to maintainers.

Open review items to address (newest first):

github-actions: Add guided gh aw fix diagnostic for restricted tools.bash on engines that ignore allow-listing #51102 (review)

Branch refresh was requested.

Run context: https://github.com/github/gh-aw/actions/runs/31200043159

Generated by 👨‍🍳 PR Sous Chef · gpt54 · 10.3 AIC · ⊞ 5.9K · ◷
_{Comment /souschef to run again}

Done

pelikhan · 2026-08-07T17:26:51Z

@copilot codex now support specifying bash: false

…ive tools from imports - Use fmt.Sprintf(%q) in describeBashRestriction to prevent terminal output spoofing via ANSI control sequences or embedded newlines in YAML-sourced command strings - Add ApplyWithContext optional field to Codemod struct for codemods that need the workflow file path to resolve effective configuration from imports/includes - Update fix_command.go to call ApplyWithContext (if set) instead of Apply so the bash allowlist codemod receives the file path at runtime - Add resolveEffectiveBashTools helper: resolves imports via parser.ProcessImportsFromFrontmatterWithSource and markdown includes via parser.ExpandIncludesWithManifest, then merges with top-level tools; fast path skips resolution when the top-level tools already declares a bash key (it wins) - Update tests: fix expected error string for quoted commands; add control-character and newline spoofing test cases; add ApplyWithContext import-sourced restriction test Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Copilot · 2026-08-07T17:42:06Z

@copilot Please address the current review feedback, refresh the branch if GitHub can update it cleanly, and run the pr-finisher skill before handing back to maintainers.

Open review items to addre...

Addressed both review items in commit Fix bash allowlist codemod: quote commands with %q and resolve effective tools from imports:

Terminal-spoofing (line 73): describeBashRestriction now formats each command string with %q instead of %v, so ANSI escape sequences and embedded newlines are rendered as Go escape sequences in the error output. Two new test cases cover \x1b and \n in command names.

Import resolution (line 29): Added an optional ApplyWithContext field to the Codemod struct (signature func(content, frontmatter, filePath)). fix_command.go calls it in preference to Apply when processing real files. The bash allowlist codemod sets ApplyWithContext and uses it to resolve the effective tools map via parser.ProcessImportsFromFrontmatterWithSource + parser.ExpandIncludesWithManifest + parser.MergeTools, so restrictions introduced solely through imports are caught. A fast path skips resolution when the top-level tools already declares a bash key (it wins in the merge). A new ApplyWithContext-based test writes actual import files to disk and verifies the guided error fires when the restriction comes only from the import.

…e-read fix, test clarity - Add EnginesWithCapability(predicate) to EngineRegistry so callers can derive the supported-engine list from the registry rather than hardcoding it; this addresses the grill-with-docs thread that flagged 'copilot, claude, or gemini' as fragile as new engines gain BashCommandAllowlist support - Use EnginesWithCapability in the guided error message in applyBashAllowlistUnsupportedEngineCheck so the list stays registry-accurate - Extract bashVal := effectiveTools["bash"] once before HasBashExplicitRestriction so describeBashRestriction reads the same value (codebase-design thread fix) - Rename 'default engine' test case to explicitly call out that it relies on copilot being the default engine and having BashCommandAllowlist (tdd thread fix) - Update test assertion to check for individual engine names rather than the full hardcoded string to remain correct as the sorted list evolves Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

gh-aw-bot · 2026-08-07T18:02:13Z

@copilot Please triage this PR, refresh the branch if possible, and run the pr-finisher skill before handing back to maintainers.

Unresolved review threads to re-check:

github-actions: Add guided gh aw fix diagnostic for restricted tools.bash on engines that ignore allow-listing #51102 (comment)
github-actions: Add guided gh aw fix diagnostic for restricted tools.bash on engines that ignore allow-listing #51102 (comment)
github-actions: Add guided gh aw fix diagnostic for restricted tools.bash on engines that ignore allow-listing #51102 (comment)
copilot-pull-request-reviewer: Add guided gh aw fix diagnostic for restricted tools.bash on engines that ignore allow-listing #51102 (comment)
copilot-pull-request-reviewer: Add guided gh aw fix diagnostic for restricted tools.bash on engines that ignore allow-listing #51102 (comment)

Run context: https://github.com/github/gh-aw/actions/runs/31204567664

Generated by 👨‍🍳 PR Sous Chef · gpt54 · 10.2 AIC · ⊞ 8.4K · ◷
_{Comment /souschef to run again}

github-actions · 2026-08-07T21:53:00Z

🎉 This pull request is included in a new release.

Release: v0.86.1

Initial plan

5175c41

Copilot AI assigned Copilot and pelikhan Aug 7, 2026

Copilot started work on behalf of pelikhan August 7, 2026 14:39 View session

Copilot AI linked an issue Aug 7, 2026 that may be closed by this pull request

[aw-compat] Missing codemod: engine:codex + restricted tools.bash allow-list fails --strict, gh aw fix does nothing #51045

Closed

Copilot AI and others added 2 commits August 7, 2026 14:45

Initial plan for codex bash allow-list codemod

8d51ba6

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Add guided codemod for restricted tools.bash on engines that ignore it

546ecea

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Add codemod for engine.codex with tools.bash allow-list~~ Add guided gh aw fix diagnostic for restricted tools.bash on engines that ignore allow-listing Aug 7, 2026

Copilot AI requested a review from pelikhan August 7, 2026 14:57

Copilot finished work on behalf of pelikhan August 7, 2026 14:57

pelikhan marked this pull request as ready for review August 7, 2026 16:37

Copilot AI balanced review requested due to automatic review settings August 7, 2026 16:37

Copilot started reviewing on behalf of pelikhan August 7, 2026 16:38 View session

Copilot AI reviewed Aug 7, 2026

View reviewed changes

Comment thread pkg/cli/codemod_bash_allowlist_unsupported_engine.go Outdated

Comment thread pkg/cli/codemod_bash_allowlist_unsupported_engine.go Outdated

github-actions Bot reviewed Aug 7, 2026

View reviewed changes

github-actions Bot approved these changes Aug 7, 2026

View reviewed changes

docs(adr): draft ADR-51102 for guided bash allow-list codemod on unsu…

c6d3d78

…pported engines

github-actions Bot added the needs-review label Aug 7, 2026

github-actions Bot previously requested changes Aug 7, 2026

View reviewed changes

Comment thread pkg/cli/codemod_bash_allowlist_unsupported_engine.go Outdated

Comment thread pkg/cli/codemod_bash_allowlist_unsupported_engine_test.go

Comment thread pkg/cli/codemod_bash_allowlist_unsupported_engine.go

Merge branch 'main' into copilot/aw-compat-missing-codemod-fix

a2d1e43

github-actions Bot mentioned this pull request Aug 7, 2026

[pr-sous-chef] PR Sous Chef run report #50950

Closed

Merge branch 'main' into copilot/aw-compat-missing-codemod-fix

cff3287

Merge branch 'main' into copilot/aw-compat-missing-codemod-fix

e6ac777

Copilot started work on behalf of gh-aw-bot August 7, 2026 17:27 View session

Copilot finished work on behalf of gh-aw-bot August 7, 2026 17:48

Copilot AI requested a review from gh-aw-bot August 7, 2026 17:48

Copilot started work on behalf of pelikhan August 7, 2026 17:49 View session

Merge branch 'main' into copilot/aw-compat-missing-codemod-fix

af5b893

pelikhan merged commit e7c9001 into main Aug 7, 2026

pelikhan deleted the copilot/aw-compat-missing-codemod-fix branch August 7, 2026 18:02

Copilot stopped work on behalf of pelikhan due to an error August 7, 2026 18:21
The session was cancelled by the user.

github-actions Bot mentioned this pull request Aug 7, 2026

Smoke Test: MAI-Code-1-Flash - 31220061988 #51200

Closed

This was referenced Aug 7, 2026

Smoke Test: MAI-Code-1-Flash - 31222506585 #51219

Closed

Smoke Test: MAI-Code-1-Flash - 31231539109 #51260

Closed

Copilot AI mentioned this pull request Aug 8, 2026

Verify aw-compat audit findings (#51044) are already resolved — no code changes required #51424

Closed

Uh oh!

Conversation

Copilot AI commented Aug 7, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Behavior

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Review details

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Aug 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Aug 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Aug 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Aug 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Review

Uh oh!

github-actions Bot commented Aug 7, 2026

🧪 Test Quality Sentinel Report

Quality Highlights

Verdict

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Aug 7, 2026

🏗️ Design Decision Gate — ADR Required

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Skills-Based Review 🧠

Key Themes

Positive Highlights

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gh-aw-bot commented Aug 7, 2026

Uh oh!

pelikhan commented Aug 7, 2026

Uh oh!

Copilot AI commented Aug 7, 2026

Uh oh!

gh-aw-bot commented Aug 7, 2026

Uh oh!

github-actions Bot commented Aug 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Copilot AI commented Aug 7, 2026 •

edited by github-actions Bot

Loading

github-actions Bot commented Aug 7, 2026 •

edited

Loading

github-actions Bot commented Aug 7, 2026 •

edited

Loading

github-actions Bot commented Aug 7, 2026 •

edited

Loading

github-actions Bot commented Aug 7, 2026 •

edited

Loading