Skip to content

[cli-tools-test] compile: unknown tool names in frontmatter are silently ignored without warning #24835

Description

@github-actions

Problem Description

When a workflow markdown file contains an unknown/invalid tool name in the tools: frontmatter section, the compiler silently ignores it without emitting any validation error or warning. The user receives no indication that their tool configuration was not applied.

Steps to Reproduce

  1. Create a workflow markdown file with an unknown tool in the tools: section:
---
engine: copilot
on:
  schedule:
    - cron: "0 0 * * *"
tools:
  nonexistent-tool-xyz:
    version: "1.0"
network:
  allowed:
    - github.com
---

This is a test workflow.
```

2. Compile the workflow using `gh aw compile` or the MCP `compile` tool.

### Expected Behavior

The compiler should emit a **validation warning** (or error in strict mode) indicating that `nonexistent-tool-xyz` is not a recognized tool and was ignored. Something like:

```
WARNING: tools.nonexistent-tool-xyz: unknown tool name, will be ignored

Actual Behavior

The workflow compiles successfully with no errors and no warnings. The unknown tool entry is completely silent — it does not appear in the generated .lock.yml file and there is no indication to the user that their configuration was not applied.

Environment

  • Repository: github/gh-aw
  • Run ID: §24019945745
  • Date: 2026-04-06
  • Test: Daily CLI Tools Exploratory Tester

Impact

  • Severity: Medium
  • Frequency: Always (reproducible with any unknown tool name)
  • Workaround: None — users have no way to know their tool configuration is being ignored unless they manually inspect the generated lock file

This is especially problematic because:

  1. Users may spend time debugging why a tool isn't working without realizing the name is wrong
  2. Typos in built-in tool names (e.g., playwrigjht instead of playwright) would also be silently ignored
  3. There is no feedback loop to help users discover valid tool names

Additional Context

Tested with nonexistent-tool-xyz as the tool name. The generated lock file contained no trace of the tool configuration. Strict mode (strict: true) does not catch this case either — the behavior is the same regardless of strict mode setting.

References:

Generated by Daily CLI Tools Exploratory Tester · ● 1.9M ·

  • expires on Apr 13, 2026, 5:34 AM UTC

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions