Skip to content

[aw-failures] Systemic stale-lock on main — 4+ workflows hard-fail at activation (committed lock/hash drift, incl. HEAD 66c9b2f) #44735

Description

@github-actions

Recompile every workflow lock file on main and gate merges on check-workflow-drift — committed .lock.yml files carry a frontmatter hash that no longer matches their .md at HEAD, hard-failing 4+ workflows at the activation step before any agent runs.

Problem statement

Multiple agentic workflows fail immediately at the Check workflow lock file activation step with:

ERR_CONFIG: Lock file '.github/workflows/<name>.lock.yml' is outdated!
The workflow file '<name>.md' frontmatter has changed. Run 'gh aw compile' to regenerate the lock file.

The agent job is skipped; no work is done and no safe output is produced. This is not self-healing: the Copilot PR NLP Analysis run failed at 66c9b2f — the current main HEAD — so the drift is committed, not a stale-checkout artifact.

Failure cluster (last 6h)

Workflow Run Branch / event SHA
Copilot PR Conversation NLP Analysis §29088572869 main / schedule 66c9b2f (HEAD)
[aw] Failure Investigator (6h) §29078152775 main / schedule d520c64b
Auto-Triage Issues §29077454106 main / schedule d520c64b
Daily News §29084339057 main / schedule d520c64b
Impeccable Skills Reviewer §29086731450 PR / pull_request 0fcba1de

Parent #44684 tracks only the Failure Investigator instance; this issue captures the systemic blast radius across ≥4 workflows on main.

Probable root cause

Committed .lock.yml files were regenerated by a gh aw compiler version whose frontmatter-hash output differs from the version CI uses to validate — or a .md frontmatter edit landed without a matching recompile. .md and .lock.yml share a commit, so the lightweight scripts/check-stale-lock-files.sh gate (which only inspects working-tree/staged edits) does not catch this committed hash drift; only the recompile-based check-workflow-drift.sh does.

Proposed remediation

  1. Run gh aw compile on main with the CI-pinned gh-aw version, commit the regenerated .lock.yml files.
  2. Wire make check-workflow-drift (full recompile diff) into required CI so committed lock/hash drift blocks merge — the current check-stale-lock-files early gate misses it.
  3. Pin/align the gh-aw compiler version between local recompiles and CI validation to stop hash-algorithm drift from re-introducing this.

Success criteria / verification

  • Every agentic workflow passes the Check workflow lock file activation step on main.
  • make check-workflow-drift is green in CI and required for merge.
  • No new ERR_CONFIG stale-lock activation failures in the next 6h investigator window.

Analyzed runs: 29088572869, 29078152775, 29077454106, 29084339057, 29086731450. Parent: #44684.
Related to #44684

Generated by 🔍 [aw] Failure Investigator (6h) · 205.7 AIC · ⌖ 17.7 AIC · ⊞ 5.2K · ◷

  • expires on Jul 17, 2026, 5:37 AM UTC-08:00

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions