Skip to content

MEASURED: a lockfile policy kills 77% of dead workflows — the wiped allowlist is only 10%, and the policy is invisible to the API #657

Description

@hyperpolymath

The estate's dominant CI killer is a lockfile policy, not the wiped allowlist

A census of the 45 repos with a wiped Actions allowlist read the server-rendered rejection annotation on every one of their 265 dead workflow runs. The result overturns the working model — including the one I was operating on earlier today.

Proximate cause of every dead run (n = 265, measured, not inferred)

cause count share annotation text
LOCKFILE-MISSING 126 47.5% "Workflow must use a lockfile. Run gh actions pin to generate one."
LOCKFILE-INVALID 78 29.4% "The lockfile could not be validated. Regenerate it by running gh actions-lock."
ALLOWLIST 27 10.2% "…is not allowed in hyperpolymath/X because all actions must be from a repository owned by hyperpolymath, created by GitHub, or verified in the GitHub Marketplace."
UNEXPLAINED (no annotation rendered) 21 7.9%
ACTOR + lockfile 6 2.3% "Actor is not allowed to trigger Actions workflows"
SHA-PINNING 4 1.5% rejection names only GitHub-owned refs → the full-length-SHA clause
PERMISSION-SCOPE 3 1.1% reusable requests actions: read, caller grants none

Lockfile policy = 204 of 265 (77%). The wiped allowlist — which I had been treating as the main event — accounts for 10%.

⚠️ The policy is live, and invisible to the REST API

Verified independently on hyperpolymath/anamnesis, run 33019781000, 2026-08-26T22:27:53Z — under two hours old at the time of writing:

Workflow must use a lockfile. Run `gh actions pin` to generate one.
The following workflows are missing a lockfile:
  - .github/workflows/governance.yml
  - hyperpolymath/standards/.github/workflows/governance-reusable.yml@81dbf2dd

That same repo reports:

{"enabled": true, "allowed_actions": "selected", "sha_pinning_required": false}

sha_pinning_required is false and the lockfile is still required. These are separate policies. Checked and found not to expose it: /actions/permissions, /actions/permissions/workflow, /actions/permissions/selected-actions, /actions/permissions/artifact-and-log-retention, /users/{u}/settings/actions.

So this policy — the single largest cause of dead CI in the estate — cannot be read from the API at all, only from the run-page banner. That is why it has gone unattributed for so long.

⚠️ Worth checking directly: a lockfile policy was reportedly switched off earlier. It is demonstrably still enforcing. Either the change did not apply at the level that matters, or it was re-enabled, or a different toggle was moved.

Two corrections to the prevailing model

1. Same-owner actions are allowed even with the allowlist wiped. The rejection template itself says "from a repository owned by hyperpolymath". Confirmed empirically: ubicity (wiped allowlist) ran hyperpolymath/standards/mirror-reusable.yml@<SHA> today and reached failure, not startup_failure. So hyperpolymath/standards/* and hyperpolymath/cicd-suite/* are not allowlist casualties.

2. Restoring the allowlist will not revive most of these repos. 89 dead workflows across 37 repos reference zero non-verified third-party actions and would stay dead after a perfect restore. Largest: lithoglyph 13, boj-server 8.

Conversely, 143 of the 238 non-allowlist deaths carry latent allowlist exposure (56 direct, 87 only via a standards/*-reusable.yml whose internals use blocked actions). Fixing their lockfile moves them to the next wall, not to green. Both fixes are needed; neither alone suffices.

Scope

45 repos, all confirmed allowed_actions: "selected" + patterns_allowed: [] individually. 727 workflow entries enumerated (max 33 in any repo, so no page-size truncation); 65 have never run; 265 dead. 44 of 45 have at least one dead workflow — ubicity is the only one with none.

A loose end that matters

The lockfile rule's source is unidentified and its behaviour is inconsistent. sha_pinning_required does not predict it:

  • anamnesis — flag false, refused for a missing lockfile (8 of 8 deaths)
  • ubicity — flag true, no lockfile anywhere in its tree, ran today, zero startup failures

Dead-rate is 140/320 in the flag-false cohort vs 125/407 in the flag-true cohort — the pinning flag correlates negatively with death. Whatever gates the lockfile requirement, it is not that flag.

⚠️ The 27 allowlist deaths are a floor, not a ceiling

GitHub renders one rejection per run. A workflow violating both the lockfile rule and the allowlist shows only the lockfile message. So allowlist damage is masked behind lockfile damage, and the true allowlist number can only be known after the lockfile issue is resolved.

Suggested order

  1. Find and settle the lockfile policy — 77% of the damage, and it is live right now. Until this is resolved, nothing else can be measured cleanly, because it masks every other cause.
  2. Then restore the 45 wiped allowlists and re-measure — the allowlist number will rise from 27.
  3. Then handle the residue: 89 workflows with neither cause, plus the 4 GitHub-owned actions refused for not being SHA-pinned (boj-server/pages.yml, tropical-types/lean.yml).

Not checked

The 7 private blog-drafts annotations (unauthenticated curl cannot read a private repo's run page); 14 public runs where GitHub rendered no annotation at all; which third-party actions are genuinely Marketplace-verified (proxied by "matches one of the 118 patterns a healthy repo carries"); the source/scope of the lockfile policy itself; the 65 never-run workflows; non-default branches.

Metadata

Metadata

Assignees

No one assigned

    Labels

    cicdCI/CD: workflows, actions, lockfiles, pins, runners, release gatesgovernancePolicy, rulesets, standards, compliance, and their enforcementresearchOpen investigation; the outcome is knowledge, not codescope:estateAffects many or all repos across the estate

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions