Skip to content

Integration CI flake: live-DB jobs collide on fixed host ports #724

Description

@coderdan

Summary

The Supabase/Drizzle v3 integration jobs fail intermittently in CI — not on anything in the PR, but in the Start step before any test runs:

Error response from daemon: failed to set up container networking:
failed to bind host port for 0.0.0.0:55430:172.18.0.3:3000/tcp: address already in use

Both integration-supabase.yml and integration-drizzle.yml bring up docker-compose stacks binding fixed host ports (55430 / 55432 / 55433) on shared Blacksmith runners with no concurrency guard. When two such jobs land on the same host at once — or a container leaks onto a reused runner from a hard-killed prior run — the second can't bind the port and the job fails. A re-run with no code change passes, confirming the flake.

The contention crosses workflows too: the Drizzle supabase matrix leg brings up the same supabase compose (55430/55433) as the Supabase workflow, so they can collide with each other.

Expected

Live-DB jobs sharing a compose/port group should serialize rather than collide, and leaked containers from a prior run should be cleared before up.

Resolution

Fixed in #641:

  • Job-level concurrency keyed by compose variant (integration-live-db-${{ matrix.db }}), keyed by port group not ref, cancel-in-progress: false so live-DB jobs queue rather than cancel. Drizzle supabase leg shares the Supabase key; the postgres leg (55432) keeps its own and still runs in parallel.
  • Pre-up cleanup (docker compose … down -v --remove-orphans || true) as belt-and-braces for leaked containers.

CI-tooling only — no changeset, no skill updates.

Follow-up

The fully robust fix is to stop binding fixed host ports (per-run compose project + ephemeral ports, read the mapped port in the harness) — larger change, deferred.

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions