Skip to content

refactor(test): make all Docker tests hermetic#439

Open
sbalabanov wants to merge 1 commit into
mainfrom
e2e-hermetic-2
Open

refactor(test): make all Docker tests hermetic#439
sbalabanov wants to merge 1 commit into
mainfrom
e2e-hermetic-2

Conversation

@sbalabanov

Copy link
Copy Markdown
Contributor

Intent

Make every Docker-based test fully hermetic: all inputs are explicit Bazel data dependencies, no test resolves the repository root, and the Bazel sandbox is stricter than before.

Note: branch is e2e-hermetic-2 because e2e-hermetic (#438) was taken by a concurrent attempt at the same task; this PR is an independent implementation.

Problem

Docker suites resolved the source checkout at runtime — testutil.FindRepoRoot walked runfile symlinks of MODULE.bazel/go.mod (declared as data solely for that purpose) or shelled out to git — and handed the checkout to docker compose as the image build context. The images COPY'd pre-built .docker-bin/* binaries produced by make build-*-linux outside Bazel's action graph, so bazel test results depended on undeclared, stale-able state.

Changes

Hermetic inputs

  • New go_cross_binary targets (gateway_linux, orchestrator_linux, runway_linux, stovepipe_linux): Bazel builds the exact Linux binary that lands in each image; tests depend on them as data.
  • testutil.WithBuildContext stages a minimal docker build context (binary + Dockerfile + configs, all from declared runfiles) into a test temp dir; compose's REPO_ROOT variable now points at that staged context. Existing compose files and local-* Makefile flows are unchanged.
  • testutil.Runfile resolves compose files from runfiles; FindRepoRoot is deleted and every //:MODULE.bazel / //:go.mod data dep is removed.
  • Image tag prefix derives from TEST_TARGET (stable per target) instead of the checkout path, so parallel targets no longer share tags.
  • Compose V2 plugin is preferred over standalone docker-compose (legacy V1 lacks up --wait); DOCKER_CONFIG falls back to the invoking user's config dir so user-level CLI plugins are found despite Bazel's scratch HOME.

Stricter sandbox (beneficial now that tests are hermetic)

  • --incompatible_strict_action_env: static action/test environment — no client env leaks into results.
  • --sandbox_default_allow_network=false: sandboxed actions get their own network namespace by default; Docker tests opt back in via tags = ["requires-network"].
  • Dropped the external tag: hermetic test results are now cacheable — unchanged integration/e2e suites skip entirely (verified locally: (cached) PASSED).
  • DOCKER_HOST / SKIP_CLEANUP are explicit --test_env pass-throughs.

Makefile / docs

  • integration-test* / e2e-test no longer depend on build-all-linux (still used by local-* compose flows).
  • Updated AGENTS.md / TESTING.md guidance.

Test Plan

  • bazel test on all 9 Docker suites passes locally (5 MySQL-only + gateway + orchestrator + stovepipe + both e2e)
  • make test (79 unit tests) passes
  • make lint, make check-tidy, make check-gazelle pass
  • Test-result caching verified (second run: (cached) PASSED)
  • CI green on all required checks

🤖 Generated with Claude Code

Docker-based integration/e2e tests previously resolved the repository root
(via MODULE.bazel / go.mod runfile symlinks or git) and handed it to docker
compose as the image build context, relying on pre-built .docker-bin binaries
produced by `make build-*-linux` outside of Bazel's action graph.

Now every input is an explicit Bazel data dependency:

- Each service gets a `go_cross_binary` `<name>_linux` target; tests depend on
  it directly, so Bazel builds the exact binary that lands in the image.
- testutil.WithBuildContext stages a minimal docker build context (binary,
  Dockerfile, configs) from declared runfiles into a temp dir; compose's
  REPO_ROOT variable points at that staged context, never at the checkout.
- testutil.Runfile resolves the compose file from runfiles; FindRepoRoot and
  all MODULE.bazel / go.mod data dependencies are removed.
- Image tag prefix derives from TEST_TARGET instead of the checkout path.
- Compose V2 plugin is preferred over a standalone docker-compose binary
  (legacy V1 lacks `up --wait`); DOCKER_CONFIG falls back to the invoking
  user's config dir so user-level CLI plugins are found under Bazel's
  scratch HOME.

Stricter sandbox settings now that tests are hermetic:

- --incompatible_strict_action_env: static action/test environment.
- --sandbox_default_allow_network=false: sandboxed actions get no network;
  Docker tests opt back in via tags = ["requires-network"].
- The `external` tag is dropped, so passing integration/e2e results are
  cached and re-run only when a declared input changes.

`make integration-test` / `make e2e-test` no longer need build-all-linux;
the local docker-compose flows (local-*) still use it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@sbalabanov
sbalabanov marked this pull request as ready for review July 24, 2026 17:10
@sbalabanov
sbalabanov requested review from a team and behinddwalls as code owners July 24, 2026 17:10
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants