Skip to content

perf(ci): 2-phase CI latency optimization and bootstrap.d test harness caching - #37

Merged
nilbot merged 9 commits into
masterfrom
feat/ci-latency-and-test-harness-optimization
Aug 29, 2026
Merged

perf(ci): 2-phase CI latency optimization and bootstrap.d test harness caching#37
nilbot merged 9 commits into
masterfrom
feat/ci-latency-and-test-harness-optimization

Conversation

@nilbot

@nilbot nilbot commented Aug 29, 2026

Copy link
Copy Markdown
Owner

Summary & Performance Optimization Phases

This PR delivers a comprehensive two-phase optimization of GitHub Actions CI (verify.yml) and local test suite latency, driving overall CI critical-path turnaround time down from ~6.2 minutes to ~1.3–2.0 minutes without compromising test hermeticity, coverage, or containment.


Phase 1: Core Test Compilation Caching & Matrix Scoping

  • Test Harness Pre-compilation & Per-Test Cache Seeding (bootstrap.d/main_test.go):
    • Introduces one-time build cache warming in TestMain to compile bootstrap once per test process into a temporary shared location.
    • Seeds the binary into isolated per-test $home/cache/dotfiles-bootstrap/<key>/ directories for standard command tests (plan, apply, check, migrate).
    • Eliminates ~45 redundant go build -trimpath sequential compilations per test run.
    • Preserves dedicated cold-build and failure-mode testing via explicit runShimCold and runShimColdEnv helpers.
  • Scoping Gitleaks Installation (.github/workflows/verify.yml):
    • Adds if: matrix.module == 'agents' to the install gitleaks, checksum-verified step in the test matrix so bootstrap.d matrix legs skip redundant tool downloads and installations.

Phase 2: Universal Alt-Checkout Seeding & Container Package Caching

  • Universal Dynamic Cache Key Derivation (bootstrap.d/main_test.go):
    • Implements bootstrapCacheKey(root string) matching ./bootstrap's POSIX cksum algorithm.
    • Automatically seeds the precompiled binary for altCheckout(t) integration tests modifying non-Go files (links.manifest, Brewfile, configs), eliminating an additional 7–8 full Go compilations per run.
    • Updates TestCacheIsKeyedOnTheCheckout to use runShimColdEnv for its secondary checkout, guaranteeing genuine cold-build and cache isolation verification.
  • Stage-Zero Package Manager Caching (.github/workflows/verify.yml):
    • Adds pinned actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 to the linux-stage-zero container matrix.
    • Caches /var/cache/apt/archives (Debian) and /var/cache/pacman/pkg (Arch Linux) across workflow runs to eliminate network bottlenecking during stageZero package installation.

Validation & Quality Invariants

  • agents suite: 100% PASS
  • bootstrap.d suite: 100% PASS in ~1.8s (down from ~150s)
  • Race detector (go test -race): 100% PASS across all modules
  • Synthetic $HOME containment: 100% PASS (0 leaked files outside isolated sandbox)
  • Restrictive umask (umask 077): 100% PASS
  • Production ./bootstrap script remains completely untouched.

@nilbot nilbot changed the title perf(ci): optimize bootstrap.d test compilation and scope gitleaks in verify matrix perf(ci): 2-phase CI latency optimization and bootstrap.d test harness caching Aug 29, 2026
@nilbot

nilbot commented Aug 29, 2026

Copy link
Copy Markdown
Owner Author

📊 CI Performance Benchmark Results

Metric / Job Initial Baseline (Avg of 29 runs) Phase 1 Run (33264574909) Phase 2 Run (33266725774) Total Improvement
Total CI Wall-Clock Turnaround 372.0s (6m 12s) 203.0s (3m 23s) 143.0s (2m 23s) -229.0s (~62% reduction)
hygiene 347.3s (5m 47s) 194.0s (3m 14s) 88.0s (1m 28s) -259.3s (74.7% faster)
suites must not write to $HOME 177.2s 100.0s 48.0s -129.2s (72.9% faster)
bootstrap.d under restrictive umask 153.7s 75.0s 23.0s -130.7s (85.0% faster)
test (ubuntu-latest, bootstrap.d) 338.0s (5m 38s) 165.0s (2m 45s) 83.0s (1m 23s) -255.0s (75.4% faster)
test step 153.2s 72.0s 26.0s -127.2s (83.0% faster)
test -race step 161.6s 82.0s 35.0s -126.6s (78.3% faster)
test (macos-latest, bootstrap.d) 328.1s (5m 28s) 159.0s (2m 39s) 115.0s (1m 55s) -213.1s (65.0% faster)
test step 144.1s 63.0s 30.0s -114.1s (79.2% faster)
test -race step 145.8s 64.0s 41.0s -104.8s (71.9% faster)
All Gate Checks Success Success Success (100% Green) Zero Regressions

@nilbot
nilbot merged commit 95222ce into master Aug 29, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant