test(integration): fix flaky tests with root-cause changes by ValentaTomas · Pull Request #2713 · e2b-dev/runtime

ValentaTomas · 2026-05-18T20:13:58Z

Summary

Address 5 flaky integration tests that have been blocking PRs (50–77% flake rate in main per Codecov). Each fix is targeted at the actual root cause, not just a longer timeout.

TestTeamMetrics: Eventually returned as soon as any metric existed, but the assertions required SandboxStartRate > 0 and ConcurrentSandboxes > 0, which lag behind first non-empty response due to ClickHouse aggregation. Moved the assertions inside the loop; bumped the window to 60s.
TestUpdateNetworkConfig pause/resume subtests: a one-shot curl after resume races whatever in-VM/proxy warmup happens after the snapshot is restored. Resume returns 201 but the egress proxy / in-sandbox networking can lag. New verifyConnectivityEventually helper polls connectivity checks for 30s.
TestBindLocalhost: fixed 5s sleep was insufficient for python's http.server to start under CI load, producing 502s. Replaced with WaitForStatus poll. Also extended WaitForStatus's cap from 5s→60s so any caller starting an in-sandbox service has time to come up.
TestListDir: parent created the shared sandbox with the 30s default timeout, which expired while the four parallel subtests waited to be scheduled (failures came in 0.01–0.03s = "sandbox gone"). Bumped to 120s.
TestSandboxMemoryIntegrity subtests: dd of /dev/urandom + pause/resume iterations routinely exceed the 30s default timeout (the tmpfs_hash subtest alone runs ~44s). Bumped all three subtests to 300s.

Note on TestUpdateNetworkConfig

Traced the resume path: pool.Get → ConfigureInternet is synchronous before resume returns, so the firewall rule install itself is not async. The remaining flake is likely either in-VM warmup or a race I couldn't pin down without deeper investigation. The Eventually retry is defensive — open to a follow-up if a real bug is identified.

cursor · 2026-05-18T20:14:05Z

PR Summary

Low Risk
Low risk since changes are limited to integration test logic and retry/timeout helpers, with no production code paths affected. Main risk is slightly longer CI runtime or masking real regressions if the new waits are too permissive.

Overview
Fixes several flaky integration tests where assertions ran before metrics/networking/services were actually ready by moving success conditions into polling loops and replacing fixed sleeps with Eventually/WaitForStatus retries. Increases sandbox/test timeouts for slower CI cases (team metrics aggregation, pause/resume connectivity checks, envd filesystem and memory integrity tests) so tests stop failing due to premature sandbox expiry or service startup lag.

^{Reviewed by Cursor Bugbot for commit 4a44dda. Bugbot is set up for automated code reviews on this repo. Configure here.}

codecov · 2026-05-18T20:15:32Z

❌ 4 Tests Failed:

Tests completed	Failed	Passed	Skipped
2628	4	2624	5

View the full list of 8 ❄️ flaky test(s)

github.com/e2b-dev/infra/tests/integration/internal/tests/api/metrics::TestSandboxMetrics

Flake rate in main: 56.34% (Passed 241 times, Failed 311 times)

Stack Traces | 9.9s run time

=== RUN   TestSandboxMetrics
=== PAUSE TestSandboxMetrics
=== CONT  TestSandboxMetrics
    sandbox_metrics_test.go:44: 
        	Error Trace:	.../api/metrics/sandbox_metrics_test.go:44
        	Error:      	Should NOT be empty, but was 0
        	Test:       	TestSandboxMetrics
--- FAIL: TestSandboxMetrics (9.90s)

github.com/e2b-dev/infra/tests/integration/internal/tests/api/sandboxes::TestFuseNonRootAccess

Flake rate in main: 54.68% (Passed 242 times, Failed 292 times)

Stack Traces | 4.59s run time

=== RUN   TestFuseNonRootAccess
=== PAUSE TestFuseNonRootAccess
=== CONT  TestFuseNonRootAccess
    sandbox_fuse_test.go:45: Command [test] output: event:{start:{pid:1263}}
    sandbox_fuse_test.go:45: Command [test] output: event:{end:{exited:true status:"exit status 0"}}
    sandbox_fuse_test.go:45: Command [test] completed successfully in sandbox ir0lq3daic7baos75pk1u
Executing command test in sandbox ir0lq3daic7baos75pk1u
    sandbox_fuse_test.go:49: Command [test] output: event:{start:{pid:1264}}
    sandbox_fuse_test.go:50: 
        	Error Trace:	.../api/sandboxes/sandbox_fuse_test.go:50
        	Error:      	Received unexpected error:
        	            	failed to execute command test in sandbox ir0lq3daic7baos75pk1u: invalid_argument: protocol error: incomplete envelope: unexpected EOF
        	Test:       	TestFuseNonRootAccess
        	Messages:   	Non-root user should be able to write to /dev/fuse
--- FAIL: TestFuseNonRootAccess (4.59s)

github.com/e2b-dev/infra/tests/integration/internal/tests/api/sandboxes::TestUpdateNetworkConfig

Flake rate in main: 76.82% (Passed 249 times, Failed 825 times)

Stack Traces | 191s run time

=== RUN   TestUpdateNetworkConfig
=== PAUSE TestUpdateNetworkConfig
=== CONT  TestUpdateNetworkConfig
--- FAIL: TestUpdateNetworkConfig (191.00s)

github.com/e2b-dev/infra/tests/integration/internal/tests/api/sandboxes::TestUpdateNetworkConfig/pause_resume_preserves_allow_internet_access_false

Flake rate in main: 77.32% (Passed 240 times, Failed 818 times)

Stack Traces | 32s run time

=== RUN   TestUpdateNetworkConfig/pause_resume_preserves_allow_internet_access_false
Executing command curl in sandbox ibq3190camnfyd1w5gz32
    sandbox_network_update_test.go:399: Command [curl] output: event:{start:{pid:1360}}
    sandbox_network_update_test.go:399: Command [curl] output: event:{end:{exit_code:35 exited:true status:"exit status 35" error:"exit status 35"}}
Executing command curl in sandbox ibq3190camnfyd1w5gz32
    sandbox_network_update_test.go:399: Command [curl] output: event:{start:{pid:1361}}
    sandbox_network_update_test.go:399: Command [curl] output: event:{end:{exit_code:35 exited:true status:"exit status 35" error:"exit status 35"}}
Executing command curl in sandbox io20m88qrbasz8d66vsbq
    sandbox_network_update_test.go:28: Command [curl] output: event:{start:{pid:1362}}
    sandbox_network_update_test.go:28: Command [curl] output: event:{data:{stdout:"HTTP/2 302 \r\nx-content-type-options: nosniff\r\nlocation: https://dns.google/\r\ndate: Mon, 18 May 2026 20:58:39 GMT\r\ncontent-type: text/html; charset=UTF-8\r\nserver: HTTP server (unknown)\r\ncontent-length: 216\r\nx-xss-protection: 0\r\nx-frame-options: SAMEORIGIN\r\nalt-svc: h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000\r\n\r\n"}}
    sandbox_network_update_test.go:28: Command [curl] output: event:{end:{exited:true status:"exit status 0"}}
    sandbox_network_update_test.go:28: Command [curl] completed successfully in sandbox ibq3190camnfyd1w5gz32
Executing command curl in sandbox ibq3190camnfyd1w5gz32
    sandbox_network_update_test.go:28: Command [curl] output: event:{start:{pid:1363}}
    sandbox_network_update_test.go:28: Command [curl] output: event:{data:{stdout:"HTTP/2 302 \r\nx-content-type-options: nosniff\r\nlocation: https://dns.google/\r\ndate: Mon, 18 May 2026 20:58:40 GMT\r\ncontent-type: text/html; charset=UTF-8\r\nserver: HTTP server (unknown)\r\ncontent-length: 216\r\nx-xss-protection: 0\r\nx-frame-options: SAMEORIGIN\r\nalt-svc: h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000\r\n\r\n"}}
    sandbox_network_update_test.go:28: Command [curl] output: event:{end:{exited:true status:"exit status 0"}}
    sandbox_network_update_test.go:28: Command [curl] completed successfully in sandbox ibq3190camnfyd1w5gz32
Executing command curl in sandbox ibq3190camnfyd1w5gz32
    sandbox_network_update_test.go:28: Command [curl] output: event:{start:{pid:1364}}
    sandbox_network_update_test.go:28: Command [curl] output: event:{data:{stdout:"HTTP/2 302 \r\nx-content-type-options: nosniff\r\nlocation: https://dns.google/\r\ndate: Mon, 18 May 2026 20:58:41 GMT\r\ncontent-type: text/html; charset=UTF-8\r\nserver: HTTP server (unknown)\r\ncontent-length: 216\r\nx-xss-protection: 0\r\nx-frame-options: SAMEORIGIN\r\nalt-svc: h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000\r\n\r\n"}}
    sandbox_network_update_test.go:28: Command [curl] output: event:{end:{exited:true status:"exit status 0"}}
    sandbox_network_update_test.go:28: Command [curl] completed successfully in sandbox ibq3190camnfyd1w5gz32
Executing command curl in sandbox ibq3190camnfyd1w5gz32
    sandbox_network_update_test.go:28: Command [curl] output: event:{start:{pid:1365}}
    sandbox_network_update_test.go:28: Command [curl] output: event:{data:{stdout:"HTTP/2 302 \r\nx-content-type-options: nosniff\r\nlocation: https://dns.google/\r\ndate: Mon, 18 May 2026 20:58:42 GMT\r\ncontent-type: text/html; charset=UTF-8\r\nserver: HTTP server (unknown)\r\ncontent-length: 216\r\nx-xss-protection: 0\r\nx-frame-options: SAMEORIGIN\r\nalt-svc: h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000\r\n\r\n"}}
    sandbox_network_update_test.go:28: Command [curl] output: event:{end:{exited:true status:"exit status 0"}}
    sandbox_network_update_test.go:28: Command [curl] completed successfully in sandbox ibq3190camnfyd1w5gz32
Executing command curl in sandbox ibq3190camnfyd1w5gz32
    sandbox_network_update_test.go:28: Command [curl] output: event:{start:{pid:1366}}
    sandbox_network_update_test.go:28: Command [curl] output: event:{data:{stdout:"HTTP/2 302 \r\nx-content-type-options: nosniff\r\nlocation: https://dns.google/\r\ndate: Mon, 18 May 2026 20:58:43 GMT\r\ncontent-type: text/html; charset=UTF-8\r\nserver: HTTP server (unknown)\r\ncontent-length: 216\r\nx-xss-protection: 0\r\nx-frame-options: SAMEORIGIN\r\nalt-svc: h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000\r\n\r\n"}}
    sandbox_network_update_test.go:28: Command [curl] output: event:{end:{exited:true status:"exit status 0"}}
    sandbox_network_update_test.go:28: Command [curl] completed successfully in sandbox ibq3190camnfyd1w5gz32
Executing command curl in sandbox ibq3190camnfyd1w5gz32
    sandbox_network_update_test.go:28: Command [curl] output: event:{start:{pid:1367}}
    sandbox_network_update_test.go:28: Command [curl] output: event:{data:{stdout:"HTTP/2 302 \r\nx-content-type-options: nosniff\r\nlocation: https://dns.google/\r\ndate: Mon, 18 May 2026 20:58:44 GMT\r\ncontent-type: text/html; charset=UTF-8\r\nserver: HTTP server (unknown)\r\ncontent-length: 216\r\nx-xss-protection: 0\r\nx-frame-options: SAMEORIGIN\r\nalt-svc: h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000\r\n\r\n"}}
    sandbox_network_update_test.go:28: Command [curl] output: event:{end:{exited:true status:"exit status 0"}}
    sandbox_network_update_test.go:28: Command [curl] completed successfully in sandbox ibq3190camnfyd1w5gz32
Executing command curl in sandbox ibq3190camnfyd1w5gz32
    sandbox_network_update_test.go:28: Command [curl] output: event:{start:{pid:1368}}
    sandbox_network_update_test.go:28: Command [curl] output: event:{data:{stdout:"HTTP/2 302 \r\nx-content-type-options: nosniff\r\nlocation: https://dns.google/\r\ndate: Mon, 18 May 2026 20:58:45 GMT\r\ncontent-type: text/html; charset=UTF-8\r\nserver: HTTP server (unknown)\r\ncontent-length: 216\r\nx-xss-protection: 0\r\nx-frame-options: SAMEORIGIN\r\nalt-svc: h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000\r\n\r\n"}}
    sandbox_network_update_test.go:28: Command [curl] output: event:{end:{exited:true status:"exit status 0"}}
    sandbox_network_update_test.go:28: Command [curl] completed successfully in sandbox ibq3190camnfyd1w5gz32
Executing command curl in sandbox ibq3190camnfyd1w5gz32
    sandbox_network_update_test.go:28: Command [curl] output: event:{start:{pid:1369}}
    sandbox_network_update_test.go:28: Command [curl] output: event:{data:{stdout:"HTTP/2 302 \r\nx-content-type-options: nosniff\r\nlocation: https://dns.google/\r\ndate: Mon, 18 May 2026 20:58:46 GMT\r\ncontent-type: text/html; charset=UTF-8\r\nserver: HTTP server (unknown)\r\ncontent-length: 216\r\nx-xss-protection: 0\r\nx-frame-options: SAMEORIGIN\r\nalt-svc: h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000\r\n\r\n"}}
    sandbox_network_update_test.go:28: Command [curl] output: event:{end:{exited:true status:"exit status 0"}}
    sandbox_network_update_test.go:28: Command [curl] completed successfully in sandbox ibq3190camnfyd1w5gz32
    sandbox_network_update_test.go:28: Command [curl] output: event:{start:{pid:1370}}
    sandbox_network_update_test.go:28: Command [curl] output: event:{data:{stdout:"HTTP/2 302 \r\nx-content-type-options: nosniff\r\nlocation: https://dns.google/\r\ndate: Mon, 18 May 2026 20:58:47 GMT\r\ncontent-type: text/html; charset=UTF-8\r\nserver: HTTP server (unknown)\r\ncontent-length: 216\r\nx-xss-protection: 0\r\nx-frame-options: SAMEORIGIN\r\nalt-svc: h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000\r\n\r\n"}}
    sandbox_network_update_test.go:28: Command [curl] output: event:{end:{exited:true status:"exit status 0"}}
    sandbox_network_update_test.go:28: Command [curl] completed successfully in sandbox ibq3190camnfyd1w5gz32
Executing command curl in sandbox ibq3190camnfyd1w5gz32
    sandbox_network_update_test.go:28: Command [curl] output: event:{start:{pid:1371}}
    sandbox_network_update_test.go:28: Command [curl] output: event:{data:{stdout:"HTTP/2 302 \r\nx-content-type-options: nosniff\r\nlocation: https://dns.google/\r\ndate: Mon, 18 May 2026 20:58:48 GMT\r\ncontent-type: text/html; charset=UTF-8\r\nserver: HTTP server (unknown)\r\ncontent-length: 216\r\nx-xss-protection: 0\r\nx-frame-options: SAMEORIGIN\r\nalt-svc: h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000\r\n\r\n"}}
    sandbox_network_update_test.go:28: Command [curl] output: event:{end:{exited:true status:"exit status 0"}}
    sandbox_network_update_test.go:28: Command [curl] completed successfully in sandbox ibq3190camnfyd1w5gz32
    sandbox_network_update_test.go:28: Command [curl] output: event:{start:{pid:1372}}
    sandbox_network_update_test.go:28: Command [curl] output: event:{data:{stdout:"HTTP/2 302 \r\nx-content-type-options: nosniff\r\nlocation: https://dns.google/\r\ndate: Mon, 18 May 2026 20:58:49 GMT\r\ncontent-type: text/html; charset=UTF-8\r\nserver: HTTP server (unknown)\r\ncontent-length: 216\r\nx-xss-protection: 0\r\nx-frame-options: SAMEORIGIN\r\nalt-svc: h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000\r\n\r\n"}}
    sandbox_network_update_test.go:28: Command [curl] output: event:{end:{exited:true status:"exit status 0"}}
    sandbox_network_update_test.go:28: Command [curl] completed successfully in sandbox ibq3190camnfyd1w5gz32
Executing command curl in sandbox ipo54616rbwbqk36w98of
    sandbox_network_update_test.go:28: Command [curl] output: event:{start:{pid:1373}}
    sandbox_network_update_test.go:28: Command [curl] output: event:{data:{stdout:"HTTP/2 302 \r\nx-content-type-options: nosniff\r\nlocation: https://dns.google/\r\ndate: Mon, 18 May 2026 20:58:50 GMT\r\ncontent-type: text/html; charset=UTF-8\r\nserver: HTTP server (unknown)\r\ncontent-length: 216\r\nx-xss-protection: 0\r\nx-frame-options: SAMEORIGIN\r\nalt-svc: h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000\r\n\r\n"}}
    sandbox_network_update_test.go:28: Command [curl] output: event:{end:{exited:true status:"exit status 0"}}
    sandbox_network_update_test.go:28: Command [curl] completed successfully in sandbox ibq3190camnfyd1w5gz32
    sandbox_network_update_test.go:28: Command [curl] output: event:{start:{pid:1374}}
    sandbox_network_update_test.go:28: Command [curl] output: event:{data:{stdout:"HTTP/2 302 \r\nx-content-type-options: nosniff\r\nlocation: https://dns.google/\r\ndate: Mon, 18 May 2026 20:58:51 GMT\r\ncontent-type: text/html; charset=UTF-8\r\nserver: HTTP server (unknown)\r\ncontent-length: 216\r\nx-xss-protection: 0\r\nx-frame-options: SAMEORIGIN\r\nalt-svc: h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000\r\n\r\n"}}
    sandbox_network_update_test.go:28: Command [curl] output: event:{end:{exited:true status:"exit status 0"}}
    sandbox_network_update_test.go:28: Command [curl] completed successfully in sandbox ibq3190camnfyd1w5gz32
    sandbox_network_update_test.go:28: Command [curl] output: event:{start:{pid:1375}}
    sandbox_network_update_test.go:28: Command [curl] output: event:{data:{stdout:"HTTP/2 302 \r\nx-content-type-options: nosniff\r\nlocation: https://dns.google/\r\ndate: Mon, 18 May 2026 20:58:52 GMT\r\ncontent-type: text/html; charset=UTF-8\r\nserver: HTTP server (unknown)\r\ncontent-length: 216\r\nx-xss-protection: 0\r\nx-frame-options: SAMEORIGIN\r\nalt-svc: h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000\r\n\r\n"}}
    sandbox_network_update_test.go:28: Command [curl] output: event:{end:{exited:true status:"exit status 0"}}
    sandbox_network_update_test.go:28: Command [curl] completed successfully in sandbox ibq3190camnfyd1w5gz32
    sandbox_network_update_test.go:28: Command [curl] output: event:{start:{pid:1376}}
    sandbox_network_update_test.go:28: Command [curl] output: event:{data:{stdout:"HTTP/2 302 \r\nx-content-type-options: nosniff\r\nlocation: https://dns.google/\r\ndate: Mon, 18 May 2026 20:58:53 GMT\r\ncontent-type: text/html; charset=UTF-8\r\nserver: HTTP server (unknown)\r\ncontent-length: 216\r\nx-xss-protection: 0\r\nx-frame-options: SAMEORIGIN\r\nalt-svc: h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000\r\n\r\n"}}
    sandbox_network_update_test.go:28: Command [curl] output: event:{end:{exited:true status:"exit status 0"}}
    sandbox_network_update_test.go:28: Command [curl] completed successfully in sandbox ibq3190camnfyd1w5gz32
Executing command curl in sandbox il6yxgkm8pb24a03lxom7
    sandbox_network_update_test.go:417: 
        	Error Trace:	.../api/sandboxes/sandbox_network_update_test.go:26
        	            				.../api/sandboxes/sandbox_network_update_test.go:417
        	Error:      	Condition never satisfied
        	Test:       	TestUpdateNetworkConfig/pause_resume_preserves_allow_internet_access_false
        	Messages:   	connectivity did not match expected state in time
--- FAIL: TestUpdateNetworkConfig/pause_resume_preserves_allow_internet_access_false (32.02s)

github.com/e2b-dev/infra/tests/integration/internal/tests/api/templates::TestTemplateBuildENV

Flake rate in main: 59.32% (Passed 238 times, Failed 347 times)

Stack Traces | 0s run time

=== RUN   TestTemplateBuildENV
=== PAUSE TestTemplateBuildENV
=== CONT  TestTemplateBuildENV
--- FAIL: TestTemplateBuildENV (0.00s)

github.com/e2b-dev/infra/tests/integration/internal/tests/api/templates::TestTemplateBuildENV/ENV_with_multiline_value

Flake rate in main: 59.83% (Passed 231 times, Failed 344 times)

Stack Traces | 7.08s run time

=== RUN   TestTemplateBuildENV/ENV_with_multiline_value
=== PAUSE TestTemplateBuildENV/ENV_with_multiline_value
=== CONT  TestTemplateBuildENV/ENV_with_multiline_value
    build_template_test.go:134: test-ubuntu-env-multiline: [info] Building template hem4zt0ti6l3bgeewfcn/b988b0f1-1e47-4f57-92f9-255662b73812
    build_template_test.go:134: test-ubuntu-env-multiline: [info] CACHED [base] FROM ubuntu:22.04 [ffd709f131f42dfab282de47a91dd2c139e900c1c11fc574b49b517a05ef0a32]
    build_template_test.go:134: test-ubuntu-env-multiline: [info] CACHED [base] DEFAULT USER user [90bdd4afa342293c931373351bf578872dec9179214ba3e8bf9edba311466213]
    build_template_test.go:134: test-ubuntu-env-multiline: [info] [builder 1/2] ENV MULTILINE line1
        line2
        line3 [e93da3f3765f20eb6407c336b9e4e0b9321d994ec5f6cb547743a2a4070eed23]
    build_template_test.go:134: test-ubuntu-env-multiline: [info] [builder 2/2] RUN [[ $(echo "$MULTILINE" | wc -l) -eq 3 ]] || exit 1 [477610d61cdf858776262d3331809539bcbcf16f706aac18515a57337bae1786]
    build_template_test.go:134: test-ubuntu-env-multiline: [error] Build failed: failed to run command '[[ $(echo "$MULTILINE" | wc -l) -eq 3 ]] || exit 1': exit status 1
    build_template_test.go:374: Build failed: {<nil> failed to run command '[[ $(echo "$MULTILINE" | wc -l) -eq 3 ]] || exit 1': exit status 1 0xc000568c10}
--- FAIL: TestTemplateBuildENV/ENV_with_multiline_value (7.08s)

github.com/e2b-dev/infra/tests/integration/internal/tests/orchestrator::TestSandboxMemoryIntegrity

Flake rate in main: 66.07% (Passed 247 times, Failed 481 times)

Stack Traces | 62.1s run time

=== RUN   TestSandboxMemoryIntegrity
=== PAUSE TestSandboxMemoryIntegrity
=== CONT  TestSandboxMemoryIntegrity
    sandbox_memory_integrity_test.go:26: Build failed: {<nil> build was cancelled <nil>}
--- FAIL: TestSandboxMemoryIntegrity (62.13s)

github.com/e2b-dev/infra/tests/integration/internal/tests/orchestrator::TestSandboxMemoryIntegrity/tmpfs_hash

Flake rate in main: 66.71% (Passed 237 times, Failed 475 times)

Stack Traces | 30.9s run time

=== RUN   TestSandboxMemoryIntegrity/tmpfs_hash
=== PAUSE TestSandboxMemoryIntegrity/tmpfs_hash
=== CONT  TestSandboxMemoryIntegrity/tmpfs_hash
    sandbox_memory_integrity_test.go:69: Command [bash] output: event:{start:{pid:1262}}
Executing command bash in sandbox iefmrjqemw1i5eal35aeg (user: root)
    sandbox_memory_integrity_test.go:69: Command [bash] output: event:{data:{stdout:"Total memory: 985 MB\nUsed memory before tmpfs mount: 185 MB\nFree memory before tmpfs mount: 800 MB\nMemory to use in integrity test (80% of free, min 64MB): 640 MB\n"}}
Executing command bash in sandbox iefmrjqemw1i5eal35aeg (user: root)
    sandbox_memory_integrity_test.go:69: Command [bash] output: event:{data:{stderr:"640+0 records in\n640+0 records out\n671088640 bytes (671 MB, 640 MiB) copied, 3.37234 s, 199 MB/s\n"}}
    sandbox_memory_integrity_test.go:69: Command [bash] output: event:{data:{stderr:"\t"}}
    sandbox_memory_integrity_test.go:69: Command [bash] output: event:{data:{stderr:"C"}}
    sandbox_memory_integrity_test.go:69: Command [bash] output: event:{data:{stderr:"o"}}
    sandbox_memory_integrity_test.go:69: Command [bash] output: event:{data:{stderr:"mmand being timed: \"dd if=/dev/urandom of=/mnt/testfile bs=1M count=640\"\n\tUser time (seconds): 0.00\n\tSystem time (seconds): 3.35\n\tPercent of CPU this job got: 99%\n\tElapsed (wall clock) time (h:mm:ss or m:ss): 0:03.37\n\tAverage shared text size (kbytes): 0\n\tAverage unshared data size (kbytes): 0\n\tAverage stack size (kbytes): 0\n\tAverage total size (kbytes): 0\n\tMaximum resident set size (kbytes): 2712\n\tAverage resident set size (kbytes): 0\n\tMajor (requiring I/O) page faults: 2\n\tMinor (reclaiming a frame) page faults: 344\n\tVoluntary context switches: 3\n\tInvoluntary context switches: 15\n\tSwaps: 0\n\tFile system inputs: 176\n\tFile system outputs: 0\n\tSocket messages sent: 0\n\tSocket messages received: 0\n\tSignals delivered: 0\n\tPage size (bytes): 4096\n\tExit status: 0\n"}}
    sandbox_memory_integrity_test.go:69: Command [bash] output: event:{data:{stdout:"Used memory after tmpfs mount and file fill: 830 MB\n"}}
    sandbox_memory_integrity_test.go:69: Command [bash] output: event:{end:{exited:true status:"exit status 0"}}
    sandbox_memory_integrity_test.go:69: Command [bash] completed successfully in sandbox ir5mmqfwnka1b142ph3nx
Executing command bash in sandbox ir5mmqfwnka1b142ph3nx (user: root)
    sandbox_memory_integrity_test.go:73: Command [bash] output: event:{start:{pid:1278}}
    sandbox_memory_integrity_test.go:73: Command [bash] output: event:{data:{stdout:"94833ebf9a3d5b807a85a44d5be726fdeb1d7f8bce8d23d5328941b34a27be99\n"}}
    sandbox_memory_integrity_test.go:73: Command [bash] output: event:{end:{exited:true status:"exit status 0"}}
    sandbox_memory_integrity_test.go:73: Command [bash] completed successfully in sandbox ir5mmqfwnka1b142ph3nx
Executing command bash in sandbox ir5mmqfwnka1b142ph3nx (user: root)
    sandbox_memory_integrity_test.go:98: Command [bash] output: event:{start:{pid:1281}}
    sandbox_memory_integrity_test.go:99: 
        	Error Trace:	.../tests/orchestrator/sandbox_memory_integrity_test.go:99
        	Error:      	Received unexpected error:
        	            	failed to execute command bash in sandbox ir5mmqfwnka1b142ph3nx: invalid_argument: protocol error: incomplete envelope: unexpected EOF
        	Test:       	TestSandboxMemoryIntegrity/tmpfs_hash
--- FAIL: TestSandboxMemoryIntegrity/tmpfs_hash (30.93s)

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}

gemini-code-assist

Code Review

Using require assertions inside the Eventually block in team_metrics_test.go causes the test to terminate immediately on the first failure, which prevents the intended retry logic from executing. Replacing these assertions with conditional checks that return false ensures the loop continues until the timeout is reached.

e2b-atlantis · 2026-05-18T20:21:21Z

Plan Error

cannot run "plan": the default workspace at path . is currently locked for this pull request by "plan".
Wait until the previous command is complete and try again

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d214bac024

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 19b11995bb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

- TestTeamMetrics: Eventually returned on any metrics, but assertions required start-rate and concurrent-sandboxes signals that lag behind the first non-empty response (ClickHouse aggregation). Move the assertions inside the loop and bump the window to 60s. - TestUpdateNetworkConfig pause/resume subtests: firewall rule reapply on resume is async; a one-shot curl races it. New helper verifyConnectivityEventually polls connectivity checks for 30s. - TestBindLocalhost: fixed 5s sleep was insufficient for python's http.server to start under CI load. Replaced with WaitForStatus poll (now extended to 60s total so any caller starting a service has time to come up). - TestListDir: parent sandbox used the 30s default timeout, which expired while the four parallel subtests waited to be scheduled. Bumped to 120s. - TestSandboxMemoryIntegrity subtests: dd + pause/resume iterations routinely exceed the 30s default timeout (tmpfs_hash alone takes ~44s). Bumped all three subtests to 300s.

cla-bot Bot added the cla-signed label May 18, 2026

e2b-request-same-site-reviewers Bot assigned djeebus May 18, 2026

gemini-code-assist Bot reviewed May 18, 2026

View reviewed changes

Comment thread tests/integration/internal/tests/api/metrics/team_metrics_test.go Outdated

ValentaTomas marked this pull request as ready for review May 18, 2026 20:21

ValentaTomas force-pushed the test/fix-flaky-integration-tests branch from 14c86e9 to af67a80 Compare May 18, 2026 20:21

ValentaTomas requested review from dobrac and jakubno as code owners May 18, 2026 20:21

ValentaTomas force-pushed the test/fix-flaky-integration-tests branch from af67a80 to d214bac Compare May 18, 2026 20:25

chatgpt-codex-connector Bot reviewed May 18, 2026

View reviewed changes

Comment thread tests/integration/internal/tests/api/sandboxes/sandbox_network_update_test.go Outdated

ValentaTomas force-pushed the test/fix-flaky-integration-tests branch from d214bac to 19b1199 Compare May 18, 2026 20:35

claude Bot reviewed May 18, 2026

View reviewed changes

Comment thread tests/integration/internal/tests/api/sandboxes/sandbox_network_update_test.go

chatgpt-codex-connector Bot reviewed May 18, 2026

View reviewed changes

Comment thread tests/integration/internal/utils/request.go Outdated

ValentaTomas force-pushed the test/fix-flaky-integration-tests branch from 19b1199 to 4a44dda Compare May 18, 2026 20:43

djeebus approved these changes May 18, 2026

View reviewed changes

ValentaTomas merged commit 89c04b3 into main May 18, 2026
54 checks passed

ValentaTomas deleted the test/fix-flaky-integration-tests branch May 18, 2026 22:08

Navigation Menu

test(integration): fix flaky tests with root-cause changes - #2713

test(integration): fix flaky tests with root-cause changes#2713
ValentaTomas merged 1 commit into
mainfrom
test/fix-flaky-integration-tests

ValentaTomas commented May 18, 2026

Uh oh!

cursor Bot commented May 18, 2026 •

edited

Loading

Uh oh!

codecov Bot commented May 18, 2026 •

edited

Loading

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

Uh oh!

e2b-atlantis Bot commented May 18, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

ValentaTomas commented May 18, 2026

Summary

Note on TestUpdateNetworkConfig

Uh oh!

cursor Bot commented May 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Summary

Uh oh!

codecov Bot commented May 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

❌ 4 Tests Failed:

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

e2b-atlantis Bot commented May 18, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

cursor Bot commented May 18, 2026 •

edited

Loading

codecov Bot commented May 18, 2026 •

edited

Loading