Skip to content

feat(envd): iptables DNAT backend for IPv4 port forwarding - #2697

Closed
ValentaTomas wants to merge 7 commits into
mainfrom
feat/envd-port-forwarder-iptables
Closed

ValentaTomas wants to merge 7 commits into
mainfrom
feat/envd-port-forwarder-iptables

Conversation

@ValentaTomas

Copy link
Copy Markdown
Member

Opt-in alternative to per-port socat for IPv4 listeners (IPv6 keeps socat). Toggled by ENVD_PORT_FORWARDER_IPV4_IPTABLES=1.

Stacks on #2693.

…hysteresis

gopsutil's net.Connections walks /proc/<pid>/fd for every process every
second to attach PIDs to TCP sockets. The port forwarder never used the
PID for anything; it was a key component of the (pid, port) lookup map
but only because that's what the gopsutil call happened to provide. On a
sandbox running ~100 processes × ~20 FDs that's thousands of readlink
syscalls per scan.

- Replace the gopsutil call with a tiny /proc/net/tcp{,6} parser that
  extracts (family, ip, port, status) for LISTEN sockets only. No /proc
  walk; one read per protocol per scan.
- Drop the PID component of the forwarder's map key (family-ip-port is
  enough — if a different process binds the same address the forwarded
  path is still correct).
- Add a 3-scan hysteresis: a listener that disappears for a single scan
  no longer drops its socat. Absorbs short flickers from things like
  next.js HMR rebuilds and docker compose restarts.

macOS handler tests build via a `proc_listeners_other.go` no-op stub.
@cla-bot cla-bot Bot added the cla-signed label May 17, 2026
@cursor

cursor Bot commented May 17, 2026 •

Copy link
Copy Markdown

PR Summary

Medium Risk
Touches low-level networking by adding iptables rule management and /proc parsing; failures or privilege issues can break IPv4 port forwarding or leave stale NAT rules behind.

Overview
Adds an opt-in IPv4 port-forwarding path that installs iptables DNAT rules and enables route_localnet, while keeping socat for IPv6 and as a fallback. The forwarder now de-dupes dual-stack listeners by keying on port and delays teardown with a missed-scan threshold, but Scanner.Processes appears unused and the non-Linux iptables stub can report success without actually forwarding if enabled.

Reviewed by Cursor Bugbot for commit 6b4f194. Bugbot is set up for automated code reviews on this repo. Configure here.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

Ignoring errors from listListeningSockets can cause the scanner to broadcast an empty list, which may lead to the unintended termination of all active port forwardings during transient filesystem errors. The forwarder fails to clean up persistent iptables rules when the subscriber channel closes, resulting in kernel resource leaks. Furthermore, the iptables backend appends rules without verifying if they already exist, causing redundant rules to accumulate across service restarts.

Comment thread packages/envd/internal/port/scan.go Outdated
Comment thread packages/envd/internal/port/forward.go
Comment thread packages/envd/internal/port/iptables_linux.go
@codecov

codecov Bot commented May 17, 2026 •

Copy link
Copy Markdown

❌ 9 Tests Failed:

Tests completed Failed Passed Skipped
2627 9 2618 5
View the full list of 9 ❄️ flaky test(s)
github.com/e2b-dev/infra/tests/integration/internal/tests/api/metrics::TestTeamMetrics

Flake rate in main: 71.70% (Passed 221 times, Failed 560 times)

Stack Traces | 0.56s run time
=== RUN   TestTeamMetrics
=== PAUSE TestTeamMetrics
=== CONT  TestTeamMetrics
    team_metrics_test.go:61: 
        	Error Trace:	.../api/metrics/team_metrics_test.go:61
        	Error:      	Should be true
        	Test:       	TestTeamMetrics
        	Messages:   	MaxConcurrentSandboxes should be >= 0
--- FAIL: TestTeamMetrics (0.56s)
github.com/e2b-dev/infra/tests/integration/internal/tests/api/sandboxes::TestUpdateNetworkConfig

Flake rate in main: 76.85% (Passed 231 times, Failed 767 times)

Stack Traces | 34.3s run time
=== RUN   TestUpdateNetworkConfig
=== PAUSE TestUpdateNetworkConfig
=== CONT  TestUpdateNetworkConfig
--- FAIL: TestUpdateNetworkConfig (34.30s)
github.com/e2b-dev/infra/tests/integration/internal/tests/api/sandboxes::TestUpdateNetworkConfig/pause_resume_preserves_allow_internet_access_false

Flake rate in main: 77.39% (Passed 222 times, Failed 760 times)

Stack Traces | 2.22s run time
=== RUN   TestUpdateNetworkConfig/pause_resume_preserves_allow_internet_access_false
Executing command curl in sandbox i777wlz3nxoe8tjsryzgb
    sandbox_network_update_test.go:372: Command [curl] output: event:{start:{pid:1359}}
    sandbox_network_update_test.go:372: Command [curl] output: event:{end:{exit_code:35 exited:true status:"exit status 35" error:"exit status 35"}}
Executing command curl in sandbox i777wlz3nxoe8tjsryzgb
    sandbox_network_update_test.go:372: Command [curl] output: event:{start:{pid:1360}}
    sandbox_network_update_test.go:372: Command [curl] output: event:{end:{exit_code:35 exited:true status:"exit status 35" error:"exit status 35"}}
Executing command curl in sandbox izeb8ndeu5m1jaz23mo86
    sandbox_network_update_test.go:391: Command [curl] output: event:{start:{pid:1361}}
    sandbox_network_update_test.go:391: Command [curl] output: event:{data:{stdout:"HTTP/2 302 \r\nx-content-type-options: nosniff\r\nlocation: https://dns.google/\r\ndate: Sun, 17 May 2026 11:26:20 GMT\r\ncontent-type: text/html; charset=UTF-8\r\nserver: HTTP server (unknown)\r\ncontent-length: 216\r\nx-xss-protection: 0\r\nx-frame-options: SAMEORIGIN\r\nalt-svc: h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000\r\n\r\n"}}
    sandbox_network_update_test.go:391: Command [curl] output: event:{end:{exited:true status:"exit status 0"}}
    sandbox_network_update_test.go:391: Command [curl] completed successfully in sandbox i777wlz3nxoe8tjsryzgb
    sandbox_network_update_test.go:391: 
        	Error Trace:	.../api/sandboxes/sandbox_network_out_test.go:74
        	            				.../api/sandboxes/sandbox_network_update_test.go:60
        	            				.../api/sandboxes/sandbox_network_update_test.go:391
        	Error:      	An error is expected but got nil.
        	Test:       	TestUpdateNetworkConfig/pause_resume_preserves_allow_internet_access_false
        	Messages:   	https://8.8.8.8 should be blocked
--- FAIL: TestUpdateNetworkConfig/pause_resume_preserves_allow_internet_access_false (2.22s)
github.com/e2b-dev/infra/tests/integration/internal/tests/envd::TestBindLocalhost

Flake rate in main: 56.36% (Passed 391 times, Failed 505 times)

Stack Traces | 0s run time
=== RUN   TestBindLocalhost
=== PAUSE TestBindLocalhost
=== CONT  TestBindLocalhost
--- FAIL: TestBindLocalhost (0.00s)
github.com/e2b-dev/infra/tests/integration/internal/tests/envd::TestBindLocalhost/bind_0_0_0_0

Flake rate in main: 63.07% (Passed 219 times, Failed 374 times)

Stack Traces | 16.5s run time
=== RUN   TestBindLocalhost/bind_0_0_0_0
=== PAUSE TestBindLocalhost/bind_0_0_0_0
=== CONT  TestBindLocalhost/bind_0_0_0_0
    localhost_bind_test.go:69: Command [python] output: event:{start:{pid:1262}}
Executing command /bin/bash in sandbox issp02gu36tqjkprvi79v
    localhost_bind_test.go:90: 
        	Error Trace:	.../tests/envd/localhost_bind_test.go:90
        	Error:      	Not equal: 
        	            	expected: 200
        	            	actual  : 502
        	Test:       	TestBindLocalhost/bind_0_0_0_0
        	Messages:   	Unexpected status code 502 for bind address 0.0.0.0
--- FAIL: TestBindLocalhost/bind_0_0_0_0 (16.46s)
github.com/e2b-dev/infra/tests/integration/internal/tests/envd::TestBindLocalhost/bind_::1

Flake rate in main: 64.45% (Passed 219 times, Failed 397 times)

Stack Traces | 12.2s run time
=== RUN   TestBindLocalhost/bind_::1
=== PAUSE TestBindLocalhost/bind_::1
=== CONT  TestBindLocalhost/bind_::1
Executing command python in sandbox i5fh9disettdii4qgzjj6
    localhost_bind_test.go:69: Command [python] output: event:{start:{pid:1262}}
    localhost_bind_test.go:90: 
        	Error Trace:	.../tests/envd/localhost_bind_test.go:90
        	Error:      	Not equal: 
        	            	expected: 200
        	            	actual  : 502
        	Test:       	TestBindLocalhost/bind_::1
        	Messages:   	Unexpected status code 502 for bind address ::1
--- FAIL: TestBindLocalhost/bind_::1 (12.19s)
github.com/e2b-dev/infra/tests/integration/internal/tests/envd::TestBindLocalhost/bind_localhost

Flake rate in main: 64.33% (Passed 219 times, Failed 395 times)

Stack Traces | 7.24s run time
=== RUN   TestBindLocalhost/bind_localhost
=== PAUSE TestBindLocalhost/bind_localhost
=== CONT  TestBindLocalhost/bind_localhost
Executing command python in sandbox ikxz5c3cryiai3z98th2i
    localhost_bind_test.go:69: Command [python] output: event:{start:{pid:1261}}
    localhost_bind_test.go:90: 
        	Error Trace:	.../tests/envd/localhost_bind_test.go:90
        	Error:      	Not equal: 
        	            	expected: 200
        	            	actual  : 502
        	Test:       	TestBindLocalhost/bind_localhost
        	Messages:   	Unexpected status code 502 for bind address localhost
--- FAIL: TestBindLocalhost/bind_localhost (7.24s)
github.com/e2b-dev/infra/tests/integration/internal/tests/orchestrator::TestSandboxMemoryIntegrity

Flake rate in main: 66.17% (Passed 229 times, Failed 448 times)

Stack Traces | 81.2s run time
=== RUN   TestSandboxMemoryIntegrity
=== PAUSE TestSandboxMemoryIntegrity
=== CONT  TestSandboxMemoryIntegrity
    sandbox_memory_integrity_test.go:26: Build completed successfully
--- FAIL: TestSandboxMemoryIntegrity (81.15s)
github.com/e2b-dev/infra/tests/integration/internal/tests/orchestrator::TestSandboxMemoryIntegrity/tmpfs_hash

Flake rate in main: 66.87% (Passed 219 times, Failed 442 times)

Stack Traces | 50.5s run time
=== RUN   TestSandboxMemoryIntegrity/tmpfs_hash
=== PAUSE TestSandboxMemoryIntegrity/tmpfs_hash
=== CONT  TestSandboxMemoryIntegrity/tmpfs_hash
    sandbox_memory_integrity_test.go:70: Command [bash] output: event:{start:{pid:1251}}
Executing command bash in sandbox icwty917wzdmc2r8pmbrh (user: root)
    sandbox_memory_integrity_test.go:70: Command [bash] output: event:{data:{stdout:"Total memory: 985 MB\nUsed memory before tmpfs mount: 187 MB\nFree memory before tmpfs mount: 797 MB\nMemory to use in integrity test (80% of free, min 64MB): 637 MB\n"}}
    sandbox_memory_integrity_test.go:70: Command [bash] output: event:{data:{stderr:"637+0 records in\n637+0 records out\n667942912 bytes (668 MB, 637 MiB) copied, 24.2088 s, 27.6 MB/s\n"}}
    sandbox_memory_integrity_test.go:70: Command [bash] output: event:{data:{stderr:"\tCommand being timed: \"dd if=/dev/urandom of=/mnt/testfile bs=1M count=637\"\n\tUser time (seconds): 0.00\n\tSystem time (seconds): 23.77\n\tPercent of CPU this job got: 97%\n\tElapsed (wall clock) time (h:mm:ss or m:ss): 0:24.42\n\tAverage shared text size (kbytes): 0\n\tAverage unshared data size (kbytes): 0\n\tAverage stack size (kbytes): 0\n\tAverage total size (kbytes): 0\n\tMaximum resident set size (kbytes): 2616\n\tAverage resident set size (kbytes): 0\n\tMajor (requiring I/O) page faults: 3\n\tMinor (reclaiming a frame) page faults: 341\n\tVoluntary context switches: 4\n\tInvoluntary context switches: 110\n\tSwaps: 0\n\tFile system inputs: 176\n\tFile system outputs: 0\n\tSocket messages sent: 0\n\tSocket messages received: 0\n\tSignals delivered: 0\n\tPage size (bytes): 4096\n\tExit status: 0\n"}}
    sandbox_memory_integrity_test.go:70: Command [bash] output: event:{data:{stdout:"Used memory after tmpfs mount and file fill: 827 MB\n"}}
    sandbox_memory_integrity_test.go:70: Command [bash] output: event:{end:{exited:true status:"exit status 0"}}
    sandbox_memory_integrity_test.go:70: Command [bash] completed successfully in sandbox il967021fuwl0qs44oprt
Executing command bash in sandbox il967021fuwl0qs44oprt (user: root)
    sandbox_memory_integrity_test.go:74: Command [bash] output: event:{start:{pid:1268}}
    sandbox_memory_integrity_test.go:74: Command [bash] output: event:{data:{stdout:"7edf1a1c0fc255a5a90c911c864e5b560a1ac91c433404274697c633f71fcf5e\n"}}
    sandbox_memory_integrity_test.go:74: Command [bash] output: event:{end:{exited:true status:"exit status 0"}}
    sandbox_memory_integrity_test.go:74: Command [bash] completed successfully in sandbox il967021fuwl0qs44oprt
Executing command bash in sandbox il967021fuwl0qs44oprt (user: root)
    sandbox_memory_integrity_test.go:99: Command [bash] output: event:{start:{pid:1271}}
    sandbox_memory_integrity_test.go:100: 
        	Error Trace:	.../tests/orchestrator/sandbox_memory_integrity_test.go:100
        	Error:      	Received unexpected error:
        	            	failed to execute command bash in sandbox il967021fuwl0qs44oprt: invalid_argument: protocol error: incomplete envelope: unexpected EOF
        	Test:       	TestSandboxMemoryIntegrity/tmpfs_hash
--- FAIL: TestSandboxMemoryIntegrity/tmpfs_hash (50.50s)

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

Opt-in alternative to per-port socat for IPv4 listeners. IPv6 keeps
socat. Toggled by ENVD_PORT_FORWARDER_IPV4_IPTABLES=1.

- iptablesBackend adds/removes PREROUTING DNAT rules from sourceIP:port
  to 127.0.0.1:port.
- route_localnet=1 enabled at forwarder start.
- Falls back to socat if iptables fails.
@ValentaTomas
ValentaTomas force-pushed the feat/envd-port-forwarder-iptables branch from fec454e to cbb07f7 Compare May 17, 2026 10:01
@ValentaTomas

Copy link
Copy Markdown
Member Author

Superseded by a provision-time blanket DNAT rule (see follow-up PR). One rule covers all ports without envd-side iptables management.

@ValentaTomas
ValentaTomas deleted the feat/envd-port-forwarder-iptables branch May 18, 2026 01:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants