Skip to content

feat(rootfs): isolate envd in a dedicated network namespace #2700

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
ValentaTomas wants to merge 2 commits into from
Closed

feat(rootfs): isolate envd in a dedicated network namespace #2700

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
9937e85
feat(rootfs): isolate envd in a dedicated network namespace
ValentaTomas May 17, 2026
91fea54
refactor(netns): declarative veth via systemd-networkd, ExecStart cha…
ValentaTomas May 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions packages/orchestrator/pkg/template/build/core/rootfs/files/e2b-netns.service.tpl
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
{{- /*gotype:github.com/e2b-dev/infra/packages/orchestrator/pkg/template/build/core/rootfs.templateModel*/ -}}
{{ .WriteFile "/etc/systemd/system/e2b-netns.service" 0o644 }}

[Unit]
Description=E2B envd network namespace setup
DefaultDependencies=no
After=systemd-networkd.service
Requires=systemd-networkd.service
Before=network.target sysinit.target

[Service]
Type=oneshot
RemainAfterExit=yes

# Create the namespace and move eth0 + the envd-side veth peer into it.
# `-` prefix ignores failure so re-runs across reboots are idempotent.
ExecStart=-/sbin/ip netns add envd-ns
ExecStart=-/sbin/ip link set eth0 netns envd-ns
ExecStart=-/sbin/ip link set veth-envd netns envd-ns

# Configure interfaces inside envd-ns (systemd-networkd doesn't run there).
ExecStart=-/sbin/ip -n envd-ns link set lo up
ExecStart=-/sbin/ip -n envd-ns addr add 169.254.0.21/30 dev eth0
ExecStart=-/sbin/ip -n envd-ns link set eth0 up
ExecStart=-/sbin/ip -n envd-ns route add default via 169.254.0.22
ExecStart=-/sbin/ip -n envd-ns addr add 192.168.250.2/30 dev veth-envd
ExecStart=-/sbin/ip -n envd-ns link set veth-envd up
ExecStart=/sbin/ip netns exec envd-ns sysctl -qw net.ipv4.ip_forward=1

# Idempotent iptables: -C check before -A. Wrapped in sh because there is no
# declarative systemd directive for "add rule if not present".
ExecStart=/bin/sh -c 'ip netns exec envd-ns iptables -t nat -C POSTROUTING -o eth0 -j MASQUERADE 2>/dev/null || ip netns exec envd-ns iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE'
ExecStart=/bin/sh -c 'ip netns exec envd-ns iptables -t nat -C PREROUTING -i eth0 -p tcp --dport 49983 -j RETURN 2>/dev/null || ip netns exec envd-ns iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 49983 -j RETURN'
ExecStart=/bin/sh -c 'ip netns exec envd-ns iptables -t nat -C PREROUTING -i eth0 -p tcp -j DNAT --to-destination 192.168.250.1 2>/dev/null || ip netns exec envd-ns iptables -t nat -A PREROUTING -i eth0 -p tcp -j DNAT --to-destination 192.168.250.1'

[Install]
WantedBy=sysinit.target
9 changes: 9 additions & 0 deletions packages/orchestrator/pkg/template/build/core/rootfs/files/e2b-veth-customer.network.tpl
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
{{- /*gotype:github.com/e2b-dev/infra/packages/orchestrator/pkg/template/build/core/rootfs.templateModel*/ -}}
{{ .WriteFile "/etc/systemd/network/10-e2b-veth-customer.network" 0o644 }}

[Match]
Name=veth-customer

[Network]
Address=192.168.250.1/30
Gateway=192.168.250.2
9 changes: 9 additions & 0 deletions packages/orchestrator/pkg/template/build/core/rootfs/files/e2b-veth.netdev.tpl
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
{{- /*gotype:github.com/e2b-dev/infra/packages/orchestrator/pkg/template/build/core/rootfs.templateModel*/ -}}
{{ .WriteFile "/etc/systemd/network/10-e2b-veth.netdev" 0o644 }}

[NetDev]
Name=veth-customer
Kind=veth

[Peer]
Name=veth-envd
6 changes: 5 additions & 1 deletion packages/orchestrator/pkg/template/build/core/rootfs/files/envd.service.tpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@

[Unit]
Description=Env Daemon Service
After=multi-user.target
After=multi-user.target e2b-netns.service
Requires=e2b-netns.service
# Disable rate limiting; retry forever
StartLimitIntervalSec=0

Expand All @@ -12,6 +13,9 @@ Type=simple
Restart=always
User=root
Group=root
# Run envd inside the dedicated network namespace set up by e2b-netns.service.
# Customer iptables in the default namespace cannot reach this namespace.
NetworkNamespacePath=/run/netns/envd-ns

@cursor cursor Bot May 17, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Child processes spawned by envd inherit wrong network namespace

High Severity

NetworkNamespacePath=/run/netns/envd-ns causes all child processes forked by envd (socat port forwarders, user terminal sessions, command execution) to inherit envd-ns. The existing socat port forwarder binds to 169.254.0.21:port in envd-ns and tries to connect to localhost:port in envd-ns — but customer processes listen in the default namespace's loopback, which is unreachable from envd-ns. Additionally, any user-facing process spawning (terminals, exec) would place the user process in the wrong network namespace where veth-customer (192.168.250.1) isn't visible.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 91fea54. Configure here.

Environment=GOTRACEBACK=all
LimitCORE=infinity
ExecStartPre=/bin/sh -c 'mountpoint -q /etc/ssl/certs || (mkdir -p /run/e2b/certs && mount --bind /run/e2b/certs /etc/ssl/certs) && ([ -s /etc/ssl/certs/ca-certificates.crt ] || update-ca-certificates)'
Expand Down
2 changes: 2 additions & 0 deletions packages/orchestrator/pkg/template/build/core/rootfs/rootfs.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -249,6 +249,8 @@ func additionalOCILayers(
"etc/systemd/system/multi-user.target.wants/envd.service": "etc/systemd/system/envd.service",
// Enable chrony service autostart
"etc/systemd/system/multi-user.target.wants/chrony.service": "etc/systemd/system/chrony.service",
// Enable envd netns setup at boot (must run before envd.service)
"etc/systemd/system/sysinit.target.wants/e2b-netns.service": "etc/systemd/system/e2b-netns.service",
},
)
if err != nil {
Expand Down
2 changes: 1 addition & 1 deletion packages/orchestrator/pkg/template/build/core/rootfs/rootfs_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ func TestAdditionalOCILayers(t *testing.T) {

keysIter := maps.Keys(actualFiles)
keys := slices.Collect(keysIter)
assert.Len(t, keys, 14)
assert.Len(t, keys, 17)
assert.Equal(t, "e2b.local", actualFiles["etc/hostname"])
assert.Equal(t, "nameserver 8.8.8.8", actualFiles["etc/resolv.conf"])

Expand Down
Loading