feat: Always-on proxy-sidecar egress enforcement (enforce-redirect), remove flag#407
Conversation
…ct), remove flag PR #406 gated proxy-sidecar egress enforcement behind proxy.egressEnforcement (default off) because the only enforcement available was fail-closed DROP, which breaks agents that ignore HTTP_PROXY. kagenti-extensions#485 adds the missing half — a transparent listener + MODE=enforce-redirect that CAPTURES bypass egress (REDIRECT to the transparent listener) instead of dropping it — so enforcement no longer breaks anything and can be always-on. This wires the operator to that: - constants: add ProxyInitModeEnforceRedirect; keep ProxyInitModeEnforceDrop as a documented no-transparent-listener fallback the operator no longer emits. - container_builder: enforce-redirect branch emits MODE / PROXY_UID / CLUSTER_CIDRS / TRANSPARENT_PORT (the new ProxyConfig.TransparentPort, default 8082, matching the authbridge listener.transparent_proxy_addr preset). - pod_mutator: the proxy-sidecar / lite branch now ALWAYS injects the enforce-redirect proxy-init (no flag check); envoy-sidecar still uses redirect. - config: REMOVE ProxyConfig.EgressEnforcement (+ its constants, validation, and the Helm value). ClusterCIDRs is now validated unconditionally (always consumed by the always-on guard), and TransparentPort gets a port-range check. - Helm values: replace egressEnforcement with transparentPort; update comments. Tests updated: builder emits enforce-redirect env incl. TRANSPARENT_PORT; proxy-sidecar always injects proxy-init (enforce-redirect); ClusterCIDRs always validated; TransparentPort validated; webhook envtest expects proxy-init. Depends on kagenti-extensions#485 (transparent listener + enforce-redirect mode) being released in the proxy-init + authbridge images. The install chart owns image versions; this PR does not pin them. Do not merge until those images ship. Preconditions for the always-on guarantee (documented, not enforced here): - UID separation — agents must not run as Proxy.UID (1337) nor hold CAP_SETUID, or they could bypass the --uid-owner exemption. Confirm via PSA restricted/SCC or an admission check. - Name-vs-IP policy — captured traffic gates on the recovered SNI/Host name (kagenti-extensions#485), which an agent controls independently of the IP, so it is reliable against a cooperative/misconfigured agent but not a hostile one. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Hai Huang <huang195@gmail.com>
The operator now only ever emits enforce-redirect, and kagenti-extensions removes the enforce-drop MODE from the init script (superseded by the transparent listener capturing bypass egress instead of dropping it). Remove the dead ProxyInitModeEnforceDrop constant, its BuildProxyInitContainer branch, and its test; update stale comments. The builder now handles redirect | enforce-redirect, with the fail-closed default for unknown modes. Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Hai Huang <huang195@gmail.com>
mrsabath
left a comment
There was a problem hiding this comment.
Clean refactor — removes the proxy.egressEnforcement gate and makes proxy-sidecar/lite egress enforcement always-on, renaming the mode enforce-drop → enforce-redirect (captures bypass egress via REDIRECT to AuthBridge's transparent listener instead of dropping it).
I verified the cross-repo env-var contract against kagenti-extensions#485 and it matches exactly: MODE=enforce-redirect, TRANSPARENT_PORT (default 8082), PROXY_UID, and CLUSTER_CIDRS are all produced here and consumed by the init script there, and the authbridge transparent_proxy_addr default :8082 lines up with the operator's TransparentPort default. The flag removal, de-nested unconditional ClusterCIDRs validation, new TransparentPort range check, and the test rewrites all look correct.
Areas reviewed: Go (config/injector/tests), Helm values, cross-repo contract, commit conventions
Commits: 2, both signed-off (DCO ✅)
CI: all green (Build, Unit, Integration, E2E, Lint, CodeQL, Trivy, Helm lint, …)
Dependency gate: the "do not merge until extensions#485 ships in proxy-init + authbridge images" warning is accurate — extensions#485 is still OPEN, so keep the merge gate.
One non-blocking suggestion inline. LGTM otherwise. 无懈可击 — flawless, not a chink to attack.
| // MUST override this (e.g. OCP services 172.30.0.0/16, pods 10.128.0.0/14 — | ||
| // 172.30/16 is outside 10/8) or in-cluster service traffic will be dropped. | ||
| // Egress enforcement is always-on for proxy-sidecar / lite, so this is | ||
| // always consumed there; envoy-sidecar (transparent redirect) does not use it. |
There was a problem hiding this comment.
suggestion (non-blocking): EgressEnforcement is removed from the values schema, but the config loader uses plain yaml.Unmarshal (no KnownFields(true)) — so a stale proxy.egressEnforcement: left in an operator's existing values override is silently ignored rather than erroring. That's the benign failure mode (enforcement just becomes always-on anyway), so no upgrade-time crash. Still worth a one-liner in the release/upgrade notes so operators aren't surprised the knob no longer does anything.
On managed OpenShift platforms (ROSA HCP), iptables is unavailable in init containers due to missing kernel modules (iptable_nat) and SELinux restrictions. PR rossoctl#407 made enforce-redirect always-on in proxy-sidecar mode, which breaks pod creation on these platforms. Add an egressEnforcement field (enforce-redirect | none) to the AgentRuntime CR and namespace authbridge-runtime-config ConfigMap, following the same resolution chain as authBridgeMode and mtlsMode: 1. AgentRuntime CR .spec.egressEnforcement (per-workload) 2. Namespace authbridge-runtime-config egressEnforcement (namespace) 3. "enforce-redirect" (cluster default, preserves current behavior) When set to "none", proxy-init is not injected in proxy-sidecar/lite modes. Egress relies on HTTP_PROXY (cooperative) + inbound AuthBridge on destinations + NetworkPolicy. Unknown values fail closed to enforce-redirect. envoy-sidecar mode is unaffected — its proxy-init (redirect mode) is structural and always injected regardless of this setting. Ref: RHAIENG-5702 Ref: rossoctl/cortex#502 Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Akram <akram.benaissi@gmail.com>
On managed OpenShift platforms (ROSA HCP), iptables is unavailable in init containers due to missing kernel modules (iptable_nat) and SELinux restrictions. PR rossoctl#407 made enforce-redirect always-on in proxy-sidecar mode, which breaks pod creation on these platforms. Add an egressEnforcement field (enforce-redirect | none) to the AgentRuntime CR and namespace authbridge-runtime-config ConfigMap, following the same resolution chain as authBridgeMode and mtlsMode: 1. AgentRuntime CR .spec.egressEnforcement (per-workload) 2. Namespace authbridge-runtime-config egressEnforcement (namespace) 3. "enforce-redirect" (cluster default, preserves current behavior) When set to "none", proxy-init is not injected in proxy-sidecar/lite modes. Egress relies on HTTP_PROXY (cooperative) + inbound AuthBridge on destinations + NetworkPolicy. Unknown values fail closed to enforce-redirect. envoy-sidecar mode is unaffected — its proxy-init (redirect mode) is structural and always injected regardless of this setting. Ref: RHAIENG-5702 Ref: rossoctl/cortex#502 Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Akram <akram.benaissi@gmail.com>
On managed OpenShift platforms (ROSA HCP), iptables is unavailable in init containers due to missing kernel modules (iptable_nat) and SELinux restrictions. PR rossoctl#407 made enforce-redirect always-on in proxy-sidecar mode, which breaks pod creation on these platforms. Add an egressEnforcement field (enforce-redirect | none) to the AgentRuntime CR and namespace authbridge-runtime-config ConfigMap, following the same resolution chain as authBridgeMode and mtlsMode: 1. AgentRuntime CR .spec.egressEnforcement (per-workload) 2. Namespace authbridge-runtime-config egressEnforcement (namespace) 3. "enforce-redirect" (cluster default, preserves current behavior) When set to "none", proxy-init is not injected in proxy-sidecar/lite modes. Egress relies on HTTP_PROXY (cooperative) + inbound AuthBridge on destinations + NetworkPolicy. Unknown values fail closed to enforce-redirect. envoy-sidecar mode is unaffected — its proxy-init (redirect mode) is structural and always injected regardless of this setting. Ref: RHAIENG-5702 Ref: rossoctl/cortex#502 Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com> Signed-off-by: Akram <akram.benaissi@gmail.com>
Summary
Makes proxy-sidecar egress enforcement always-on and removes the
proxy.egressEnforcementflag.PR #406 gated enforcement behind
egressEnforcement(defaultoff) because the only enforcement available was fail-closed DROP, which breaks agents that ignoreHTTP_PROXY. kagenti-extensions#485 adds the missing half — a transparent listener +MODE=enforce-redirectthat captures bypass egress (REDIRECT to the transparent listener, recovering the original destination viaSO_ORIGINAL_DST) instead of dropping it. Because nothing breaks, enforcement can be on by default with no knob.Changes
ProxyInitModeEnforceRedirect; keepProxyInitModeEnforceDropas a documented fallback the operator no longer emits.MODE/PROXY_UID/CLUSTER_CIDRS/TRANSPARENT_PORT(newProxyConfig.TransparentPort, default8082, matching the authbridgelistener.transparent_proxy_addrpreset).redirect. Idempotent via the existingcontainerExistsguard.ProxyConfig.EgressEnforcement(+ its constants, validation, and the Helm value).ClusterCIDRsis now validated unconditionally (always consumed by the always-on guard);TransparentPortgets a port-range check.egressEnforcementwithtransparentPort; update comments. Rendered platform-defaults ConfigMap verified.Tests
TRANSPARENT_PORT;ClusterCIDRsalways validated; newTransparentPortvalidation test;go build/go vet/gofmt/ fullgo test ./internal/webhook/...green. The onegolangci-lintfinding (pod_mutator.go:187goconst) is pre-existing and unrelated.Depends on kagenti-extensions#485 being released in the
proxy-initandauthbridge/authbridge-liteimages (the init script'senforce-redirectmode + the proxy's transparent listener). The install chart owns image versions; this PR does not pin them.Preconditions for the always-on guarantee (documented, not enforced here)
Proxy.UID(1337) nor holdCAP_SETUID, else they bypass the--uid-ownerexemption. Confirm via PSArestricted/SCC or an admission check.Assisted-By: Claude (Anthropic AI) noreply@anthropic.com