Skip to content

feat: Always-on proxy-sidecar egress enforcement (enforce-redirect), remove flag#407

Merged
huang195 merged 2 commits into
mainfrom
feat/proxy-sidecar-enforce-redirect
Jun 6, 2026
Merged

feat: Always-on proxy-sidecar egress enforcement (enforce-redirect), remove flag#407
huang195 merged 2 commits into
mainfrom
feat/proxy-sidecar-enforce-redirect

Conversation

@huang195

@huang195 huang195 commented Jun 5, 2026

Copy link
Copy Markdown
Member

Summary

Makes proxy-sidecar egress enforcement always-on and removes the proxy.egressEnforcement flag.

PR #406 gated enforcement behind egressEnforcement (default off) because the only enforcement available was fail-closed DROP, which breaks agents that ignore HTTP_PROXY. kagenti-extensions#485 adds the missing half — a transparent listener + MODE=enforce-redirect that captures bypass egress (REDIRECT to the transparent listener, recovering the original destination via SO_ORIGINAL_DST) instead of dropping it. Because nothing breaks, enforcement can be on by default with no knob.

Changes

  • constants: add ProxyInitModeEnforceRedirect; keep ProxyInitModeEnforceDrop as a documented fallback the operator no longer emits.
  • container_builder: enforce-redirect branch emits MODE / PROXY_UID / CLUSTER_CIDRS / TRANSPARENT_PORT (new ProxyConfig.TransparentPort, default 8082, matching the authbridge listener.transparent_proxy_addr preset).
  • pod_mutator: the proxy-sidecar / lite branch now always injects the enforce-redirect proxy-init (no flag check); envoy-sidecar still uses redirect. Idempotent via the existing containerExists guard.
  • config: remove ProxyConfig.EgressEnforcement (+ its constants, validation, and the Helm value). ClusterCIDRs is now validated unconditionally (always consumed by the always-on guard); TransparentPort gets a port-range check.
  • Helm values: replace egressEnforcement with transparentPort; update comments. Rendered platform-defaults ConfigMap verified.

Tests

  • builder emits enforce-redirect env incl. TRANSPARENT_PORT;
  • proxy-sidecar always injects proxy-init (enforce-redirect), idempotent; default-off subtest removed;
  • ClusterCIDRs always validated; new TransparentPort validation test;
  • webhook envtest now expects proxy-init in proxy-sidecar defaults.

go build / go vet / gofmt / full go test ./internal/webhook/... green. The one golangci-lint finding (pod_mutator.go:187 goconst) is pre-existing and unrelated.

⚠️ Do not merge until images ship

Depends on kagenti-extensions#485 being released in the proxy-init and authbridge/authbridge-lite images (the init script's enforce-redirect mode + the proxy's transparent listener). The install chart owns image versions; this PR does not pin them.

Preconditions for the always-on guarantee (documented, not enforced here)

  • UID separation — agents must not run as Proxy.UID (1337) nor hold CAP_SETUID, else they bypass the --uid-owner exemption. Confirm via PSA restricted/SCC or an admission check.
  • Name-vs-IP policy — captured traffic gates on the recovered SNI/Host name (extensions#485), which an agent controls independently of the IP it dials; reliable against a cooperative/misconfigured agent, not a hostile one. Hard enforcement would need IP-set allowlists or SNI/cert cross-checks.

Assisted-By: Claude (Anthropic AI) noreply@anthropic.com

…ct), remove flag

PR #406 gated proxy-sidecar egress enforcement behind proxy.egressEnforcement
(default off) because the only enforcement available was fail-closed DROP, which
breaks agents that ignore HTTP_PROXY. kagenti-extensions#485 adds the missing
half — a transparent listener + MODE=enforce-redirect that CAPTURES bypass
egress (REDIRECT to the transparent listener) instead of dropping it — so
enforcement no longer breaks anything and can be always-on.

This wires the operator to that:
- constants: add ProxyInitModeEnforceRedirect; keep ProxyInitModeEnforceDrop as
  a documented no-transparent-listener fallback the operator no longer emits.
- container_builder: enforce-redirect branch emits MODE / PROXY_UID /
  CLUSTER_CIDRS / TRANSPARENT_PORT (the new ProxyConfig.TransparentPort, default
  8082, matching the authbridge listener.transparent_proxy_addr preset).
- pod_mutator: the proxy-sidecar / lite branch now ALWAYS injects the
  enforce-redirect proxy-init (no flag check); envoy-sidecar still uses redirect.
- config: REMOVE ProxyConfig.EgressEnforcement (+ its constants, validation, and
  the Helm value). ClusterCIDRs is now validated unconditionally (always consumed
  by the always-on guard), and TransparentPort gets a port-range check.
- Helm values: replace egressEnforcement with transparentPort; update comments.

Tests updated: builder emits enforce-redirect env incl. TRANSPARENT_PORT;
proxy-sidecar always injects proxy-init (enforce-redirect); ClusterCIDRs always
validated; TransparentPort validated; webhook envtest expects proxy-init.

Depends on kagenti-extensions#485 (transparent listener + enforce-redirect mode)
being released in the proxy-init + authbridge images. The install chart owns
image versions; this PR does not pin them. Do not merge until those images ship.

Preconditions for the always-on guarantee (documented, not enforced here):
- UID separation — agents must not run as Proxy.UID (1337) nor hold CAP_SETUID,
  or they could bypass the --uid-owner exemption. Confirm via PSA restricted/SCC
  or an admission check.
- Name-vs-IP policy — captured traffic gates on the recovered SNI/Host name
  (kagenti-extensions#485), which an agent controls independently of the IP, so
  it is reliable against a cooperative/misconfigured agent but not a hostile one.

Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com>
Signed-off-by: Hai Huang <huang195@gmail.com>
The operator now only ever emits enforce-redirect, and kagenti-extensions
removes the enforce-drop MODE from the init script (superseded by the
transparent listener capturing bypass egress instead of dropping it).
Remove the dead ProxyInitModeEnforceDrop constant, its BuildProxyInitContainer
branch, and its test; update stale comments. The builder now handles
redirect | enforce-redirect, with the fail-closed default for unknown modes.

Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com>
Signed-off-by: Hai Huang <huang195@gmail.com>

@mrsabath mrsabath left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clean refactor — removes the proxy.egressEnforcement gate and makes proxy-sidecar/lite egress enforcement always-on, renaming the mode enforce-dropenforce-redirect (captures bypass egress via REDIRECT to AuthBridge's transparent listener instead of dropping it).

I verified the cross-repo env-var contract against kagenti-extensions#485 and it matches exactly: MODE=enforce-redirect, TRANSPARENT_PORT (default 8082), PROXY_UID, and CLUSTER_CIDRS are all produced here and consumed by the init script there, and the authbridge transparent_proxy_addr default :8082 lines up with the operator's TransparentPort default. The flag removal, de-nested unconditional ClusterCIDRs validation, new TransparentPort range check, and the test rewrites all look correct.

Areas reviewed: Go (config/injector/tests), Helm values, cross-repo contract, commit conventions
Commits: 2, both signed-off (DCO ✅)
CI: all green (Build, Unit, Integration, E2E, Lint, CodeQL, Trivy, Helm lint, …)
Dependency gate: the "do not merge until extensions#485 ships in proxy-init + authbridge images" warning is accurate — extensions#485 is still OPEN, so keep the merge gate.

One non-blocking suggestion inline. LGTM otherwise. 无懈可击 — flawless, not a chink to attack.

// MUST override this (e.g. OCP services 172.30.0.0/16, pods 10.128.0.0/14 —
// 172.30/16 is outside 10/8) or in-cluster service traffic will be dropped.
// Egress enforcement is always-on for proxy-sidecar / lite, so this is
// always consumed there; envoy-sidecar (transparent redirect) does not use it.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (non-blocking): EgressEnforcement is removed from the values schema, but the config loader uses plain yaml.Unmarshal (no KnownFields(true)) — so a stale proxy.egressEnforcement: left in an operator's existing values override is silently ignored rather than erroring. That's the benign failure mode (enforcement just becomes always-on anyway), so no upgrade-time crash. Still worth a one-liner in the release/upgrade notes so operators aren't surprised the knob no longer does anything.

@huang195
huang195 merged commit 7bba888 into main Jun 6, 2026
15 checks passed
@github-project-automation github-project-automation Bot moved this from New /:ToDo to Done in Rossoctl Issue Prioritization Jun 6, 2026
@huang195
huang195 deleted the feat/proxy-sidecar-enforce-redirect branch June 6, 2026 03:02
akram added a commit to akram/kagenti-operator that referenced this pull request Jun 16, 2026
On managed OpenShift platforms (ROSA HCP), iptables is unavailable in
init containers due to missing kernel modules (iptable_nat) and SELinux
restrictions. PR rossoctl#407 made enforce-redirect always-on in proxy-sidecar
mode, which breaks pod creation on these platforms.

Add an egressEnforcement field (enforce-redirect | none) to the
AgentRuntime CR and namespace authbridge-runtime-config ConfigMap,
following the same resolution chain as authBridgeMode and mtlsMode:

  1. AgentRuntime CR .spec.egressEnforcement (per-workload)
  2. Namespace authbridge-runtime-config egressEnforcement (namespace)
  3. "enforce-redirect" (cluster default, preserves current behavior)

When set to "none", proxy-init is not injected in proxy-sidecar/lite
modes. Egress relies on HTTP_PROXY (cooperative) + inbound AuthBridge
on destinations + NetworkPolicy. Unknown values fail closed to
enforce-redirect.

envoy-sidecar mode is unaffected — its proxy-init (redirect mode) is
structural and always injected regardless of this setting.

Ref: RHAIENG-5702
Ref: rossoctl/cortex#502

Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com>
Signed-off-by: Akram <akram.benaissi@gmail.com>
akram added a commit to akram/kagenti-operator that referenced this pull request Jun 16, 2026
On managed OpenShift platforms (ROSA HCP), iptables is unavailable in
init containers due to missing kernel modules (iptable_nat) and SELinux
restrictions. PR rossoctl#407 made enforce-redirect always-on in proxy-sidecar
mode, which breaks pod creation on these platforms.

Add an egressEnforcement field (enforce-redirect | none) to the
AgentRuntime CR and namespace authbridge-runtime-config ConfigMap,
following the same resolution chain as authBridgeMode and mtlsMode:

  1. AgentRuntime CR .spec.egressEnforcement (per-workload)
  2. Namespace authbridge-runtime-config egressEnforcement (namespace)
  3. "enforce-redirect" (cluster default, preserves current behavior)

When set to "none", proxy-init is not injected in proxy-sidecar/lite
modes. Egress relies on HTTP_PROXY (cooperative) + inbound AuthBridge
on destinations + NetworkPolicy. Unknown values fail closed to
enforce-redirect.

envoy-sidecar mode is unaffected — its proxy-init (redirect mode) is
structural and always injected regardless of this setting.

Ref: RHAIENG-5702
Ref: rossoctl/cortex#502

Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com>
Signed-off-by: Akram <akram.benaissi@gmail.com>
akram added a commit to akram/kagenti-operator that referenced this pull request Jun 16, 2026
On managed OpenShift platforms (ROSA HCP), iptables is unavailable in
init containers due to missing kernel modules (iptable_nat) and SELinux
restrictions. PR rossoctl#407 made enforce-redirect always-on in proxy-sidecar
mode, which breaks pod creation on these platforms.

Add an egressEnforcement field (enforce-redirect | none) to the
AgentRuntime CR and namespace authbridge-runtime-config ConfigMap,
following the same resolution chain as authBridgeMode and mtlsMode:

  1. AgentRuntime CR .spec.egressEnforcement (per-workload)
  2. Namespace authbridge-runtime-config egressEnforcement (namespace)
  3. "enforce-redirect" (cluster default, preserves current behavior)

When set to "none", proxy-init is not injected in proxy-sidecar/lite
modes. Egress relies on HTTP_PROXY (cooperative) + inbound AuthBridge
on destinations + NetworkPolicy. Unknown values fail closed to
enforce-redirect.

envoy-sidecar mode is unaffected — its proxy-init (redirect mode) is
structural and always injected regardless of this setting.

Ref: RHAIENG-5702
Ref: rossoctl/cortex#502

Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com>
Signed-off-by: Akram <akram.benaissi@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants