Deploy ziti-management service and enable Ziti in the gateway

[rowan-stein]

User Request

Enable Ziti in the gateway. This issue covers the bootstrap Terraform changes to:

1. Deploy the ziti-management service (currently not deployed)
2. Provision a PostgreSQL database for ziti-management
3. Enable Ziti in the gateway by passing env vars through the existing env list in Helm values

Specification

Current Behavior

• The ziti-management service repo exists (agynio/ziti-management) with a Helm chart and container image, but no ArgoCD application deploys it.
• The gateway ArgoCD application does not pass any Ziti configuration.
• The Ziti infrastructure (controller, router, services, identities, policies) is fully provisioned in stacks/ziti/.

Target Behavior

Deploy ziti-management with its database, then enable Ziti in the gateway by passing env vars directly.

Changes Required

All changes are in stacks/platform/.

1. stacks/platform/variables.tf

Add these variables (follow existing patterns):

variable "ziti_management_chart_version" {
  type        = string
  description = "Version of the ziti-management Helm chart published to GHCR"
  default     = "0.1.0"
}

variable "ziti_management_image_tag" {
  type        = string
  description = "Optional override for the ziti-management container image tag"
  default     = ""
}

variable "ziti_management_db_password" {
  type        = string
  description = "Password for the ziti-management PostgreSQL database user"
  default     = "ziti_management"
  sensitive   = true
}

variable "ziti_management_db_pvc_size" {
  type        = string
  description = "Persistent volume claim size for the ziti-management PostgreSQL primary"
  default     = "5Gi"
}

2. stacks/platform/main.tf — locals block

Add to the locals block, following existing patterns:

Resolved image tag (in the resolved_* section):

  resolved_ziti_management_image_tag = trimspace(var.ziti_management_image_tag) != "" ? var.ziti_management_image_tag : var.ziti_management_chart_version

Chart name (in the chart name section):

  ziti_management_chart_name = "agynio/charts/ziti-management"

DB values (in the DB values section, follow agents_db_values pattern):

  ziti_management_db_values = yamlencode({
    fullnameOverride = "ziti-management-db"
    postgres = {
      database = "ziti_management"
      username = "ziti_management"
      password = var.ziti_management_db_password
      pgdata   = "/var/lib/postgresql/data/pgdata"
    }
    persistence = {
      size                    = var.ziti_management_db_pvc_size
      mountPath               = "/var/lib/postgresql/data"
      volumeClaimTemplateName = "data"
    }
    probes = {
      readiness = {
        execCommand = ["pg_isready", "-U", "ziti_management", "-d", "ziti_management"]
      }
      liveness = {
        execCommand = ["pg_isready", "-U", "ziti_management", "-d", "ziti_management"]
      }
    }
  })

Service values (in the service values section, follow agents_values pattern):

  ziti_management_values = yamlencode({
    fullnameOverride = "ziti-management"
    image = {
      repository = "ghcr.io/agynio/ziti-management"
      tag        = local.resolved_ziti_management_image_tag
      pullPolicy = "IfNotPresent"
    }
    env = [
      {
        name  = "DATABASE_URL"
        value = format("postgresql://ziti_management:%s@ziti-management-db:5432/ziti_management?sslmode=disable", var.ziti_management_db_password)
      },
      {
        name  = "ZITI_CONTROLLER_URL"
        value = format("https://ziti-mgmt.%s:%d/edge/management/v1", local.base_domain, local.ingress_port)
      },
    ]
  })

3. stacks/platform/main.tf — new ArgoCD applications

ziti-management DB (sync-wave "8", follow agents_db pattern):

resource "argocd_application" "ziti_management_db" {
  depends_on = [argocd_repository.litellm_repo]
  wait       = true

  metadata {
    name      = "ziti-management-db"
    namespace = "argocd"
    annotations = {
      "argocd.argoproj.io/sync-wave" = "8"
    }
  }

  spec {
    project = "default"
    source {
      repo_url        = local.postgres_chart_repo_host
      chart           = local.postgres_chart_name
      target_revision = var.postgres_chart_version
      helm {
        values = local.ziti_management_db_values
      }
    }
    destination {
      server    = var.destination_server
      namespace = var.platform_namespace
    }
    sync_policy {
      automated {
        prune       = false
        self_heal   = true
        allow_empty = false
      }
      sync_options = local.postgres_sync_options
    }
  }
}

ziti-management service (sync-wave "17", depends on its DB):

resource "argocd_application" "ziti_management" {
  depends_on = [
    argocd_repository.litellm_repo,
    argocd_application.ziti_management_db,
  ]

  metadata {
    name      = "ziti-management"
    namespace = "argocd"
    annotations = {
      "argocd.argoproj.io/sync-wave" = "17"
    }
  }

  spec {
    project = "default"
    source {
      repo_url        = local.platform_chart_repo_host
      chart           = local.ziti_management_chart_name
      target_revision = var.ziti_management_chart_version
      helm {
        values = local.ziti_management_values
      }
    }
    destination {
      server    = var.destination_server
      namespace = var.platform_namespace
    }
    sync_policy {
      dynamic "automated" {
        for_each = var.argocd_automated_sync_enabled ? [1] : []
        content {
          prune       = var.argocd_prune_enabled
          self_heal   = var.argocd_self_heal_enabled
          allow_empty = false
        }
      }
      sync_options = local.default_sync_options
    }
  }
}

4. stacks/platform/main.tf — update gateway ArgoCD application

Add argocd_application.ziti_management to depends_on and pass Ziti env vars through the existing env list (same pattern used by agents, users, etc.):

resource "argocd_application" "gateway" {
  depends_on = [argocd_application.llm, argocd_application.ziti_management]
  ...
      helm {
        values = yamlencode({
          replicaCount = 1
          image = {
            tag = local.resolved_gateway_image_tag
          }
          gateway = {
            oidcIssuerUrl   = var.oidc_issuer_url
            oidcClientId    = var.oidc_client_id
            usersGrpcTarget = "users:50051"
          }
          env = [
            {
              name  = "ZITI_ENABLED"
              value = "true"
            },
            {
              name  = "ZITI_MANAGEMENT_GRPC_TARGET"
              value = "ziti-management:50051"
            },
          ]
        })
      }

Prerequisites

• The ziti-certs Kubernetes Secret must exist in the platform namespace with the enrolled TLS credentials for the ziti-management identity.

Notes

• No changes to the gateway Helm chart are needed — the env list is supported by the service-base library chart used by all services.
• ZITI_LEASE_RENEWAL_INTERVAL is intentionally omitted — the Go default of 2m is correct.
• ZITI_MANAGEMENT_GRPC_TARGET is set explicitly for clarity even though ziti-management:50051 is the Go default.

[rowan-stein]

Additional changes needed: ziti-certs secret provisioning

The CI health check fails because the ziti-management pod can't mount the ziti-certs K8s Secret — it doesn't exist. This secret must be provisioned via enrollment of the ziti_management identity (converting the enrollment JWT into TLS certs).

Root cause

• The stacks/ziti/main.tf creates ziti_identity.ziti_management and stores its enrollment JWT in ziti-management-enrollment secret (in ziti namespace).
• The ziti-management Helm chart expects a ziti-certs secret (in platform namespace) with keys tls.crt, tls.key, ca.crt — the enrolled identity credentials.
• No step currently converts the JWT into enrolled certs.

Required changes

1. .github/workflows/bootstrap.yml — Add ziti CLI installation step (before the apply step):

- name: Install OpenZiti CLI
  run: |
    curl -sS https://get.openziti.io/install.bash | sudo bash -s -- openziti

2. apply.sh — Add enroll-ziti-management step between stack:ziti and stack:data:

The step should:

1. Check if ziti-certs secret already exists in platform namespace → skip if so (enrollment JWT is one-time-use)
2. Ensure platform namespace exists (create if not)
3. Extract enrollment JWT from ziti-management-enrollment secret in ziti namespace
4. Run ziti edge enroll --jwt <jwt-file> --out <identity-json>
5. Extract PEM cert, key, and CA from the identity JSON (format: {"id": {"key": "pem:...", "cert": "pem:...", "ca": "pem:..."}})
6. Create ziti-certs secret in platform namespace with keys tls.crt, tls.key, ca.crt
7. Clean up temp files

Reference

• apply.sh already follows this pattern for the ziti admin password (extracting from k8s secrets between stacks)
• The ziti-management chart values show the expected mount: configMounts[0].sourceName: ziti-certs, mountPath: /etc/ziti
• Architecture doc confirms: ziti-management uses "Certificate-based auth using a long-lived infrastructure credential provisioned at deployment"