Skip to content

Deploy ziti-management and enable Ziti - #153

Merged
Benkovichnikita merged 10 commits into
mainfrom
noa/issue-152
Mar 23, 2026
Merged

Deploy ziti-management and enable Ziti#153
Benkovichnikita merged 10 commits into
mainfrom
noa/issue-152

Conversation

@casey-brooks

@casey-brooks casey-brooks commented Mar 23, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add ziti-management variables, locals, and Helm values in platform stack
  • deploy ziti-management db and app Argo CD applications
  • enable Ziti settings for the gateway chart

Testing

  • ./apply.sh -y (fails: GHCR access denied for agynio/charts/ziti-management:0.1.0)
  • terraform fmt -check -recursive stacks

Closes #152

@casey-brooks
casey-brooks requested a review from a team as a code owner March 23, 2026 01:27
@casey-brooks

casey-brooks commented Mar 23, 2026

Copy link
Copy Markdown
Contributor Author

Test & Lint Summary

  • ./apply.sh -y (fails: GHCR access denied for agynio/charts/ziti-management:0.1.0)
  • terraform fmt -check -recursive stacks

noa-lucent
noa-lucent previously approved these changes Mar 23, 2026
View reviewed changes

@noa-lucent noa-lucent left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clean, well-structured PR that follows existing patterns closely. All five spec items from #152 are addressed:

  1. ✅ Variables in variables.tf — chart version, image tag, DB password, PVC size
  2. ✅ Locals — resolved image tag, chart name, DB values, service values
  3. ✅ ziti_management_db ArgoCD application (sync-wave 8, wait = true, timeouts)
  4. ✅ ziti_management ArgoCD application (sync-wave 17, dynamic automated sync)
  5. ✅ Gateway updated with depends_on and Ziti Helm values

One minor comment on PVC size default/description consistency. No blocking issues.

Comment thread stacks/platform/variables.tf
@casey-brooks

casey-brooks commented Mar 23, 2026

Copy link
Copy Markdown
Contributor Author

Summary

  • updated ziti-management DB PVC size default and description to match other DB settings

Test & Lint Summary

  • ./apply.sh -y (fails: ghcr.io/agynio/charts/ziti-management:0.1.0 not found)
  • terraform fmt -check -recursive stacks

noa-lucent
noa-lucent previously approved these changes Mar 23, 2026
View reviewed changes

@noa-lucent noa-lucent left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified the fix: PVC default updated to 5Gi and description aligned to the established pattern. Previous comment resolved. LGTM — ready to merge.

@casey-brooks

casey-brooks commented Mar 23, 2026

Copy link
Copy Markdown
Contributor Author

Summary

  • pass Ziti config to gateway via env list and restore gateway values to match main

Test & Lint Summary

  • ./apply.sh -y (fails: ghcr.io/agynio/charts/ziti-management:0.1.0 not found)
  • terraform fmt -check -recursive stacks

noa-lucent
noa-lucent previously approved these changes Mar 23, 2026
View reviewed changes

@noa-lucent noa-lucent left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified the updated approach. Ziti configuration moved from custom gateway Helm values (zitiEnabled, zitiManagementGrpcTarget) to the standard env list (ZITI_ENABLED, ZITI_MANAGEMENT_GRPC_TARGET). This aligns with the established pattern used by all other services (agents, users, ziti-management, etc.). The gateway block is cleanly restored to its original three fields. LGTM.

@casey-brooks

casey-brooks commented Mar 23, 2026

Copy link
Copy Markdown
Contributor Author

Summary

  • install OpenZiti CLI in CI before apply
  • add enroll-ziti-management step to create ziti-certs from enrollment JWT

Test & Lint Summary

  • ./apply.sh -y (fails: ziti edge enroll token is expired)
  • terraform fmt -check -recursive stacks

@casey-brooks

casey-brooks commented Mar 23, 2026

Copy link
Copy Markdown
Contributor Author

Summary

  • install OpenZiti CLI in CI before apply
  • add enroll-ziti-management step to create ziti-certs from the enrollment JWT

Test & Lint Summary

  • ./apply.sh -y
  • ./.github/scripts/verify_platform_health.sh (timed out: platform-server CrashLoopBackOff (EACCES /mnt/graph/graph), ziti-management CreateContainerConfigError: runAsNonRoot with non-numeric user)
  • terraform fmt -check -recursive stacks

@rowan-stein
fix: move ziti-management enrollment after platform stack
967fff3 
The enrollment step was creating the platform namespace before
Terraform, causing "namespaces platform already exists" error.
Move the step to after stack:platform so the namespace already
exists, and remove the redundant kubectl create namespace call.
@rowan-stein
fix: add numeric runAsUser/runAsGroup for ziti-management
e39197a 
The container image uses a named user (app) which causes
CreateContainerConfigError with runAsNonRoot. Set explicit
runAsUser=100 and runAsGroup=101 matching other services.
@rowan-stein
fix: add CoreDNS rewrite for ziti-mgmt in-cluster resolution
4a249be 
ziti-management pod needs to reach the Ziti controller management
API at ziti-mgmt.<domain>. Without a CoreDNS rewrite, this resolves
to 127.0.0.1 (loopback) inside the cluster, causing immediate
connection refused and CrashLoopBackOff.

Add rewrite rule mapping ziti-mgmt.<domain> to the controller
management service in the ziti namespace.
@rowan-stein
fix: add missing ZITI_* env vars to ziti-management Helm values
8f77ca8 
Helm env array override replaces defaults entirely. The chart
defaults include ZITI_CERT_FILE, ZITI_KEY_FILE, and ZITI_CA_FILE
which are required for config.FromEnv(). Without them the container
exits immediately with "ZITI_CERT_FILE must be set".
@rowan-stein

rowan-stein commented Mar 23, 2026

Copy link
Copy Markdown
Collaborator

CI is now green ✅ after fixing three issues:

  1. Namespace conflict — Moved enrollment step to after stack:platform (commit 967fff3)
  2. CreateContainerConfigError — Added runAsUser=100/runAsGroup=101 to security context (commit e39197a)
  3. CoreDNS rewrite — Added ziti-mgmt → ziti-controller-mgmt.ziti.svc.cluster.local rewrite (commit 4a249be)
  4. Missing env vars — Added ZITI_CERT_FILE, ZITI_KEY_FILE, ZITI_CA_FILE to Helm values (commit 8f77ca8)

Requesting fresh review as previous approvals were dismissed by the new commits.

noa-lucent
noa-lucent previously approved these changes Mar 23, 2026
View reviewed changes

@noa-lucent noa-lucent left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed after the four CI-fix commits. All changes look correct and well-integrated:

  1. Enrollment step (apply.sh) — Correctly placed after stack:platform, idempotent (ziti-certs existence check), proper cleanup with trap/unset pattern. One trailing whitespace nit noted.
  2. Security context (ziti_management_values) — runAsUser=100/runAsGroup=101 with full hardening (readOnlyRootFilesystem, drop ALL, seccomp) follows the established pattern for other services in this file.
  3. CoreDNS rewrite (stacks/deps/main.tf) — ziti-mgmt rewrite added with clear comment; existing block comment updated to reflect the change.
  4. Cert env vars (ZITI_CERT_FILE, ZITI_KEY_FILE, ZITI_CA_FILE) — Correctly added to ziti_management_values.
  5. OpenZiti CLI install (.github/workflows/bootstrap.yml) — Consistent with the existing unpinned curl-to-bash pattern used for k3d.

All five spec items from #152 remain correctly addressed. The CI fixes are appropriate and don't introduce any structural concerns.

LGTM — one nit on trailing blank lines, not blocking.

Comment thread apply.sh Outdated
@rowan-stein
refactor: remove redundant ZITI_MANAGEMENT_GRPC_TARGET from gateway env
4a2e967 
The gateway code defaults to "ziti-management:50051" which matches
what we were setting explicitly. No need to pass it.
noa-lucent
noa-lucent previously approved these changes Mar 23, 2026
View reviewed changes

@noa-lucent noa-lucent left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified the new commit: ZITI_MANAGEMENT_GRPC_TARGET correctly removed from the gateway env list since the Go service defaults to ziti-management:50051. The gateway now passes only ZITI_ENABLED=true, which is clean and sufficient.

Previous nit (trailing blank lines in apply.sh) resolved — not blocking.

All #152 requirements remain fully addressed. LGTM.

@rowan-stein
feat(ziti): switch to self-enrollment, remove ziti CLI dependency
eba7511 
- Update ziti-management chart to v0.2.0 (adds startup self-enrollment)
- Add PVC persistence, enrollment JWT secret mount, Recreate strategy
- Change cert paths from /etc/ziti/ to /var/lib/ziti/ (writable PVC)
- Add ZITI_ENROLLMENT_JWT_FILE env var
- Create enrollment JWT secret in platform namespace via ziti remote state
- Remove enroll-ziti-management step from apply.sh
- Remove 'Install OpenZiti CLI' step from CI workflow
@rowan-stein

rowan-stein commented Mar 23, 2026

Copy link
Copy Markdown
Collaborator

Self-enrollment update pushed

Switched ziti-management to v0.2.0 with startup self-enrollment, eliminating the ziti CLI dependency:

Added:

  • updateStrategy: Recreate (single instance, PVC requires RWO)
  • PVC persistence for cert storage (/var/lib/ziti/)
  • Enrollment JWT secret mount (/etc/ziti-enrollment/)
  • ZITI_ENROLLMENT_JWT_FILE env var
  • ziti_management_enrollment_token output from ziti stack
  • terraform_remote_state.ziti in platform stack
  • kubernetes_secret_v1.ziti_management_enrollment in platform namespace

Removed:

  • enroll-ziti-management step from apply.sh (52 lines)
  • "Install OpenZiti CLI" step from CI workflow
  • ziti CLI dependency entirely

noa-lucent
noa-lucent approved these changes Mar 23, 2026
View reviewed changes

@noa-lucent noa-lucent left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified the self-enrollment refactor. This is a clean improvement over the previous shell-based approach:

  • Removed: enroll-ziti-management step in apply.sh (52 lines), OpenZiti CLI install in CI workflow, ziti-certs secret — all fully cleaned up with no stale references.
  • Added: terraform_remote_state.ziti to pass the enrollment JWT, kubernetes_secret_v1.ziti_management_enrollment in the platform namespace, and updated Helm values with configMounts (JWT), persistence (PVC for self-enrolled certs), updateStrategy = Recreate, and ZITI_ENROLLMENT_JWT_FILE env var.
  • Updated: Cert paths correctly moved from /etc/ziti/ → /var/lib/ziti/ (PVC-backed, writable by the app).
  • Chart version bumped to 0.2.0.

All #152 requirements remain fully addressed. One cosmetic nit on a trailing blank line, not blocking. LGTM.

Comment thread apply.sh
run_stack "platform"
step_end "stack:platform"


@noa-lucent noa-lucent Mar 23, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] Extra blank line — the rest of the script uses a single blank line between sections.

@Benkovichnikita
Benkovichnikita merged commit f7d2f07 into main Mar 23, 2026
1 check passed
casey-brooks added a commit that referenced this pull request Apr 27, 2026
@casey-brooks @rowan-stein
Deploy ziti-management and enable Ziti (#153)
04e4b1b 
* feat(platform): add ziti management

* fix(platform): update ziti db pvc size

* fix(platform): pass ziti env vars

* feat(bootstrap): enroll ziti management

* fix: move ziti-management enrollment after platform stack

The enrollment step was creating the platform namespace before
Terraform, causing "namespaces platform already exists" error.
Move the step to after stack:platform so the namespace already
exists, and remove the redundant kubectl create namespace call.

* fix: add numeric runAsUser/runAsGroup for ziti-management

The container image uses a named user (app) which causes
CreateContainerConfigError with runAsNonRoot. Set explicit
runAsUser=100 and runAsGroup=101 matching other services.

* fix: add CoreDNS rewrite for ziti-mgmt in-cluster resolution

ziti-management pod needs to reach the Ziti controller management
API at ziti-mgmt.<domain>. Without a CoreDNS rewrite, this resolves
to 127.0.0.1 (loopback) inside the cluster, causing immediate
connection refused and CrashLoopBackOff.

Add rewrite rule mapping ziti-mgmt.<domain> to the controller
management service in the ziti namespace.

* fix: add missing ZITI_* env vars to ziti-management Helm values

Helm env array override replaces defaults entirely. The chart
defaults include ZITI_CERT_FILE, ZITI_KEY_FILE, and ZITI_CA_FILE
which are required for config.FromEnv(). Without them the container
exits immediately with "ZITI_CERT_FILE must be set".

* refactor: remove redundant ZITI_MANAGEMENT_GRPC_TARGET from gateway env

The gateway code defaults to "ziti-management:50051" which matches
what we were setting explicitly. No need to pass it.

* feat(ziti): switch to self-enrollment, remove ziti CLI dependency

- Update ziti-management chart to v0.2.0 (adds startup self-enrollment)
- Add PVC persistence, enrollment JWT secret mount, Recreate strategy
- Change cert paths from /etc/ziti/ to /var/lib/ziti/ (writable PVC)
- Add ZITI_ENROLLMENT_JWT_FILE env var
- Create enrollment JWT secret in platform namespace via ziti remote state
- Remove enroll-ziti-management step from apply.sh
- Remove 'Install OpenZiti CLI' step from CI workflow

---------

Co-authored-by: Rowan Stein <rowan.stein@agyn.io>
This was referenced Jun 11, 2026
Closed
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Benkovichnikita Benkovichnikita Benkovichnikita approved these changes
@noa-lucent noa-lucent noa-lucent approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Deploy ziti-management service and enable Ziti in the gateway

4 participants

@casey-brooks @rowan-stein @Benkovichnikita @noa-lucent