diff --git a/content/en/docs/next/install/ansible.md b/content/en/docs/next/install/ansible.md index 9c9cc9c1..9d8f02a0 100644 --- a/content/en/docs/next/install/ansible.md +++ b/content/en/docs/next/install/ansible.md @@ -178,7 +178,7 @@ The playbook performs the following steps automatically: | --- | --- | --- | | `cozystack_api_server_host` | *(required)* | Internal IP of the control-plane node. | | `cozystack_chart_version` | `{{< version-pin "cozystack_version" >}}` | Version of the Cozystack Helm chart. **Pin this explicitly.** | -| `cozystack_platform_variant` | `isp-full-generic` | Platform variant: `default`, `isp-full`, `isp-hosted`, `isp-full-generic`. | +| `cozystack_platform_variant` | `isp-full-generic` | Platform variant: `default`, `isp-full`, `isp-hosted`, `isp-full-generic`, `isp-slim`, `isp-slim-generic`, `isp-hosted-slim`. On k3s use a `-generic` variant: `isp-full-generic`, or `isp-slim-generic` for a minimal install. | | `cozystack_root_host` | `""` | Domain for Cozystack services. Leave empty to skip publishing configuration. | ### Networking diff --git a/content/en/docs/next/install/cozystack/kubernetes-distribution.md b/content/en/docs/next/install/cozystack/kubernetes-distribution.md index 02f26a2d..95b7b815 100644 --- a/content/en/docs/next/install/cozystack/kubernetes-distribution.md +++ b/content/en/docs/next/install/cozystack/kubernetes-distribution.md @@ -92,9 +92,12 @@ PackageSource: cozystack.cozystack-platform Available variants: 1. default 2. isp-full - 3. isp-full-generic - 4. isp-hosted -Select variant (1-4): 1 + 3. isp-hosted + 4. isp-full-generic + 5. isp-slim + 6. isp-slim-generic + 7. isp-hosted-slim +Select variant (1-7): 1 ``` After the platform package is installed, all other PackageSources become available: diff --git a/content/en/docs/next/install/kubernetes/generic.md b/content/en/docs/next/install/kubernetes/generic.md index bd63e29b..ce0caa60 100644 --- a/content/en/docs/next/install/kubernetes/generic.md +++ b/content/en/docs/next/install/kubernetes/generic.md @@ -6,7 +6,7 @@ weight: 50 --- This guide explains how to deploy Cozystack on generic Kubernetes distributions such as k3s, kubeadm, or RKE2. -While Talos Linux remains the recommended platform for production deployments, Cozystack supports deployment on other Kubernetes distributions using the `isp-full-generic` bundle. +While Talos Linux remains the recommended platform for production deployments, Cozystack supports deployment on other Kubernetes distributions using the `isp-full-generic` bundle, or the minimal `isp-slim-generic` bundle (see [Variants]({{% ref "/docs/next/operations/configuration/variants#isp-slim-generic" %}})). ## When to Use Generic Kubernetes @@ -160,7 +160,7 @@ Initialize the cluster without the default CNI: kubeadm init --config kubeadm-config.yaml --skip-phases=addon/kube-proxy ``` -Do not install a CNI plugin after `kubeadm init` — Cozystack will deploy Kube-OVN and Cilium automatically. +Do not install a CNI plugin after `kubeadm init` — Cozystack will deploy Kube-OVN and Cilium automatically (Cilium alone on `isp-slim-generic`). {{% /tab %}} {{% tab name="RKE2" %}} @@ -208,7 +208,7 @@ The manifest includes the operator deployment, the `cozystack-operator-config` C After the operator starts and reconciles the `PackageSource`, create a `Package` resource to trigger the platform installation. {{% alert color="warning" %}} -:warning: **Important**: The `podCIDR` and `serviceCIDR` values **must match** your Kubernetes cluster configuration. +:warning: **Important**: The `podCIDR` and `serviceCIDR` values **must match** your Kubernetes cluster configuration. On `isp-slim-generic` they are not used, but the cluster must allocate pod CIDRs to nodes (`networking.podSubnet` in the kubeadm config above). Different distributions use different defaults: - **k3s**: `10.42.0.0/16` (pods), `10.43.0.0/16` (services) diff --git a/content/en/docs/next/operations/configuration/platform-package.md b/content/en/docs/next/operations/configuration/platform-package.md index 2bb9f2e5..b8a49ea4 100644 --- a/content/en/docs/next/operations/configuration/platform-package.md +++ b/content/en/docs/next/operations/configuration/platform-package.md @@ -50,7 +50,7 @@ spec: | Field | Description | | --- | --- | -| `spec.variant` | Variant to use for installation (e.g., `isp-full`, `isp-full-generic`, `isp-hosted`, `distro-full`). | +| `spec.variant` | Variant to use for installation (e.g., `isp-full`, `isp-full-generic`, `isp-hosted`, `isp-slim`, `isp-slim-generic`, `isp-hosted-slim`). | ### Platform values (`spec.components.platform.values.*`) @@ -93,10 +93,10 @@ spec: | Value | Default | Description | | --- | --- | --- | | `networking.clusterDomain` | `"cozy.local"` | Internal cluster domain name. | -| `networking.podCIDR` | `"10.244.0.0/16"` | The pod subnet used by Pods to assign IPs. | -| `networking.podGateway` | `"10.244.0.1"` | The gateway address for the pod subnet. | -| `networking.serviceCIDR` | `"10.96.0.0/16"` | The service subnet used by Services to assign IPs. | -| `networking.joinCIDR` | `"100.64.0.0/16"` | The `join` subnet for network communication between the Node and Pod. Follow the [kube-ovn] documentation to learn more. | +| `networking.podCIDR` | `"10.244.0.0/16"` | The pod subnet used by Pods to assign IPs. Used by Kube-OVN only: ignored on `isp-hosted` and the slim variants. | +| `networking.podGateway` | `"10.244.0.1"` | The gateway address for the pod subnet. Kube-OVN only. | +| `networking.serviceCIDR` | `"10.96.0.0/16"` | The service subnet used by Services to assign IPs. Kube-OVN only. | +| `networking.joinCIDR` | `"100.64.0.0/16"` | The `join` subnet for network communication between the Node and Pod. Follow the [kube-ovn] documentation to learn more. Kube-OVN only. | | `networking.kubeovn.MASTER_NODES` | `""` | Comma-separated list of KubeOVN master node IPs. By default, KubeOVN uses `lookup` to find control-plane nodes by label `node-role.kubernetes.io/control-plane`. On fresh clusters, lookup may return empty results. Set this to override. | #### Bundles @@ -104,7 +104,7 @@ spec: | Value | Default | Description | | --- | --- | --- | | `bundles.system.enabled` | `false` | Enable the system bundle. Managed by the operator based on `spec.variant`. | -| `bundles.system.variant` | `"isp-full"` | System bundle variant. Options: `isp-full`, `isp-full-generic`, `isp-hosted`. Managed by the operator based on `spec.variant`. | +| `bundles.system.variant` | `"isp-full"` | System bundle variant. Options: `isp-full`, `isp-full-generic`, `isp-hosted`, `isp-slim`, `isp-slim-generic`, `isp-hosted-slim`. Managed by the operator based on `spec.variant`. | | `bundles.iaas.enabled` | `false` | Enable the IaaS bundle. Managed by the operator based on `spec.variant`. | | `bundles.paas.enabled` | `false` | Enable the PaaS bundle. Managed by the operator based on `spec.variant`. | | `bundles.naas.enabled` | `false` | Enable the NaaS bundle. Managed by the operator based on `spec.variant`. | @@ -127,7 +127,7 @@ Platform-wide Gateway API integration. The actual per-tenant Gateway is material | Value | Default | Description | | --- | --- | --- | | `gateway.enabled` | `false` | Enable Gateway API support across the platform. When `true`, cert-manager `ClusterIssuer`s use an `http01.gatewayHTTPRoute` solver attached to the publishing tenant's Gateway, and exposed services (`dashboard`, `keycloak`, `grafana`, `alerta`, `harbor`, `bucket`, `cozystack-api`, `vm-exportproxy`, `cdi-uploadproxy`) render `HTTPRoute`/`TLSRoute` instead of `Ingress`. Materialising the actual per-tenant Gateway still requires an owning tenant to set `tenant.spec.gateway: true`. | -| `gateway.http2` | `true` | Advertise HTTP/2 via TLS ALPN (`h2`, then `http/1.1`) on every Gateway API listener served by the bundled Cilium dataplane. Browsers negotiate HTTP/2 exclusively through ALPN, so with this off every client silently falls back to HTTP/1.1 — the pre-Gateway ingress-nginx path advertised `h2` out of the box, hence on by default. Affects only the client↔gateway hop: gateway↔backend connections stay HTTP/1.1 unless a `Service` opts in per [GEP-1911](https://gateway-api.sigs.k8s.io/geps/gep-1911/) by declaring `appProtocol: kubernetes.io/h2c` on its port (that backend-protocol support is switched on together with ALPN). Maps to Cilium's cluster-wide `enable-gateway-api-alpn` agent setting, so it covers the root and all tenant Gateways at once, with no per-Gateway granularity; only effective on bundles where Cozystack manages Cilium (`isp-full`, `isp-full-generic`). Flipping it re-rolls the `cilium` DaemonSet on the next platform upgrade, the same disruption profile as any other Cilium config change. | +| `gateway.http2` | `true` | Advertise HTTP/2 via TLS ALPN (`h2`, then `http/1.1`) on every Gateway API listener served by the bundled Cilium dataplane. Browsers negotiate HTTP/2 exclusively through ALPN, so with this off every client silently falls back to HTTP/1.1 — the pre-Gateway ingress-nginx path advertised `h2` out of the box, hence on by default. Affects only the client↔gateway hop: gateway↔backend connections stay HTTP/1.1 unless a `Service` opts in per [GEP-1911](https://gateway-api.sigs.k8s.io/geps/gep-1911/) by declaring `appProtocol: kubernetes.io/h2c` on its port (that backend-protocol support is switched on together with ALPN). Maps to Cilium's cluster-wide `enable-gateway-api-alpn` agent setting, so it covers the root and all tenant Gateways at once, with no per-Gateway granularity; only effective on bundles where Cozystack manages Cilium (`isp-full`, `isp-full-generic`, `isp-slim`, `isp-slim-generic`). Flipping it re-rolls the `cilium` DaemonSet on the next platform upgrade, the same disruption profile as any other Cilium config change. | | `gateway.className` | `"cilium"` | The `GatewayClass` every tenant Gateway uses unless the tenant names another one via `tenant.spec.gatewayClass`. Nothing checks the name against the classes the cluster has installed — a name no controller claims produces a `Gateway` that is created but never programmed, surfacing as `Ready=False` with reason `GatewayNotAccepted` on the `TenantGateway`. Changing it while a tenant pins the outgoing name fails that tenant's gateway release, because the set a tenant may name is built from the *current* default; add the outgoing class to `gateway.tenantSelectableClasses` first. | | `gateway.tenantSelectableClasses` | `[]` | Additional `GatewayClass` names a tenant may select for its own Gateway with `tenant.spec.gatewayClass`. The set a tenant may name is this list plus the current `gateway.className`, so a tenant may always name the default explicitly; anything else fails that tenant's own gateway release at render time, naming the class and the allowed set, and reaches no other tenant. Empty means no tenant can pick anything but the default. The allowlist exists because `Tenant` is tenant-writable while a `GatewayClass` is cluster-scoped. | | `gateway.edgeTerminatedClasses` | `[]` | `GatewayClass` names whose provider terminates TLS upstream of the Gateway. Membership here is the only thing that puts a tenant into `edge` cert mode, which renders port-80 listeners only and issues no `Issuer` and no `Certificate`; it wins over `publishing.certificates.wildcardSecretName` and over the solver. Nothing verifies the assertion — listing a class whose provider does *not* terminate TLS makes its Gateways serve every application hostname over plain HTTP, with no redirect and no certificate. Setting `gateway.className` to a class in this list also puts the publishing tenant on it, which unpublishes whichever of the TLS-passthrough endpoints (Kubernetes API, VM export, CDI upload) are published at all — each renders its `TLSRoute` only when `gateway.enabled` is on and its name is in `publishing.exposedServices`. Nothing fails at render time and nothing reports it: the controller writes no route conditions in this mode and removes none, so an orphaned `TLSRoute` can still show the `Accepted=True` it was given under the previous mode. | diff --git a/content/en/docs/next/operations/configuration/variants.md b/content/en/docs/next/operations/configuration/variants.md index d456ecd7..82ac58da 100644 --- a/content/en/docs/next/operations/configuration/variants.md +++ b/content/en/docs/next/operations/configuration/variants.md @@ -29,19 +29,20 @@ or need full manual control over installed packages. ## Variants Overview -| Component | [default] | [isp-full] | [isp-full-generic] | [isp-hosted] | -|:------------------------------|:-----------------------|:-----------------------|:-----------------------|:-----------------------| -| [Managed Kubernetes][k8s] | | ✔ | ✔ | | -| [Managed Applications][apps] | | ✔ | ✔ | ✔ | -| [Virtual Machines][vm] | | ✔ | ✔ | | -| Cozystack Dashboard (UI) | | ✔ | ✔ | ✔ | -| [Cozystack API][api] | | ✔ | ✔ | ✔ | -| [Kubernetes Operators] | | ✔ | ✔ | ✔ | -| [Monitoring subsystem] | | ✔ | ✔ | ✔ | -| Storage subsystem | | [LINSTOR] | [LINSTOR] | | -| Networking subsystem | | [Kube-OVN] + [Cilium] | [Kube-OVN] + [Cilium] | | -| Virtualization subsystem | | [KubeVirt] | [KubeVirt] | | -| OS and [Kubernetes] subsystem | | [Talos Linux] | | | +| Component | [default] | [isp-full] | [isp-full-generic] | [isp-hosted] | [isp-slim] | [isp-slim-generic] | [isp-hosted-slim] | +|:------------------------------|:-----------------------|:-----------------------|:-----------------------|:-----------------------|:-----------------------|:-----------------------|:-----------------------| +| [Managed Kubernetes][k8s] | | ✔ | ✔ | | | | | +| [Managed Applications][apps] | | ✔ | ✔ | ✔ | opt-in | opt-in | opt-in | +| [Virtual Machines][vm] | | ✔ | ✔ | | | | | +| Cozystack Dashboard (UI) | | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | +| [Cozystack API][api] | | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | +| [Kubernetes Operators] | | ✔ | ✔ | ✔ | opt-in | opt-in | opt-in | +| [Monitoring subsystem] | | ✔ | ✔ | ✔ | opt-in | opt-in | opt-in | +| Backups | | ✔ | ✔ | ✔ | opt-in | opt-in | opt-in | +| Storage subsystem | | [LINSTOR] | [LINSTOR] | | [LINSTOR] | [LINSTOR] | | +| Networking subsystem | | [Kube-OVN] + [Cilium] | [Kube-OVN] + [Cilium] | | [Cilium] | [Cilium] | | +| Virtualization subsystem | | [KubeVirt] | [KubeVirt] | | | | | +| OS and [Kubernetes] subsystem | | [Talos Linux] | | | [Talos Linux] | | | [apps]: {{% ref "/docs/next/applications" %}} [vm]: {{% ref "/docs/next/virtualization" %}} @@ -60,6 +61,9 @@ or need full manual control over installed packages. [isp-full]: {{% ref "/docs/next/operations/configuration/variants#isp-full" %}} [isp-full-generic]: {{% ref "/docs/next/operations/configuration/variants#isp-full-generic" %}} [isp-hosted]: {{% ref "/docs/next/operations/configuration/variants#isp-hosted" %}} +[isp-slim]: {{% ref "/docs/next/operations/configuration/variants#isp-slim" %}} +[isp-slim-generic]: {{% ref "/docs/next/operations/configuration/variants#isp-slim-generic" %}} +[isp-hosted-slim]: {{% ref "/docs/next/operations/configuration/variants#isp-hosted-slim" %}} ## Choosing the Right Variant @@ -189,6 +193,97 @@ spec: - dashboard ``` +### `isp-slim` + +`isp-slim` is the minimal counterpart of `isp-full` for Talos Linux, meant for small installations such as arm64 clusters, labs and edge sites. It installs the base platform only: Cilium networking, LINSTOR storage, the Cozystack API and dashboard, tenants, ingress and Gateway API. Everything else, including managed applications, their operators, monitoring and backups, is opt-in (with `authentication.oidc.enabled`, Keycloak, `cozystack.keycloak-operator` and `cozystack.postgres-operator` are installed as well): add the packages you need to `bundles.enabledPackages` (see [Enabling components on slim variants](#enabling-components-on-slim-variants)). + +Virtualization and managed Kubernetes are not available on slim variants: enabling the `iaas` bundle fails the render. Use `isp-full` or `isp-full-generic` if you need them. + +Networking runs on Cilium alone, without Kube-OVN, so `networking.podCIDR`, `networking.podGateway`, `networking.serviceCIDR` and `networking.joinCIDR` are not used: pods get addresses from the pod CIDR Kubernetes assigns to each node. `networking.encryption.enabled` is not supported on `isp-slim` and `isp-slim-generic` and fails the render. + +MetalLB is not installed. `LoadBalancer` Services get their addresses from Cilium: create a pool and an L2 announcement policy, for example: + +```yaml +apiVersion: cilium.io/v2 +kind: CiliumLoadBalancerIPPool +metadata: + name: default +spec: + blocks: + - start: 192.0.2.10 + stop: 192.0.2.20 +--- +apiVersion: cilium.io/v2alpha1 +kind: CiliumL2AnnouncementPolicy +metadata: + name: default +spec: + loadBalancerIPs: true +``` + +With `publishing.externalIPs` set, the host ingress needs no load balancer at all, but a Gateway still creates a `LoadBalancer` Service and needs the pool. MetalLB is still available as an opt-in package, `cozystack.metallb`. + +Example configuration: + +```yaml +apiVersion: cozystack.io/v1alpha1 +kind: Package +metadata: + name: cozystack.cozystack-platform +spec: + variant: isp-slim + components: + platform: + values: + publishing: + host: "example.org" + apiServerEndpoint: "https://192.168.100.10:6443" + exposedServices: + - api + - dashboard +``` + +### `isp-slim-generic` + +`isp-slim-generic` is the same minimal platform as `isp-slim`, with the same Cilium-only networking, for generic Kubernetes distributions such as k3s, kubeadm or RKE2. Node requirements are those of `isp-full-generic`; see the [Generic Kubernetes guide]({{% ref "/docs/next/install/kubernetes/generic" %}}) and set `variant: isp-slim-generic` in the Platform Package. + +### `isp-hosted-slim` + +`isp-hosted-slim` is the minimal counterpart of `isp-hosted`: the host cluster provides CNI and storage, and Cozystack installs only the API, dashboard, tenants, ingress and Gateway API. Cluster requirements are those of `isp-hosted`. Managed applications are opt-in, as on the other slim variants. + +### Enabling components on slim variants + +On a slim variant, a package is installed only when it is listed in `bundles.enabledPackages` of the Platform Package. A package does not pull in its dependencies: list every package from its row below, otherwise the Package stays in `DependenciesNotReady`. + +```yaml +apiVersion: cozystack.io/v1alpha1 +kind: Package +metadata: + name: cozystack.cozystack-platform +spec: + variant: isp-slim + components: + platform: + values: + bundles: + enabledPackages: + - cozystack.postgres-operator + - cozystack.postgres-application +``` + +| Component | Packages to add to `bundles.enabledPackages` | +| --- | --- | +| A managed application, for example PostgreSQL | The application and its operator: `cozystack.postgres-application`, `cozystack.postgres-operator`. The same pattern applies to MariaDB, Kafka, ClickHouse, FoundationDB, RabbitMQ, Redis, MongoDB and OpenSearch. Valkey uses `cozystack.redis-operator`; Harbor also needs `cozystack.postgres-operator`, `cozystack.redis-operator` and `cozystack.seaweedfs-application`, and its tenant needs SeaweedFS enabled for the registry bucket. NATS, OpenBao, Qdrant, `cozystack.tcp-balancer-application` and `cozystack.vpn-application` need no operator. | +| Monitoring | `cozystack.monitoring-application`, `cozystack.grafana-operator`, `cozystack.postgres-operator`, `cozystack.monitoring-agents`, `cozystack.metrics-server`, `cozystack.vertical-pod-autoscaler`. Also set `monitoring.rootEnabled: true` in the platform values and `spec.monitoring: true` on the root Tenant. | +| etcd for tenants | `cozystack.etcd-application`, `cozystack.etcd-operator`, `cozystack.vertical-pod-autoscaler` | +| SeaweedFS and buckets | `cozystack.seaweedfs-application`, `cozystack.bucket-application` | +| Backups | `cozystack.backupstrategy-controller`, `cozystack.backup-controller`, `cozystack.velero`, `cozystack.bucket-application`, `cozystack.monitoring-agents`, `cozystack.metrics-server`, `cozystack.vertical-pod-autoscaler`. The default backup bucket lives in the root Tenant, so also add `cozystack.seaweedfs-application` and set `spec.seaweedfs: true` on the root Tenant. | +| Metrics API (`kubectl top`, HPA) | `cozystack.metrics-server` | +| Multus | `cozystack.multus`. Not available on `isp-hosted-slim`. | +| MetalLB | `cozystack.metallb`. Not available on `isp-hosted-slim`. | + +Slim variants are meant for new installations. Do not switch a running `isp-full` or `isp-full-generic` cluster to a slim variant: the networking Package moves from Kube-OVN with Cilium to Cilium alone, Kube-OVN is removed, and running pods lose networking. The other platform components stay installed (their Packages carry `helm.sh/resource-policy: keep`), so the switch does not make the cluster smaller. + ## Learn More For a full list of configuration options for each variant, refer to the