Deploy a multi-zone global control plane

Prerequisites

To set up a multi-zone deployment we will need to:

Set up the global control plane
Set up the zone control planes
Verify control plane connectivity
Ensure mTLS is enabled for the multi-zone meshes

Usage

Set up the global control plane

The global control plane must run on a dedicated cluster (unless using “Universal on Kubernetes” mode), and cannot be assigned to a zone.

Kubernetes

Helm

Universal on Kubernetes using Helm

Universal

The global control plane on Kubernetes must reside on its own Kubernetes cluster, to keep its resources separate from the resources the zone control planes create during synchronization.

Run:

kumactl install control-plane --mode=global | kubectl apply -f -

Find the external IP and port of the global-remote-sync service in the kong-mesh-system namespace:

kubectl get services -n kong-mesh-system
NAMESPACE     NAME                   TYPE           CLUSTER-IP      EXTERNAL-IP      PORT(S)                                                                  AGE
kong-mesh-system   global-remote-sync     LoadBalancer   10.105.9.10     35.226.196.103   5685:30685/TCP                                                           89s
kong-mesh-system   kong-mesh-control-plane     ClusterIP      10.105.12.133   <none>           5681/TCP,443/TCP,5676/TCP,5677/TCP,5678/TCP,5679/TCP,5682/TCP,5653/UDP   90s

In this example the value is 35.226.196.103:5685. You pass this as the value of <global-kds-address> when you set up the zone control planes.

Set the controlPlane.mode value to global in the chart (values.yaml), then install. On the command line, run:
```
helm install kong-mesh --create-namespace --namespace kong-mesh-system --set controlPlane.mode=global kong-mesh/kong-mesh
```
Or you can edit the chart and pass the file to the helm install kong-mesh command. To get the default values, run:
```
helm show values kuma/kuma
```

Find the external IP and port of the global-remote-sync service in the kong-mesh-system namespace:

kubectl get services -n kong-mesh-system
NAMESPACE     NAME                   TYPE           CLUSTER-IP      EXTERNAL-IP      PORT(S)                                                                  AGE
kong-mesh-system   global-remote-sync     LoadBalancer   10.105.9.10     35.226.196.103   5685:30685/TCP                                                           89s
kong-mesh-system   kong-mesh-control-plane     ClusterIP      10.105.12.133   <none>           5681/TCP,443/TCP,5676/TCP,5677/TCP,5678/TCP,5679/TCP,5682/TCP,5653/UDP   90s

By default, it’s exposed on port 5685. In this example the value is 35.226.196.103:5685. You pass this as the value of <global-kds-address> when you set up the zone control planes.

Running global control plane in “Universal on Kubernetes” mode means using PostgreSQL as storage instead of Kubernetes. It means that failover / HA / reliability characteristics will change. Please read Kubernetes and PostgreSQL docs for more details.

Set controlPlane.environment=universal and controlPlane.mode=global in the chart (values.yaml).

Define Kubernetes secrets with database sensitive information

apiVersion: v1
kind: Secret
metadata:
  name: your-secret-name
type: Opaque
data:
  POSTGRES_DB: ...
  POSTGRES_HOST_RW: ...
  POSTGRES_USER: ...
  POSTGRES_PASSWORD: ...

Set controlPlane.secrets with database sensitive information

# ...
    secrets:
      postgresDb:
        Secret: your-secret-name
        Key: POSTGRES_DB
        Env: KUMA_STORE_POSTGRES_DB_NAME
      postgresHost:
        Secret: your-secret-name
        Key: POSTGRES_HOST_RW
        Env: KUMA_STORE_POSTGRES_HOST
      postgrestUser:
        Secret: your-secret-name
        Key: POSTGRES_USER
        Env: KUMA_STORE_POSTGRES_USER
      postgresPassword:
        Secret: your-secret-name
        Key: POSTGRES_PASSWORD
        Env: KUMA_STORE_POSTGRES_PASSWORD

Optionally set postgres with TLS settings

  # Postgres' settings for universal control plane on k8s
  postgres:
    # -- Postgres port, password should be provided as a secret reference in "controlPlane.secrets"
    # with the Env value "KUMA_STORE_POSTGRES_PASSWORD".
    # Example:
    # controlPlane:
    #   secrets:
    #     - Secret: postgres-postgresql
    #       Key: postgresql-password
    #       Env: KUMA_STORE_POSTGRES_PASSWORD
    port: "5432"
    # TLS settings
    tls:
      # -- Mode of TLS connection. Available values are: "disable", "verifyNone", "verifyCa", "verifyFull"
      mode: disable # ENV: KUMA_STORE_POSTGRES_TLS_MODE
      # -- Whether to disable SNI the postgres `sslsni` option.
      disableSSLSNI: false # ENV: KUMA_STORE_POSTGRES_TLS_DISABLE_SSLSNI
      # -- Secret name that contains the ca.crt
      caSecretName:
      # -- Secret name that contains the client tls.crt, tls.key
      secretName:

Run helm install

 helm install kong-mesh -f values.yaml --skip-crds --create-namespace --namespace kong-mesh-system kong-mesh/kong-mesh

Find the external IP and port of the global-remote-sync service in the kong-mesh-system namespace:

 kubectl get services -n kong-mesh-system
 NAMESPACE     NAME                   TYPE           CLUSTER-IP      EXTERNAL-IP      PORT(S)                                                                  AGE
 kong-mesh-system   global-remote-sync     LoadBalancer   10.105.9.10     35.226.196.103   5685:30685/TCP                                                           89s
 kong-mesh-system   kong-mesh-control-plane     ClusterIP      10.105.12.133   <none>           5681/TCP,443/TCP,5676/TCP,5677/TCP,5678/TCP,5679/TCP,5682/TCP,5653/UDP   90s

In this example the value is 35.226.196.103:5685. You pass this as the value of <global-kds-address> when you set up the zone control planes.

Set up the global control plane, and add the global environment variable:
```
KUMA_MODE=global kuma-cp run
```

Set up the zone control planes

You need the following values to pass to each zone control plane setup:

zone – the zone name. An arbitrary string. This value registers the zone control plane with the global control plane.
kds-global-address – the external IP and port of the global control plane.

Kubernetes

Helm

Universal

Without zone egress: