Secure access across services

This page explains how secure access is provided across a Kong Mesh deployment:

• security between the data plane proxies and the Kong Mesh control plane
• security between the user and the Kong Mesh control plane
• security between the Kong Mesh control planes (Multizone)

Kong Mesh stores autogenerated certificates and other files in a working directory. The default value for this directory is $HOME/.kuma. You can change the working directory by setting the KUMA_GENERAL_WORK_DIR environment variable.

This section is not to be confused with the mTLS policy that you can apply to a Mesh to secure service-to-service traffic.

A data plane proxy connects to the control plane for its configuration, including mTLS certificates described in the following sections.

Encrypted communication

Because the data plane proxy and the control plane exchange sensitive information, the communication needs to be encrypted by TLS. By default, the control plane’s server that is consumed by the data plane proxy is secured by TLS with autogenerated certificates.

It’s recommended that the data plane proxy verifies the identity of the control plane. To do so, data plane proxies need to obtain the CA that was used to generate the certificate by which the control plane’s server is secured. Note, this CA is not the same CA for service-to-service communication.

To override autogenerated certificates

If overridden, Kong Mesh uses the certificates to protect not only data plane proxy to control plane traffic but also user to control plane traffic and control plane to control plane traffic.

1) Prepare certificates

Generate a TLS pair with a PKI of your choice and store it in PEM-encoded format in /tmp/tls.crt, /tmp/tls.key. Store the CA that was used to sign this pair in /tmp/ca.crt.

You can also use kumactl to generate self-signed certs:

kumactl generate tls-certificate \
  --type=server \
  --hostname=<KUMA_CP_DNS_NAME> \
  --cert-file=/tmp/tls.crt \
  --key-file=/tmp/tls.key

Since “tls.crt” is a self-signed cert, it’s also a CA:

cp /tmp/tls.crt /tmp/ca.crt

2) Configure the control plane with generated certs:

kubectl create secret generic general-tls-certs -n kong-mesh-system \
  --from-file=tls.crt=/tmp/tls.crt \
  --from-file=tls.key=/tmp/tls.key \
  --from-file=ca.crt=/tmp/ca.crt

kumactl install control-plane \
  --set "kuma.controlPlane.tls.general.secretName=general-tls-certs" \
  --set "kuma.controlPlane.tls.general.caBundle=$(cat /tmp/ca.crt | base64)" \
  | kubectl apply -f -

# Before installing Kong Mesh with Helm, configure your local Helm repository:
# https://developer.konghq.com/mesh/2.11/production/cp-deployment/kubernetes/#helm
helm install \
  --create-namespace \
  --namespace kong-mesh-system \
  --set "kuma.controlPlane.tls.general.secretName=general-tls-certs" \
  --set "kuma.controlPlane.tls.general.caBundle=$(cat /tmp/ca.crt | base64)" \
  kong-mesh kong-mesh/kong-mesh

KUMA_GENERAL_TLS_CERT_FILE=/tmp/tls.crt \
  KUMA_GENERAL_TLS_KEY_FILE=/tmp/tls.key \
  kuma-cp run

  Warning  FailedCreate  3m39s (x18 over 14m)  replicaset-controller  Error creating: Internal error occurred: failed calling webhook "namespace-kuma-injector.kuma.io": could not get REST client: unable to load root certificates: unable to parse bytes as PEM block

kuma-dp run \
  --cp-address=https://<KUMA_CP_DNS_NAME>:5678 \
  --ca-cert-file=/tmp/ca.crt \
  --dataplane-file=dp.yaml \
  --dataplane-token-file=/tmp/kuma-dp-redis-1-token

Authentication

See Data plane proxy authentication and Zone proxy authentication.

You can enable TLS on the Monitoring Assignment Discovery Service. By default, it uses the same certificate used for CP to DP communication. This is the certificate configured with the --tls-general options. You can enable it by using the KUMA_MONITORING_ASSIGNMENT_SERVER_TLS_ENABLED=true environment variable.

kubectl create secret generic general-tls-certs -n kong-mesh-system \
  --from-file=tls.crt=/tmp/tls.crt \
  --from-file=tls.key=/tmp/tls.key \
  --from-file=ca.crt=/tmp/ca.crt

kumactl install control-plane \
  --set "kuma.controlPlane.tls.general.secretName=general-tls-certs" \
  --set "kuma.controlPlane.tls.general.caBundle=$(cat /tmp/ca.crt | base64)" \
  --set "kuma.controlPlane.envVars.KUMA_MONITORING_ASSIGNMENT_SERVER_TLS_ENABLED=true" \
  | kubectl apply -f -

# Before installing Kong Mesh with Helm, configure your local Helm repository:
# https://developer.konghq.com/mesh/2.11/production/cp-deployment/kubernetes/#helm
helm install \
  --create-namespace \
  --namespace kong-mesh-system \
  --set "kuma.controlPlane.tls.general.secretName=general-tls-certs" \
  --set "kuma.controlPlane.tls.general.caBundle=$(cat /tmp/ca.crt | base64)" \
  --set "kuma.controlPlane.envVars.KUMA_MONITORING_ASSIGNMENT_SERVER_TLS_ENABLED=true" \
  kong-mesh kong-mesh/kong-mesh

KUMA_MONITORING_ASSIGNMENT_SERVER_TLS_CERT_FILE=/tmp/tls.crt \
  KUMA_MONITORING_ASSIGNMENT_SERVER_TLS_KEY_FILE=/tmp/tls.key \
  KUMA_MONITORING_ASSIGNMENT_SERVER_TLS_ENABLED=true \
  kuma-cp run

Now you can configure Kuma’s Prometheus SD with the correct TLS configuration using the Prometheus docs.

Users and automation tools can interact with the control plane via the API Server using tools like curl or kumactl. API Server is exposed by default on :5681 on HTTP and :5682 on HTTPS.

Encrypted communication

The API Server HTTPS server is secured by default by autogenerated certificates.

To override autogenerated certificates.

1) Prepare certificates. Generate TLS pair with a PKI of your choice and store it in PEM-encoded format in /tmp/tls.crt, /tmp/tls.key. Store the CA that was used to sign this pair in /tmp/ca.crt

You can also use kumactl to generate self-signed certs:

  kumactl generate tls-certificate \
    --type=server \
    --hostname=<KUMA_CP_DNS_NAME> \
    --cert-file=/tmp/tls.crt \
    --key-file=/tmp/tls.key

Since “tls.crt” is a self-signed cert, it’s also a CA:

  cp /tmp/tls.crt /tmp/ca.crt

2) Configure the control plane with generated certificates

kubectl create secret tls api-server-tls -n kong-mesh-system \
  --cert=/tmp/tls.crt \
  --key=/tmp/tls.key

kumactl install control-plane \
  --set "kuma.controlPlane.tls.apiServer.secretName=api-server-tls" \
  | kubectl apply -f -

# Before installing Kong Mesh with Helm, configure your local Helm repository:
# https://developer.konghq.com/mesh/2.11/production/cp-deployment/kubernetes/#helm
helm install \
  --create-namespace \
  --namespace kong-mesh-system \
  --set "kuma.controlPlane.tls.apiServer.secretName=api-server-tls" \
  kong-mesh kong-mesh/kong-mesh

KUMA_API_SERVER_HTTPS_TLS_CERT_FILE=/tmp/tls.crt \
  KUMA_API_SERVER_HTTPS_TLS_KEY_FILE=/tmp/tls.key \
  kuma-cp run

If you receive an error like the following, make sure you are using a supported PEM certificate and that the certificate doesn’t contain incomplete or corrupted data:

  Warning  FailedCreate  3m39s (x18 over 14m)  replicaset-controller  Error creating: Internal error occurred: failed calling webhook "namespace-kuma-injector.kuma.io": could not get REST client: unable to load root certificates: unable to parse bytes as PEM block

3) Configure secure connection using the kumactl CLI:

  kumactl config control-planes add \
    --name=<NAME> \
    --address=https://<KUMA_CP_DNS_NAME>:5682 \
    --ca-cert-file=/tmp/ca.crt \

You can also hide the HTTP version of API Server by binding it to localhost KUMA_API_SERVER_HTTP_INTERFACE: 127.0.0.1 or by disabling it altogether KUMA_API_SERVER_HTTP_ENABLED: false

Authentication

See API Server authentication.

A zone control plane connects to a global control plane for policies configuration.

Encrypted communication

Because the global control plane and the zone control plane exchange sensitive information, the communication needs to be encrypted by TLS. By default, the global control plane’s server that is consumed by the zone control plane is secured by TLS with autogenerated certificates.

It’s recommended that the zone control plane verifies the identity of the global control plane. To do so, zone control planes need to obtain the CA that was used to generate the certificate by which the control plane’s server is secured.

To override autogenerated certificates

1) Prepare certificates

Generate TLS pair with a PKI of your choice and store it in PEM-encoded format in /tmp/tls.crt, /tmp/tls.key. Store the CA that was used to sign this pair in /tmp/ca.crt

You can also use kumactl to generate self-signed certs:

kumactl generate tls-certificate \
  --type=server \
  --hostname=<CROSS_ZONE_KUMA_CP_DNS_NAME> \
  --cert-file=/tmp/tls.crt \
  --key-file=/tmp/tls.key

Since “tls.crt” is a self-signed cert, it’s also a CA:

cp /tmp/tls.crt /tmp/ca.crt

2) Configure global control plane

kubectl create secret tls kds-server-tls -n kong-mesh-system \
  --cert=/tmp/tls.crt \
  --key=/tmp/tls.key

kumactl install control-plane \
  --set "kuma.controlPlane.tls.kdsGlobalServer.secretName=kds-server-tls" \
  | kubectl apply -f -

# Before installing Kong Mesh with Helm, configure your local Helm repository:
# https://developer.konghq.com/mesh/2.11/production/cp-deployment/kubernetes/#helm
helm install \
  --create-namespace \
  --namespace kong-mesh-system \
  --set "kuma.controlPlane.tls.kdsGlobalServer.secretName=kds-server-tls" \
  kong-mesh kong-mesh/kong-mesh

KUMA_MULTIZONE_GLOBAL_KDS_TLS_CERT_FILE=/tmp/tls.crt \
  KUMA_MULTIZONE_GLOBAL_KDS_TLS_KEY_FILE=/tmp/tls.key \
  KUMA_MODE=global \
  kuma-cp run

3) Configure the zone control plane

kubectl create secret generic kds-ca-certs -n kong-mesh-system \
  --from-file=ca.crt=/tmp/ca.crt

kumactl install control-plane \
  --set "kuma.controlPlane.tls.kdsZoneClient.secretName=kds-ca-certs" \
  | kubectl apply -f -

# Before installing Kong Mesh with Helm, configure your local Helm repository:
# https://developer.konghq.com/mesh/2.11/production/cp-deployment/kubernetes/#helm
helm install \
  --create-namespace \
  --namespace kong-mesh-system \
  --set "kuma.controlPlane.tls.kdsZoneClient.secretName=kds-ca-certs" \
  kong-mesh kong-mesh/kong-mesh

KUMA_MULTIZONE_ZONE_KDS_ROOT_CA_FILE=/tmp/ca.crt \
  KUMA_MODE=zone \
  KUMA_MULTIZONE_ZONE_GLOBAL_ADDRESS=grpcs://<CROSS_ZONE_KUMA_CP_DNS_NAME>:5685 \
  kuma-cp run

Authentication

Define firewall rules on the global control plane to only accept connections from known IPs of the zone control planes.

Third-party extensions, cloud implementations or commercial offerings may be extending the authentication support.