You are browsing documentation for an older version. See the latest documentation here.

Kubernetes annotations and labels

This page provides a complete list of all the annotations you can specify when you run Kong Mesh in Kubernetes mode.

Labels

kuma.io/sidecar-injection

Enable or disable sidecar injection.

Example

Used on the namespace it will inject the sidecar in all pods created in the namespace:

apiVersion: v1
kind: Namespace
metadata:
 name: default
 labels:
   kuma.io/sidecar-injection: enabled
[...]

Used on a deployment using pod template it will inject the sidecar in all pods managed by this deployment:

apiVersion: v1
kind: Deployment
metadata:
  name: my-deployment
spec:
  template:
    metadata:
      labels:
        kuma.io/sidecar-injection: enabled
[...]

Labeling pods or deployments will take precedence on the namespace annotation.

kuma.io/system-namespace

This label is used to indicate the Namespace that Kuma stores its secrets in. It’s automatically set on the Namespace the Helm chart is installed into by a Job started by Helm.

Annotations

kuma.io/mesh

Associate Pods with a particular Mesh. Annotation value must be the name of a Mesh resource.

Example

It can be used on an entire namespace:

apiVersion: v1
kind: Namespace
metadata:
 name: default
 annotations:
   kuma.io/mesh: default
[...]

It can be used on a pod:

apiVersion: v1
kind: Pod
metadata:
 name: backend
 annotations:
   kuma.io/mesh: default
[...]

Annotating pods or deployments will take precedence on the namespace annotation.

kuma.io/gateway

Lets you specify the Pod should run in gateway mode. Inbound listeners are not generated.

Example

apiVersion: apps/v1
kind: Deployment
metadata:
  name: gateway
spec:
  selector:
    matchLabels:
      app: gateway
  template:
    metadata:
      labels:
        app: gateway
      annotations:
        kuma.io/gateway: enabled
[...]

kuma.io/ingress

Marks the Pod as the Zone Ingress. Needed for multizone communication – provides the entry point for traffic from other zones.

Example

apiVersion: v1
kind: Pod
metadata:
 name: zone-ingress
 annotations:
   kuma.io/ingress: enabled
[...]

kuma.io/ingress-public-address

Specifies the public address for Ingress. If not provided, Kong Mesh picks the address from the Ingress Service.

Example

apiVersion: v1
kind: Pod
metadata:
 name: zone-ingress
 annotations:
   kuma.io/ingress: enabled
   kuma.io/ingress-public-address: custom-address.com
[...]

kuma.io/ingress-public-port

Specifies the public port for Ingress. If not provided, Kong Mesh picks the port from the Ingress Service.

Example

apiVersion: v1
kind: Pod
metadata:
 name: zone-ingress
 annotations:
   kuma.io/ingress: enabled
   kuma.io/ingress-public-port: "1234"
[...]

kuma.io/direct-access-services

Defines a comma-separated list of Services that can be accessed directly.

Example

apiVersion: v1
kind: Pod
metadata:
  name: example
  annotations:
    kuma.io/direct-access-services: test-app_playground_svc_80,test-app_playground_svc_443
    kuma.io/transparent-proxying: enabled
    kuma.io/transparent-proxying-inbound-port: [...]
    kuma.io/transparent-proxying-outbound-port: [...]

When you provide this annotation, Kong Mesh generates a listener for each IP address and redirects traffic through a direct-access cluster that’s configured to encrypt connections.

These listeners are needed because transparent proxy and mTLS assume a single IP per cluster (for example, the ClusterIP of a Kubernetes Service). If you pass requests to direct IP addresses, Envoy considers them unknown destinations and manages them in passthrough mode – which means they’re not encrypted with mTLS. The direct-access cluster enables encryption anyway.

WARNING: You should specify this annotation only if you really need it. Generating listeners for every endpoint makes the xDS snapshot very large.

kuma.io/virtual-probes

Enables automatic converting of HttpGet probes to virtual probes. The virtual probe is served on a sub-path of the insecure port specified with kuma.io/virtual-probes-port – for example, :8080/health/readiness -> :9000/8080/health/readiness, where 9000 is the value of the kuma.io/virtual-probes-port annotation.

Example

apiVersion: v1
kind: Pod
metadata:
  name: example
  annotations:
    kuma.io/virtual-probes: enabled
    kuma.io/virtual-probes-port: "9000"
[...]

kuma.io/virtual-probes-port

Specifies the insecure port for listening on virtual probes.

kuma.io/sidecar-env-vars

Semicolon (;) separated list of environment variables for the Kong Mesh sidecar.

Example

apiVersion: v1
kind: Pod
metadata:
  name: example
  annotations:
    kuma.io/sidecar-env-vars: TEST1=1;TEST2=2

kuma.io/container-patches

Specifies the list of names of ContainerPatch resources to be applied on kuma-init and kuma-sidecar containers.

More information about how to use ContainerPatch you can find at Custom Container Configuration.

Example

It can be used on a resource describing workload (i.e. Deployment, DaemonSet or Pod):

apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: kong-mesh-system
  name: example
spec:
  replicas: 1
  selector:
    matchLabels:
      app: example
  template:
    metadata:
      labels:
        app: example
      annotations:
        kuma.io/container-patches: container-patch-1,container-patch-2
    spec: [...]

prometheus.metrics.kuma.io/port

Lets you override the Mesh-wide default port that Prometheus should scrape metrics from.

Example

apiVersion: v1
kind: Pod
metadata:
  name: example
  annotations:
    prometheus.metrics.kuma.io/port: "1234"

prometheus.metrics.kuma.io/path

Lets you override the Mesh-wide default path that Prometheus should scrape metrics from.

Example

apiVersion: v1
kind: Pod
metadata:
  name: example
  annotations:
    prometheus.metrics.kuma.io/path: "/custom-metrics"

kuma.io/builtindns

Tells the sidecar to use its builtin DNS server.

Example

apiVersion: v1
kind: Pod
metadata:
  name: example
  annotations:
    kuma.io/builtindns: enabled

kuma.io/builtindnsport

Port the builtin DNS server should listen on for DNS queries.

Example

apiVersion: v1
kind: Pod
metadata:
  name: example
  annotations:
    kuma.io/builtindns: enabled
    kuma.io/builtindnsport: "15053"

kuma.io/ignore

A boolean to mark a resource as ignored by Kong Mesh. It currently only works for services. This is useful when transitioning to Kong Mesh or to temporarily ignore some entities.

Example

apiVersion: v1
kind: Service
metadata:
  name: example
  annotations:
    kuma.io/ignore: "true"

traffic.kuma.io/exclude-inbound-ports

List of inbound ports to exclude from traffic interception by the Kong Mesh sidecar.

Example

apiVersion: v1
kind: Pod
metadata:
  name: example
  annotations:
    traffic.kuma.io/exclude-inbound-ports: "1234,1235"

traffic.kuma.io/exclude-outbound-ports

List of outbound ports to exclude from traffic interception by the Kong Mesh sidecar.

Example

apiVersion: v1
kind: Pod
metadata:
  name: example
  annotations:
    traffic.kuma.io/exclude-outbound-ports: "1234,1235"

kuma.io/transparent-proxying-experimental-engine

Enable or disable experimental transparent proxy engine on Pod. Default is disabled.

Example

apiVersion: v1
kind: Pod
metadata:
  name: example
  annotations:
    kuma.io/transparent-proxying-experimental-engine: enabled

kuma.io/envoy-admin-port

Specifies the port for Envoy Admin API. If not set, default admin port 9901 will be used.

Example

apiVersion: v1
kind: Pod
metadata:
  name: example
  annotations:
    kuma.io/envoy-admin-port: "8801"

kuma.io/envoy-log-level

Specifies the log level for Envoy system logs to enable. The available log levels are trace, debug, info, warning/warn, error, critical, off. The default is info.

Example

apiVersion: v1
kind: Pod
metadata:
  name: example
  annotations:
    kuma.io/envoy-log-level: "warning"

kuma.io/service-account-token-volume

Volume (specified in the pod spec) containing a service account token for Kong Mesh to inject into the sidecar.

Example

apiVersion: v1
kind: Pod
metadata:
  name: example
  annotations:
    kuma.io/service-account-token-volume: "token-vol"
spec:
  automountServiceAccountToken: false
  serviceAccount: example
  containers:
    - image: busybox
      name: busybox
  volumes:
    - name: token-vol
      projected:
        sources:
          - serviceAccountToken:
              expirationSeconds: 7200
              path: token
              audience: "https://kubernetes.default.svc"
          - configMap:
              items:
                - key: ca.crt
                  path: ca.crt
              name: kube-root-ca.crt
          - downwardAPI:
              items:
                - fieldRef:
                    apiVersion: v1
                    fieldPath: metadata.namespace
                  path: namespace

kuma.io/transparent-proxying-reachable-services

A comma separated list of kuma.io/service to indicate which services this communicates with. For more details see the reachable services docs.

Example

apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-app
  namespace: kuma-example
spec:
  ...
  template:
    metadata:
      ...
      annotations:
        # a comma separated list of kuma.io/service values
        kuma.io/transparent-proxying-reachable-services: "redis_kuma-demo_svc_6379,elastic_kuma-demo_svc_9200"
    spec:
      containers:
        ...

kuma.io/transparent-proxying-ebpf

When transparent proxy is installed with ebpf mode, you can disable it for particular workloads if necessary.

For more details see the transparent proxying with ebpf docs.

Example

apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-app
  namespace: kuma-example
spec:
  [...]
  template:
    metadata:
      [...]
      annotations:
        kuma.io/transparent-proxying-ebpf: disabled
    spec:
      containers:
        [...]

kuma.io/transparent-proxying-ebpf-bpf-fs-path

Path to BPF FS if different than default (/sys/fs/bpf)

For more details see the transparent proxying with ebpf docs.

Example

apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-app
  namespace: kuma-example
spec:
  [...]
  template:
    metadata:
      [...]
      annotations:
        kuma.io/transparent-proxying-ebpf-bpf-fs-path: /custom/bpffs/path
    spec:
      containers:
        [...]

kuma.io/transparent-proxying-ebpf-cgroup-path

cgroup2 path if different than default (/sys/fs/cgroup)

For more details see the transparent proxying with ebpf docs.

Example

apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-app
  namespace: kuma-example
spec:
  [...]
  template:
    metadata:
      [...]
      annotations:
        kuma.io/transparent-proxying-ebpf-cgroup-path: /custom/cgroup2/path
    spec:
      containers:
        [...]

kuma.io/transparent-proxying-ebpf-programs-source-path

Custom path for ebpf programs to be loaded when installing transparent proxy

For more details see the transparent proxying with ebpf docs.

Example

apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-app
  namespace: kuma-example
spec:
  [...]
  template:
    metadata:
      [...]
      annotations:
        kuma.io/transparent-proxying-ebpf-programs-source-path: /custom/ebpf/programs/source/path
    spec:
      containers:
        [...]

kuma.io/transparent-proxying-ebpf-tc-attach-iface

Name of the network interface which should be used to attach to it TC-related eBPF programs. By default Kong Mesh will use first, non-loopback interface it’ll find.

For more details see the transparent proxying with ebpf docs.

Example

apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-app
  namespace: kuma-example
spec:
  [...]
  template:
    metadata:
      [...]
      annotations:
        kuma.io/transparent-proxying-ebpf-tc-attach-iface: eth3
    spec:
      containers:
        [...]

prometheus.metrics.kuma.io/aggregate-<name>-enabled

Define if kuma-dp should scrape metrics from the application that has been defined in the Mesh configuration. Default value: true. For more details see the applications metrics docs

apiVersion: v1
kind: Pod
metadata:
  name: example
  annotations:
    prometheus.metrics.kuma.io/aggregate-app-enabled: "false"
spec: ...

prometheus.metrics.kuma.io/aggregate-<name>-path

Define path, which kuma-dp sidecar has to scrape for prometheus metrics. Default value: /metrics. For more details see the applications metrics docs

Example

apiVersion: v1
kind: Pod
metadata:
  name: example
  annotations:
    prometheus.metrics.kuma.io/aggregate-app-path: "/stats"
spec: ...

prometheus.metrics.kuma.io/aggregate-<name>-port

Define port, which kuma-dp sidecar has to scrape for prometheus metrics. For more details see the applications metrics docs

Example

apiVersion: v1
kind: Pod
metadata:
  name: example
  annotations:
    prometheus.metrics.kuma.io/aggregate-app-port: "1234"
spec: ...

kuma.io/transparent-proxying-inbound-v6-port

Define the port to use for IPv6 traffic. To turn off IPv6 set this to 0.

Example

apiVersion: v1
kind: Pod
metadata:
  name: example
  annotations:
    kuma.io/transparent-proxying-inbound-v6-port: "0"
spec: ...

kuma.io/sidecar-drain-time

Allows specifying drain time of Kong Mesh DP sidecar. The default value is 30s. The default could be changed using the control-plane configuration or KUMA_RUNTIME_KUBERNETES_INJECTOR_SIDECAR_CONTAINER_DRAIN_TIME env.

Example

apiVersion: v1
kind: Pod
metadata:
  name: example
  annotations:
    kuma.io/sidecar-drain-time: "10s"
spec: ...