You are browsing unreleased documentation.
See the latest documentation here.
Configuring your Mesh and multi-tenancy
This resource describes a very important concept in Kong Mesh, and that is the ability of creating multiple isolated service meshes within the same Kong Mesh cluster which in turn make Kong Mesh a very simple and easy project to operate in environments where more than one mesh is required based on security, segmentation or governance requirements.
Typically, we would want to create a Mesh
per line of business, per team, per application or per environment or for any other reason. Typically multiple meshes are being created so that a service mesh can be adopted by an organization with a gradual roll-out that doesn’t require all the teams and their applications to coordinate with each other, or as an extra layer of security and segmentation for our services so that - for example - policies applied to one Mesh
do not affect another Mesh
.
Mesh
is the parent resource of every other resource in Kong Mesh, including:
In order to use Kong Mesh at least one Mesh
must exist, and there is no limit to the number of Meshes that can be created. When a data plane proxy connects to the control plane (kuma-cp
) it specifies to what Mesh
resource it belongs: a data plane proxy can only belong to one Mesh
at a time.
When starting a new Kong Mesh cluster from scratch a default
Mesh is being created automatically.
Besides the ability of being able to create virtual service mesh, a Mesh
resource will also be used for:
To support cross-mesh communication an intermediate API Gateway must be used. Kong Mesh checkout Kong Mesh’s builtin gateway to set this up.
Previously, observability and locality awareness were configured within the Mesh
object.
However, for enhanced flexibility and granular control, these configurations have been extracted into separate policies:
MeshAccessLog
, MeshTrace
and MeshMetric
for observability, and MeshLoadBalancingStrategy
for locality awareness.
This separation allows for more fine-grained adjustments of each aspect, ensuring that observability and locality awareness are tailored to specific requirements.
Usage
The easiest way to create a Mesh
is to specify its name
. The name of a Mesh must be unique.
apiVersion: kuma.io/v1alpha1
kind: Mesh
metadata:
name: default
We will apply the configuration with kubectl apply -f [..]
.
We will apply the configuration with kumactl apply -f [..]
or via the HTTP API.
Creating resources in a Mesh
It is possible to determine to what Mesh
other resources belong to in the following ways.
Data plane proxies
Every time we start a data plane proxy, we need to specify to what Mesh
it belongs, this can be done in the following way:
By using the kuma.io/mesh
annotation in a Deployment
, like:
apiVersion: apps/v1
kind: Deployment
metadata:
name: example-app
namespace: kuma-example
spec:
...
template:
metadata:
...
annotations:
# indicate to Kong Mesh what is the Mesh that the data plane proxy belongs to
kuma.io/mesh: default
spec:
containers:
...
A Mesh
may span multiple Kubernetes namespaces. Any Kong Mesh resource in the cluster which
specifies a particular Mesh
will be part of that Mesh
.
By using the -m
or --mesh
argument when running kuma-dp
, for example:
kuma-dp run \
--name=backend-1 \
--mesh=default \
--cp-address=https://127.0.0.1:5678 \
--dataplane-token-file=/tmp/kuma-dp-backend-1-token
You can control which data plane proxies are allowed to join the mesh using mesh constraints.
Policies
When creating new Policies we also must specify to what Mesh
they belong. This can be done in the following way:
By using the kuma.io/mesh
label, like:
apiVersion: kuma.io/v1alpha1
kind: MeshHTTPRoute
metadata:
name: route-1
namespace: kong-mesh-system
labels:
kuma.io/mesh: default # indicate to Kong Mesh what is the Mesh that the resource belongs to
spec:
...
Kong Mesh consumes all Policies on the cluster and joins each to an individual Mesh
, identified by this property.
By using the mesh
property, like:
type: MeshHTTPRoute
name: route-1
mesh: default # indicate to Kong Mesh what is the Mesh that the resource belongs to
...
Skipping default resource creation
By default, to help users get started we create the following default policies:
apiVersion: kuma.io/v1alpha1
kind: MeshTimeout
metadata:
name: mesh-gateways-timeout-all-default
namespace: kong-mesh-system
labels:
kuma.io/mesh: default
spec:
targetRef:
kind: MeshSubset
proxyTypes:
- Gateway
to:
- targetRef:
kind: Mesh
default:
idleTimeout: 1h
http:
streamIdleTimeout: 5s
from:
- targetRef:
kind: Mesh
default:
idleTimeout: 5m
http:
streamIdleTimeout: 5s
requestHeadersTimeout: 500ms
---
apiVersion: kuma.io/v1alpha1
kind: MeshTimeout
metadata:
name: mesh-timeout-all-default
namespace: kong-mesh-system
labels:
kuma.io/mesh: default
spec:
targetRef:
kind: Mesh
proxyTypes:
- Sidecar
to:
- targetRef:
kind: Mesh
default:
connectionTimeout: 5s
idleTimeout: 1h
http:
requestTimeout: 15s
streamIdleTimeout: 30m
from:
- targetRef:
kind: Mesh
default:
connectionTimeout: 10s
idleTimeout: 2h
http:
requestTimeout: 0s
streamIdleTimeout: 1h
maxStreamDuration: 0s
---
apiVersion: kuma.io/v1alpha1
kind: MeshRetry
metadata:
name: mesh-retry-all-default
namespace: kong-mesh-system
labels:
kuma.io/mesh: default
spec:
targetRef:
kind: Mesh
to:
- targetRef:
kind: Mesh
default:
tcp:
maxConnectAttempt: 5
http:
numRetries: 5
perTryTimeout: 16s
backOff:
baseInterval: 25ms
maxInterval: 250ms
grpc:
numRetries: 5
perTryTimeout: 16s
backOff:
baseInterval: 25ms
maxInterval: 250ms
---
apiVersion: kuma.io/v1alpha1
kind: MeshCircuitBreaker
metadata:
name: mesh-circuit-breaker-all-default
namespace: kong-mesh-system
labels:
kuma.io/mesh: default
spec:
targetRef:
kind: Mesh
to:
- targetRef:
kind: Mesh
default:
connectionLimits:
maxConnections: 1024
maxPendingRequests: 1024
maxRetries: 3
maxRequests: 1024
type: MeshTimeout
mesh: default
name: mesh-gateways-timeout-all-default
spec:
targetRef:
kind: MeshSubset
proxyTypes:
- Gateway
to:
- targetRef:
kind: Mesh
default:
idleTimeout: 1h
http:
streamIdleTimeout: 5s
from:
- targetRef:
kind: Mesh
default:
idleTimeout: 5m
http:
streamIdleTimeout: 5s
requestHeadersTimeout: 500ms
---
type: MeshTimeout
mesh: default
name: mesh-timeout-all-default
spec:
targetRef:
kind: Mesh
proxyTypes:
- Sidecar
to:
- targetRef:
kind: Mesh
default:
connectionTimeout: 5s
idleTimeout: 1h
http:
requestTimeout: 15s
streamIdleTimeout: 30m
from:
- targetRef:
kind: Mesh
default:
connectionTimeout: 10s
idleTimeout: 2h
http:
requestTimeout: 0s
streamIdleTimeout: 1h
maxStreamDuration: 0s
---
type: MeshRetry
mesh: default
name: mesh-retry-all-default
spec:
targetRef:
kind: Mesh
to:
- targetRef:
kind: Mesh
default:
tcp:
maxConnectAttempt: 5
http:
numRetries: 5
perTryTimeout: 16s
backOff:
baseInterval: 25ms
maxInterval: 250ms
grpc:
numRetries: 5
perTryTimeout: 16s
backOff:
baseInterval: 25ms
maxInterval: 250ms
---
type: MeshCircuitBreaker
mesh: default
name: mesh-circuit-breaker-all-default
spec:
targetRef:
kind: Mesh
to:
- targetRef:
kind: Mesh
default:
connectionLimits:
maxConnections: 1024
maxPendingRequests: 1024
maxRetries: 3
maxRequests: 1024
If you want to not have these policies be added on creation of the mesh set the configuration: skipCreatingInitialPolicies
:
apiVersion: kuma.io/v1alpha1
kind: Mesh
metadata:
name: default
spec:
skipCreatingInitialPolicies: ['*']
type: Mesh
name: default
skipCreatingInitialPolicies: ['*']
You can also skip creating the default mesh by setting the control-plane configuration: KUMA_DEFAULTS_SKIP_MESH_CREATION=true
.
All options