Configure transparent proxying
In order to automatically intercept traffic from and to a service through a kuma-dp
data plane proxy instance, Kong Mesh utilizes a transparent proxying using iptables
.
Transparent proxying helps with a smoother rollout of a Service Mesh to a current deployment by preserving existing service naming and as the result - avoid changes to the application code.
Kubernetes
On Kubernetes kuma-dp
leverages transparent proxying automatically via iptables
installed with kuma-init
container or CNI.
All incoming and outgoing traffic is automatically intercepted by kuma-dp
without having to change the application code.
Kong Mesh integrates with a service naming provided by Kubernetes DNS as well as providing its own Kong Mesh DNS for multi-zone service naming.
Universal
On Universal kuma-dp
leverages the data plane proxy specification associated to it for receiving incoming requests on a pre-defined port.
In order to enable transparent-proxy the Zone Control Plane must exist on a seperate server. Running the Zone Control Plane with Postgres does not function with transparent-proxy on the same machine.
There are several advantages for using transparent proxying in universal mode:
- Simpler Dataplane resource, as the
outbound
section becomes obsolete and can be skipped. - Universal service naming with
.mesh
DNS domain instead of explicit outbound likehttps://localhost:10001
. - Support for hostnames of your choice using VirtualOutbounds that lets you preserve existing service naming.
- Better service manageability (security, tracing).
Setting up the service host
Prerequisites:
-
kuma-dp
,envoy
, andcoredns
must run on the worker node – that is, the node that runs your service mesh workload. -
coredns
must be in the PATH so thatkuma-dp
can access it.- You can also set the location with the
--dns-coredns-path
flag tokuma-dp
.
- You can also set the location with the
Kong Mesh comes with kumactl
executable which can help us to prepare the host. Due to the wide variety of Linux setup options, these steps may vary and may need to be adjusted for the specifics of the particular deployment.
The host that will run the kuma-dp
process in transparent proxying mode needs to be prepared with the following steps executed as root
:
-
Create a new dedicated user on the machine.
useradd -u 5678 -U kuma-dp
-
Redirect all the relevant inbound, outbound and DNS traffic to the Kong Mesh data plane proxy (if you’re running any other services on that machine you need to adjust the comma separated lists of
--exclude-inbound-ports
and--exclude-outbound-ports
accordingly).kumactl install transparent-proxy \ --kuma-dp-user kuma-dp \ --redirect-dns \ --exclude-inbound-ports 22
Please note that this command will change the host’s
iptables
rules.The command excludes port
22
, so you can SSH to the machine withoutkuma-dp
running.
The changes won’t persist over restarts. You need to either add this command to your start scripts or use firewalld.
Data plane proxy resource
In transparent proxying mode, the Dataplane
resource should omit the networking.outbound
section and use networking.transparentProxying
section instead.
type: Dataplane
mesh: default
name: {{ name }}
networking:
address: {{ address }}
inbound:
- port: {{ port }}
tags:
kuma.io/service: demo-client
transparentProxying:
redirectPortInbound: 15006
redirectPortOutbound: 15001
The ports illustrated above are the default ones that kumactl install transparent-proxy
will set. These can be changed using the relevant flags to that command.
Invoking the Kong Mesh data plane
It is important that the
kuma-dp
process runs with the same system user that was passed tokumactl install transparent-proxy --kuma-dp-user
. The service itself should run with any other user thankuma-dp
. Otherwise, it won’t be able to leverage transparent proxying.
When systemd is used, this can be done with an entry User=kuma-dp
in the [Service]
section of the service file.
When starting kuma-dp
with a script or some other automation instead, we can use runuser
with the aforementioned yaml resource as follows:
runuser -u kuma-dp -- \
/usr/bin/kuma-dp run \
--cp-address=https://<IP or hostname of CP>:5678 \
--dataplane-token-file=/kuma/token-demo \
--dataplane-file=/kuma/dpyaml-demo \
--dataplane-var name=dp-demo \
--dataplane-var address=<IP of VM> \
--dataplane-var port=<Port of the service> \
--binary-path /usr/local/bin/envoy
You can now reach the service on the same IP and port as before installing transparent proxy, but now the traffic goes through Envoy. At the same time, you can now connect to services using Kong Mesh DNS.
firewalld support
If you run firewalld
to manage firewalls and wrap iptables, add the --store-firewalld
flag to kumactl install transparent-proxy
. This persists the relevant rules across host restarts. The changes are stored in /etc/firewalld/direct.xml
. There is no uninstall command for this feature.
Upgrades
Before upgrading to the next version of Kong Mesh, it’s best to clean existing iptables
rules and only then replace the kumactl
binary.
You can clean the rules either by restarting the host or by running following commands
Executing these commands will remove all
iptables
rules, including those created by Kong Mesh and any other applications or services.
iptables --table nat --flush
iptables --table raw --flush
ip6tables --table nat --flush
ip6tables --table raw --flush
iptables --table nat --delete-chain
iptables --table raw --delete-chain
ip6tables --table nat --delete-chain
ip6tables --table raw --delete-chain
In the future release, kumactl
will ship with uninstall
command.
Configuration
Intercepted traffic
Reachable Services
By default, every data plane proxy in the mesh follows every other data plane proxy. This may lead to performance problems in larger deployments of the mesh. It is highly recommended to define a list of services that your service connects to.
Transparent Proxy with eBPF (experimental)
Starting from Kong Mesh 2.0 you can set up transparent proxy to use eBPF instead of iptables.
To use Transparent Proxy with eBPF your environment has to use
Kernel >= 5.7
and havecgroup2
available