Customizing load-balancing behavior with KongUpstreamPolicy

In this guide, we will learn how to use KongUpstreamPolicy resource to control proxy load-balancing behavior.

Deploy an echo service

To proxy requests, you need an upstream application to send a request to. Deploying this echo server provides a simple application that returns information about the Pod it’s running in:

kubectl apply -f https://docs.konghq.com/assets/kubernetes-ingress-controller/examples/echo-service.yaml

The results should look like this:

service/echo created
deployment.apps/echo created

Deploy additional echo replicas

To demonstrate Kong’s load balancing functionality we need multiple echo Pods. Scale out the echo deployment.

kubectl scale --replicas 2 deployment echo

Add routing configuration

Create routing configuration to proxy /echo requests to the echo server:

echo "
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: echo
  annotations:
    konghq.com/strip-path: 'true'
spec:
  parentRefs:
  - name: kong
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /echo
    backendRefs:
    - name: echo
      kind: Service
      port: 1027
" | kubectl apply -f -

echo "
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: echo
  annotations:
    konghq.com/strip-path: 'true'
spec:
  ingressClassName: kong
  rules:
  - http:
      paths:
      - path: /echo
        pathType: ImplementationSpecific
        backend:
          service:
            name: echo
            port:
              number: 1027
" | kubectl apply -f -

The results should look like this:

httproute.gateway.networking.k8s.io/echo created

ingress.networking.k8s.io/echo created

Test the routing rule:

curl -i $PROXY_IP/echo

The results should look like this:

HTTP/1.1 200 OK
Content-Type: text/plain; charset=utf-8
Content-Length: 140
Connection: keep-alive
Date: Fri, 21 Apr 2023 12:24:55 GMT
X-Kong-Upstream-Latency: 0
X-Kong-Proxy-Latency: 1
Via: kong/3.2.2

Welcome, you are connected to node docker-desktop.
Running on Pod echo-7f87468b8c-tzzv6.
In namespace default.
With IP address 10.1.0.237.
...

If everything is deployed correctly, you should see the above response. This verifies that Kong Gateway can correctly route traffic to an application running inside Kubernetes.

Use KongUpstreamPolicy with a Service resource

By default, Kong will round-robin requests between upstream replicas. If you run curl -s $PROXY_IP/echo | grep "Pod" repeatedly, you should see the reported Pod name alternate between two values.

You can configure the Kong upstream associated with the Service to use a different load balancing strategy, such as consistently sending requests to the same upstream based on a header value (please see the KongUpstreamPolicy reference for the full list of supported algorithms and their configuration options).

To modify these behaviours, let’s first create a KongUpstreamPolicy resource defining the new behaviour:

echo '
apiVersion: configuration.konghq.com/v1beta1
kind: KongUpstreamPolicy
metadata:
  name: sample-customization
spec:
  algorithm: consistent-hashing
  hashOn:
    header: x-lb
  hashOnFallback:
    input: ip
  ' | kubectl apply -f -

kongupstreampolicy.configuration.konghq.com/test created

Now, let’s associate this KongUpstreamPolicy resource with our Service resource using the konghq.com/upstream-policy annotation.

kubectl patch service echo -p '{"metadata":{"annotations":{"konghq.com/upstream-policy":"sample-customization"}}}'

service/echo patched

With consistent hashing and client IP fallback, sending repeated requests without any x-lb header now sends them to the same Pod:

for n in {1..5}; do curl -s $PROXY_IP/echo | grep "Pod"; done

Running on Pod echo-965f7cf84-frpjc.
Running on Pod echo-965f7cf84-frpjc.
Running on Pod echo-965f7cf84-frpjc.
Running on Pod echo-965f7cf84-frpjc.
Running on Pod echo-965f7cf84-frpjc.

If you add the header, Kong hashes its value and distributes it to the same replica when using the same value:

for n in {1..3}; do
  curl -s $PROXY_IP/echo -H "x-lb: foo" | grep "Pod";
  curl -s $PROXY_IP/echo -H "x-lb: bar" | grep "Pod";
  curl -s $PROXY_IP/echo -H "x-lb: baz" | grep "Pod";
done

Running on Pod echo-965f7cf84-wlvw9.
Running on Pod echo-965f7cf84-frpjc.
Running on Pod echo-965f7cf84-wlvw9.
Running on Pod echo-965f7cf84-wlvw9.
Running on Pod echo-965f7cf84-frpjc.
Running on Pod echo-965f7cf84-wlvw9.
Running on Pod echo-965f7cf84-wlvw9.
Running on Pod echo-965f7cf84-frpjc.
Running on Pod echo-965f7cf84-wlvw9.

Increasing the replicas redistributes some subsequent requests onto the new replica:

kubectl scale --replicas 3 deployment echo

deployment.apps/echo patched

for n in {1..3}; do
  curl -s $PROXY_IP/echo -H "x-lb: foo" | grep "Pod";
  curl -s $PROXY_IP/echo -H "x-lb: bar" | grep "Pod";
  curl -s $PROXY_IP/echo -H "x-lb: baz" | grep "Pod";
done

Running on Pod echo-965f7cf84-5h56p.
Running on Pod echo-965f7cf84-5h56p.
Running on Pod echo-965f7cf84-wlvw9.
Running on Pod echo-965f7cf84-5h56p.
Running on Pod echo-965f7cf84-5h56p.
Running on Pod echo-965f7cf84-wlvw9.
Running on Pod echo-965f7cf84-5h56p.
Running on Pod echo-965f7cf84-5h56p.
Running on Pod echo-965f7cf84-wlvw9.

Kong’s load balancer doesn’t directly distribute requests to each of the Service’s Endpoints. It first distributes them evenly across a number of equal-size buckets. These buckets are then distributed across the available Endpoints according to their weight. For Ingresses, however, there is only one Service, and the controller assigns each Endpoint (represented by a Kong upstream target) equal weight. In this case, requests are evenly hashed across all Endpoints.

Gateway API HTTPRoute rules support distributing traffic across multiple Services. The rule can assign weights to the Services to change the proportion of requests an individual Service receives. In Kong’s implementation, all Endpoints of a Service have the same weight. Kong calculates a per-Endpoint upstream target weight such that the aggregate target weight of the Endpoints is equal to the proportion indicated by the HTTPRoute weight.

For example, say you have two Services with the following configuration:

• One Service has four Endpoints
• The other Service has two Endpoints
• Each Service has weight 50 in the HTTPRoute

The targets created for the two-Endpoint Service have double the weight of the targets created for the four-Endpoint Service (two weight 16 targets and four weight 8 targets). Scaling the four-Endpoint Service to eight would halve the weight of its targets (two weight 16 targets and eight weight 4 targets).

KongUpstreamPolicy can also configure upstream health checking behavior as well. See the KongUpstreamPolicy reference for the health check fields.