Kubernetes Gateway API

Kong Mesh supports Kubernetes Gateway API for configuring built-in gateway as well as traffic routing using the experimental GAMMA routing spec.

Installation

Kubernetes Gateway API is still beta, therefore Kong Mesh’s integration provides the same level of stability.

Gateway API Gateways aren’t supported in multi-zone. To use the builtin Gateway, you need to use the MeshGateway resources.

Install the Gateway API CRDs.

The Gateway API CRDs aren’t available in Kubernetes by default yet. You must first install the experimental release that includes those CRDs as well as the admission webhook.
Enable Gateway API support.
- With kumactl, use the --experimental-gatewayapi flag.
- With Helm, use the kuma.experimental.gatewayAPI=true value.

Gateways

Install the counter demo.

kumactl install demo | kubectl apply -f -

Add a Gateway.

The Gateway resource represents the proxy instance that handles traffic for a set of Gateway API routes.

Every Gateway refers to a GatewayClass. The GatewayClass represents the class of Gateway, in this case Kong Mesh’s builtin edge gateway, and points to a controller that should manage these Gateways. It can also hold additional configuration.

Standard install

Custom install

For Helm and kumactl installations, a GatewayClass named kuma is automatically installed if the Gateway API CRDs are present.

If you’ve installed Kong Mesh some other way, you can create your own GatewayClass using the controllerName: gateways.kuma.io/controller:

echo "apiVersion: gateway.networking.k8s.io/v1beta1
kind: GatewayClass
metadata:
  name: kuma
spec:
  controllerName: gateways.kuma.io/controller
" | kubectl apply -f -

echo "apiVersion: gateway.networking.k8s.io/v1beta1
kind: Gateway
metadata:
  name: kuma
  namespace: kuma-demo
spec:
  gatewayClassName: kuma
  listeners:
  - name: proxy
    port: 8080
    protocol: HTTP
" | kubectl apply -f -

When a user applies a Gateway resource, Kong Mesh automatically creates a Deployment of built-in gateways with a corresponding Service.

kubectl get pods -n kuma-demo
NAME                          READY   STATUS    RESTARTS   AGE
redis-59c9d56fc-6gcbc         2/2     Running   0          2m8s
demo-app-5845d6447b-v7npw     2/2     Running   0          2m8s
kuma-4j6wr-58998b5576-25wl6   1/1     Running   0          30s
   
kubectl get svc -n kuma-demo
NAME         TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
redis        ClusterIP      10.43.223.223   <none>        6379/TCP         3m27s
demo-app     ClusterIP      10.43.216.203   <none>        5000/TCP         3m27s
kuma-pfh4s   LoadBalancer   10.43.122.93    172.20.0.3    8080:30627/TCP   87s

The Gateway is now accessible using the external address 172.20.0.3:8080.

Add an HTTPRoute.

HTTPRoute resources contain a set of matching criteria for HTTP requests and upstream Services to route those requests to.

echo "apiVersion: gateway.networking.k8s.io/v1beta1
kind: HTTPRoute
metadata:
  name: echo
  namespace: kuma-demo
spec:
  parentRefs:
  - group: gateway.networking.k8s.io
    kind: Gateway
    name: kuma
    namespace: kuma-demo
  rules:
  - backendRefs:
    - kind: Service
      name: demo-app
      port: 5000
      weight: 1
    matches:
    - path:
        type: PathPrefix
        value: /
" | kubectl apply -f -

After creating an HTTPRoute, accessing / forwards a request to the demo app:

curl 172.20.0.3:8080/ -i

HTTP/1.1 200 OK
x-powered-by: Express
accept-ranges: bytes
cache-control: public, max-age=0
last-modified: Tue, 20 Oct 2020 17:16:41 GMT
etag: W/"2b91-175470350a8"
content-type: text/html; charset=UTF-8
content-length: 11153
date: Fri, 18 Mar 2022 11:33:29 GMT
x-envoy-upstream-service-time: 2
server: Kuma Gateway

<html>
<head>
...

TLS termination

Gateway API supports TLS termination by using standard kubernetes.io/tls Secrets.

Here is an example

apiVersion: v1
kind: Secret
metadata:
  name: secret-tls
  namespace: kuma-demo
type: kubernetes.io/tls
data:
  tls.crt: "MIIEOzCCAyO..." # redacted
  tls.key: "MIIEowIBAAKC..." # redacted

apiVersion: gateway.networking.k8s.io/v1beta1
kind: Gateway
metadata:
  name: kuma
  namespace: kuma-demo
spec:
  gatewayClassName: kuma
  listeners:
    - name: proxy
      port: 8080
      hostname: test.kuma.io
      protocol: HTTPS
      tls:
        certificateRefs:
          - name: secret-tls

Under the hood, Kong Mesh CP copies the Secret to kong-mesh-system namespace and converts it to Kong Mesh secret . It tracks all the changes to the secret and deletes it upon deletion of the original secret.

Customization

Gateway API provides the parametersRef field on GatewayClass.spec to provide additional, implementation-specific configuration to Gateways. When using Gateway API with Kong Mesh, you can refer to a MeshGatewayConfig resource:

apiVersion: gateway.networking.k8s.io/v1beta1
kind: GatewayClass
metadata:
  name: kuma
spec:
  controllerName: gateways.kuma.io/controller
  parametersRef:
    kind: MeshGatewayConfig
    group: kuma.io
    name: kuma

This resource has the same structure as the MeshGatewayInstance resource except that the tags field is optional. With a MeshGatewayConfig you can then customize the generated Service and Deployment resources.

Multi-mesh

You can specify a Mesh for Gateway and HTTPRoute resources by setting the kuma.io/mesh annotation Note that HTTPRoutes must also have the annotation to reference a Gateway from a non-default Mesh.

Cross-mesh

Cross-mesh gateways are supported with Gateway API. You’ll just need to create a corresponding GatewayClass pointing to a MeshGatewayConfig that sets crossMesh: true:

---
apiVersion: gateway.networking.k8s.io/v1beta1
kind: GatewayClass
metadata:
  name: kuma-cross-mesh
spec:
  controllerName: gateways.kuma.io/controller
  parametersRef:
    group: kuma.io
    kind: MeshGatewayConfig
    name: default-cross-mesh
---
apiVersion: kuma.io/v1alpha1
kind: MeshGatewayConfig
metadata:
  name: default-cross-mesh
spec:
  crossMesh: true

and then reference it in your Gateway:

apiVersion: gateway.networking.k8s.io/v1beta1
kind: Gateway
metadata:
  name: kuma
  namespace: default
spec:
  gatewayClassName: kuma-cross-mesh

Multi-zone

Gateway API isn’t supported with multi-zone deployments, use Kong Mesh’s MeshGateways/MeshGatewayRoutes instead.

Service to service routing

Kong Mesh also supports routing between services with HTTPRoute in conformance with the GAMMA specifications.

GAMMA is a Gateway API subproject focused on mesh implementations of Gateway API and extending the Gateway API resources to mesh use cases.

GAMMA itself as well as GAMMA in Kuma are experimental!

The key feature of HTTPRoute for mesh routing is specifying a Kubernetes Service as the parentRef, as opposed to a Gateway. All requests to this Service are then filtered and routed as specified in the HTTPRoute.

apiVersion: gateway.networking.k8s.io/v1beta1
kind: HTTPRoute
metadata:
  name: canary-demo-app
  namespace: kuma-demo
spec:
  parentRefs:
    - name: demo-app
      port: 5000
      kind: Service
  rules:
    - backendRefs:
        - name: demo-app-v1
          port: 5000
        - name: demo-app-v2
          port: 5000

The namespace of the HTTPRoute is key. If the route’s namespace and the parentRef’s namespace are identical, Kong Mesh applies the route to requests from all workloads. If the route’s namespace differs from its parentRef’s namespace, the HTTPRoute applies only to requests from workloads in the route’s namespace.

Remember to tag your Service ports with appProtocol: http to use them in an HTTPRoute!

Because of how Kuma maps resources at the moment, the combination of the HTTPRoutes name and namespace and the parent Service name and namespace must be no more than 249 characters.

How it works

Kong Mesh includes controllers that reconcile Gateway API CRDs and convert them into the corresponding Kong Mesh CRDs. This is why in the GUI, Kong Mesh MeshGateways/MeshGatewayRoutes/MeshHTTPRoutes are visible and not Kubernetes Gateway API resources.

Kubernetes Gateway API resources serve as the source of truth for Kong Mesh gateways and routes. Any edits to the corresponding Kong Mesh resources are overwritten.