Kong custom plugin distribution with KongPluginInstallation

Kong Gateway Operator can install Kong custom plugins packaged as container images. This guide shows how to package, install, and use a custom plugin in Kong Gateway instances managed by the Kong Gateway Operator.

helm repo add kong https://charts.konghq.com
helm repo update kong

helm upgrade --install kgo kong/gateway-operator -n kong-system --create-namespace  \
  --set image.tag=1.6 \
  --set env.ENABLE_CONTROLLER_KONGPLUGININSTALLATION=true

kubectl -n kong-system wait --for=condition=Available=true --timeout=120s deployment/kgo-gateway-operator-controller-manager

Create a custom plugin

For details about plugin development for Kong Gateway, see the Plugin Development guide.

1. Create a directory with plugin code.

   If you already have a real plugin, you can skip this step.

    mkdir myheader
    echo 'local MyHeader = {}
   
    MyHeader.PRIORITY = 1000
    MyHeader.VERSION = "1.0.0"
   
    function MyHeader:header_filter(conf)
      -- do custom logic here
      kong.response.set_header("myheader", conf.header_value)
    end
   
    return MyHeader
    ' > myheader/handler.lua
   
    echo 'return {
      name = "myheader",
      fields = {
        { config = {
            type = "record",
            fields = {
              { header_value = { type = "string", default = "roar", }, },
            },
        }, },
      }
    }
    ' > myheader/schema.lua
   

   The directory should now look like this:

    tree myheader
   

    myheader
    ├── handler.lua
    └── schema.lua
   
    0 directories, 2 files
   

2. Build a container image that includes the plugin code.

   Plugin-related files should be at the root of the image, so the Dockerfile for the plugin would look like this:

   echo 'FROM scratch
   
   COPY myheader /
   ' > Dockerfile
   

   where myheader is a directory that contains handler.lua and schema.lua.

   Build the image:

   docker build -t myheader:1.0.0 .
   

   Next, push the image to a public or private registry available to the Kubernetes cluster where Kong Gateway Operator is running.

   docker tag myheader:1.0.0 <YOUR-REGISTRY-ADDRESS>/myheader:1.0.0
   docker push <YOUR-REGISTRY-ADDRESS>/myheader:1.0.0
   

   In this example, the plugin is available in the public registry (Docker Hub) as kong/plugin-example:1.0.0. The following steps use the same source.

3. Install the plugin using the KongPluginInstallation resource. This resource makes the plugin available for instances of Kong Gateway resources.

   echo '
   kind: KongPluginInstallation
   apiVersion: gateway-operator.konghq.com/v1alpha1
   metadata:
     name: custom-plugin-myheader
   spec:
     image: kong/plugin-example:1.0.0
   ' | kubectl apply -f -
   

   Learn more about the KongPluginInstallation resource in the CRD reference documentation.

   Verify that the plugin is fetched and available by examining the status of the KongPluginInstallation resource:

   kubectl get kongplugininstallations.gateway-operator.konghq.com -o jsonpath-as-json='{.items[*].status}'
   

   The output should look like this:

   [
     {
         "conditions": [
             {
                 "lastTransitionTime": "2024-10-09T19:39:39Z",
                 "message": "plugin successfully saved in cluster as ConfigMap",
                 "observedGeneration": 1,
                 "reason": "Ready",
                 "status": "True",
                 "type": "Accepted"
             }
         ],
         "underlyingConfigMapName": "custom-plugin-myheader-hnzf9"
     }
   ]
   

   In case of problems, respective conditions or respective resources will provide more information.

   The KongPluginInstallation resource creates a ConfigMap with the plugin content. Additional ConfigMaps are created when a plugin is referenced by other resources. The operator automatically manages the lifecycle of all these ConfigMaps.

4. Make the plugin available in a Gateway resource by referencing it in the spec.dataPlaneOptions.spec.pluginsToInstall field of the GatewayConfiguration resource. Plugins can be referenced across namespaces without any additional configuration.

   echo '
   kind: GatewayConfiguration
   apiVersion: gateway-operator.konghq.com/v1beta1
   metadata:
     name: kong
     namespace: default
   spec:
     dataPlaneOptions:
       deployment:
         replicas: 2
         podTemplateSpec:
           spec:
             containers:
               - name: proxy
                 image: kong/kong-gateway:3.10.0.1
       pluginsToInstall:
         - name: custom-plugin-myheader
     controlPlaneOptions:
       deployment:
         podTemplateSpec:
           spec:
             containers:
               - name: controller
                 image: kong/kubernetes-ingress-controller:3.4.4
   ---
   apiVersion: gateway.networking.k8s.io/v1
   kind: GatewayClass
   metadata:
     name: kong
   spec:
     controllerName: konghq.com/gateway-operator
     parametersRef:
       group: gateway-operator.konghq.com
       kind: GatewayConfiguration
       name: kong
       namespace: default
   ---
   apiVersion: gateway.networking.k8s.io/v1
   kind: Gateway
   metadata:
     name: kong
     namespace: default
   spec:
     gatewayClassName: kong
     listeners:
       - name: http
         protocol: HTTP
         port: 80
   ' | kubectl apply -f -
   

5. Deploy an example service and expose it by configuring HTTPRoute with the custom plugin:

   kubectl apply -f https://docs.konghq.com/assets/kubernetes-ingress-controller/examples/echo-service.yaml
   

   Next, add the HTTPRoute with the custom plugin. The configuration of the plugin is provided with the KongPlugin CRD, where the field plugin is set to the name of the KongPluginInstallation resource.

   echo '
   apiVersion: configuration.konghq.com/v1
   kind: KongPlugin
   metadata:
     name: myheader
   plugin: custom-plugin-myheader
   config:
     header_value: my-first-plugin
   ---
   apiVersion: gateway.networking.k8s.io/v1
   kind: HTTPRoute
   metadata:
     name: httproute-echo
     namespace: default
     annotations:
       konghq.com/strip-path: "true"
       konghq.com/plugins: myheader
   spec:
     parentRefs:
       - name: kong
     rules:
       - matches:
           - path:
               type: PathPrefix
               value: /echo
         backendRefs:
           - name: echo
             kind: Service
             port: 1027
   ' | kubectl apply -f -
   

   This example HTTPRoute routes requests to the echo service and applies the plugin to responses.

6. Ensure that everything is up and running and make a request to the service.

   To call the API, fetch the PROXY_IP for the Gateway:

   export PROXY_IP=$(kubectl get gateway kong -o jsonpath='{.status.addresses[0].value}')
   

   Make a curl request to the service:

   curl -I $PROXY_IP/echo
   

   The response should include the custom header set by the plugin:

   HTTP/1.1 200 OK
   Content-Type: text/plain; charset=utf-8
   Content-Length: 61
   Connection: keep-alive
   Date: Wed, 09 Oct 2024 20:21:23 GMT
   Server: kong/3.8.0.0-enterprise-edition
   myheader: my-first-plugin
   X-Kong-Upstream-Latency: 3
   X-Kong-Proxy-Latency: 0
   Via: 1.1 kong/3.8.0.0-enterprise-edition
   X-Kong-Request-Id: 6eec26150170fe3547bc1a4a20e93d74
   

Troubleshooting

Everything needed to debug problematic KongPluginInstallations can be found in the status of the respective resource.

For instance, if the plugin referenced in the GatewayConfiguration does not exist, examine the status of the Gateway resource:

kubectl get gateway -o jsonpath-as-json='{.items[*].status.conditions}'

Pay attention to the status of the resources:

[
    [
        {
            "lastTransitionTime": "2024-10-10T15:58:52Z",
            "message": "All listeners are accepted.",
            "observedGeneration": 1,
            "reason": "Accepted",
            "status": "True",
            "type": "Accepted"
        },
        {
            "lastTransitionTime": "2024-10-10T15:58:52Z",
            "message": "There are other conditions that are not yet ready",
            "observedGeneration": 1,
            "reason": "Pending",
            "status": "False",
            "type": "Programmed"
        },
        {
            "lastTransitionTime": "2024-10-10T15:58:52Z",
            "message": "Waiting for the resource to become ready",
            "observedGeneration": 1,
            "reason": "WaitingToBecomeReady",
            "status": "False",
            "type": "DataPlaneReady"
        },
        {
            "lastTransitionTime": "2024-10-10T15:58:52Z",
            "message": "Resource has been created",
            "observedGeneration": 1,
            "reason": "ResourceCreatedOrUpdated",
            "status": "False",
            "type": "ControlPlaneReady"
        }
    ]
]

In this case, the DataPlane is not ready. You can now look at the DataPlane’s status:

kubectl get dataplanes.gateway-operator.konghq.com -o jsonpath-as-json='{.items[*].status.conditions}'

This provides more information about the specific resource:

[
    [
        {
            "lastTransitionTime": "2024-10-10T15:58:54Z",
            "message": "referenced KongPluginInstallation default/additional-custom-plugin-4 not found",
            "observedGeneration": 1,
            "reason": "ReferencedResourcesNotAvailable",
            "status": "False",
            "type": "Ready"
        }
    ]
]

In this case, you can see that KongPluginInstallation in the namespace default with the name additional-custom-plugin-4 is not found.

Following the approach described above, you can troubleshoot any issues related to the KongPluginInstallation resource. Sometimes troubleshooting may lead to examining the status field of the KongPluginInstallation resource itself (for example, referencing a non-existent image or an image that doesn’t contain a valid plugin).