Kubernetes Ingress Controller Deployment
The Kubernetes Ingress Controller is designed to be deployed in a variety of ways
based on uses-cases. This document explains various components involved
and choices one can make as per the specific use-case.
- Kubernetes Resources:
Various Kubernetes resouces required to run the Kubernetes Ingress Controller.
- Deployment options:
A high-level explanantion of choices that one should consider and customize
the deployment to best serve a specific use case.
The following resources are used to run the Kubernetes Ingress Controller:
These resources are created if the reference deployment manifests are used
to deploy the Kubernetes Ingress Controller.
The resources are explained below for users to gain an understanding of how
they are used, so that they can be tweaked as necessary for a specific use-case.
The Kubernetes Ingress Controller can be deployed in any namespace.
If Kubernetes Ingress Controller is being used to proxy traffic for all namespaces
in a Kubernetes cluster, which is generally the case,
it is recommended that it is installed in a dedicated
kong namespace but it is not required to do so.
The example deployments present in this repository automatically create a
namespace and deploy resources into that namespace.
The Ingress resource in Kubernetes is a fairly narrow and ambiguous API, and
doesn’t offer resources to describe the specifics of proxying.
To overcome this limitation, custom resources are used as an
“extension” to the existing Ingress API.
A few custom resources are bundled with the Kubernetes Ingress Controller to
configure settings that are specific to Kong and provide fine-grained control
over the proxying behavior.
Please refer to the custom resources
concept document for details.
The Kubernetes Ingress Controller communciates with the Kubernetes API-server and
dynamically configures Kong to automatically load balance across pods
of a service as any service is scaled in our out.
For this reason, it requires RBAC permissions to access resources stored
in Kubernetes object store.
It needs read permissions (get,list,watch)
on the following Kubernetes resources:
By default, the controller listens for events and above resources across
all namespaces and will need access to these resources at the cluster level
In addition to these, it needs:
- Create a ConfigMap and read and update ConfigMap for to facilitate
leader-election. Please read this document
for more details.
- Update permission on the Ingress resource to update the status of
the Ingress resource.
If the Ingress Controller is listening for events on a single namespace,
these permissions can be updated to restrict these permissions to a specific
In addition to these, it is necessary to create a
has the above permissions. The Ingress Controller Pod then has this
ServiceAccount association. This gives the Ingress Controller process
necessary authentication and authorization tokens to communicate with the
rbac.yaml contains the permissions
needed for the Ingress Controller to operate correctly.
Ingress Controller deployment
Kong Ingress deployment consists of the Ingress Controller deployed alongside
Kong. The deployment will be different depending on if a database is being
used or not.
The deployment(s) is the core which actually runs the Kubernetes Ingress Controller.
See the database section below for details.
Kong Proxy service
Once the Kubernetes Ingress Controller is deployed, one service is needed to
expose Kong outside the Kubernetes cluster so that it can receive all traffic
that is destined for the cluster and route it appropriately.
kong-proxy is a Kubernetes service which points to the Kong pods which are
capable of proxying request traffic. This service will be usually of type
LoadBalancer, however it is not required to be such.
The IP address of this service should be used to configure DNS records
of all the domains that Kong should be proxying, to route the traffic to Kong.
Database deployment and migration
The Kubernetes Ingress Controller can run with or without a database.
If a database is being deployed, then following resources are required:
StatefulSet which runs a Postgresql pod backed with a
to store Kong’s configuration.
- An internal
Service which resolves to the Postgresql pod. This ensures
that Kong can find the Postgresql instance using DNS inside
the Kubernetes cluster.
- A batch
Job to run schema migrations. This is required to be executed once
to install bootstrap Kong’s database schema.
Please note that on an any upgrade for Kong version, another
need to be created if the newer version contains any migrations.
To figure out if you should be using a database or not, please refer to the
database section below.
Following are the difference options to consider while deploying the
Kubernetes Ingress Controller for your specific use case:
Kubernetes Service Types
Once deployed, any Ingress Controller needs to be exposed outside the
Kubernetes cluster to start accepting external traffic.
Service abstraction is used to expose any application
to the rest of the cluster or outside the cluster.
If your Kubernetes cluster is running in a cloud environment, where
Load Balancers can be provisioned with relative ease, it is recommended
that you use a Service of type
LoadBalancer to expose Kong to the outside
world. For the Ingress Controller to function correctly, it is also required
that a L4 (or TCP) Load Balancer is used and not an L7 (HTTP(s)) one.
If your Kubernetes cluster doesn’t support a service of type
then it is possible to use a service of type
Until Kong 1.0, a database was required to run Kong.
Kong 1.1 introduced a new mode, DB-less, in which Kong can be configured
using a config file, and removes the need to use a database.
It is possible to deploy and run the Kubernetes Ingress Controller with or without a
database. The choice depends on the specific use-case and results in no
loss of functionality.
Without a database
In DB-less deployments, Kong’s Ingress controller runs
alongside Kong and configures Kong and dynamically configures
Kong as per the changes it receives from the Kubernetes API server.
Following figure shows how this deployment looks like:
In this deployment, only one Deployment is required, which is comprised of
a Pod with two containers, a Kong container which proxies the requests
and a controller container which configures Kong.
kong-proxy service would point to the ports of the Kong container in the
Since each pod contains a controller and a Kong container, scaling out
simply requires horizontally scaling this deployment to handle more traffic
or to add redundancy in the infrastructure.
With a Database
In a deployment where Kong is backed by a DB, the deployment architecture
is a little different.
Please refer to the below figure:
In this type of deployment, there are two types of deployments created,
separating the control and data flow:
- Control-plane: This deployment consists of a pod(s) running
the controller alongside
a Kong container, which can only configure the database. This deployment
does not proxy any traffic but only configures Kong. If multiple
replicas of this pod are running, a leader election process will ensure
that only one of the pods is configuring Kong’s database at a time.
- Data-plane: This deployment consists of pods running a
single Kong container which can proxy traffic based on the configuration
it loads from the database. This deployment should be scaled to respond
to change in traffic profiles and add redundancy to safeguard from node
- Database: The database is used to store Kong’s configuration and propagate
changes to all the Kong pods in the cluster. All Kong containers, in the
cluster should be able to connect to this database.
A database driven deployment should be used if your use-case requires
dynamica creation of Consumers and/or credentials in Kong at a scale large
enough that the consumers will not fit entirely in memory.
Multiple Ingress Controllers
It is possible to run multiple instances of the Kubernetes Ingress Controller or
run a Kong Kubernetes Ingress Controller alongside other Ingress Controllers inside
the same Kubernetes cluster.
There are a few different ways of accomplishing this:
It is common to deploy Ingress Controllers on a cluster level, meaning
an Ingress Controller will satisfy Ingress rules created in all the namespaces
inside a Kubernetes cluster.
Use the annotation on Ingress and Custom resources to segment
the Ingress resources between multiple Ingress Controllers.
When you use another Ingress Controler, which is default for cluster
(without set any
kubernetes.io/ingress.class), be aware of using default
ingress class. There is special behavior of the default
kong ingress class,
where any ingress resource that is not annotated is picked up.
Therefore with different ingress class then
kong, you have to use that
ingress class with every Kong CRD object (plugin, consumer) which you use.
- Namespace based isolation:
Kubernetes Ingress Controller supports a deployment option where it will satisfy
Ingress resources in a specific namespace. With this model, one can deploy
a controller in multiple namespaces and they will run in an isolated manner.
- If you are using Kong Enterprise, you can run multiple Ingress Controllers
pointing to the same database and configuring different Workspaces inside
Kong Enterprise. With such a deployment, one can use either of the above
two approaches to segment Ingress resources into different Workspaces in
The Kubernetes Ingress Controller is compatible a variety of runtimes:
Kong Gateway (OSS)
This is the Open-Source Gateway runtime.
The Ingress Controller is primarily developed against releases of the
Kong Enterprise K8S
If you are a Kong Enterprise customer, you have access to two more runtimes.
The first one, Kong Enterprise K8S, is an package that takes the Open-Source
Kong Gateway and adds enterprise-only plugins to it.
You simply need to deploy Kong Enterprise K8S instead of the Open-Source
Gateway in-order to take full-advantage of enterprise plugins.
The Kubernetes Ingress Controller is also compatible with the full-blown version of
Kong Enterprise. This runtime ships with Kong Manager, Kong Portal, and a
number of other enterprise-only features.
This doc provides a high-level
overivew of the architecture.