In order to give you better service we use cookies. By continuing to use our website, you agree to the use of cookies as described in our Cookie Policy

Kong Logo
  • Request Demo
  • Install
  • Products
    • Kong Gateway
      Install Kong Open Source
      • Overview
      • Control API

        Own your Kong experience to customize your API and microservices workflows.

      • Kong Proxy

        Deliver performance needed for microservices, service mesh, and cloud native deployments.

        • Overview
        • Multi-protocol
      • Kong Plugins

        Unleash the full power of Kong with a robust ecosystem of plugins.

    • Kong Enterprise
      DemoFree Trial
      • Overview
      • Kong Manager

        Monitor and manage all your services with a consumer-grade interface.

        • Overview
      • Teams

        Organize developers and assign permissions to improve efficiency and compliance.

        • Overview
        • RBAC
        • Workspaces
      • Kong Studio

        Empower your developers with the Kong Studio Integrated Development Environment.

      • Dev Portal

        Accelerate innovation across your organization with the Kong Developer Portal.

      • Brain

        Automatically standardize documentation and create a visual map of your services.

      • Immunity

        Autonomously identify service issues with machine learning-powered anomaly detection.

      • Kong for Kubernetes

        The only fully native Kubernetes Ingress Controller for enhanced API management.

      • Enterprise Plugins

        Instantly implement policies built for global scale with Kong Enterprise Plugins.

      • Vitals

        Monitor your Kong Enterprise health and microservice API transactions traversing Kong.

  • Solutions
    • API Gateway

      Take control of your microservices traffic with the world’s most popular API gateway.

    • Kubernetes

      Own your Kubernetes cluster by extending Kong functionality as an ingress controller.

    • Service Mesh

      Inject Kong as a sidecar for your services to go from mess to mesh.

  • Plugins
  • Open Source
    • Install Kong Gateway
    • Kong Community
    • Kubernetes Ingress
    • Kuma
    • Insomnia
  • Docs
    • Get Started
      • Install Kong Gateway
      • Install Kong Enterprise
    • Learn
      • Kong Gateway
      • Kong Enterprise
      • Kong Studio
      • Plugins Hub
  • Resources
    • Learning
      • eBooks
      • Webinars
      • Briefs
      • Blog
      • API Gateway
      • Microservices
    • Community
      • Community
      • Kong Nation
      • Kong Summit
      • GitHub
    • Support
      • Enterprise Support Portal
      • FAQS
  • Company
    • About
    • Customers
    • Investors
    • Careers
    • Partners
    • Press
    • Contact
  • Request Demo
Edit this Page
Kong Kong Enterprise Kong Studio Open Sidebar
Documentation

Clustering Reference

  • 1.4.x (latest)
  • 1.3.x
  • 1.2.x
  • 1.1.x
  • 1.0.x
  • 0.14.x
  • 0.13.x
  • 0.12.x
  • 0.11.x
  • 0.10.x
  • 0.9.x
  • 0.8.x
  • 0.7.x
  • 0.6.x
  • 0.5.x
  • 0.4.x
  • 0.3.x
  • 0.2.x
Getting Started
  • Introduction
  • Five-minute quickstart
  • Configuring a Service
  • Enabling Plugins
  • Adding Consumers
Guides & References
  • Configuration reference
  • CLI reference
  • Proxy reference
  • Authentication reference
  • Load balancing reference
  • Health checks and circuit breakers reference
  • Clustering reference
  • Logging reference
  • Network & Firewall
  • Securing the Admin API
  • Plugin Development Guide
    • Introduction
    • File structure
    • Implementing custom logic
    • Plugin configuration
    • Accessing the datastore
    • Storing custom entities
    • Caching custom entities
    • Extending the Admin API
    • Writing tests
    • (un)Installing your plugin
  • Plugin Development Kit
    • kong.client
    • kong.ctx
    • kong.ip
    • kong.log
    • kong.request
    • kong.response
    • kong.service
    • kong.service.request
    • kong.service.response
    • kong.table
Admin API
  • Supported Content Types
  • Information routes
    • Retrieve node information
    • Retrieve node status
  • Service Object
    • Add Service
    • Retrieve Service
    • List Services
    • Update Service
    • Update or create Service
    • Delete Service
  • Route Object
    • Add Route
    • Retrieve Route
    • List Routes
    • List Routes associated to a Service
    • Update Route
    • Update or create Route
    • Delete Route
  • Consumer Object routes
    • Create Consumer
    • Retrieve Consumer
    • List Consumers
    • Update Consumer
    • Update or create Consumer
    • Delete Consumer
  • Plugin Object routes
    • Add Plugin
    • Retrieve Plugin
    • List all Plugins
    • Update Plugin
    • Update or Add Plugin
    • Delete Plugin
    • Retrieve Enabled Plugins
    • Retrieve Plugin Schema
  • Certificate Object routes
    • Add Certificate
    • Retrieve Certificate
    • List Certificates
    • Update Certificate
    • Update or Create Certificate
    • Delete Certificate
  • SNI Object routes
    • Add SNI
    • Retrieve SNI
    • List SNIs
    • Update SNI
    • Update or Create SNI
    • Delete SNI
  • Upstream Object routes
    • Add Upstream
    • Retrieve Upstream
    • List all Upstreams
    • Update Upstream
    • Update or create Upstream
    • Delete Upstream
    • Show Upstream health for node
  • Target Object routes
    • Add Target
    • List Targets
    • List All Targets
    • Delete Target
    • Set Target as Healthy
    • Set Target as Unhealthy
Maybe you were looking for the Enterprise Documentation instead?
Careful! You are browsing documentation for an outdated version of Kong. Go here to browse the documentation for the latest version.

Table of Contents

  • Introduction
  • What a Kong cluster does and doesn’t do
  • Single node Kong clusters
  • Multiple nodes Kong clusters
  • What is being cached?
  • How to configure database caching?
    • 1. db_update_frequency (default: 5s)
    • 2. db_update_propagation (default: 0s)
    • 3. db_cache_ttl (default: 0s)
    • 4. When using Cassandra
  • Interacting with the cache via the Admin API
    • Inspect a cached value
    • Purge a cached value
    • Purge a node’s cache

Introduction

A Kong cluster allows you to scale the system horizontally by adding more machines to handle more incoming requests. They will all share the same configuration since they point to the same database. Kong nodes pointing to the same datastore will be part of the same Kong cluster.

You need a load-balancer in front of your Kong cluster to distribute traffic across your available nodes.

What a Kong cluster does and doesn’t do

Having a Kong cluster does not mean that your clients traffic will be load-balanced across your Kong nodes out of the box. You still need a load-balancer in front of your Kong nodes to distribute your traffic. Instead, a Kong cluster means that those nodes will share the same configuration.

For performance reasons, Kong avoids database connections when proxying requests, and caches the contents of your database in memory. The cached entities include Services, Routes, Consumers, Plugins, Credentials, etc… Since those values are in memory, any change made via the Admin API of one of the nodes needs to be propagated to the other nodes.

This document describes how those cached entities are being invalidated and how to configure your Kong nodes for your use case, which lies somewhere between performance and consistency.

Back to TOC

Single node Kong clusters

A single Kong node connected to a database (Cassandra or PostgreSQL) creates a Kong cluster of one node. Any changes applied via the Admin API of this node will instantly take effect. Example:

Consider a single Kong node A. If we delete a previously registered Service:

$ curl -X DELETE http://127.0.0.1:8001/services/test-service

Then any subsequent request to A would instantly return 404 Not Found, as the node purged it from its local cache:

$ curl -i http://127.0.0.1:8000/test-service

Back to TOC

Multiple nodes Kong clusters

In a cluster of multiple Kong nodes, other nodes connected to the same database would not instantly be notified that the Service was deleted by node A. While the Service is not in the database anymore (it was deleted by node A), it is still in node B’s memory.

All nodes perform a periodic background job to synchronize with changes that may have been triggered by other nodes. The frequency of this job can be configured via:

  • db_update_frequency (default: 5 seconds)

Every db_update_frequency seconds, all running Kong nodes will poll the database for any update, and will purge the relevant entities from their cache if necessary.

If we delete a Service from node A, this change will not be effective in node B until node Bs next database poll, which will occur up to db_update_frequency seconds later (though it could happen sooner).

This makes Kong clusters eventually consistent.

Back to TOC

What is being cached?

All of the core entities such as Services, Routes, Plugins, Consumers, Credentials are cached in memory by Kong and depend on their invalidation via the polling mechanism to be updated.

Additionally, Kong also caches database misses. This means that if you configure a Service with no plugin, Kong will cache this information. Example:

On node A, we add a Service and a Route:

# node A
$ curl -X POST http://127.0.0.1:8001/services \
    --data "name=example-service" \
    --data "url=http://example.com"

$ curl -X POST http://127.0.0.1:8001/services/example-service/routes \
    --data "paths[]=/example"

(Note that we used /services/example-service/routes as a shortcut: we could have used the /routes endpoint instead, but then we would need to pass service_id as an argument, with the UUID of the new Service.)

A request to the Proxy port of both node A and B will cache this Service, and the fact that no plugin is configured on it:

# node A
$ curl http://127.0.0.1:8000/example
HTTP 200 OK
...
# node B
$ curl http://127.0.0.2:8000/example
HTTP 200 OK
...

Now, say we add a plugin to this Service via node A’s Admin API:

# node A
$ curl -X POST http://127.0.0.1:8001/services/example-service/plugins \
    --data "name=example-plugin"

Because this request was issued via node A’s Admin API, node A will locally invalidate its cache and on subsequent requests, it will detect that this API has a plugin configured.

However, node B hasn’t run a database poll yet, and still caches that this API has no plugin to run. It will be so until node B runs its database polling job.

Conclusion: all CRUD operations trigger cache invalidations. Creation (POST, PUT) will invalidate cached database misses, and update/deletion (PATCH, DELETE) will invalidate cached database hits.

Back to TOC

How to configure database caching?

You can configure 3 properties in the Kong configuration file, the most important one being db_update_frequency, which determine where your Kong nodes stand on the performance vs consistency trade off.

Kong comes with default values tuned for consistency, in order to let you experiment with its clustering capabilities while avoiding “surprises”. As you prepare a production setup, you should consider tuning those values to ensure that your performance constraints are respected.

1. db_update_frequency (default: 5s)

This value determines the frequency at which your Kong nodes will be polling the database for invalidation events. A lower value will mean that the polling job will be executed more frequently, but that your Kong nodes will keep up with changes you apply. A higher value will mean that your Kong nodes will spend less time running the polling jobs, and will focus on proxying your traffic.

Note: changes propagate through the cluster in up to db_update_frequency seconds.

Back to TOC

2. db_update_propagation (default: 0s)

If your database itself is eventually consistent (ie: Cassandra), you must configure this value. It is to ensure that the change has time to propagate across your database nodes. When set, Kong nodes receiving invalidation events from their polling jobs will delay the purging of their cache for db_update_propagation seconds.

If a Kong node connected to an eventual consistent database was not delaying the event handling, it could purge its cache, only to cache the non-updated value again (because the change hasn’t propagated through the database yet)!

You should set this value to an estimate of the amount of time your database cluster takes to propagate changes.

Note: when this value is set, changes propagate through the cluster in up to db_update_frequency + db_update_propagation seconds.

Back to TOC

3. db_cache_ttl (default: 0s)

The time (in seconds) for which Kong will cache database entities (both hits and misses). This Time-To-Live value acts as a safeguard in case a Kong node misses an invalidation event, to avoid it from running on stale data for too long. When the TTL is reached, the value will be purged from its cache, and the next database result will be cached again.

By default no data is invalidated based on this TTL (the default value is 0). This is usually fine: Kong nodes rely on invalidation events, which are handled at the db store level (Cassandra/PosgreSQL). If you are concerned that a Kong node might miss invalidation event for any reason, you should set a TTL. Otherwise the node might run with a stale value in its cache for an undefined amount of time, until the cache is manually purged, or the node is restarted.

Back to TOC

4. When using Cassandra

If you use Cassandra as your Kong database, you must set db_update_propagation to a non-zero value. Since Cassandra is eventually consistent by nature, this will ensure that Kong nodes do not prematurely invalidate their cache, only to fetch and catch a not up-to-date entity again. Kong will present you a warning logs if you did not configure this value when using Cassandra.

Additionally, you might want to configure cassandra_consistency to a value like QUORUM or LOCAL_QUORUM, to ensure that values being cached by your Kong nodes are up-to-date values from your database.

Back to TOC

Interacting with the cache via the Admin API

If for some reason, you wish to investigate the cached values, or manually invalidate a value cached by Kong (a cached hit or miss), you can do so via the Admin API /cache endpoint.

Inspect a cached value

Endpoint

/cache/{cache_key}

Response

If a value with that key is cached:

HTTP 200 OK
...
{
    ...
}

Else:

HTTP 404 Not Found

Note: retrieving the cache_key for each entity being cached by Kong is currently an undocumented process. Future versions of the Admin API will make this process easier.

Back to TOC

Purge a cached value

Endpoint

/cache/{cache_key}

Response

HTTP 204 No Content
...

Note: retrieving the cache_key for each entity being cached by Kong is currently an undocumented process. Future versions of the Admin API will make this process easier.

Back to TOC

Purge a node’s cache

Endpoint

/cache

Response

HTTP 204 No Content

Note: be wary of using this endpoint on a warm, production running node. If the node is receiving a lot of traffic, purging its cache at the same time will trigger many requests to your database, and could cause a dog-pile effect.

Back to TOC

  • Kong
    Star
  • Products
    • Kong
    • Kong Enterprise
    • Kong Studio
    • Subscriptions
  • Resources
    • Enterprise Support
    • Documentation
    • Partners
    • Webinars
    • Ebooks
  • Company
    • About
    • Customers
    • Investors
    • News
    • Careers Hiring!
    • Kong Summit
    • Contact
  • Open Source
    • Install
    • GitHub
    • Kong Nation
    • Community
  • © Kong Inc. 2019   Terms•Privacy