You are browsing documentation for an older version. See the latest documentation here.

Vitals with InfluxDB

Leveraging a time series database for Vitals data can improve request and Vitals performance in very-high traffic Kong Gateway Enterprise clusters (such as environments handling tens or hundreds of thousands of requests per second), without placing additional write load on the database backing the Kong cluster.

For information about using Kong Vitals with a database as the backend, refer to Kong Vitals.

Set up Kong Vitals with InfluxDB

Start an InfluxDB database

Production-ready InfluxDB installations should be deployed as a separate effort, but for proof-of-concept testing, running a local InfluxDB instance is possible with Docker.

1. Create a custom network to allow the containers to discover and communicate with each other.

    docker network create kong-ee-net
   

2. Start a local InfluxDB instance with Docker.

    docker run -d -p 8086:8086 \
      --network=kong-ee-net \
      --name influxdb \
      -e INFLUXDB_DB=kong \
      influxdb:1.8.4-alpine
   

   You must use InfluxDB 1.8.4-alpine because InfluxDB 2.0 will not work.

   Writing Vitals data to InfluxDB requires that the kong database is created, this is done using the INFLUXDB_DB variable.

Install Kong Gateway

If you already have a Kong Gateway instance, skip to deploying a license.

If you have not installed Kong Gateway, a Docker installation will work for the purposes of this guide.

Pull the Kong Gateway Docker image

1. Pull the following Docker image.

    docker pull kong/kong-gateway:3.3.1.1
   

   Some older Kong Gateway images are not publicly accessible. If you need a specific patch version and can’t find it on Kong’s public Docker Hub page, contact Kong Support.

   You should now have your Kong Gateway image locally.

2. Tag the image.

    docker tag kong/kong-gateway:3.3.1.1 kong-ee
   

Start the database and Kong Gateway containers

1. Start a PostgreSQL container:

    docker run -d --name kong-ee-database \
      --network=kong-ee-net \
      -p 5432:5432 \
      -e "POSTGRES_USER=kong" \
      -e "POSTGRES_DB=kong" \
      -e "POSTGRES_PASSWORD=kong" \
      postgres:9.6
   

2. Prepare the Kong database:

    docker run --rm --network=kong-ee-net \
      -e "KONG_DATABASE=postgres" \
      -e "KONG_PG_HOST=kong-ee-database" \
      -e "KONG_PG_PASSWORD=kong" \
      -e "KONG_PASSWORD=kongpass" \
      kong-ee kong migrations bootstrap
   

3. Start the gateway with Kong Manager and InfluxDB:

   Important: The settings below are intended for non-production use only, as they override the default admin_listen setting to listen for requests from any source. Do not use these settings in environments directly exposed to the internet.

   
   If you need to expose the admin_listen port to the internet in a production environment, secure it with authentication.

    docker run -d --name kong-ee --network=kong-ee-net \
      -e "KONG_DATABASE=postgres" \
      -e "KONG_PG_HOST=kong-ee-database" \
      -e "KONG_PG_PASSWORD=kong" \
      -e "KONG_PROXY_ACCESS_LOG=/dev/stdout" \
      -e "KONG_ADMIN_ACCESS_LOG=/dev/stdout" \
      -e "KONG_PROXY_ERROR_LOG=/dev/stderr" \
      -e "KONG_ADMIN_ERROR_LOG=/dev/stderr" \
      -e "KONG_ADMIN_LISTEN=0.0.0.0:8001" \
      -e "KONG_ADMIN_GUI_URL=http://localhost:8002" \
      -e "KONG_VITALS_STRATEGY=influxdb" \
      -e "KONG_VITALS_TSDB_ADDRESS=influxdb:8086" \
      -p 8000:8000 \
      -p 8443:8443 \
      -p 8001:8001 \
      -p 8444:8444 \
      -p 8002:8002 \
      -p 8445:8445 \
      -p 8003:8003 \
      -p 8004:8004 \
      kong-ee
   

   Note: For KONG_ADMIN_GUI_URL, replace localhost with with the DNS name or IP of the Docker host. KONG_ADMIN_GUI_URL should have a protocol, for example, http://.

Deploy a Kong Gateway license

If you already have a Kong Gateway Enterprise license attached to your Kong Gateway instance, skip to Configure Vitals with InfluxDB.

You will not be able to access the Kong Vitals functionality without a valid Kong Gateway Enterprise license attached to your Kong Gateway instance.

You can deploy a license file in one of the following ways:

The recommended method is using the Admin API.

Important:

• If you deploy a license using the /license endpoint on the control plane, the control plane propagates the license to connected data planes automatically.
• If you deploy a license using a KONG_LICENSE_DATA or KONG_LICENSE_PATH environment variable, the control plane does not propagate the license to data plane nodes. You must add the license to each data plane node, and each node must start with the license. The license can’t be added after starting the node.

We don’t recommend using this method in hybrid mode deployments.

Prerequisites

• You have received a license.json file from Kong.
• Kong Gateway is installed.

Deploy the license

curl -i -X POST http://localhost:8001/licenses \
  -d payload='{"license":{"payload":{"admin_seats":"1","customer":"Example Company, Inc","dataplanes":"1","license_creation_date":"2017-07-20","license_expiration_date":"2017-07-20","license_key":"00141000017ODj3AAG_a1V41000004wT0OEAU","product_subscription":"Konnect Enterprise","support_plan":"None"},"signature":"6985968131533a967fcc721244a979948b1066967f1e9cd65dbd8eeabe060fc32d894a2945f5e4a03c1cd2198c74e058ac63d28b045c2f1fcec95877bd790e1b","version":"1"}}'

{
  "created_at": 1500508800,
  "id": "30b4edb7-0847-4f65-af90-efbed8b0161f",
  "payload": "{\"license\":{\"payload\":{\"admin_seats\":\"1\",\"customer\":\"Example Company, Inc\",\"dataplanes\":\"1\",\"license_creation_date\":\"2017-07-20\",\"license_expiration_date\":\"2017-07-20\",\"license_key\":\"00141000017ODj3AAG_a1V41000004wT0OEAU\",\"product_subscription\":\"Konnect Enterprise\",\"support_plan\":\"None\"},\"signature\":\"6985968131533a967fcc721244a979948b1066967f1e9cd65dbd8eeabe060fc32d894a2945f5e4a03c1cd2198c74e058ac63d28b045c2f1fcec95877bd790e1b\",\"version\":\"1\"}}",
  "updated_at": 1500508800
}

docker run -d --name kong-gateway \
 --network=kong-net \
 ...
 -v "$(pwd)/kong-license/:/kong-license/" \
 -e "KONG_LICENSE_PATH=/kong-license/license.json" \
 kong/kong-gateway:-alpine

Configure Vitals with InfluxDB Kong Gateway

Note: If you used the configuration in Installing Kong Gateway on Docker, then you do not need to complete this step.

In addition to enabling Kong Vitals, Kong Gateway must be configured to use InfluxDB as the backing strategy for Vitals. The InfluxDB host and port must also be defined:

echo "KONG_VITALS_STRATEGY=influxdb KONG_VITALS_TSDB_ADDRESS=influxdb:8086 kong reload exit" \
| docker exec -i kong-ee /bin/sh

Note: In Hybrid Mode, configure vitals_strategy and vitals_tsdb_address on both the control plane and all data planes.

Understanding Vitals data using InfluxDB measurements

Kong Vitals records metrics in two InfluxDB measurements:

1. kong_request: Contains field values for request latencies and HTTP, and tags for various Kong entities associated with the requests (for example, the Route and Service in question).
2. kong_datastore_cache: Contains points about cache hits and misses.

To display the measurement schemas on your InfluxDB instance running in Docker:

1. Open command line in your InfluxDB Docker container.

    docker exec -it influxdb /bin/sh
   

2. Log in to the InfluxDB CLI.

    influx -precision rfc3339
   

3. Enter the InfluxQL query for returning a list of tag keys associated with the specified database.

    > SHOW TAG KEYS ON kong
   

   Example result:

    name: kong_request
    tagKey
    ------
    consumer
    hostname
    route
    service
    status_f
    wid
    workspace
   
    name: kong_datastore_cache
    tagKey
    ------
    hostname
    wid
   

4. Enter the InfluxQL query for returning the field keys and the data type of their field values.

    > SHOW FIELD KEYS ON kong
   

   Example result:

    name: kong_request
    fieldKey	         fieldType
    --------	         ---------
    kong_latency       integer
    proxy_latency      integer
    request_latency    integer
    status             integer
   
    name: kong_datastore_cache
    fieldKey  fieldType
    --------  ---------
   
    hits      integer
    misses    integer
   

The tag wid is used to differentiate the unique worker ID per host, to avoid duplicate metrics shipped at the same point in time.

As demonstrated above, the series cardinality of the kong_request measurement varies based on the cardinality of the Kong cluster configuration - a greater number of Service/Route/Consumer/Workspace combinations handled by Kong results in a greater series cardinality as written by Vitals.

Sizing an InfluxDB node/cluster for Vitals

Consult the InfluxDB sizing guidelines for reference on appropriately sizing an InfluxDB node/cluster.

Note: The query behavior when reading Vitals data falls under the “moderate” load category as defined by the InfluxDB sizing guidelines. Several GROUP BY statements and functions are used to generate the Vitals API responses, which can require significant CPU resources to execute when hundreds of thousands or millions of data points are present.

Query frequency and precision

Kong buffers Vitals metrics and writes InfluxDB points in batches to improve throughput in InfluxDB and reduce overhead in the Kong proxy path. Each Kong worker process flushes its buffer of metrics every 10 seconds or 5000 data points, whichever comes first.

Metrics points are written with microsecond (u) precision. To comply with the Vitals API, measurement values are read back grouped by second.

Note: Because of limitations in the OpenResty API, writing values with microsecond precision requires an additional syscall per request.

Managing the retention policy of the Kong database

Vitals InfluxDB data points are not downsampled or managed by a retention policy through Kong. InfluxDB operators are encouraged to manually manage the retention policy of the kong database to reduce the disk space and memory needed to manage Vitals data points.