Kong Splunk Log plugin

Community Plugin: This plugin is developed, tested, and maintained by a third-party contributor.

Kong provides many great logging tools out of the box - this is a modified version of the Kong HTTP logging plugin that has been refactored and tailored to work with Splunk.

Example Log in Splunk UI

Splunk UI screen shot

Installation

Recommended:

$ luarocks install kong-splunk-log

Other:

$ git clone https://github.com/Optum/kong-splunk-log.git /path/to/kong/plugins/kong-splunk-log
$ cd /path/to/kong/plugins/kong-splunk-log
$ luarocks make *.rockspec

Configuration

The plugin requires an environment variable SPLUNK_HOST. This is how we define the host="" Splunk field in the example log picture embedded above in our README.

Example Plugin Configuration

Splunk Config

If not already set, it can be done so as follows:

$ export SPLUNK_HOST="gateway.company.com"

One last step is to make the environment variable accessible by an nginx worker. To do this, simply add this line to your nginx.conf

env SPLUNK_HOST;

Maintainers

jeremyjpj0916
rsbrisci

Feel free to open issues, or refer to our Contribution Guidelines if you have any questions.

Configuration Reference

This plugin is not compatible with DB-less mode.

Enable the plugin on a service

Admin API

Kubernetes

Declarative (YAML)

For example, configure this plugin on a service by making the following request:

curl -X POST http://{HOST}:8001/services/{SERVICE}/plugins \
    --data "name=kong-splunk-log"  \
    --data "config.splunk_endpoint=https://hec-splunk.company.com/services/collector" \
    --data "config.splunk_access_token=aaaaaaaa-bbbb-cccc-dddd-ffffffffffff" \
    --data "config.method=POST" \
    --data "config.content_type=application/json" \
    --data "config.timeout=10000" \
    --data "config.retry_count=5" \
    --data "config.queue_size=20" \
    --data "config.flush_timeout=30" \
    --data "config.keepalive=60000"

SERVICE is the id or name of the service that this plugin configuration will target.

First, create a KongPlugin resource:

apiVersion: configuration.konghq.com/v1
kind: KongPlugin
metadata:
  name: <kong-splunk-log-example>
config: 
  splunk_endpoint: https://hec-splunk.company.com/services/collector
  splunk_access_token: aaaaaaaa-bbbb-cccc-dddd-ffffffffffff
  method: POST
  content_type: application/json
  timeout: 10000
  retry_count: 5
  queue_size: 20
  flush_timeout: 30
  keepalive: 60000
plugin: kong-splunk-log

Next, apply the KongPlugin resource to a Service by annotating the Service as follows:

apiVersion: v1
kind: Service
metadata:
  name: {SERVICE}
  labels:
    app: {SERVICE}
  annotations:
    konghq.com/plugins: <kong-splunk-log-example>
spec:
  ports:
  - port: 80
    targetPort: 80
    protocol: TCP
    name: {SERVICE}
  selector:
    app: {SERVICE}

{SERVICE} is the id or name of the service that this plugin configuration will target.

Note: The KongPlugin resource only needs to be defined once and can be applied to any service, consumer, or route in the namespace. If you want the plugin to be available cluster-wide, create the resource as a KongClusterPlugin instead of KongPlugin.

For example, configure this plugin on a service by adding this section to your declarative configuration file:

plugins:
- name: kong-splunk-log
  service: {SERVICE}
  config: 
    splunk_endpoint: https://hec-splunk.company.com/services/collector
    splunk_access_token: aaaaaaaa-bbbb-cccc-dddd-ffffffffffff
    method: POST
    content_type: application/json
    timeout: 10000
    retry_count: 5
    queue_size: 20
    flush_timeout: 30
    keepalive: 60000

SERVICE is the id or name of the service that this plugin configuration will target.

Enable the plugin on a route

Admin API

Kubernetes

Declarative (YAML)

For example, configure this plugin on a route with:

$ curl -X POST http://{HOST}:8001/routes/{ROUTE}/plugins \
    --data "name=kong-splunk-log"  \
    --data "config.splunk_endpoint=https://hec-splunk.company.com/services/collector" \
    --data "config.splunk_access_token=aaaaaaaa-bbbb-cccc-dddd-ffffffffffff" \
    --data "config.method=POST" \
    --data "config.content_type=application/json" \
    --data "config.timeout=10000" \
    --data "config.retry_count=5" \
    --data "config.queue_size=20" \
    --data "config.flush_timeout=30" \
    --data "config.keepalive=60000"

ROUTE is the id or name of the route that this plugin configuration will target.

First, create a KongPlugin resource:

apiVersion: configuration.konghq.com/v1
kind: KongPlugin
metadata:
  name: <kong-splunk-log-example>
config: 
  splunk_endpoint: https://hec-splunk.company.com/services/collector
  splunk_access_token: aaaaaaaa-bbbb-cccc-dddd-ffffffffffff
  method: POST
  content_type: application/json
  timeout: 10000
  retry_count: 5
  queue_size: 20
  flush_timeout: 30
  keepalive: 60000
plugin: kong-splunk-log

Then, apply it to an ingress (Route or Routes) by annotating the ingress as follows:

apiVersion: networking/v1beta1
kind: Ingress
metadata:
  name: {ROUTE}
  annotations:
    kubernetes.io/ingress.class: kong
    konghq.com/plugins: <kong-splunk-log-example>
spec:
  rules:
  - host: examplehostname.com
    http:
      paths:
      - path: /bar
        backend:
          serviceName: echo
          servicePort: 80

ROUTE is the id or name of the route that this plugin configuration will target.

For example, configure this plugin on a route by adding this section to your declarative configuration file:

plugins:
- name: kong-splunk-log
  route: <route>
  config: 
    splunk_endpoint: https://hec-splunk.company.com/services/collector
    splunk_access_token: aaaaaaaa-bbbb-cccc-dddd-ffffffffffff
    method: POST
    content_type: application/json
    timeout: 10000
    retry_count: 5
    queue_size: 20
    flush_timeout: 30
    keepalive: 60000

ROUTE is the id or name of the route that this plugin configuration will target.

Enable the plugin globally

A plugin which is not associated to any service, route, or consumer is considered global, and will be run on every request. Read the Plugin Reference and the Plugin Precedence sections for more information.

Admin API

Kubernetes

Declarative (YAML)

For example, configure this plugin globally with:

$ curl -X POST http://{HOST}:8001/plugins/ \
    --data "name=kong-splunk-log"  \
    --data "config.splunk_endpoint=https://hec-splunk.company.com/services/collector" \
    --data "config.splunk_access_token=aaaaaaaa-bbbb-cccc-dddd-ffffffffffff" \
    --data "config.method=POST" \
    --data "config.content_type=application/json" \
    --data "config.timeout=10000" \
    --data "config.retry_count=5" \
    --data "config.queue_size=20" \
    --data "config.flush_timeout=30" \
    --data "config.keepalive=60000"

Create a KongClusterPlugin resource and label it as global:

apiVersion: configuration.konghq.com/v1
kind: KongClusterPlugin
metadata:
  name: <global-kong-splunk-log>
  annotations:
    kubernetes.io/ingress.class: kong
  labels:
    global: \"true\"
config: 
  splunk_endpoint: https://hec-splunk.company.com/services/collector
  splunk_access_token: aaaaaaaa-bbbb-cccc-dddd-ffffffffffff
  method: POST
  content_type: application/json
  timeout: 10000
  retry_count: 5
  queue_size: 20
  flush_timeout: 30
  keepalive: 60000
plugin: kong-splunk-log

For example, configure this plugin using the plugins: entry in the declarative configuration file:

plugins:
- name: kong-splunk-log
  config: 
    splunk_endpoint: https://hec-splunk.company.com/services/collector
    splunk_access_token: aaaaaaaa-bbbb-cccc-dddd-ffffffffffff
    method: POST
    content_type: application/json
    timeout: 10000
    retry_count: 5
    queue_size: 20
    flush_timeout: 30
    keepalive: 60000

Parameters

Here's a list of all the parameters which can be used in this plugin's configuration:

Form Parameter	Description
`name` required Type: string	The name of the plugin, in this case `kong-splunk-log`.
`service.id` Type: string	The ID of the Service the plugin targets.
`route.id` Type: string	The ID of the Route the plugin targets.
`enabled` required Type: boolean Default value: `true`	Whether this plugin will be applied.
`config.splunk_endpoint` required	Splunk HTTP Event Collector(HEC) Endpoint
`config.splunk_access_token` required	Passes required Splunk header `Authorization Splunk:` `aaaaaaaa-bbbb-cccc-dddd-ffffffffffff`
`config.method` optional Default value: `POST`	HTTP Method to send to Splunk
`config.content_type` optional Default value: `application/json`	Defines the Content-Type header to send to Splunk
`config.timeout` optional Default value: `10000`	The amount of time to wait on a Splunk transaction before timing out
`config.retry_count` optional Default value: `5`	The number of attempts to retry logging an event on splunk connection errors
`config.queue_size` optional Default value: `20`	The max number of event logs the plugin may send in a single request
`config.flush_timeout` optional Default value: `30`	The time taken in seconds before flushing a queue of transactions to splunk that has not reached its max `queue_size`
`config.keepalive` optional Default value: `60000`	The amount of time to keep plugin connections with Splunk active

We recommend enabling the Splunk Logging plugin at a global level.

About this Plugin
Made by	Optum
Categories	Logging
DB-less compatible?	Yes
Support	github.com
Source	Code License (Apache-2.0)