Skip to content
|
search
Plugin Hub
header icon

HTTP Log

Send request and response logs to an HTTP server.

Configuration Reference

This plugin is compatible with DB-less mode.

In DB-less mode, you configure Kong Gateway declaratively. Therefore, the Admin API is mostly read-only. The only tasks it can perform are all related to handling the declarative config, including:

  • Setting a target's health status in the load balancer
  • Validating configurations against schemas
  • Uploading the declarative configuration using the /config endpoint

Example plugin configuration

The following examples provide some typical configurations for enabling the http-log plugin on a service.

Make the following request:

curl -X POST http://localhost:8001/services/SERVICE_NAME|SERVICE_ID/plugins \
    --data "name=http-log"  \
    --data "config.http_endpoint=http://mockbin.org/bin/:id" \
    --data "config.method=POST" \
    --data "config.timeout=1000" \
    --data "config.keepalive=1000" \
    --data "config.flush_timeout=2" \
    --data "config.retry_count=15"

Replace SERVICE_NAME|SERVICE_ID with the id or name of the service that this plugin configuration will target.

First, create a KongPlugin resource:

apiVersion: configuration.konghq.com/v1
kind: KongPlugin
metadata:
  name: http-log-example
config: 
  http_endpoint: http://mockbin.org/bin/:id
  method: POST
  timeout: 1000
  keepalive: 1000
  flush_timeout: 2
  retry_count: 15
plugin: http-log

Next, apply the KongPlugin resource to a service by annotating the service as follows:

apiVersion: v1
kind: Service
metadata:
  name: SERVICE_NAME|SERVICE_ID
  labels:
    app: SERVICE_NAME|SERVICE_ID
  annotations:
    konghq.com/plugins: http-log-example
spec:
  ports:
  - port: 80
    targetPort: 80
    protocol: TCP
    name: SERVICE_NAME|SERVICE_ID
  selector:
    app: SERVICE_NAME|SERVICE_ID

Replace SERVICE_NAME|SERVICE_ID with the id or name of the service that this plugin configuration will target.

Note: The KongPlugin resource only needs to be defined once and can be applied to any service, consumer, or route in the namespace. If you want the plugin to be available cluster-wide, create the resource as a KongClusterPlugin instead of KongPlugin.

Add this section to your declarative configuration file:

plugins:
- name: http-log
  service: SERVICE_NAME|SERVICE_ID
  config: 
    http_endpoint: http://mockbin.org/bin/:id
    method: POST
    timeout: 1000
    keepalive: 1000
    flush_timeout: 2
    retry_count: 15

Replace SERVICE_NAME|SERVICE_ID with the id or name of the service that this plugin configuration will target.

You can configure this plugin through the Konnect UI.

From the servicehub icon Service Hub, select a service version, then set up the plugin:

  1. In the Plugins section, click Add Plugin.
  2. Find and select the HTTP Log plugin.

  3. Configure the plugin’s parameters.

    You can test out the plugin with the following sample configuration:

    • Config.Http Endpoint: http://mockbin.org/bin/:id
  4. Click Create.

You can configure this plugin through the Kong Manager UI.

  1. In Kong Manager, select the workspace.
  2. From the Services section, click View for the service row.
  3. From the plugin section, click Add Plugin.
  4. Find and select the HTTP Log plugin.
  5. If the option is available, select Scoped.
  6. Add the service name and ID to the Service field if it is not already pre-filled.

  7. Configure the plugin’s parameters.

    You can test out the plugin with the following sample configuration:

    • Config.Http Endpoint: http://mockbin.org/bin/:id
  8. Click Create.

The following examples provide some typical configurations for enabling the http-log plugin on a route.

Make the following request:

curl -X POST http://localhost:8001/routes/ROUTE_NAME|ROUTE_ID/plugins \
  --data "name=http-log"  \
    --data "config.http_endpoint=http://mockbin.org/bin/:id" \
    --data "config.method=POST" \
    --data "config.timeout=1000" \
    --data "config.keepalive=1000" \
    --data "config.flush_timeout=2" \
    --data "config.retry_count=15"

Replace ROUTE_NAME|ROUTE_ID with the id or name of the route that this plugin configuration will target.

First, create a KongPlugin resource:

apiVersion: configuration.konghq.com/v1
kind: KongPlugin
metadata:
  name: http-log-example
config: 
  http_endpoint: http://mockbin.org/bin/:id
  method: POST
  timeout: 1000
  keepalive: 1000
  flush_timeout: 2
  retry_count: 15
plugin: http-log

Then, apply it to an ingress (route or routes) by annotating the ingress as follows:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ROUTE_NAME|ROUTE_ID
  annotations:
    kubernetes.io/ingress.class: kong
    konghq.com/plugins: http-log-example
spec:
  rules:
  - host: examplehostname.com
    http:
      paths:
      - path: /bar
        backend:
          service:
            name: echo
            port:
              number: 80

Replace ROUTE_NAME|ROUTE_ID with the id or name of the route that this plugin configuration will target.

Note: The KongPlugin resource only needs to be defined once and can be applied to any service, consumer, or route in the namespace. If you want the plugin to be available cluster-wide, create the resource as a KongClusterPlugin instead of KongPlugin.

Add this section to your declarative configuration file:

plugins:
- name: http-log
  route: ROUTE_NAME
  config: 
    http_endpoint: http://mockbin.org/bin/:id
    method: POST
    timeout: 1000
    keepalive: 1000
    flush_timeout: 2
    retry_count: 15

Replace ROUTE_NAME|ROUTE_ID with the id or name of the route that this plugin configuration will target.

You can configure this plugin through the Konnect UI.

From the servicehub icon Service Hub, select a service version, then set up the plugin:

  1. Select a route.
  2. In the Plugins section, click Add Plugin.
  3. Find and select the HTTP Log plugin.

  4. Configure the plugin’s parameters.

    You can test out the plugin with the following sample configuration:

    • Config.Http Endpoint: http://mockbin.org/bin/:id
  5. Click Create.

You can configure this plugin through the Kong Manager UI.

  1. In Kong Manager, select the workspace.
  2. Open Routes from the menu, then click View for the route row.
  3. From the plugin section, click Add Plugin.
  4. Find and select the HTTP Log plugin.
  5. If the option is available, select Scoped.
  6. Add the route ID if it is not already prefilled.

  7. Configure the plugin’s parameters.

    You can test out the plugin with the following sample configuration:

    • Config.Http Endpoint: http://mockbin.org/bin/:id
  8. Click Create.

The following examples provide some typical configurations for enabling the http-log plugin on a consumer.

Make the following request:

curl -X POST http://localhost:8001/consumers/CONSUMER_NAME|CONSUMER_ID/plugins \
  --data "name=http-log"  \
    --data "config.http_endpoint=http://mockbin.org/bin/:id" \
    --data "config.method=POST" \
    --data "config.timeout=1000" \
    --data "config.keepalive=1000" \
    --data "config.flush_timeout=2" \
    --data "config.retry_count=15"

Replace CONSUMER_NAME|CONSUMER_ID with the id or name of the consumer that this plugin configuration will target.

You can combine consumer.id, service.id, or route.id in the same request, to further narrow the scope of the plugin.

First, create a KongPlugin resource:

apiVersion: configuration.konghq.com/v1
kind: KongPlugin
metadata:
  name: http-log-example
config: 
  http_endpoint: http://mockbin.org/bin/:id
  method: POST
  timeout: 1000
  keepalive: 1000
  flush_timeout: 2
  retry_count: 15
plugin: http-log

Then, apply it to a consumer by annotating the KongConsumer resource as follows:

apiVersion: configuration.konghq.com/v1
kind: KongConsumer
metadata:
  name: CONSUMER_NAME|CONSUMER_ID
  annotations:
    konghq.com/plugins: http-log-example
    kubernetes.io/ingress.class: kong

Replace CONSUMER_NAME|CONSUMER_ID with the id or name of the consumer that this plugin configuration will target.

Note: The KongPlugin resource only needs to be defined once and can be applied to any service, consumer, or route in the namespace. If you want the plugin to be available cluster-wide, create the resource as a KongClusterPlugin instead of KongPlugin.

Add this section to your declarative configuration file:

plugins:
- name: http-log
  consumer: CONSUMER_NAME|CONSUMER_ID
  config: 
    http_endpoint: http://mockbin.org/bin/:id
    method: POST
    timeout: 1000
    keepalive: 1000
    flush_timeout: 2
    retry_count: 15

Replace CONSUMER_NAME|CONSUMER_ID with the id or name of the consumer that this plugin configuration will target.

You can configure this plugin through the Kong Manager UI.

  1. In Kong Manager, select the workspace.
  2. From the Consumers section, click View for the consumer row.
  3. Select the Plugins tab, then click Add Plugin.
  4. Find and select the HTTP Log plugin.
  5. If the option is available, select Scoped.
  6. Add the consumer ID if it is not already prefilled.

  7. Configure the plugin’s parameters.

    You can test out the plugin with the following sample configuration:

    • Config.Http Endpoint: http://mockbin.org/bin/:id
  8. Click Create.

A plugin which is not associated to any service, route, or consumer is considered global, and will be run on every request. Read the Plugin Reference and the Plugin Precedence sections for more information.

The following examples provide some typical configurations for enabling the http-log plugin globally.

Make the following request:

curl -X POST http://localhost:8001/plugins/ \
  --data "name=http-log"  \
    --data "config.http_endpoint=http://mockbin.org/bin/:id" \
    --data "config.method=POST" \
    --data "config.timeout=1000" \
    --data "config.keepalive=1000" \
    --data "config.flush_timeout=2" \
    --data "config.retry_count=15"

Create a KongClusterPlugin resource and label it as global:

apiVersion: configuration.konghq.com/v1
kind: KongClusterPlugin
metadata:
  name: <global-http-log>
  annotations:
    kubernetes.io/ingress.class: kong
  labels:
    global: \"true\"
config: 
  http_endpoint: http://mockbin.org/bin/:id
  method: POST
  timeout: 1000
  keepalive: 1000
  flush_timeout: 2
  retry_count: 15
plugin: http-log

Add a plugins entry in the declarative configuration file:

plugins:
- name: http-log
  config: 
    http_endpoint: http://mockbin.org/bin/:id
    method: POST
    timeout: 1000
    keepalive: 1000
    flush_timeout: 2
    retry_count: 15

You can configure this plugin through the Kong Manager UI.

  1. In Kong Manager, select the workspace.
  2. Open Plugins from the menu, then click New Plugin.
  3. Find and select the HTTP Log plugin.
  4. If the option is available, set the plugin scope to Global.

  5. Configure the plugin’s parameters.

    You can test out the plugin with the following sample configuration:

    • Config.Http Endpoint: http://mockbin.org/bin/:id
  6. Click Create.

Parameters

Here's a list of all the parameters which can be used in this plugin's configuration:

Form Parameter Description
name

required

Type: string

 The name of the plugin, in this case http-log.
instance_name

optional

Type: string

An optional custom name to identify an instance of the plugin, for example http-log_my-service.

Useful when running the same plugin in multiple contexts, for example, on multiple services.
service.name or service.id

optional

Type: string

The name or ID of the service the plugin targets.

Set one of these parameters if adding the plugin to a service through the top-level /plugins endpoint.

Not required if using /services/SERVICE_NAME|SERVICE_ID/plugins.
route.name or route.id

optional

Type: string

The name or ID of the route the plugin targets.

Set one of these parameters if adding the plugin to a route through the top-level /plugins endpoint.

Not required if using /routes/ROUTE_NAME|ROUTE_ID/plugins.
consumer.name or consumer.id

optional


Type: string

The name or ID of the consumer the plugin targets.

Set one of these parameters if adding the plugin to a consumer through the top-level /plugins endpoint.

Not required if using /consumers/CONSUMER_NAME|CONSUMER_ID/plugins.
enabled

optional

Type: boolean

Default value: true

 Whether this plugin will be applied.
config.http_endpoint

required

Type: string


The HTTP URL endpoint (including the protocol to use) to which the data is sent.

If the http_endpoint contains a username and password (for example, http://bob:password@example.com/logs), then Kong Gateway automatically includes a basic-auth Authorization header in the log requests.

If keyring database encryption is enabled, this value will be encrypted.
config.method

optional

Type: string

Default value: POST


An optional method used to send data to the HTTP server. Supported values are POST (default), PUT, and PATCH.
config.content_type

optional

Type: string

Default value: application/json


Indicates the type of data sent. The only available option is application/json.
config.timeout

optional

Type: number

Default value: 10000


An optional timeout in milliseconds when sending data to the upstream server.
config.keepalive

optional

Type: number

Default value: 60000


An optional value in milliseconds that defines how long an idle connection will live before being closed.
config.flush_timeout

optional

Type: number

Default value: 2


Optional time in seconds. If queue_size > 1, this is the max idle time before sending a log with less than queue_size records.
config.retry_count

optional

Type: integer

Default value: 10


Number of times to retry when sending data to the upstream server.
config.queue_size

optional

Type: integer

Default value: 1


Maximum number of log entries to be sent on each message to the upstream server.
config.headers

optional

Type: table

Default value: empty table


An optional table of headers included in the HTTP message to the upstream server. Values are indexed by header name, and each header name accepts a single string.

The following headers are not allowed: Host, Content-Length, Content-Type.

Note: Before version 3.0.0, the values were arrays of strings (multiple values per header name).
config.custom_fields_by_lua

optional

Type: map


A list of key-value pairs, where the key is the name of a log field and the value is a chunk of Lua code, whose return value sets or replaces the log field value.

Log format

Note: If the queue_size argument > 1, a request is logged as an array of JSON objects.

Every request is logged separately in a JSON object, separated by a new line \n, with the following format:

{
  "service": {
    "host": "httpbin.org",
    "created_at": 1614232642,
    "connect_timeout": 60000,
    "id": "167290ee-c682-4ebf-bdea-e49a3ac5e260",
    "protocol": "http",
    "read_timeout": 60000,
    "port": 80,
    "path": "/anything",
    "updated_at": 1614232642,
    "write_timeout": 60000,
    "retries": 5,
    "ws_id": "54baa5a9-23d6-41e0-9c9a-02434b010b25"
  },
  "route": {
    "id": "78f79740-c410-4fd9-a998-d0a60a99dc9b",
    "paths": [
      "/log"
    ],
    "protocols": [
      "http"
    ],
    "strip_path": true,
    "created_at": 1614232648,
    "ws_id": "54baa5a9-23d6-41e0-9c9a-02434b010b25",
    "request_buffering": true,
    "updated_at": 1614232648,
    "preserve_host": false,
    "regex_priority": 0,
    "response_buffering": true,
    "https_redirect_status_code": 426,
    "path_handling": "v0",
    "service": {
      "id": "167290ee-c682-4ebf-bdea-e49a3ac5e260"
    }
  },
  "request": {
    "querystring": {},
    "size": 138,
    "uri": "/log",
    "url": "http://localhost:8000/log",
    "headers": {
      "host": "localhost:8000",
      "accept-encoding": "gzip, deflate",
      "user-agent": "HTTPie/2.4.0",
      "accept": "*/*",
      "connection": "keep-alive"
    },
    "method": "GET"
  },
  "response": {
    "headers": {
      "content-type": "application/json",
      "date": "Thu, 25 Feb 2021 05:57:48 GMT",
      "connection": "close",
      "access-control-allow-credentials": "true",
      "content-length": "503",
      "server": "gunicorn/19.9.0",
      "via": "kong/2.2.1.0-enterprise-edition",
      "x-kong-proxy-latency": "57",
      "x-kong-upstream-latency": "457",
      "access-control-allow-origin": "*"
    },
    "status": 200,
    "size": 827
  },
  "latencies": {
    "request": 515,
    "kong": 58,
    "proxy": 457
  },
  "tries": [
    {
      "balancer_latency": 0,
      "port": 80,
      "balancer_start": 1614232668399,
      "ip": "18.211.130.98"
    }
  ],
  "client_ip": "192.168.144.1",
  "workspace": "54baa5a9-23d6-41e0-9c9a-02434b010b25",
  "upstream_uri": "/anything",
	"authenticated_entity": {
		"id": "c62c1455-9b1d-4f2d-8797-509ba83b8ae8"
	},
	"consumer": {
		"id": "ae974d6c-0f8a-4dc5-b701-fa0aa38592bd",
		"created_at": 1674035962,
		"username_lower": "foo",
		"username": "foo",
		"type": 0
	},
  "started_at": 1614232668342
}

JSON object considerations

  • service: Properties about the service associated with the requested route.
  • route: Properties about the specific route requested.
  • request: Properties about the request sent by the client.
  • response: Properties about the response sent to the client.
  • latencies: Latency data.
    • kong: The internal Kong Gateway latency that it takes to process the request. It varies based on the actual processing flow. Generally, it consists of three parts:
      • The time it took to find the right upstream.
      • The time it took to receive the whole response from upstream.
      • The time it took to run all plugins executed before the log phase.
    • request: The time in milliseconds that has elapsed between when the first bytes were read from the client and the last byte was sent to the client. This is useful for detecting slow clients.
    • proxy: The time in milliseconds that it took for the upstream to process the request. In other words, it’s the time elapsed between transferring the request to the final service and when Kong Gateway starts receiving the response.
  • tries: a list of iterations made by the load balancer for this request.
    • balancer_start: A Unix timestamp for when the balancer started.
    • ip: The IP address of the contacted balancer.
    • port: The port number of the contacted balancer.
    • balancer_latency: The latency of the balancer expressed in milliseconds.
  • client_ip: The original client IP address.
  • workspace: The UUID of the workspace associated with this request.
  • upstream_uri: The URI, including query parameters, for the configured upstream.
  • authenticated_entity: Properties about the authenticated credential (if an authentication plugin has been enabled).
  • consumer: The authenticated consumer (if an authentication plugin has been enabled).
  • started_at: The unix timestamp of when the request has started to be processed.

Log plugins enabled on services and routes contain information about the service or route.

Custom Headers

The log server that receives these messages might require extra headers, such as for authorization purposes.

...
  - name: http-log
    config:
      headers:
        Authorization: "Bearer <token>"
...

Kong process errors

This logging plugin logs HTTP request and response data, and also supports streams data (TCP, TLS, and UDP).

If you are looking for the Kong process error file (which is the nginx error file), you can find it at the following path:

{prefix}/logs/error.log

Custom Fields by Lua

The custom_fields_by_lua configuration allows for the dynamic modification of log fields using Lua code. Below is an example configuration that removes the existing route field in the logs:

curl -i -X POST --url http://kong:8001/plugins ... --data config.custom_fields_by_lua.route="return nil"

Similarly, new fields can be added:

curl -i -X POST --url http://kong:8001/plugins ... --data config.custom_fields_by_lua.header="return kong.request.get_header('h1')"

Limitations

Lua code runs in a restricted sandbox environment, whose behavior is governed by the untrusted_lua configuration properties configuration properties.

Sandboxing consists of several limitations in the way the Lua code can be executed, for heightened security.

The following functions are not available because they can be used to abuse the system:

  • string.rep: Can be used to allocate millions of bytes in one operation.
  • {set|get}metatable: Can be used to modify the metatables of global objects (strings, numbers).
  • collectgarbage: Can be abused to kill the performance of other workers.
  • _G: Is the root node which has access to all functions. It is masked by a temporary table.
  • load{file|string}: Is deemed unsafe because it can grant access to the global environment.
  • raw{get|set|equal}: Potentially unsafe because sandboxing relies on some metatable manipulation.
  • string.dump: Can display confidential server information (such as implementation of functions).
  • math.randomseed: Can affect the host system. Kong Gateway already seeds the random number generator properly.
  • All os.* (except os.clock, os.difftime, and os.time). os.execute can significantly alter the host system.
  • io.*: Provides access to the hard drive.
  • dofile|require: Provides access to the hard drive.

The exclusion of require means that plugins must only use PDK functions kong.*. The ngx.* abstraction is also available, but it is not guaranteed to be present in future versions of the plugin.

In addition to the above restrictions:

  • All the provided modules (like string or table) are read-only and can’t be modified.
  • Bytecode execution is disabled.

Further, as code runs in the context of the log phase, only PDK methods that can run in said phase can be used.

Changelog

Kong Gateway 3.0.x

  • The headers parameter now takes a single string per header name, where it previously took an array of values.

Kong Gateway 2.7.x

  • If keyring encryption is enabled, the config.http_endpoint parameter value will be encrypted.

Kong Gateway 2.4.x

  • Added the custom_fields_by_lua parameter.

Kong Gateway 2.3.x

  • Custom headers can now be specified for the log request using the headers parameter.