Skip to content
|
search
Plugin Hub
github-edit-pageEdit this page
report-issueReport an issue
header icon

File Log

By Kong Inc.

Looking for the plugin's configuration parameters? You can find them in the File Log configuration reference doc.

Append request and response data in JSON format to a log file. You can also specify streams (for example, /dev/stdout and /dev/stderr), which is especially useful when running Kong in Kubernetes.

This plugin uses blocking I/O, which could affect performance when writing to physical files on slow (spinning) disks.

Log format

Every request is logged separately in a JSON object, separated by a new line \n, with the following format:

{
  "service": {
    "host": "httpbin.org",
    "created_at": 1614232642,
    "connect_timeout": 60000,
    "id": "167290ee-c682-4ebf-bdea-e49a3ac5e260",
    "protocol": "http",
    "read_timeout": 60000,
    "port": 80,
    "path": "/anything",
    "updated_at": 1614232642,
    "write_timeout": 60000,
    "retries": 5,
    "ws_id": "54baa5a9-23d6-41e0-9c9a-02434b010b25"
  },
  "route": {
    "id": "78f79740-c410-4fd9-a998-d0a60a99dc9b",
    "paths": [
      "/log"
    ],
    "protocols": [
      "http"
    ],
    "strip_path": true,
    "created_at": 1614232648,
    "ws_id": "54baa5a9-23d6-41e0-9c9a-02434b010b25",
    "request_buffering": true,
    "updated_at": 1614232648,
    "preserve_host": false,
    "regex_priority": 0,
    "response_buffering": true,
    "https_redirect_status_code": 426,
    "path_handling": "v0",
    "service": {
      "id": "167290ee-c682-4ebf-bdea-e49a3ac5e260"
    }
  },
  "request": {
    "querystring": {},
    "size": 138,
    "uri": "/log",
    "url": "http://localhost:8000/log",
    "headers": {
      "host": "localhost:8000",
      "accept-encoding": "gzip, deflate",
      "user-agent": "HTTPie/2.4.0",
      "accept": "*/*",
      "connection": "keep-alive"
    },
    "method": "GET"
  },
  "response": {
    "headers": {
      "content-type": "application/json",
      "date": "Thu, 25 Feb 2021 05:57:48 GMT",
      "connection": "close",
      "access-control-allow-credentials": "true",
      "content-length": "503",
      "server": "gunicorn/19.9.0",
      "via": "kong/2.2.1.0-enterprise-edition",
      "x-kong-proxy-latency": "57",
      "x-kong-upstream-latency": "457",
      "access-control-allow-origin": "*"
    },
    "status": 200,
    "size": 827
  },
  "latencies": {
    "request": 515,
    "kong": 58,
    "proxy": 457
  },
  "tries": [
    {
      "balancer_latency": 0,
      "port": 80,
      "balancer_start": 1614232668399,
      "ip": "18.211.130.98"
    }
  ],
  "client_ip": "192.168.144.1",
  "workspace": "54baa5a9-23d6-41e0-9c9a-02434b010b25",
  "workspace_name": "default",
  "upstream_uri": "/anything",
	"authenticated_entity": {
		"id": "c62c1455-9b1d-4f2d-8797-509ba83b8ae8"
	},
	"consumer": {
		"id": "ae974d6c-0f8a-4dc5-b701-fa0aa38592bd",
		"created_at": 1674035962,
		"username_lower": "foo",
		"username": "foo",
		"type": 0
	},
  "started_at": 1614232668342
}

JSON object considerations

  • service: Properties about the service associated with the requested route.
  • route: Properties about the specific route requested.
  • request: Properties about the request sent by the client.
  • response: Properties about the response sent to the client.
  • latencies: Latency data.
    • kong: The internal Kong Gateway latency, in milliseconds, that it takes to process the request. It varies based on the actual processing flow. Generally, it consists of three parts:
      • The time it took to find the right upstream.
      • The time it took to receive the whole response from upstream.
      • The time it took to run all plugins executed before the log phase.
    • request: The time in milliseconds that has elapsed between when the first bytes were read from the client and the last byte was sent to the client. This is useful for detecting slow clients.
    • proxy: The time in milliseconds that it took for the upstream to process the request. In other words, it’s the time elapsed between transferring the request to the final service and when Kong Gateway starts receiving the response.
  • tries: a list of iterations made by the load balancer for this request.
    • balancer_start: A Unix timestamp for when the balancer started.
    • ip: The IP address of the contacted balancer.
    • port: The port number of the contacted balancer.
    • balancer_latency: The latency of the balancer expressed in milliseconds.
  • client_ip: The original client IP address.

  • workspace: The UUID of the workspace associated with this request.

  • workspace_name: The name of the workspace associated with this request.

  • upstream_uri: The URI, including query parameters, for the configured upstream.
  • authenticated_entity: Properties about the authenticated credential (if an authentication plugin has been enabled).
  • consumer: The authenticated consumer (if an authentication plugin has been enabled).
  • started_at: The unix timestamp of when the request has started to be processed.

Log plugins enabled on services and routes contain information about the service or route.

Kong process errors

This logging plugin logs HTTP request and response data, and also supports stream data (TCP, TLS, and UDP).

The Kong process error file is the Nginx error file. You can find it at the following path:

{prefix}/logs/error.log

Configure the prefix in kong.conf.

Custom Fields by Lua

The custom_fields_by_lua configuration allows for the dynamic modification of log fields using Lua code. Below is a snippet of an example configuration that removes the route field from the logs:

curl -i -X POST http://localhost:8001/plugins \
... 
  --data config.custom_fields_by_lua.route="return nil"

Similarly, new fields can be added:

curl -i -X POST http://localhost:8001/plugins \
... 
  --data config.custom_fields_by_lua.header="return kong.request.get_header('h1')"

Limitations

Lua code runs in a restricted sandbox environment, whose behavior is governed by the untrusted_lua configuration properties configuration properties.

Sandboxing consists of several limitations in the way the Lua code can be executed, for heightened security.

The following functions are not available because they can be used to abuse the system:

  • string.rep: Can be used to allocate millions of bytes in one operation.
  • {set|get}metatable: Can be used to modify the metatables of global objects (strings, numbers).
  • collectgarbage: Can be abused to kill the performance of other workers.
  • _G: Is the root node which has access to all functions. It is masked by a temporary table.
  • load{file|string}: Is deemed unsafe because it can grant access to the global environment.
  • raw{get|set|equal}: Potentially unsafe because sandboxing relies on some metatable manipulation.
  • string.dump: Can display confidential server information (such as implementation of functions).
  • math.randomseed: Can affect the host system. Kong Gateway already seeds the random number generator properly.
  • All os.* (except os.clock, os.difftime, and os.time). os.execute can significantly alter the host system.
  • io.*: Provides access to the hard drive.
  • dofile|require: Provides access to the hard drive.

The exclusion of require means that plugins must only use PDK functions kong.*. The ngx.* abstraction is also available, but it is not guaranteed to be present in future versions of the plugin.

In addition to the above restrictions:

  • All the provided modules (like string or table) are read-only and can’t be modified.
  • Bytecode execution is disabled.

Further, as code runs in the context of the log phase, only PDK methods that can run in said phase can be used.