Append request and response data in JSON format to a log file. You can also specify
streams (for example, /dev/stdout
and /dev/stderr
), which is especially useful
when running Kong in Kubernetes.
This plugin uses blocking I/O, which could affect performance when writing to physical files on slow (spinning) disks.
Configuration Reference
This plugin is compatible with DB-less mode.
In DB-less mode, you configure Kong Gateway declaratively. Therefore, the Admin API is mostly read-only. The only tasks it can perform are all related to handling the declarative config, including:
- Setting a target's health status in the load balancer
- Validating configurations against schemas
- Uploading the declarative configuration using the
/config
endpoint
Example plugin configuration
Parameters
Here's a list of all the parameters which can be used in this plugin's configuration:
Form Parameter | Description |
---|---|
name
required Type: string |
The name of the plugin, in this case file-log . |
service.name or service.id
Type: string |
The name or ID of the service the plugin targets.
Set one of these parameters if adding the plugin to a service through the top-level /plugins endpoint.
Not required if using /services/SERVICE_NAME|SERVICE_ID/plugins . |
route.name or route.id
Type: string |
The name or ID of the route the plugin targets.
Set one of these parameters if adding the plugin to a route through the top-level /plugins endpoint.
Not required if using /routes/ROUTE_NAME|ROUTE_ID/plugins . |
consumer.name or consumer.id
Type: string |
The name or ID of the consumer the plugin targets.
Set one of these parameters if adding the plugin to a consumer through the top-level /plugins endpoint.
Not required if using /consumers/CONSUMER_NAME|CONSUMER_ID/plugins . |
enabled
Type: boolean Default value: true |
Whether this plugin will be applied. |
config.path
required Type: string |
The file path of the output log file. The plugin creates the log file if it doesn’t exist yet. Make sure Kong has write permissions to this file. |
config.reopen
required Type: boolean Default value: false
|
Determines whether the log file is closed and reopened on every request. If the file is not reopened, and has been removed/rotated, the plugin keeps writing to the stale file descriptor, and hence loses information. |
config.custom_fields_by_lua
optional Type: map |
A list of key-value pairs, where the key is the name of a log field and the value is a chunk of Lua code, whose return value sets or replaces the log field value. Requires Kong 2.4.x or above. |
Log format
Every request is logged separately in a JSON object, separated by a new line \n
, with the following format:
{
"latencies": {
"request": 515,
"kong": 58,
"proxy": 457
},
"service": {
"host": "httpbin.org",
"created_at": 1614232642,
"connect_timeout": 60000,
"id": "167290ee-c682-4ebf-bdea-e49a3ac5e260",
"protocol": "http",
"read_timeout": 60000,
"port": 80,
"path": "/anything",
"updated_at": 1614232642,
"write_timeout": 60000,
"retries": 5,
"ws_id": "54baa5a9-23d6-41e0-9c9a-02434b010b25"
},
"request": {
"querystring": {},
"size": 138,
"uri": "/log",
"url": "http://localhost:8000/log",
"headers": {
"host": "localhost:8000",
"accept-encoding": "gzip, deflate",
"user-agent": "HTTPie/2.4.0",
"accept": "*/*",
"connection": "keep-alive"
},
"method": "GET"
},
"tries": [
{
"balancer_latency": 0,
"port": 80,
"balancer_start": 1614232668399,
"ip": "18.211.130.98"
}
],
"client_ip": "192.168.144.1",
"workspace": "54baa5a9-23d6-41e0-9c9a-02434b010b25",
"upstream_uri": "/anything",
"response": {
"headers": {
"content-type": "application/json",
"date": "Thu, 25 Feb 2021 05:57:48 GMT",
"connection": "close",
"access-control-allow-credentials": "true",
"content-length": "503",
"server": "gunicorn/19.9.0",
"via": "kong/2.2.1.0-enterprise-edition",
"x-kong-proxy-latency": "57",
"x-kong-upstream-latency": "457",
"access-control-allow-origin": "*"
},
"status": 200,
"size": 827
},
"route": {
"id": "78f79740-c410-4fd9-a998-d0a60a99dc9b",
"paths": [
"/log"
],
"protocols": [
"http"
],
"strip_path": true,
"created_at": 1614232648,
"ws_id": "54baa5a9-23d6-41e0-9c9a-02434b010b25",
"request_buffering": true,
"updated_at": 1614232648,
"preserve_host": false,
"regex_priority": 0,
"response_buffering": true,
"https_redirect_status_code": 426,
"path_handling": "v0",
"service": {
"id": "167290ee-c682-4ebf-bdea-e49a3ac5e260"
}
},
"started_at": 1614232668342
}
JSON object considerations
A few considerations on the JSON object:
request
contains properties about the request sent by the client.response
contains properties about the response sent to the client.tries
contains the list of (re)tries (successes and failures) made by the load balancer for this request.route
contains Kong Gateway properties about the specific Route requested.service
contains Kong Gateway properties about the Service associated with the requested Route.authenticated_entity
contains Kong Gateway properties about the authenticated credential (if an authentication plugin has been enabled).workspaces
contains Kong Gateway properties of the Workspaces associated with the requested Route. Only in Kong Gateway version >= 0.34.x. For Kong Gateway version >= 2.1.x, there is aws_id
on services and routes that reference the workspace ID.consumer
contains the authenticated Consumer (if an authentication plugin has been enabled).latencies
contains some data about the latencies involved:proxy
is the time it took for the final service to process the request.kong
is the internal Kong latency that it took to run all the plugins.request
is the time elapsed between the first bytes were read from the client and after the last bytes were sent to the client. Useful for detecting slow clients.
client_ip
contains the original client IP address.started_at
contains the UTC timestamp of when the request has started to be processed.
Log plugins enabled on services and routes contain information about the service or route.
Kong process errors
This logging plugin logs HTTP request and response data, and also supports streams data (TCP, TLS, and UDP).
If you are looking for the Kong process error file (which is the nginx error file), you can find it at the following path:
{prefix}/logs/error.log
Custom Fields by Lua
The custom_fields_by_lua
configuration allows for the dynamic modification of
log fields using Lua code. Below is an example configuration that removes the
existing route
field in the logs:
curl -i -X POST --url http://kong:8001/plugins ... --data config.custom_fields_by_lua.route="return nil"
Similarly, new fields can be added:
curl -i -X POST --url http://kong:8001/plugins ... --data config.custom_fields_by_lua.header="return kong.request.get_header('h1')"
Limitations
Lua code runs in a restricted sandbox environment, whose behavior is governed
by the untrusted_lua
configuration properties configuration
properties.
Sandboxing consists of several limitations in the way the Lua code can be executed, for heightened security.
The following functions are not available because they can be used to abuse the system:
string.rep
: Can be used to allocate millions of bytes in one operation.{set|get}metatable
: Can be used to modify the metatables of global objects (strings, numbers).collectgarbage
: Can be abused to kill the performance of other workers._G
: Is the root node which has access to all functions. It is masked by a temporary table.load{file|string}
: Is deemed unsafe because it can grant access to the global environment.raw{get|set|equal}
: Potentially unsafe because sandboxing relies on some metatable manipulation.string.dump
: Can display confidential server information (such as implementation of functions).math.randomseed
: Can affect the host system. Kong Gateway already seeds the random number generator properly.- All
os.*
(exceptos.clock
,os.difftime
, andos.time
).os.execute
can significantly alter the host system. io.*
: Provides access to the hard drive.dofile|require
: Provides access to the hard drive.
The exclusion of require
means that plugins must only use PDK functions kong.*
. The ngx.*
abstraction is
also available, but it is not guaranteed to be present in future versions of the plugin.
In addition to the above restrictions:
- All the provided modules (like
string
ortable
) are read-only and can’t be modified. - Bytecode execution is disabled.
Further, as code runs in the context of the log phase, only PDK methods that can run in said phase can be used.
Changelog
Kong Gateway 2.4.x
- Added
custom_fields_by_lua
configuration option.