MeshAccessLog
With the MeshAccessLog policy you can easily set up access logs on every data plane proxy in a mesh.
This policy uses a new policy matching algorithm. Do not combine with TrafficLog.
This guide assumes you have already configured your observability tools to work with Kuma. If you haven’t, see the observability docs.
targetRef
support matrix
targetRef |
Allowed kinds |
---|---|
targetRef.kind |
Mesh , MeshSubset
|
to[].targetRef.kind |
Mesh , MeshService , MeshExternalService
|
from[].targetRef.kind |
Mesh |
To learn more about the information in this table, see the matching docs.
Configuration
Format
Kuma gives you full control over the format of the access logs.
The shape of a single log record is defined by a template string that uses command operators to extract and format data about a TCP
connection or an HTTP
request.
For example:
%START_TIME% %KUMA_SOURCE_SERVICE% => %KUMA_DESTINATION_SERVICE% %DURATION%
%START_TIME%
and %KUMA_SOURCE_SERVICE%
are examples of available command operators.
All command operators defined by Envoy are supported, along with additional command operators defined by Kuma:
Command Operator | Description |
---|---|
%KUMA_MESH% |
Name of the mesh in which traffic is flowing. |
%KUMA_SOURCE_SERVICE% |
Name of a service that is the source of traffic. |
%KUMA_DESTINATION_SERVICE% |
Name of a service that is the destination of traffic. |
%KUMA_SOURCE_ADDRESS_WITHOUT_PORT% |
Address of a Dataplane that is the source of traffic. |
%KUMA_TRAFFIC_DIRECTION% |
Direction of the traffic, INBOUND , OUTBOUND , or UNSPECIFIED . |
All additional access log command operators are valid to use with both TCP
and HTTP
traffic.
Internally, Kuma determines traffic protocol based on the value of kuma.io/protocol
tag on the inbound
interface of a destination
Dataplane
.
There are two types of format
, plain
and json
.
Plain accepts a string with command operators and produces a string output.
JSON accepts a list of key-value pairs that produces a valid JSON object.
It is up to the user to decide which format type to use. Some system will automatically parse JSON logs and allow you to filter and query based on available keys.
If a command operator is specific to HTTP
traffic, such as %REQ(X?Y):Z%
or %RESP(X?Y):Z%
, in the case of TCP traffic it will be replaced by a symbol “-
” for plain
and a null
value for json
.
You can set the format.omitEmptyValues
boolean option to change this to ""
for plain
and
omit them entirely for json
.
Plain
The default format string for TCP
traffic is:
[%START_TIME%] %RESPONSE_FLAGS% %KUMA_MESH% %KUMA_SOURCE_ADDRESS_WITHOUT_PORT%(%KUMA_SOURCE_SERVICE%)->%UPSTREAM_HOST%(%KUMA_DESTINATION_SERVICE%) took %DURATION%ms, sent %BYTES_SENT% bytes, received: %BYTES_RECEIVED% bytes
The default format string for HTTP
traffic is:
[%START_TIME%] %KUMA_MESH% "%REQ(:METHOD)% %REQ(X-ENVOY-ORIGINAL-PATH?:PATH)% %PROTOCOL%" %RESPONSE_CODE% %RESPONSE_FLAGS% %BYTES_RECEIVED% %BYTES_SENT% %DURATION% %RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)% "%REQ(X-FORWARDED-FOR)%" "%REQ(USER-AGENT)%" "%REQ(X-REQUEST-ID)%" "%REQ(:AUTHORITY)%" "%KUMA_SOURCE_SERVICE%" "%KUMA_DESTINATION_SERVICE%" "%KUMA_SOURCE_ADDRESS_WITHOUT_PORT%" "%UPSTREAM_HOST%"
Example configuration:
format:
type: Plain
plain: '[%START_TIME%] %BYTES_RECEIVED%'
Example output:
[2016-04-15T20:17:00.310Z] 154
JSON
Example configuration:
format:
type: Json
json:
- key: "start_time"
value: "%START_TIME%"
- key: "bytes_received"
value: "%BYTES_RECEIVED%"
Example output:
{
"start_time": "2016-04-15T20:17:00.310Z",
"bytes_received": "154"
}
TCP configuration with default fields:
format:
type: Json
json:
- key: "start_time"
value: "%START_TIME%"
- key: "response_flags"
value: "%RESPONSE_FLAGS%"
- key: "kuma_mesh"
value: "%KUMA_MESH%"
- key: "kuma_source_address_without_port"
value: "%KUMA_SOURCE_ADDRESS_WITHOUT_PORT%"
- key: "kuma_source_service"
value: "%KUMA_SOURCE_SERVICE%"
- key: "upstream_host"
value: "%UPSTREAM_HOST%"
- key: "kuma_destination_service"
value: "%KUMA_DESTINATION_SERVICE%"
- key: "duration_ms"
value: "%DURATION%"
- key: "bytes_sent"
value: "%BYTES_SENT%"
- key: "bytes_received"
value: "%BYTES_RECEIVED%"
HTTP configuration with default fields:
format:
type: Json
json:
- key: "start_time"
value: "%START_TIME%"
- key: "kuma_mesh"
value: "%KUMA_MESH%"
- key: 'method'
value: '"%REQ(:METHOD)%'
- key: "path"
value: "%REQ(X-ENVOY-ORIGINAL-PATH?:PATH)%"
- key: 'protocol'
value: '%PROTOCOL%'
- key: "response_code"
value: "%RESPONSE_CODE%"
- key: "response_flags"
value: "%RESPONSE_FLAGS%"
- key: "bytes_received"
value: "%BYTES_RECEIVED%"
- key: "bytes_sent"
value: "%BYTES_SENT%"
- key: "duration_ms"
value: "%DURATION%"
- key: "upstream_service_time"
value: "%RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)%"
- key: 'x_forwarded_for'
value: '"%REQ(X-FORWARDED-FOR)%"'
- key: 'user_agent'
value: '"%REQ(USER-AGENT)%"'
- key: 'request_id'
value: '"%REQ(X-REQUEST-ID)%"'
- key: 'authority'
value: '"%REQ(:AUTHORITY)%"'
- key: "kuma_source_service"
value: "%KUMA_SOURCE_SERVICE%"
- key: "kuma_destination_service"
value: "%KUMA_DESTINATION_SERVICE%"
- key: "kuma_source_address_without_port"
value: "%KUMA_SOURCE_ADDRESS_WITHOUT_PORT%"
- key: "upstream_host"
value: "%UPSTREAM_HOST%"
Backends
A backend determines where the logs end up.
TCP
A TCP backend streams logs to a server via TCP protocol. You can configure a TCP backend with an address:
backends:
- type: Tcp
tcp:
address: 127.0.0.1:5000
File
A file backend streams logs to a text file. You can configure a file backend with a path:
backends:
- type: File
file:
path: /dev/stdout
OpenTelemetry
An OpenTelemetry (OTel) backend sends data to an OpenTelemetry server. You can configure an OpenTelemetry backend with an endpoint, attributes (which contain additional information about the log) and body (can be a string message, including multi-line, or it can be a structured data). Attributes and endpoints can use placeholders described in the format section.
backends:
- type: OpenTelemetry
openTelemetry:
endpoint: otel-collector:4317
body:
kvlistValue:
values:
- key: "mesh"
value:
stringValue: "%KUMA_MESH%"
attributes:
- key: "start_time"
value: "%START_TIME%"
Body
Body is of type any (defined here) and can be one of the following forms:
body:
stringValue: "%KUMA_MESH%"
body:
boolValue: true
body:
intValue: 123
body:
doubleValue: 1.2
body:
bytesValue: aGVsbG8=
body:
arrayValue:
values:
- stringValue: "%KUMA_MESH%"
body:
kvlistValue:
values:
- key: "mesh"
value:
stringValue: "%KUMA_MESH%"
Examples
Log outgoing traffic from specific frontend version to a backend service
apiVersion: kuma.io/v1alpha1
kind: MeshAccessLog
metadata:
name: frontend-to-backend
namespace: kuma-demo
labels:
kuma.io/mesh: default
spec:
targetRef:
kind: MeshSubset
tags:
app: frontend
version: canary
to:
- targetRef:
kind: MeshService
name: backend
namespace: kuma-demo
sectionName: http
default:
backends:
- type: File
file:
path: "/dev/stdout"
Logging to multiple backends
This configuration logs to three backends: TCP, file and OpenTelemetry.
apiVersion: kuma.io/v1alpha1
kind: MeshAccessLog
metadata:
name: multiple-backends
namespace: kuma-demo
labels:
kuma.io/mesh: default
spec:
from:
- targetRef:
kind: Mesh
default:
backends:
- type: Tcp
tcp:
address: 127.0.0.1:5000
format:
type: Json
json:
- key: start_time
value: "%START_TIME%"
- type: File
file:
path: "/dev/stdout"
format:
type: Plain
plain: "[%START_TIME%]"
- type: OpenTelemetry
openTelemetry:
endpoint: otel-collector:4317
body:
kvlistValue:
values:
- key: mesh
value:
stringValue: "%KUMA_MESH%"
attributes:
- key: start_time
value: "%START_TIME%"
Log all incoming and outgoing traffic
For this use case we recommend creating two separate policies. One for incoming traffic:
apiVersion: kuma.io/v1alpha1
kind: MeshAccessLog
metadata:
name: all-incoming-traffic
namespace: kuma-demo
labels:
kuma.io/mesh: default
spec:
from:
- targetRef:
kind: Mesh
default:
backends:
- type: File
file:
path: "/dev/stdout"
And one for outgoing traffic:
apiVersion: kuma.io/v1alpha1
kind: MeshAccessLog
metadata:
name: all-outgoing-traffic
namespace: kuma-demo
labels:
kuma.io/mesh: default
spec:
to:
- targetRef:
kind: Mesh
default:
backends:
- type: File
file:
path: "/dev/stdout"
Logging traffic going outside the Mesh
To target ExternalServices
, use MeshService
as the targetRef
kind with name
set to
the kuma.io/service
value.
To target other non-mesh traffic, for example passthrough traffic, use Mesh
as the targetRef
kind. In this case, %KUMA_DESTINATION_SERVICE%
is set to external
.
Select a built-in gateway
You can select a built-in gateway using the kuma.io/service
value. A current limitation is that traffic routed from a gateway to a service is logged by that gateway as having destination "*"
.
All policy options
Spec is the specification of the Kuma MeshAccessLog resource.
Type: object
Properties
- from
- From list makes a match between clients and corresponding configurations
- Type:
array
- Items
- Type:
object
- Properties
- default
- Default is a configuration specific to the group of clients referenced in'targetRef'
- Type:
object
- Properties
- backends
- Type:
array
- Items
- Type:
object
- Properties
- file
- FileBackend defines configuration for file based access logs
- Type:
object
- Properties
- format
- Format of access logs. Placeholders available onhttps://www.envoyproxy.io/docs/envoy/latest/configuration/observability/accesslog/usage#command-operators_
- Type:
object
- Properties
- json
- Type:
array
- Items
- Type:
object
- Properties
- key
- Type:
string
- Type:
- value
- Type:
string
- Type:
- key
- Type:
- omitEmptyValues
- Type:
boolean
- Default: false
- Type:
- plain
- Type:
string
- Type:
- type
required
- Type:
string
- The value is restricted to the following:
- "Plain"
- "Json"
- Type:
- json
- path
required
- Path to a file that logs will be written to
- Type:
string
- Length: ≥ 1
- format
- openTelemetry
- Defines an OpenTelemetry logging backend.
- Type:
object
- Properties
- attributes
- Attributes can contain placeholders available onhttps://www.envoyproxy.io/docs/envoy/latest/configuration/observability/accesslog/usage#command-operators_
- Type:
array
- Items
- Type:
object
- Properties
- key
- Type:
string
- Type:
- value
- Type:
string
- Type:
- key
- body
- Body is a raw string or an OTLP any value as described athttps://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/logs/data-model.md#field-bodyIt can contain placeholders available onhttps://www.envoyproxy.io/docs/envoy/latest/configuration/observability/accesslog/usage#command-operators_
- endpoint
required
- Endpoint of OpenTelemetry collector. An empty port defaults to 4317.
- Type:
string
- Length: ≥ 1
- attributes
- tcp
- TCPBackend defines a TCP logging backend.
- Type:
object
- Properties
- address
required
- Address of the TCP logging backend
- Type:
string
- Length: ≥ 1
- format
- Format of access logs. Placeholders available onhttps://www.envoyproxy.io/docs/envoy/latest/configuration/observability/accesslog/usage#command-operators_
- Type:
object
- Properties
- json
- Type:
array
- Items
- Type:
object
- Properties
- key
- Type:
string
- Type:
- value
- Type:
string
- Type:
- key
- Type:
- omitEmptyValues
- Type:
boolean
- Default: false
- Type:
- plain
- Type:
string
- Type:
- type
required
- Type:
string
- The value is restricted to the following:
- "Plain"
- "Json"
- Type:
- json
- address
- type
required
- Type:
string
- The value is restricted to the following:
- "Tcp"
- "File"
- "OpenTelemetry"
- Type:
- file
- Type:
- backends
- targetRef
required
- TargetRef is a reference to the resource that represents a group ofclients.
- Type:
object
- Properties
- kind
- Kind of the referenced resource
- Type:
string
- The value is restricted to the following:
- "Mesh"
- "MeshSubset"
- "MeshGateway"
- "MeshService"
- "MeshExternalService"
- "MeshMultiZoneService"
- "MeshServiceSubset"
- "MeshHTTPRoute"
- labels
- Labels are used to select group of MeshServices that match labels. Either Labels orName and Namespace can be used.
- Type:
object
- This schema accepts additional properties.
- Properties
- mesh
- Mesh is reserved for future use to identify cross mesh resources.
- Type:
string
- name
- Name of the referenced resource. Can only be used with kinds:
MeshService
,MeshServiceSubset
andMeshGatewayRoute
- Type:
string
- Name of the referenced resource. Can only be used with kinds:
- namespace
- Namespace specifies the namespace of target resource. If empty only resources in policy namespacewill be targeted.
- Type:
string
- proxyTypes
- ProxyTypes specifies the data plane types that are subject to the policy. When not specified,all data plane types are targeted by the policy.
- Type:
array
- Item Count: ≥ 1
- Items
- Type:
string
- The value is restricted to the following:
- "Sidecar"
- "Gateway"
- sectionName
- SectionName is used to target specific section of resource.For example, you can target port from MeshService.ports[] by its name. Only traffic to this port will be affected.
- Type:
string
- tags
- Tags used to select a subset of proxies by tags. Can only be used with kinds
MeshSubset
andMeshServiceSubset
- Type:
object
- This schema accepts additional properties.
- Properties
- Tags used to select a subset of proxies by tags. Can only be used with kinds
- kind
- default
- targetRef
- TargetRef is a reference to the resource the policy takes an effect on.The resource could be either a real store object or virtual resourcedefined in-place.
- Type:
object
- Properties
- kind
- Kind of the referenced resource
- Type:
string
- The value is restricted to the following:
- "Mesh"
- "MeshSubset"
- "MeshGateway"
- "MeshService"
- "MeshExternalService"
- "MeshMultiZoneService"
- "MeshServiceSubset"
- "MeshHTTPRoute"
- labels
- Labels are used to select group of MeshServices that match labels. Either Labels orName and Namespace can be used.
- Type:
object
- This schema accepts additional properties.
- Properties
- mesh
- Mesh is reserved for future use to identify cross mesh resources.
- Type:
string
- name
- Name of the referenced resource. Can only be used with kinds:
MeshService
,MeshServiceSubset
andMeshGatewayRoute
- Type:
string
- Name of the referenced resource. Can only be used with kinds:
- namespace
- Namespace specifies the namespace of target resource. If empty only resources in policy namespacewill be targeted.
- Type:
string
- proxyTypes
- ProxyTypes specifies the data plane types that are subject to the policy. When not specified,all data plane types are targeted by the policy.
- Type:
array
- Item Count: ≥ 1
- Items
- Type:
string
- The value is restricted to the following:
- "Sidecar"
- "Gateway"
- sectionName
- SectionName is used to target specific section of resource.For example, you can target port from MeshService.ports[] by its name. Only traffic to this port will be affected.
- Type:
string
- tags
- Tags used to select a subset of proxies by tags. Can only be used with kinds
MeshSubset
andMeshServiceSubset
- Type:
object
- This schema accepts additional properties.
- Properties
- Tags used to select a subset of proxies by tags. Can only be used with kinds
- kind
- to
- To list makes a match between the consumed services and corresponding configurations
- Type:
array
- Items
- Type:
object
- Properties
- default
- Default is a configuration specific to the group of destinations referenced in'targetRef'
- Type:
object
- Properties
- backends
- Type:
array
- Items
- Type:
object
- Properties
- file
- FileBackend defines configuration for file based access logs
- Type:
object
- Properties
- format
- Format of access logs. Placeholders available onhttps://www.envoyproxy.io/docs/envoy/latest/configuration/observability/accesslog/usage#command-operators_
- Type:
object
- Properties
- json
- Type:
array
- Items
- Type:
object
- Properties
- key
- Type:
string
- Type:
- value
- Type:
string
- Type:
- key
- Type:
- omitEmptyValues
- Type:
boolean
- Default: false
- Type:
- plain
- Type:
string
- Type:
- type
required
- Type:
string
- The value is restricted to the following:
- "Plain"
- "Json"
- Type:
- json
- path
required
- Path to a file that logs will be written to
- Type:
string
- Length: ≥ 1
- format
- openTelemetry
- Defines an OpenTelemetry logging backend.
- Type:
object
- Properties
- attributes
- Attributes can contain placeholders available onhttps://www.envoyproxy.io/docs/envoy/latest/configuration/observability/accesslog/usage#command-operators_
- Type:
array
- Items
- Type:
object
- Properties
- key
- Type:
string
- Type:
- value
- Type:
string
- Type:
- key
- body
- Body is a raw string or an OTLP any value as described athttps://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/logs/data-model.md#field-bodyIt can contain placeholders available onhttps://www.envoyproxy.io/docs/envoy/latest/configuration/observability/accesslog/usage#command-operators_
- endpoint
required
- Endpoint of OpenTelemetry collector. An empty port defaults to 4317.
- Type:
string
- Length: ≥ 1
- attributes
- tcp
- TCPBackend defines a TCP logging backend.
- Type:
object
- Properties
- address
required
- Address of the TCP logging backend
- Type:
string
- Length: ≥ 1
- format
- Format of access logs. Placeholders available onhttps://www.envoyproxy.io/docs/envoy/latest/configuration/observability/accesslog/usage#command-operators_
- Type:
object
- Properties
- json
- Type:
array
- Items
- Type:
object
- Properties
- key
- Type:
string
- Type:
- value
- Type:
string
- Type:
- key
- Type:
- omitEmptyValues
- Type:
boolean
- Default: false
- Type:
- plain
- Type:
string
- Type:
- type
required
- Type:
string
- The value is restricted to the following:
- "Plain"
- "Json"
- Type:
- json
- address
- type
required
- Type:
string
- The value is restricted to the following:
- "Tcp"
- "File"
- "OpenTelemetry"
- Type:
- file
- Type:
- backends
- targetRef
required
- TargetRef is a reference to the resource that represents a group ofdestinations.
- Type:
object
- Properties
- kind
- Kind of the referenced resource
- Type:
string
- The value is restricted to the following:
- "Mesh"
- "MeshSubset"
- "MeshGateway"
- "MeshService"
- "MeshExternalService"
- "MeshMultiZoneService"
- "MeshServiceSubset"
- "MeshHTTPRoute"
- labels
- Labels are used to select group of MeshServices that match labels. Either Labels orName and Namespace can be used.
- Type:
object
- This schema accepts additional properties.
- Properties
- mesh
- Mesh is reserved for future use to identify cross mesh resources.
- Type:
string
- name
- Name of the referenced resource. Can only be used with kinds:
MeshService
,MeshServiceSubset
andMeshGatewayRoute
- Type:
string
- Name of the referenced resource. Can only be used with kinds:
- namespace
- Namespace specifies the namespace of target resource. If empty only resources in policy namespacewill be targeted.
- Type:
string
- proxyTypes
- ProxyTypes specifies the data plane types that are subject to the policy. When not specified,all data plane types are targeted by the policy.
- Type:
array
- Item Count: ≥ 1
- Items
- Type:
string
- The value is restricted to the following:
- "Sidecar"
- "Gateway"
- sectionName
- SectionName is used to target specific section of resource.For example, you can target port from MeshService.ports[] by its name. Only traffic to this port will be affected.
- Type:
string
- tags
- Tags used to select a subset of proxies by tags. Can only be used with kinds
MeshSubset
andMeshServiceSubset
- Type:
object
- This schema accepts additional properties.
- Properties
- Tags used to select a subset of proxies by tags. Can only be used with kinds
- kind
- default
Generated with json-schema-md-doc