Skip to content
Kong Logo | Kong Docs Logo
search
  • We're Hiring!
  • Docs
    • Kong Gateway
    • Kong Konnect
    • Kong Mesh
    • Plugin Hub
    • decK
    • Kubernetes Ingress Controller
    • Insomnia
    • Kuma

    • Docs contribution guidelines
  • Plugin Hub
  • Support
  • Community
  • Kong Academy
Get a Demo Start Free Trial
  • Kong Gateway
  • Kong Konnect
  • Kong Mesh
  • Plugin Hub
  • decK
  • Kubernetes Ingress Controller
  • Insomnia
  • Kuma

  • Docs contribution guidelines
  • 2.1.x (latest)
  • 2.0.x
  • 1.9.x
  • 1.8.x
  • 1.7.x
  • 1.6.x
  • 1.5.x
  • 1.4.x
  • 1.3.x
  • 1.2.x
  • 1.1.x
  • 1.0.x
    • Introduction to Kong Mesh
    • What is Service Mesh?
    • How Kong Mesh works
    • Deployments
    • Version support policy
    • Stability
    • Release notes
    • Installation Options
    • Kubernetes
    • Helm
    • OpenShift
    • Docker
    • Amazon ECS
    • Amazon Linux
    • Red Hat
    • CentOS
    • Debian
    • Ubuntu
    • macOS
    • Windows
    • Explore Kong Mesh with the Kubernetes demo app
    • Explore Kong Mesh with the Universal demo app
    • Standalone deployment
    • Multi-zone deployment
    • License
    • Overview
    • Data plane proxy
    • Data plane on Kubernetes
    • Data plane on Universal
    • Gateway
    • Zone Ingress
    • Zone Egress
    • CLI
    • GUI
    • Observability
    • Inspect API
    • Kubernetes Gateway API
    • Networking
    • Service Discovery
    • DNS
    • Kong Mesh CNI
    • Transparent Proxying
    • IPv6 support
    • Secure access across Kong Mesh components
    • Secrets
    • Kong Mesh API Access Control
    • API server authentication
    • Data plane proxy authentication
    • Zone proxy authentication
    • Data plane proxy membership
    • Dataplane Health
    • Fine-tuning
    • Control Plane Configuration
    • Upgrades
    • Requirements
    • Introduction
    • General notes about Kong Mesh policies
    • Applying Policies
    • How Kong Mesh chooses the right policy to apply
    • Understanding TargetRef policies
    • Protocol support in Kong Mesh
    • Mesh
    • Mutual TLS
    • Traffic Permissions
    • Traffic Route
    • Traffic Metrics
    • Traffic Trace
    • Traffic Log
    • Locality-aware Load Balancing
    • Fault Injection
    • Health Check
    • Circuit Breaker
    • Proxy Template
    • External Service
    • Retry
    • Timeout
    • Rate Limit
    • Virtual Outbound
    • MeshGateway
    • MeshGatewayRoute
    • Service Health Probes
    • MeshTrace (Beta)
    • MeshAccessLog (Beta)
    • MeshTrafficPermission (Beta)
    • Overview
    • HashiCorp Vault CA
    • Amazon ACM Private CA
    • cert-manager Private CA
    • OPA policy support
    • Multi-zone authentication
    • FIPS support
    • Certificate Authority rotation
    • Role-Based Access Control
    • UBI Images
    • Windows Support
    • Auditing
    • HTTP API
    • Annotations and labels in Kubernetes mode
    • Kong Mesh data collection
      • Mesh
      • CircuitBreaker
      • ExternalService
      • FaultInjection
      • HealthCheck
      • MeshGateway
      • MeshGatewayRoute
      • ProxyTemplate
      • RateLimit
      • Retry
      • Timeout
      • TrafficLog
      • TrafficPermission
      • TrafficRoute
      • TrafficTrace
      • VirtualOutbound
      • Dataplane
      • ZoneEgress
      • ZoneIngress
      • kuma-cp
      • kuma-dp
      • kumactl
    • Kuma-cp configuration reference

github-edit-pageEdit this page

report-issueReport an issue

enterprise-switcher-iconSwitch to OSS

On this page
  • targetRef support matrix
  • Configuration
    • Format
    • Backends
  • Examples
    • Log outgoing traffic from specific frontend version to a backend service
    • Logging to multiple backends
  • Log all incoming and outgoing traffic
  • Logging traffic going outside the Mesh
  • Select a built-in gateway
Kong Mesh
2.0.x
  • Home
  • Kong Mesh
  • Policies
  • MeshAccessLog (beta)
You are browsing documentation for an outdated version. See the latest documentation here.

MeshAccessLog (beta)

With the MeshAccessLog policy you can easily set up access logs on every data plane proxy in a mesh.

This policy uses a new policy matching algorithm and is in beta state. It should not be combined with TrafficLog.

This guide assumes you have already configured your observability tools to work with Kuma. If you haven’t, see the observability docs.

targetRef support matrix

targetRef.kind top level to from
Mesh ✅ ✅ ✅
MeshSubset ✅ ❌ ❌
MeshService ✅ ✅ ❌
MeshServiceSubset ✅ ❌ ❌

To learn more about the information in this table, see the matching docs.

Configuration

Format

Kuma gives you full control over the format of the access logs.

The shape of a single log record is defined by a template string that uses command operators to extract and format data about a TCP connection or an HTTP request.

For example:

%START_TIME% %KUMA_SOURCE_SERVICE% => %KUMA_DESTINATION_SERVICE% %DURATION%

%START_TIME% and %KUMA_SOURCE_SERVICE% are examples of available command operators.

All command operators defined by Envoy are supported, along with additional command operators defined by Kuma:

Command Operator Description
%KUMA_MESH% Name of the mesh in which traffic is flowing.
%KUMA_SOURCE_SERVICE% Name of a service that is the source of traffic.
%KUMA_DESTINATION_SERVICE% Name of a service that is the destination of traffic.
%KUMA_SOURCE_ADDRESS_WITHOUT_PORT% Address of a Dataplane that is the source of traffic.
%KUMA_TRAFFIC_DIRECTION% Direction of the traffic, INBOUND, OUTBOUND, or UNSPECIFIED.

All additional access log command operators are valid to use with both TCP and HTTP traffic.

If a command operator is specific to HTTP traffic, such as %REQ(X?Y):Z% or %RESP(X?Y):Z%, it will be replaced by a symbol “-” in case of TCP traffic.

Internally, Kuma determines traffic protocol based on the value of kuma.io/protocol tag on the inbound interface of a destination Dataplane.

There are two types of format, plain and json.

Plain accepts a string with command operators and produces a string output.

JSON accepts a list of key-value pairs that produces a valid JSON object.

It is up to the user to decide which format type to use. Some system will automatically parse JSON logs and allow you to filter and query based on available keys.

Plain

The default format string for TCP traffic is:

[%START_TIME%] %RESPONSE_FLAGS% %KUMA_MESH% %KUMA_SOURCE_ADDRESS_WITHOUT_PORT%(%KUMA_SOURCE_SERVICE%)->%UPSTREAM_HOST%(%KUMA_DESTINATION_SERVICE%) took %DURATION%ms, sent %BYTES_SENT% bytes, received: %BYTES_RECEIVED% bytes

The default format string for HTTP traffic is:

[%START_TIME%] %KUMA_MESH% "%REQ(:METHOD)% %REQ(X-ENVOY-ORIGINAL-PATH?:PATH)% %PROTOCOL%" %RESPONSE_CODE% %RESPONSE_FLAGS% %BYTES_RECEIVED% %BYTES_SENT% %DURATION% %RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)% "%REQ(X-FORWARDED-FOR)%" "%REQ(USER-AGENT)%" "%REQ(X-REQUEST-ID)%" "%REQ(:AUTHORITY)%" "%KUMA_SOURCE_SERVICE%" "%KUMA_DESTINATION_SERVICE%" "%KUMA_SOURCE_ADDRESS_WITHOUT_PORT%" "%UPSTREAM_HOST%"

Example configuration:

format:
  plain: '[%START_TIME%] %BYTES_RECEIVED%'

Example output:

[2016-04-15T20:17:00.310Z] 154

JSON

Example configuration:

format:
  json:
    - key: "start_time"
      value: "%START_TIME%"
    - key: "bytes_received"
      value: "%BYTES_RECEIVED%"

Example output:

{
  "start_time": "2016-04-15T20:17:00.310Z",
  "bytes_received": "154"
}
TCP configuration with default fields:
format:
  json:
    - key: "start_time"
      value: "%START_TIME%"
    - key: "response_flags"
      value: "%RESPONSE_FLAGS%"
    - key: "kuma_mesh"
      value: "%KUMA_MESH%"
    - key: "kuma_source_address_without_port"
      value: "%KUMA_SOURCE_ADDRESS_WITHOUT_PORT%"
    - key: "kuma_source_service"
      value: "%KUMA_SOURCE_SERVICE%"
    - key: "upstream_host"
      value: "%UPSTREAM_HOST%"
    - key: "kuma_destination_service"
      value: "%KUMA_DESTINATION_SERVICE%"
    - key: "duration_ms"
      value: "%DURATION%"
    - key: "bytes_sent"
      value: "%BYTES_SENT%"
    - key: "bytes_received"
      value: "%BYTES_RECEIVED%"
HTTP configuration with default fields:
format:
  json:
    - key: "start_time"
      value: "%START_TIME%"
    - key: "kuma_mesh"
      value: "%KUMA_MESH%"
    - key: 'method'
      value: '"%REQ(:METHOD)%'
    - key: "path"
      value: "%REQ(X-ENVOY-ORIGINAL-PATH?:PATH)%"
    - key: 'protocol'
      value: '%PROTOCOL%'
    - key: "response_code"
      value: "%RESPONSE_CODE%"
    - key: "response_flags"
      value: "%RESPONSE_FLAGS%"
    - key: "bytes_received"
      value: "%BYTES_RECEIVED%"
    - key: "bytes_sent"
      value: "%BYTES_SENT%"
    - key: "duration_ms"
      value: "%DURATION%"
    - key: "upstream_service_time"
      value: "%RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)%"
    - key: 'x_forwarded_for'
      value: '"%REQ(X-FORWARDED-FOR)%"'
    - key: 'user_agent'
      value: '"%REQ(USER-AGENT)%"'
    - key: 'request_id'
      value: '"%REQ(X-REQUEST-ID)%"'
    - key: 'authority'
      value: '"%REQ(:AUTHORITY)%"'
    - key: "kuma_source_service"
      value: "%KUMA_SOURCE_SERVICE%"
    - key: "kuma_destination_service"
      value: "%KUMA_DESTINATION_SERVICE%"
    - key: "kuma_source_address_without_port"
      value: "%KUMA_SOURCE_ADDRESS_WITHOUT_PORT%"
    - key: "upstream_host"
      value: "%UPSTREAM_HOST%"

Backends

A backend determines where the logs end up.

TCP

A TCP backend streams logs to a server via TCP protocol. You can configure a TCP backend with an address:

backends:
  - tcp:
      address: 127.0.0.1:5000

File

A file backend streams logs to a text file. You can configure a file backend with a path:

backends:
  - file:
      path: /tmp/access.log

Examples

Log outgoing traffic from specific frontend version to a backend service

Kubernetes
Universal
apiVersion: kuma.io/v1alpha1
kind: MeshAccessLog
metadata:
  name: default
  namespace: kong-mesh-system
  labels:
    kuma.io/mesh: default # optional, defaults to `default` if it isn't configured
spec:
  targetRef:
    kind: MeshServiceSubset
    name: frontend
    tags:
      version: canary
  to:
    - targetRef:
        kind: MeshService
        name: backend
      default:
        backends:
          - file:
              path: /tmp/access.log

Apply the configuration with kubectl apply -f [..].

type: MeshAccessLog
name: default
mesh: default
spec:
  targetRef:
    kind: MeshServiceSubset
    name: frontend
    tags:
      version: canary
  to:
    - targetRef:
        kind: MeshService
        name: backend
      default:
        backends:
          - file:
              path: /tmp/access.log

Apply the configuration with kumactl apply -f [..] or with the HTTP API.

Logging to multiple backends

This configuration logs to two backends: TCP and file.

Kubernetes
Universal
apiVersion: kuma.io/v1alpha1
kind: MeshAccessLog
metadata:
  name: default
  namespace: kong-mesh-system
  labels:
    kuma.io/mesh: default # optional, defaults to `default` if it isn't configured
spec:
  targetRef:
    kind: Mesh
  from:
    - targetRef:
        kind: Mesh
      default:
        backends:
          - file:
              address: 127.0.0.1:5000
              format:
                json:
                  - key: "start_time"
                    value: "%START_TIME%"
          - file:
              path: /tmp/access.log
              format:
                plain: '[%START_TIME%]'

Apply the configuration with kubectl apply -f [..].

type: MeshAccessLog
name: default
mesh: default
spec:
  targetRef:
    kind: Mesh
  from:
    - targetRef:
        kind: Mesh
      default:
        backends:
          - file:
              address: 127.0.0.1:5000
              format:
                json:
                  - key: "start_time"
                    value: "%START_TIME%"
          - file:
              path: /tmp/access.log
              format:
                plain: '[%START_TIME%]'

Apply the configuration with kumactl apply -f [..] or with the HTTP API.

Log all incoming and outgoing traffic

Kubernetes
Universal
apiVersion: kuma.io/v1alpha1
kind: MeshAccessLog
metadata:
  name: default
  namespace: kong-mesh-system
  labels:
    kuma.io/mesh: default # optional, defaults to `default` if it isn't configured
spec:
  targetRef:
    kind: Mesh
  from: # delete this section if you don't want to log incoming traffic
    - targetRef:
        kind: Mesh
      default:
        backends:
          - file:
              path: /tmp/access.log
  to: # delete this section if you don't want to log outgoing traffic
    - targetRef:
        kind: Mesh
      default:
        backends:
          - file:
              path: /tmp/access.log

Apply the configuration with kubectl apply -f [..].

type: MeshAccessLog
name: default
mesh: default
spec:
  targetRef:
    kind: Mesh
  from: # delete this section if you don't want to log incoming traffic
    - targetRef:
        kind: Mesh
      default:
        backends:
          - file:
              path: /tmp/access.log
  to: # delete this section if you don't want to log outgoing traffic
    - targetRef:
        kind: Mesh
      default:
        backends:
          - file:
              path: /tmp/access.log

Apply the configuration with kumactl apply -f [..] or with the HTTP API.

Logging traffic going outside the Mesh

To target ExternalServices, use MeshService as the targetRef kind with name set to
the kuma.io/service value.

To target other non-mesh traffic, i.e. passthrough traffic, use Mesh as the targetRef kind. In this case, %KUMA_DESTINATION_SERVICE% is set to external.

Select a built-in gateway

You can select a built-in gateway using the kuma.io/service value. A current limitation is that traffic routed from a gateway to a service is logged by that gateway as having destination "*".

Thank you for your feedback.
Was this page useful?
  • Kong
    THE CLOUD CONNECTIVITY COMPANY

    Kong powers reliable digital connections across APIs, hybrid and multi-cloud environments.

    • Company
    • Customers
    • Events
    • Investors
    • Careers Hiring!
    • Partners
    • Press
    • Contact
  • Products
    • Kong Konnect
    • Kong Gateway
    • Kong Mesh
    • Get Started
    • Pricing
  • Resources
    • eBooks
    • Webinars
    • Briefs
    • Blog
    • API Gateway
    • Microservices
  • Open Source
    • Install Kong Gateway
    • Kong Community
    • Kubernetes Ingress
    • Kuma
    • Insomnia
  • Solutions
    • Decentralize
    • Secure & Govern
    • Create a Dev Platform
    • API Gateway
    • Kubernetes
    • Service Mesh
Star
  • Terms•Privacy
© Kong Inc. 2023