Set up Dynatrace with OpenTelemetry - Plugin

Set up the OpenTelemetry plugin to send logs and metrics to Dynatrace.

Prerequisites

Install the contrib version of the OpenTelemetry Collector. The contrib version is necessary for generating metrics.
Kong Gateway 3.8+

Configure Kong Gateway

Set the following parameters in your kong.conf file:

tracing_instrumentations = all
tracing_sampling_rate = 1.0

Configure the OpenTelemetry plugin

Adjust the {OPENTELEMETRY_COLLECTOR} variable with your own collector endpoint:

Enable on a service

Enable on a route

Enable on a consumer

Enable globally

Kong Admin API

Konnect API

Kubernetes

Declarative (YAML)

Konnect Terraform

Make the following request:

curl -X POST http://localhost:8001/services/{serviceName|Id}/plugins \
    --header "accept: application/json" \
    --header "Content-Type: application/json" \
    --data '
    {
  "name": "opentelemetry",
  "config": {
    "traces_endpoint": "https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/traces",
    "logs_endpoint": "https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/logs",
    "resource_attributes": {
      "service.name": "kong-dev"
    }
  }
}
    '

Replace SERVICE_NAME|ID with the id or name of the service that this plugin configuration will target.

Make the following request, substituting your own access token, region, control plane ID, and service ID:

curl -X POST \
https://{us|eu}.api.konghq.com/v2/control-planes/{controlPlaneId}/core-entities/services/{serviceId}/plugins \
    --header "accept: application/json" \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer TOKEN" \
    --data '{"name":"opentelemetry","config":{"traces_endpoint":"https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/traces","logs_endpoint":"https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/logs","resource_attributes":{"service.name":"kong-dev"}}}'

See the Konnect API reference to learn about region-specific URLs and personal access tokens.

First, create a KongPlugin resource:

echo "
apiVersion: configuration.konghq.com/v1
kind: KongPlugin
metadata:
  name: opentelemetry-example
plugin: opentelemetry
config:
  traces_endpoint: https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/traces
  logs_endpoint: https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/logs
  resource_attributes:
    service.name: kong-dev
" | kubectl apply -f -

Next, apply the KongPlugin resource to an ingress by annotating the service as follows:

kubectl annotate service SERVICE_NAME konghq.com/plugins=opentelemetry-example

Replace SERVICE_NAME with the name of the service that this plugin configuration will target. You can see your available ingresses by running kubectl get service.

Note: The KongPlugin resource only needs to be defined once and can be applied to any service, consumer, or route in the namespace. If you want the plugin to be available cluster-wide, create the resource as a KongClusterPlugin instead of KongPlugin.

Add this section to your declarative configuration file:

plugins:
- name: opentelemetry
  service: SERVICE_NAME|ID
  config:
    traces_endpoint: https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/traces
    logs_endpoint: https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/logs
    resource_attributes:
      service.name: kong-dev

Replace SERVICE_NAME|ID with the id or name of the service that this plugin configuration will target.

Prerequisite: Configure your Personal Access Token

terraform {
  required_providers {
    konnect = {
      source  = "kong/konnect"
    }
  }
}

provider "konnect" {
  personal_access_token = "kpat_YOUR_TOKEN"
  server_url            = "https://us.api.konghq.com/"
}

Add the following to your Terraform configuration to create a Konnect Gateway Plugin:

resource "konnect_gateway_plugin_opentelemetry" "my_opentelemetry" {
  enabled = true

  config = {
    traces_endpoint = "https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/traces"
    logs_endpoint = "https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/logs"
    resource_attributes = {
      service.name = "kong-dev"
    }
  }

  control_plane_id = konnect_gateway_control_plane.my_konnect_cp.id
  service = {
    id = konnect_gateway_service.my_service.id
  }
}

Kong Admin API

Konnect API

Kubernetes

Declarative (YAML)

Konnect Terraform

Make the following request:

curl -X POST http://localhost:8001/routes/{routeName|Id}/plugins \
    --header "accept: application/json" \
    --header "Content-Type: application/json" \
    --data '
    {
  "name": "opentelemetry",
  "config": {
    "traces_endpoint": "https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/traces",
    "logs_endpoint": "https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/logs",
    "resource_attributes": {
      "service.name": "kong-dev"
    }
  }
}
    '

Replace ROUTE_NAME|ID with the id or name of the route that this plugin configuration will target.

Make the following request, substituting your own access token, region, control plane ID, and route ID:

curl -X POST \
https://{us|eu}.api.konghq.com/v2/control-planes/{controlPlaneId}/core-entities/routes/{routeId}/plugins \
    --header "accept: application/json" \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer TOKEN" \
    --data '{"name":"opentelemetry","config":{"traces_endpoint":"https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/traces","logs_endpoint":"https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/logs","resource_attributes":{"service.name":"kong-dev"}}}'

See the Konnect API reference to learn about region-specific URLs and personal access tokens.

First, create a KongPlugin resource:

echo "
apiVersion: configuration.konghq.com/v1
kind: KongPlugin
metadata:
  name: opentelemetry-example
plugin: opentelemetry
config:
  traces_endpoint: https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/traces
  logs_endpoint: https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/logs
  resource_attributes:
    service.name: kong-dev
" | kubectl apply -f -

Next, apply the KongPlugin resource to an ingress by annotating the ingress as follows:

kubectl annotate ingress INGRESS_NAME konghq.com/plugins=opentelemetry-example

Replace INGRESS_NAME with the name of the ingress that this plugin configuration will target. You can see your available ingresses by running kubectl get ingress.

Note: The KongPlugin resource only needs to be defined once and can be applied to any service, consumer, or route in the namespace. If you want the plugin to be available cluster-wide, create the resource as a KongClusterPlugin instead of KongPlugin.

Add this section to your declarative configuration file:

plugins:
- name: opentelemetry
  route: ROUTE_NAME|ID
  config:
    traces_endpoint: https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/traces
    logs_endpoint: https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/logs
    resource_attributes:
      service.name: kong-dev

Replace ROUTE_NAME|ID with the id or name of the route that this plugin configuration will target.

Prerequisite: Configure your Personal Access Token

terraform {
  required_providers {
    konnect = {
      source  = "kong/konnect"
    }
  }
}

provider "konnect" {
  personal_access_token = "kpat_YOUR_TOKEN"
  server_url            = "https://us.api.konghq.com/"
}

Add the following to your Terraform configuration to create a Konnect Gateway Plugin:

resource "konnect_gateway_plugin_opentelemetry" "my_opentelemetry" {
  enabled = true

  config = {
    traces_endpoint = "https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/traces"
    logs_endpoint = "https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/logs"
    resource_attributes = {
      service.name = "kong-dev"
    }
  }

  control_plane_id = konnect_gateway_control_plane.my_konnect_cp.id
  route = {
    id = konnect_gateway_route.my_route.id
  }
}

Kong Admin API

Konnect API

Kubernetes

Declarative (YAML)

Konnect Terraform

Make the following request:

curl -X POST http://localhost:8001/consumers/{consumerName|Id}/plugins \
    --header "accept: application/json" \
    --header "Content-Type: application/json" \
    --data '
    {
  "name": "opentelemetry",
  "config": {
    "traces_endpoint": "https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/traces",
    "logs_endpoint": "https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/logs",
    "resource_attributes": {
      "service.name": "kong-dev"
    }
  }
}
    '

Replace CONSUMER_NAME|ID with the id or name of the consumer that this plugin configuration will target.

Make the following request, substituting your own access token, region, control plane ID, and consumer ID:

curl -X POST \
https://{us|eu}.api.konghq.com/v2/control-planes/{controlPlaneId}/core-entities/consumers/{consumerId}/plugins \
    --header "accept: application/json" \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer TOKEN" \
    --data '{"name":"opentelemetry","config":{"traces_endpoint":"https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/traces","logs_endpoint":"https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/logs","resource_attributes":{"service.name":"kong-dev"}}}'

See the Konnect API reference to learn about region-specific URLs and personal access tokens.

First, create a KongPlugin resource:

echo "
apiVersion: configuration.konghq.com/v1
kind: KongPlugin
metadata:
  name: opentelemetry-example
plugin: opentelemetry
config:
  traces_endpoint: https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/traces
  logs_endpoint: https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/logs
  resource_attributes:
    service.name: kong-dev
" | kubectl apply -f -

Next, apply the KongPlugin resource to an ingress by annotating the KongConsumer object as follows:

kubectl annotate KongConsumer CONSUMER_NAME konghq.com/plugins=opentelemetry-example

Replace CONSUMER_NAME with the name of the consumer that this plugin configuration will target. You can see your available consumers by running kubectl get KongConsumer.

To learn more about KongConsumer objects, see Provisioning Consumers and Credentials.

Note: The KongPlugin resource only needs to be defined once and can be applied to any service, consumer, or route in the namespace. If you want the plugin to be available cluster-wide, create the resource as a KongClusterPlugin instead of KongPlugin.

Add this section to your declarative configuration file:

plugins:
- name: opentelemetry
  consumer: CONSUMER_NAME|ID
  config:
    traces_endpoint: https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/traces
    logs_endpoint: https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/logs
    resource_attributes:
      service.name: kong-dev

Replace CONSUMER_NAME|ID with the id or name of the consumer that this plugin configuration will target.

Prerequisite: Configure your Personal Access Token

terraform {
  required_providers {
    konnect = {
      source  = "kong/konnect"
    }
  }
}

provider "konnect" {
  personal_access_token = "kpat_YOUR_TOKEN"
  server_url            = "https://us.api.konghq.com/"
}

Add the following to your Terraform configuration to create a Konnect Gateway Plugin:

resource "konnect_gateway_plugin_opentelemetry" "my_opentelemetry" {
  enabled = true

  config = {
    traces_endpoint = "https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/traces"
    logs_endpoint = "https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/logs"
    resource_attributes = {
      service.name = "kong-dev"
    }
  }

  control_plane_id = konnect_gateway_control_plane.my_konnect_cp.id
  consumer = {
    id = konnect_gateway_consumer.my_consumer.id
  }
}

Kong Admin API

Konnect API

Kubernetes

Declarative (YAML)

Konnect Terraform

Make the following request:

curl -X POST http://localhost:8001/plugins/ \
    --header "accept: application/json" \
    --header "Content-Type: application/json" \
    --data '
    {
  "name": "opentelemetry",
  "config": {
    "traces_endpoint": "https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/traces",
    "logs_endpoint": "https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/logs",
    "resource_attributes": {
      "service.name": "kong-dev"
    }
  }
}
    '

Make the following request, substituting your own access token, region, and control plane ID:

curl -X POST \
https://{us|eu}.api.konghq.com/v2/control-planes/{controlPlaneId}/core-entities/plugins/ \
    --header "accept: application/json" \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer TOKEN" \
    --data '{"name":"opentelemetry","config":{"traces_endpoint":"https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/traces","logs_endpoint":"https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/logs","resource_attributes":{"service.name":"kong-dev"}}}'

See the Konnect API reference to learn about region-specific URLs and personal access tokens.

Create a KongClusterPlugin resource and label it as global:

apiVersion: configuration.konghq.com/v1
kind: KongClusterPlugin
metadata:
  name: <global-opentelemetry>
  annotations:
    kubernetes.io/ingress.class: kong
  labels:
    global: "true"
config:
  traces_endpoint: https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/traces
  logs_endpoint: https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/logs
  resource_attributes:
    service.name: kong-dev
plugin: opentelemetry

Add a plugins entry in the declarative configuration file:

plugins:
- name: opentelemetry
  config:
    traces_endpoint: https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/traces
    logs_endpoint: https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/logs
    resource_attributes:
      service.name: kong-dev

Prerequisite: Configure your Personal Access Token

terraform {
  required_providers {
    konnect = {
      source  = "kong/konnect"
    }
  }
}

provider "konnect" {
  personal_access_token = "kpat_YOUR_TOKEN"
  server_url            = "https://us.api.konghq.com/"
}

Add the following to your Terraform configuration to create a Konnect Gateway Plugin:

resource "konnect_gateway_plugin_opentelemetry" "my_opentelemetry" {
  enabled = true

  config = {
    traces_endpoint = "https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/traces"
    logs_endpoint = "https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/logs"
    resource_attributes = {
      service.name = "kong-dev"
    }
  }

  control_plane_id = konnect_gateway_control_plane.my_konnect_cp.id
}

Configure the OpenTelemetry Collector

Configure your OpenTelemetry Collector to send data to the Dynatrace environment.

The following example OpenTelemetry configuration shows how to export traces and logs:

receivers:
  otlp:
    protocols:
      http:
        endpoint: 0.0.0.0:4318

exporters:
  otlphttp:
    endpoint: "https://{your-environment-id}.live.dynatrace.com/api/v2/otlp"
    headers: 
      "Authorization": "Api-Token <your-api-token>"

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: []
      exporters: [otlphttp]
    logs:
      receivers: [otlp]
      processors: []
      exporters: [otlphttp]

Export application span metrics

To include span metrics for application traces, configure the collector exporters section of the OpenTelemetry Collector configuration file:

connectors:
  spanmetrics:
    dimensions:
      - name: http.method
        default: GET
      - name: http.status_code
      - name: http.route
    exclude_dimensions:
      - status.code
    metrics_flush_interval: 15s
    histogram:
      disable: false

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: []
      exporters: [spanmetrics]
    metrics:
      receivers: [spanmetrics]
      processors: []
      exporters: [otlphttp]

More information

Previous Basic config examples for OpenTelemetry

Next Get Started with the OpenTelemetry plugin