Set up Dynatrace with OpenTelemetry

Premium Partner

Uses: Kong Gateway decK

Deployment Platform:

Prerequisites

Tracing environment variables

Set the following Dynatrace tracing variables before you configure the Data Plane:

export KONG_TRACING_INSTRUMENTATIONS=all
export KONG_TRACING_SAMPLING_RATE=1.0

Copied to clipboard!

Kong Konnect

This is a Konnect tutorial and requires a Konnect personal access token.

Create a new personal access token by opening the Konnect PAT page and selecting Generate Token.
Export your token to an environment variable:
```
 export KONNECT_TOKEN='YOUR_KONNECT_PAT'
```
Copied to clipboard!

Run the quickstart script to automatically provision a Control Plane and Data Plane, and configure your environment:

 curl -Ls https://get.konghq.com/quickstart | bash -s -- -k $KONNECT_TOKEN -e KONG_TRACING_INSTRUMENTATIONS -e KONG_TRACING_SAMPLING_RATE --deck-output

Copied to clipboard!

This sets up a Konnect Control Plane named quickstart, provisions a local Data Plane, and prints out the following environment variable exports:

 export DECK_KONNECT_TOKEN=$KONNECT_TOKEN
 export DECK_KONNECT_CONTROL_PLANE_NAME=quickstart
 export KONNECT_CONTROL_PLANE_URL=https://us.api.konghq.com
 export KONNECT_PROXY_URL='http://localhost:8000'

Copied to clipboard!

Copy and paste these into your terminal to configure your session.

Kong Gateway running

This tutorial requires Kong Gateway Enterprise. If you don’t have Kong Gateway set up yet, you can use the quickstart script with an enterprise license to get an instance of Kong Gateway running almost instantly.

Export your license to an environment variable:
```
 export KONG_LICENSE_DATA='LICENSE-CONTENTS-GO-HERE'
```
Copied to clipboard!

Run the quickstart script:

curl -Ls https://get.konghq.com/quickstart | bash -s -- -e KONG_LICENSE_DATA \
  -e KONG_TRACING_INSTRUMENTATIONS \
  -e KONG_TRACING_SAMPLING_RATE

Copied to clipboard!

Once Kong Gateway is ready, you will see the following message:

 Kong Gateway Ready

decK

Required entities

For this tutorial, you’ll need Kong Gateway entities, like Gateway Services and Routes, pre-configured. These entities are essential for Kong Gateway to function but installing them isn’t the focus of this guide. Follow these steps to pre-configure them:

Run the following command:

echo '
_format_version: "3.0"
services:
  - name: example-service
    url: http://httpbin.konghq.com/anything
routes:
  - name: example-route
    paths:
    - "/anything"
    service:
      name: example-service
' | deck gateway apply -

      
        
      
    
Copied to clipboard!

To learn more about entities, you can read our entities documentation.

Dynatrace

This tutorial requires you to have a Dynatrace SaaS account.

In Dynatrace, find your environment ID.
Generate an API token with the openTelemetryTrace.ingest and metrics.ingest scopes.

Export those values as environment variables:

export DECK_DYNATRACE_ENVIRONMENT_ID='ENVIRONMENT-ID-HERE'
export DECK_DYNATRACE_API_TOKEN='TOKEN-HERE'

Copied to clipboard!

Enable the OTEL plugin

In this tutorial, we’ll be configuring the OpenTelemetry plugin to send Kong Gateway traces and logs to Dynatrace SaaS. This configuration is good for testing purposes, but we recommend using a collector, like Dynatrace Collector, in production environments.

Enable the OTEL plugin with Dynatrace settings configured:

echo '
_format_version: "3.0"
plugins:
  - name: opentelemetry
    config:
      traces_endpoint: https://${{ env "DECK_DYNATRACE_ENVIRONMENT_ID" }}.live.dynatrace.com/api/v2/otlp/v1/traces
      logs_endpoint: https://${{ env "DECK_DYNATRACE_ENVIRONMENT_ID" }}.live.dynatrace.com/api/v2/otlp/v1/logs
      resource_attributes:
        service.name: kong-dev
      headers:
        Authorization: Api-Token ${{ env "DECK_DYNATRACE_API_TOKEN" }}
' | deck gateway apply -

      
        
      
    
Copied to clipboard!

Validate

Send a POST request to generate traffic that we can use to validate that Dynatrace is receiving the traces:

 curl -X POST "$KONNECT_PROXY_URL/anything" \
     -H "Accept: application/json"\
     -H "Content-Type: application/json"

Copied to clipboard!

In the Dynatrace UI, navigate to Distributed Traces and search for Service name of kong-dev. You should see a trace for the request you just sent. Sometimes it can take a few seconds to display.

Cleanup

Clean up Konnect environment

If you created a new control plane and want to conserve your free trial credits or avoid unnecessary charges, delete the new control plane used in this tutorial.

Destroy the Kong Gateway container

FAQs

How do I export application span metrics?

Kong Gateway relies on the OpenTelemetry Collector to calculate the metrics based on the traces the OpenTelemetry plugin generates.

To include span metrics for application traces, configure the collector exporters section of the OpenTelemetry Collector configuration file:

connectors:
spanmetrics:
  dimensions:
    - name: http.method
      default: GET
    - name: http.status_code
    - name: http.route
  exclude_dimensions:
    - status.code
  metrics_flush_interval: 15s
  histogram:
    disable: false

service:
pipelines:
  traces:
    receivers: [otlp]
    processors: []
    exporters: [spanmetrics]
  metrics:
    receivers: [spanmetrics]
    processors: []
    exporters: [otlphttp]

      
        
      
    
Copied to clipboard!

I want to use the Dynatrace Collector between Kong Gateway and Dynatrace, how do I configure that?

Make an otel-collector-config.yaml file with the following configuration:

receivers:
  otlp:
    protocols:
      http:
        endpoint: 0.0.0.0:4318

exporters:
  otlphttp:
    endpoint: "https://{yourEnvironmentId}.live.dynatrace.com/api/v2/otlp"
    headers: 
      "Authorization": "Api-Token <your-api-token>"

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: []
      exporters: [otlphttp]
    logs:
      receivers: [otlp]
      processors: []
      exporters: [otlphttp]

      
        
      
    
Copied to clipboard!

I’m getting a 200 OK when I make requests, but nothing is showing up in Dynatrace, how do I fix this?

I’m getting a Missing authorization parameter., context: ngx.timer in Kong Gateway logs when I send a request after configuring the OpenTelemetry plugin with Dynatrace, how do I fix this?