Note: The OpenTelemetry plugin’s tracing capabilities only work when Kong Gateway’s tracing_instrumentations
configuration is enabled.
The OpenTelemetry plugin is fully compatible with the OpenTelemetry specification and can be used with any OpenTelemetry compatible backend.
There are two ways to set up an OpenTelemetry backend:
Set up Kong Gateway
The OpenTelemetry tracing capability supported by this plugin requires the following Kong Gateway configuration:
-
tracing_instrumentations = all
: Enables all possible tracing instrumentations.
Valid values can be found in Kong’s configuration reference.
-
tracing_sampling_rate = 1.0
: Tracing instrumentation sampling rate.
Tracer samples a fixed percentage of all spans following the sampling rate.
Set the sampling rate to a lower value to reduce the impact of the instrumentation on Kong Gateway’s proxy performance in production.
Set up an OpenTelemetry compatible backend
This section is optional if you are using a OpenTelemetry compatible APM vendor.
All the supported vendors are listed in the OpenTelemetry’s Vendor support.
Jaeger natively supports OpenTelemetry starting with v1.35 and can be used with the OpenTelemetry plugin.
Deploy a Jaeger instance with Docker:
docker run --name jaeger \
-e COLLECTOR_OTLP_ENABLED=true \
-p 16686:16686 \
-p 4317:4317 \
-p 4318:4318 \
jaegertracing/all-in-one:1.36
-
The COLLECTOR_OTLP_ENABLED
environment variable must be set to true
to enable the OpenTelemetry Collector.
-
The 4318
port is the OTLP/HTTP port and the 4317
port is the OTLP/GRPC port that isn’t supported by the OpenTelemetry plugin yet.
Set up an OpenTelemetry Collector
This section is required if you are using an incompatible OpenTelemetry APM vendor.
Create a config file (otelcol.yaml
) for the OpenTelemetry Collector:
receivers:
otlp:
protocols:
grpc:
http:
processors:
batch:
exporters:
logging:
loglevel: debug
zipkin:
endpoint: "http://some.url:9411/api/v2/spans"
tls:
insecure: true
service:
pipelines:
traces:
receivers: [otlp]
processors: [batch]
exporters: [logging, zipkin]
logs:
receivers: [otlp]
processors: [batch]
exporters: [logging]
Run the OpenTelemetry Collector with Docker:
docker run --name opentelemetry-collector \
-p 4317:4317 \
-p 4318:4318 \
-p 55679:55679 \
-v $(pwd)/otelcol.yaml:/etc/otel-collector-config.yaml \
otel/opentelemetry-collector-contrib:0.52.0 \
--config=/etc/otel-collector-config.yaml
See the OpenTelemetry Collector documentation for more information.
Enable on a service
Enable on a route
Enable on a consumer
Enable globally
Kong Admin API
Konnect API
Kubernetes
Declarative (YAML)
Konnect Terraform
Make the following request:
curl -X POST http://localhost:8001/services/{serviceName|Id}/plugins \
--header "accept: application/json" \
--header "Content-Type: application/json" \
--data '
{
"name": "opentelemetry",
"config": {
"traces_endpoint": "http://<opentelemetry-backend>:4318/v1/traces",
"logs_endpoint": "http://<opentelemetry-backend>:4318/v1/logs",
"resource_attributes": {
"service.name": "kong-dev"
}
}
}
'
Replace SERVICE_NAME|ID
with the id
or name
of the service that this plugin configuration will target.
Make the following request, substituting your own access token, region, control plane ID, and service ID:
curl -X POST \
https://{us|eu}.api.konghq.com/v2/control-planes/{controlPlaneId}/core-entities/services/{serviceId}/plugins \
--header "accept: application/json" \
--header "Content-Type: application/json" \
--header "Authorization: Bearer TOKEN" \
--data '{"name":"opentelemetry","config":{"traces_endpoint":"http://<opentelemetry-backend>:4318/v1/traces","logs_endpoint":"http://<opentelemetry-backend>:4318/v1/logs","resource_attributes":{"service.name":"kong-dev"}}}'
See the Konnect API reference to learn about region-specific URLs and personal access tokens.
First, create a KongPlugin
resource:
echo "
apiVersion: configuration.konghq.com/v1
kind: KongPlugin
metadata:
name: opentelemetry-example
plugin: opentelemetry
config:
traces_endpoint: http://<opentelemetry-backend>:4318/v1/traces
logs_endpoint: http://<opentelemetry-backend>:4318/v1/logs
resource_attributes:
service.name: kong-dev
" | kubectl apply -f -
Next, apply the KongPlugin
resource to an ingress by annotating the service
as follows:
kubectl annotate service SERVICE_NAME konghq.com/plugins=opentelemetry-example
Replace SERVICE_NAME
with the name of the service that this plugin configuration will target.
You can see your available ingresses by running kubectl get service
.
Note: The KongPlugin resource only needs to be defined once
and can be applied to any service, consumer, or route in the namespace. If you
want the plugin to be available cluster-wide, create the resource as a
KongClusterPlugin
instead of KongPlugin
.
Add this section to your declarative configuration file:
plugins:
- name: opentelemetry
service: SERVICE_NAME|ID
config:
traces_endpoint: http://<opentelemetry-backend>:4318/v1/traces
logs_endpoint: http://<opentelemetry-backend>:4318/v1/logs
resource_attributes:
service.name: kong-dev
Replace SERVICE_NAME|ID
with the id
or name
of the service that this plugin configuration will target.
Prerequisite: Configure your Personal Access Token
terraform {
required_providers {
konnect = {
source = "kong/konnect"
}
}
}
provider "konnect" {
personal_access_token = "kpat_YOUR_TOKEN"
server_url = "https://us.api.konghq.com/"
}
Add the following to your Terraform configuration to create a Konnect Gateway Plugin:
resource "konnect_gateway_plugin_opentelemetry" "my_opentelemetry" {
enabled = true
config = {
traces_endpoint = "http://<opentelemetry-backend>:4318/v1/traces"
logs_endpoint = "http://<opentelemetry-backend>:4318/v1/logs"
resource_attributes = {
service.name = "kong-dev"
}
}
control_plane_id = konnect_gateway_control_plane.my_konnect_cp.id
service = {
id = konnect_gateway_service.my_service.id
}
}
Kong Admin API
Konnect API
Kubernetes
Declarative (YAML)
Konnect Terraform
Make the following request:
curl -X POST http://localhost:8001/routes/{routeName|Id}/plugins \
--header "accept: application/json" \
--header "Content-Type: application/json" \
--data '
{
"name": "opentelemetry",
"config": {
"traces_endpoint": "http://<opentelemetry-backend>:4318/v1/traces",
"logs_endpoint": "http://<opentelemetry-backend>:4318/v1/logs",
"resource_attributes": {
"service.name": "kong-dev"
}
}
}
'
Replace ROUTE_NAME|ID
with the id
or name
of the route that this plugin configuration will target.
Make the following request, substituting your own access token, region, control plane ID, and route ID:
curl -X POST \
https://{us|eu}.api.konghq.com/v2/control-planes/{controlPlaneId}/core-entities/routes/{routeId}/plugins \
--header "accept: application/json" \
--header "Content-Type: application/json" \
--header "Authorization: Bearer TOKEN" \
--data '{"name":"opentelemetry","config":{"traces_endpoint":"http://<opentelemetry-backend>:4318/v1/traces","logs_endpoint":"http://<opentelemetry-backend>:4318/v1/logs","resource_attributes":{"service.name":"kong-dev"}}}'
See the Konnect API reference to learn about region-specific URLs and personal access tokens.
First, create a KongPlugin
resource:
echo "
apiVersion: configuration.konghq.com/v1
kind: KongPlugin
metadata:
name: opentelemetry-example
plugin: opentelemetry
config:
traces_endpoint: http://<opentelemetry-backend>:4318/v1/traces
logs_endpoint: http://<opentelemetry-backend>:4318/v1/logs
resource_attributes:
service.name: kong-dev
" | kubectl apply -f -
Next, apply the KongPlugin
resource to an ingress by annotating the ingress
as follows:
kubectl annotate ingress INGRESS_NAME konghq.com/plugins=opentelemetry-example
Replace INGRESS_NAME
with the name of the ingress that this plugin configuration will target.
You can see your available ingresses by running kubectl get ingress
.
Note: The KongPlugin resource only needs to be defined once
and can be applied to any service, consumer, or route in the namespace. If you
want the plugin to be available cluster-wide, create the resource as a
KongClusterPlugin
instead of KongPlugin
.
Add this section to your declarative configuration file:
plugins:
- name: opentelemetry
route: ROUTE_NAME|ID
config:
traces_endpoint: http://<opentelemetry-backend>:4318/v1/traces
logs_endpoint: http://<opentelemetry-backend>:4318/v1/logs
resource_attributes:
service.name: kong-dev
Replace ROUTE_NAME|ID
with the id
or name
of the route that this plugin configuration
will target.
Prerequisite: Configure your Personal Access Token
terraform {
required_providers {
konnect = {
source = "kong/konnect"
}
}
}
provider "konnect" {
personal_access_token = "kpat_YOUR_TOKEN"
server_url = "https://us.api.konghq.com/"
}
Add the following to your Terraform configuration to create a Konnect Gateway Plugin:
resource "konnect_gateway_plugin_opentelemetry" "my_opentelemetry" {
enabled = true
config = {
traces_endpoint = "http://<opentelemetry-backend>:4318/v1/traces"
logs_endpoint = "http://<opentelemetry-backend>:4318/v1/logs"
resource_attributes = {
service.name = "kong-dev"
}
}
control_plane_id = konnect_gateway_control_plane.my_konnect_cp.id
route = {
id = konnect_gateway_route.my_route.id
}
}
Kong Admin API
Konnect API
Kubernetes
Declarative (YAML)
Konnect Terraform
Make the following request:
curl -X POST http://localhost:8001/consumers/{consumerName|Id}/plugins \
--header "accept: application/json" \
--header "Content-Type: application/json" \
--data '
{
"name": "opentelemetry",
"config": {
"traces_endpoint": "http://<opentelemetry-backend>:4318/v1/traces",
"logs_endpoint": "http://<opentelemetry-backend>:4318/v1/logs",
"resource_attributes": {
"service.name": "kong-dev"
}
}
}
'
Replace CONSUMER_NAME|ID
with the id
or name
of the consumer group that this plugin configuration will target.
Make the following request, substituting your own access token, region, control plane ID, and consumer ID:
curl -X POST \
https://{us|eu}.api.konghq.com/v2/control-planes/{controlPlaneId}/core-entities/consumers/{consumerId}/plugins \
--header "accept: application/json" \
--header "Content-Type: application/json" \
--header "Authorization: Bearer TOKEN" \
--data '{"name":"opentelemetry","config":{"traces_endpoint":"http://<opentelemetry-backend>:4318/v1/traces","logs_endpoint":"http://<opentelemetry-backend>:4318/v1/logs","resource_attributes":{"service.name":"kong-dev"}}}'
See the Konnect API reference to learn about region-specific URLs and personal access tokens.
First, create a KongPlugin
resource:
echo "
apiVersion: configuration.konghq.com/v1
kind: KongPlugin
metadata:
name: opentelemetry-example
plugin: opentelemetry
config:
traces_endpoint: http://<opentelemetry-backend>:4318/v1/traces
logs_endpoint: http://<opentelemetry-backend>:4318/v1/logs
resource_attributes:
service.name: kong-dev
" | kubectl apply -f -
Next, apply the KongPlugin
resource to an ingress by annotating the KongConsumer
object as follows:
kubectl annotate KongConsumer CONSUMER_NAME konghq.com/plugins=opentelemetry-example
Replace CONSUMER_NAME
with the name of the consumer that this plugin configuration will target.
You can see your available consumers by running kubectl get KongConsumer
.
To learn more about KongConsumer
objects, see Provisioning Consumers and Credentials
.
Note: The KongPlugin resource only needs to be defined once
and can be applied to any service, consumer, or route in the namespace. If you
want the plugin to be available cluster-wide, create the resource as a
KongClusterPlugin
instead of KongPlugin
.
Add this section to your declarative configuration file:
plugins:
- name: opentelemetry
consumer: CONSUMER_NAME|ID
config:
traces_endpoint: http://<opentelemetry-backend>:4318/v1/traces
logs_endpoint: http://<opentelemetry-backend>:4318/v1/logs
resource_attributes:
service.name: kong-dev
Replace CONSUMER_NAME|ID
with the id
or name
of the consumer that this plugin configuration will target.
Prerequisite: Configure your Personal Access Token
terraform {
required_providers {
konnect = {
source = "kong/konnect"
}
}
}
provider "konnect" {
personal_access_token = "kpat_YOUR_TOKEN"
server_url = "https://us.api.konghq.com/"
}
Add the following to your Terraform configuration to create a Konnect Gateway Plugin:
resource "konnect_gateway_plugin_opentelemetry" "my_opentelemetry" {
enabled = true
config = {
traces_endpoint = "http://<opentelemetry-backend>:4318/v1/traces"
logs_endpoint = "http://<opentelemetry-backend>:4318/v1/logs"
resource_attributes = {
service.name = "kong-dev"
}
}
control_plane_id = konnect_gateway_control_plane.my_konnect_cp.id
consumer = {
id = konnect_gateway_consumer.my_consumer.id
}
}
Kong Admin API
Konnect API
Kubernetes
Declarative (YAML)
Konnect Terraform
Make the following request:
curl -X POST http://localhost:8001/plugins/ \
--header "accept: application/json" \
--header "Content-Type: application/json" \
--data '
{
"name": "opentelemetry",
"config": {
"traces_endpoint": "http://<opentelemetry-backend>:4318/v1/traces",
"logs_endpoint": "http://<opentelemetry-backend>:4318/v1/logs",
"resource_attributes": {
"service.name": "kong-dev"
}
}
}
'
Make the following request, substituting your own access token, region, and control plane ID:
curl -X POST \
https://{us|eu}.api.konghq.com/v2/control-planes/{controlPlaneId}/core-entities/plugins/ \
--header "accept: application/json" \
--header "Content-Type: application/json" \
--header "Authorization: Bearer TOKEN" \
--data '{"name":"opentelemetry","config":{"traces_endpoint":"http://<opentelemetry-backend>:4318/v1/traces","logs_endpoint":"http://<opentelemetry-backend>:4318/v1/logs","resource_attributes":{"service.name":"kong-dev"}}}'
See the Konnect API reference to learn about region-specific URLs and personal access tokens.
Create a KongClusterPlugin
resource and label it as global:
apiVersion: configuration.konghq.com/v1
kind: KongClusterPlugin
metadata:
name: <global-opentelemetry>
annotations:
kubernetes.io/ingress.class: kong
labels:
global: "true"
config:
traces_endpoint: http://<opentelemetry-backend>:4318/v1/traces
logs_endpoint: http://<opentelemetry-backend>:4318/v1/logs
resource_attributes:
service.name: kong-dev
plugin: opentelemetry
Add a plugins
entry in the declarative
configuration file:
plugins:
- name: opentelemetry
config:
traces_endpoint: http://<opentelemetry-backend>:4318/v1/traces
logs_endpoint: http://<opentelemetry-backend>:4318/v1/logs
resource_attributes:
service.name: kong-dev
Prerequisite: Configure your Personal Access Token
terraform {
required_providers {
konnect = {
source = "kong/konnect"
}
}
}
provider "konnect" {
personal_access_token = "kpat_YOUR_TOKEN"
server_url = "https://us.api.konghq.com/"
}
Add the following to your Terraform configuration to create a Konnect Gateway Plugin:
resource "konnect_gateway_plugin_opentelemetry" "my_opentelemetry" {
enabled = true
config = {
traces_endpoint = "http://<opentelemetry-backend>:4318/v1/traces"
logs_endpoint = "http://<opentelemetry-backend>:4318/v1/logs"
resource_attributes = {
service.name = "kong-dev"
}
}
control_plane_id = konnect_gateway_control_plane.my_konnect_cp.id
}