Collect metrics with Datadog and the Prometheus plugin

Before you configure the Datadog Agent to scrape metrics from Kong Gateway, you first need to enable the Prometheus plugin.

The following configuration enables the plugin globally and exports status code metrics, like the total number of HTTP requests:

echo '
_format_version: "3.0"
plugins:
  - name: prometheus
    config:
      status_code_metrics: true
' | deck gateway apply -

Now that the Prometheus plugin is configured, you can configure the Datadog Agent to scrape Kong Gateway metrics.

Create the conf.yaml file:

touch ./.datadog-agent/conf.d/openmetrics.d/conf.yaml

This command uses the macOS directory location. For other distributions, see Datadog’s Agent configuration directory.

Copy and paste the following configuration in the conf.yaml file:

instances:
  - openmetrics_endpoint: http://localhost:8001/metrics
    namespace: kong
    metrics:
      - kong.*

instances:
  - openmetrics_endpoint: http://localhost:8100/metrics
    namespace: kong
    metrics:
      - kong.*

This configuration pulls all the kong. prefixed metrics from the Kong Gateway metrics endpoint (http://localhost:8001/metrics).

This configuration pulls all the kong. prefixed metrics from the Kong Gateway Status API metrics endpoint (http://localhost:8100/metrics).

Important: If you’re running Kong Gateway and the Datadog Agent in Docker, you’ll need to replace localhost in the config.yaml with the name of the Kong Gateway container. Also, both containers need to be running on the same network for them to communicate.

You must restart the agent to start collecting metrics:

launchctl stop com.datadoghq.agent

launchctl start com.datadoghq.agent

This example is for macOS, see Datadog’s Agent commands documentation for all restart commands.

The Datadog Agent may take a few minutes to restart.

Now, you can validate that Datadog can scrape Kong Gateway metrics by first sending requests to generate metrics:

for _  in {1..10}; do
 curl "$KONNECT_PROXY_URL/anything"; done

for _  in {1..10}; do
 curl "http://localhost:8000/anything"; done

You can validate that Kong Gateway is sending metrics to Datadog by running the following:

curl -X GET "$DD_SITE_API_URL/api/v1/search?q=kong.kong_http_requests.count" \
-H "Accept: application/json" \
-H "DD-API-KEY: $DD_API_KEY" \
-H "DD-APPLICATION-KEY: $DD_APP_KEY"

You should get the following response:

{"results":{"metrics":["kong.kong_http_requests.count"],"hosts":[]}}

Alternatively, you can navigate to the Metrics Explorer page in the Datadog UI and search for kong.kong_http_requests.count. You should see the 10 requests that you just sent.