Skip to content
Kong Logo | Kong Docs Logo
search
  • We're Hiring!
  • Docs
    • Kong Gateway
    • Kong Konnect
    • Kong Mesh
    • Plugin Hub
    • decK
    • Kubernetes Ingress Controller
    • Insomnia
    • Kuma

    • Docs contribution guidelines
  • Plugin Hub
  • Support
  • Community
  • Kong Academy
Get a Demo Start Free Trial
  • Kong Gateway
  • Kong Konnect
  • Kong Mesh
  • Plugin Hub
  • decK
  • Kubernetes Ingress Controller
  • Insomnia
  • Kuma

  • Docs contribution guidelines
  • 3.2.x (latest)
  • 3.1.x
  • 3.0.x
  • 2.8.x
  • 2.7.x
  • 2.6.x
  • Older Enterprise versions (2.1-2.5)
  • Older OSS versions (2.1-2.5)
  • Archive (pre-2.1)
    • Overview of Kong Gateway
      • Version Support Policy
      • Supported Installation Options
      • Supported Linux Distributions
    • Stability
    • Release Notes
      • Services
        • Overview
        • Configure Routes with Expressions
      • Upstreams
      • Plugins
      • Routing Traffic
      • Load Balancing
      • Health Checks and Circuit Breakers
      • Kong Performance Testing
    • Glossary
    • Get Kong
    • Services and Routes
    • Rate Limiting
    • Proxy Caching
    • Key Authentication
    • Load-Balancing
      • Overview
        • Overview
        • Deploy Kong Gateway in Hybrid mode
      • DB-less Deployment
      • Traditional
      • Overview
        • Helm
        • OpenShift with Helm
        • kubectl apply
        • Kubernetes Deployment Options
        • Using docker run
        • Build your own Docker images
        • Amazon Linux
        • Debian
        • Red Hat
        • Ubuntu
      • Running Kong as a non-root user
      • Securing the Admin API
      • Using systemd
      • Start Kong Gateway Securely
      • Programatically Creating Admins
      • Enabling RBAC
      • Overview
      • Download your License
      • Deploy Enterprise License
      • Using the License API
      • Monitor Licenses Usage
      • Default Ports
      • DNS Considerations
      • Network and Firewall
      • CP/DP Communication through a Forward Proxy
    • Kong Configuration File
    • Environment Variables
    • Serving a Website and APIs from Kong
      • Overview
      • Prometheus
      • StatsD
      • Datadog
      • Overview
      • Writing a Custom Trace Exporter
      • Tracing API Reference
    • Resource Sizing Guidelines
    • Security Update Process
    • Blue-Green Deployments
    • Canary Deployments
    • Clustering Reference
      • Log Reference
      • Dynamic log level updates
      • Customize Gateway Logs
      • Upgrade Kong Gateway 3.1.x
      • Migrate from OSS to Enterprise
    • Overview
      • Overview
      • Metrics
      • Analytics with InfluxDB
      • Analytics with Prometheus
      • Estimate Analytics Storage in PostgreSQL
      • Overview
      • Getting Started
      • Advanced Usage
        • Overview
        • Environment Variables
        • AWS Secrets Manager
        • Google Secrets Manager
        • Hashicorp Vault
        • Securing the Database with AWS Secrets Manager
      • Reference Format
      • Overview
      • Get Started with Dynamic Plugin Ordering
      • Overview
      • Enable the Dev Portal
      • Publish an OpenAPI Spec
      • Structure and File Types
      • Themes Files
      • Working with Templates
      • Using the Editor
        • Basic Auth
        • Key Auth
        • OIDC
        • Sessions
        • Adding Custom Registration Fields
        • Manage Developers
        • Developer Roles and Content Permissions
        • Authorization Provider Strategy
        • Enable Application Registration
        • Enable Key Authentication for Application Registration
          • External OAuth2 Support
          • Set up Okta and Kong for External Oauth
          • Set up Azure AD and Kong for External Authentication
        • Manage Applications
        • Theme Editing
        • Migrating Templates Between Workspaces
        • Markdown Rendering Module
        • Customizing Portal Emails
        • Adding and Using JavaScript Assets
        • Single Page App in Dev Portal
        • Alternate OpenAPI Renderer
      • SMTP
      • Workspaces
      • Helpers CLI
      • Portal API Documentation
    • Audit Logging
    • Keyring and Data Encryption
    • Workspaces
    • Consumer Groups
    • Event Hooks
    • FIPS 140-2
    • Overview
    • Enable Kong Manager
      • Services and Routes
      • Rate Limiting
      • Proxy Caching
      • Authentication with Consumers
      • Load Balancing
      • Overview
      • Create a Super Admin
      • Workspaces and Teams
      • Reset Passwords and RBAC Tokens
      • Basic Auth
        • Configure LDAP
        • LDAP Service Directory Mapping
        • Configure OIDC
        • OIDC Authenticated Group Mapping
      • Sessions
        • Overview
        • Enable RBAC
        • Add a Role and Permissions
        • Create a User
        • Create an Admin
    • Networking Configuration
    • Workspaces
    • Create Consumer Groups
    • Sending Email
    • Overview
    • File Structure
    • Implementing Custom Logic
    • Plugin Configuration
    • Accessing the Data Store
    • Storing Custom Entities
    • Caching Custom Entities
    • Extending the Admin API
    • Writing Tests
    • (un)Installing your Plugin
      • Overview
      • kong.client
      • kong.client.tls
      • kong.cluster
      • kong.ctx
      • kong.ip
      • kong.jwe
      • kong.log
      • kong.nginx
      • kong.node
      • kong.request
      • kong.response
      • kong.router
      • kong.service
      • kong.service.request
      • kong.service.response
      • kong.table
      • kong.tracing
      • kong.vault
      • kong.websocket.client
      • kong.websocket.upstream
      • Go
      • Javascript
      • Python
      • Running Plugins in Containers
      • External Plugin Performance
    • Overview
        • Overview
        • OpenID Connect with Curity
        • OpenID Connect with Azure AD
        • OpenID Connect with Google
        • OpenID Connect with Okta
        • OpenID Connect with Auth0
        • OpenID Connect with Cognito
      • Authentication Reference
      • Allow Multiple Authentication Plugins
    • Rate Limiting Plugin
      • Add a Body Value
    • GraphQL
      • gRPC Plugins
      • Configure a gRPC service
    • Overview
    • Information Routes
    • Health Routes
    • Tags
    • Debug Routes
    • Services
    • Routes
    • Consumers
    • Plugins
    • Certificates
    • CA Certificates
    • SNIs
    • Upstreams
    • Targets
    • Vaults
    • Keys
    • Licenses
    • Workspaces
    • RBAC
    • Admins
    • Developers
    • Consumer Groups
    • Event Hooks
    • Keyring and Data Encryption
    • Audit Logs
    • kong.conf
    • Injecting Nginx Directives
    • CLI
    • File Permissions Reference
    • Key Management
    • Performance Testing Framework
    • Router Expressions Language
    • FAQ

github-edit-pageEdit this page

report-issueReport an issue

enterprise-switcher-iconSwitch to OSS

On this page
  • Prerequisites
  • Configure Prometheus monitoring
  • Cleanup
  • More information
Kong Gateway
3.1.x
  • Home
  • Kong Gateway
  • Production Deployment
  • Monitoring
  • Monitoring with Prometheus
You are browsing documentation for an outdated version. See the latest documentation here.

Monitoring with Prometheus

Prometheus is a popular systems monitoring and alerting toolkit. Prometheus implements a multi-dimensional time series data model and distributed storage system where metrics data is collected via a pull model over HTTP.

Kong Gateway supports Prometheus with the Prometheus Plugin that exposes Kong Gateway performance and proxied upstream service metrics on the /metrics endpoint.

This guide will help you setup a test Kong Gateway and Prometheus service. Then you will generate sample requests to Kong Gateway and observe the collected monitoring data.

Prerequisites

This guide assumes the following tools are installed locally:

  • Docker is used to run Kong Gateway, the supporting database, and Prometheus locally.
  • curl is used to send requests to Kong Gateway. curl is pre-installed on most systems.

Configure Prometheus monitoring

  1. Install Kong Gateway:

    This step is optional if you wish to use an existing Kong Gateway installation. When using an existing Kong Gateway, you will need to modify the commands to account for network connectivity and installed Kong Gateway services and routes.

    curl -Ls https://get.konghq.com/quickstart | bash -s -- -m
    

    The -m flag instructs the script to install a mock service that is used in this guide to generate sample metrics.

    Once the Kong Gateway is ready, you will see the following message:

    ✔ Kong is ready!
    
  2. Install the Prometheus Kong Gateway plugin:

    curl -s -X POST http://localhost:8001/plugins/ \
     --data "name=prometheus" 
    

    You should receive a JSON response with the details of the installed plugin.

  3. Create a Prometheus configuration file named prometheus.yml in the current directory, and copy the following values:

    scrape_configs:
      - job_name: 'kong'
        scrape_interval: 5s
        static_configs:
          - targets: ['kong-quickstart-gateway:8001']
    

    See the Prometheus Configuration Documentation for details on these settings.

  4. Run a Prometheus server, and pass it the configuration file created in the previous step. Prometheus will begin to scrape metrics data from Kong Gateway.

    docker run -d --name kong-quickstart-prometheus \
       --network=kong-quickstart-net -p 9090:9090 \
       -v $(PWD)/prometheus.yml:/etc/prometheus/prometheus.yml \
       prom/prometheus:latest
    
  5. Generate sample traffic to the mock service. This allows you to observe metrics generated from the StatsD plugin. The following command generates 60 requests over one minute. Run the following in a new terminal:

    for _ in {1..60}; do {curl -s localhost:8000/mock/request; sleep 1; } done
    
  6. You can view the metric data directly from Kong Gateway by querying the /metrics endpoint on the Admin API:

    curl -s localhost:8001/metrics
    

    Kong Gateway will report system wide performance metrics by default. When the Plugin has been installed and traffic is being proxied, it will record additional metrics across service, route, and upstream dimensions.

    The response will look similar to the following snippet:

    # HELP kong_bandwidth Total bandwidth in bytes consumed per service/route in Kong
    # TYPE kong_bandwidth counter
    kong_bandwidth{service="mock",route="mock",type="egress"} 13579
    kong_bandwidth{service="mock",route="mock",type="ingress"} 540
    # HELP kong_datastore_reachable Datastore reachable from Kong, 0 is unreachable
    # TYPE kong_datastore_reachable gauge
    kong_datastore_reachable 1
    # HELP kong_http_status HTTP status codes per service/route in Kong
    # TYPE kong_http_status counter
    kong_http_status{service="mock",route="mock",code="200"} 6
    # HELP kong_latency Latency added by Kong, total request time and upstream latency for each service/route in Kong
    # TYPE kong_latency histogram
    kong_latency_bucket{service="mock",route="mock",type="kong",le="1"} 4
    kong_latency_bucket{service="mock",route="mock",type="kong",le="2"} 4
    ...
    

    See the Kong Prometheus Plugin documentation for details on the available metrics and configurations.

  7. Prometheus provides multiple ways to query collected metric data.

    You can view the Prometheus expression viewer by opening a browser to http://localhost:9090/graph.

    You can also query Prometheus directly using it’s HTTP API:

    curl -s 'localhost:9090/api/v1/query?query=kong_http_status'
    

    Prometheus also provides documentation for setting up Grafana as a visualization tool for the collected time series data.

Cleanup

Once you are done experimenting with Prometheus and Kong Gateway, you can use the following commands to stop and remove the services created in this guide:

docker stop kong-quickstart-prometheus
curl -Ls https://get.konghq.com/quickstart | bash -s -- -d

More information

  • How to monitor with StatsD provides a guide to using StatsD for monitoring with the Kong Gateway Plugin
  • See the Tracing API Reference for information on Kong Gateway’s tracing capabilities
Thank you for your feedback.
Was this page useful?
  • Kong
    THE CLOUD CONNECTIVITY COMPANY

    Kong powers reliable digital connections across APIs, hybrid and multi-cloud environments.

    • Company
    • Customers
    • Events
    • Investors
    • Careers Hiring!
    • Partners
    • Press
    • Contact
  • Products
    • Kong Konnect
    • Kong Gateway
    • Kong Mesh
    • Get Started
    • Pricing
  • Resources
    • eBooks
    • Webinars
    • Briefs
    • Blog
    • API Gateway
    • Microservices
  • Open Source
    • Install Kong Gateway
    • Kong Community
    • Kubernetes Ingress
    • Kuma
    • Insomnia
  • Solutions
    • Decentralize
    • Secure & Govern
    • Create a Dev Platform
    • API Gateway
    • Kubernetes
    • Service Mesh
Star
  • Terms•Privacy
© Kong Inc. 2023