Related Documentation
Made by
Kong Inc.
Supported Gateway Topologies
hybrid db-less traditional
Supported Konnect Deployments
hybrid cloud-gateways serverless
Compatible Protocols
grpc grpcs http https
Minimum Version
Kong Gateway - 3.11
Related Resources

The Kong Gateway Datakit plugin allows you to interact with third-party APIs. It sends requests to third-party APIs, then uses the response data to seed information for subsequent calls, either upstream or to other APIs.

Datakit allows you to create an API workflow, which can include:

  • Making calls to third-party APIs
  • Transforming or combining API responses
  • Modifying client requests and upstream service responses
  • Adjusting Kong Gateway entity configuration
  • Returning directly to users instead of proxying

Use cases for Datakit

The following are examples of common use cases for Datakit:

Use case

Description

Third-Party Auth Use internal auth within your ecosystem to inject request headers before proxying a request.
Request Multiplexing Make requests to multiple APIs and combine their responses into one response.
Manipulate Request Headers Use the Datakit plugin to dynamically adjust request headers before passing them to a third-party service.

How does the Datakit plugin work?

The following sections describes what Datakit nodes are and how they work.

The node object

The core component of Datakit is the node object. A node represents some task that consumes input data, produces output data, or a combination of the two. Datakit nodes can:

  • Read client request attributes
  • Send an external HTTP request
  • Modify the response from an upstream before sending it to the client

Most of these tasks can be performed in isolation by an existing plugin. Datakit can string together an execution plan from several nodes, connecting the output from one into the input of another:

  • Read client request attributes and use them to craft an external HTTP request
  • Send an external HTTP request and use the response to augment the service request before proxying upstream
  • Modify the response from an upstream then send it to the client using a custom jq filter to enrich the response with data from a third-party API

The following diagram shows how Datakit can be used to combine two third-party API calls into one response:

Node I/O

A Datakit node consumes data via inputs and emits data via outputs. Connecting the output of one node to the input of another is accomplished by referencing the node’s unique name in the plugin’s configuration.

For example, the following snippet establishes a connection from GET_PROPERTY -> FILTER, where GET_PROPERTY is the source node, and FILTER is the target node:

# fetch the value of `my_property` from the shared request context, if it exists
- name: GET_PROPERTY
  type: property
  property: kong.ctx.shared.my_property

- name: FILTER
  type: jq
  jq: "."
  # connect the output of `GET_PROPERTY` to the input of the `FILTER` jq node
  input: GET_PROPERTY
Copied to clipboard!

Connections can also be reflexively defined in terms of the source node. The following configuration will yield an execution plan with the same GET_PROPERTY -> FILTER connection:

# fetch the value of `my_property` from the shared request context, if it exists
- name: GET_PROPERTY
  type: property
  property: kong.ctx.shared.my_property
  # connect the output of `GET_PROPERTY` to the input of the `FILTER` jq node
  output: FILTER

- name: FILTER
  type: jq
  jq: "."
Copied to clipboard!

Some nodes have structured, object-like inputs and outputs that can be referenced by their field name in an I/O label. For example, the following configuration allows sending different outputs of API to entirely different nodes:

- name: API
  type: call
  url: https://example.com/my-json-api

- name: FILTER_BODY
  type: jq
  jq: "."
  # this node only receives the response body from the `API` node
  input: API.body

- name: FILTER_HEADERS
  type: jq
  jq: "."
  # this node only receives the respone headers from the `API` node
  input: API.headers
Copied to clipboard!

Another way to express this type of connection is by using the outputs property on the source node to select a target node for each named output:

- name: API
  type: call
  url: https://example.com/my-json-api
  outputs:
    body: FILTER_BODY
    headers: FILTER_HEADERS

- name: FILTER_BODY
  type: jq
  jq: "."

- name: FILTER_HEADERS
  type: jq
  jq: "."
Copied to clipboard!

Node outputs can be copied to any number of inputs, but each input may only be connected to one output. This configuration is correct:

- name: GET_FOO
  type: property
  property: kong.ctx.shared.foo

- name: FILTER_FOO
  type: jq
  jq: "."
  input: GET_FOO

- name: FILTER_FOO_TOO
  type: jq
  jq: "."
  input: GET_FOO
Copied to clipboard!

But this configuration will yield an error for GET_BAR.output:

- name: GET_FOO
  type: property
  property: kong.ctx.shared.foo
  output: FILTER

- name: GET_BAR
  type: property
  property: kong.ctx.shared.bar
  output: FILTER

- name: FILTER
  type: jq
  jq: "."
Copied to clipboard!

Error:

invalid connection ("GET_BAR" -> "FILTER"): conflicts with existing connection ("GET_FOO" -> "FILTER")

The jq node is especially flexible, allowing you to craft an ad-hoc input object by defining individual input fields in your configuration. Here’s an alternate version of that last configuration that actually works:

- name: GET_FOO
  type: property
  property: kong.ctx.shared.foo

- name: GET_BAR
  type: property
  property: kong.ctx.shared.bar

- name: COMBINE
  type: jq
  # jq will be fed an object with fields `foo` and `bar`
  inputs:
    foo: GET_FOO
    bar: GET_BAR
  jq: ".foo * .bar"
Copied to clipboard!

Connecting the output of a node to the input of another node establishes a dependency relationship. Datakit always ensures that a node doesn’t run until its dependencies are satisfied, which means that nodes don’t even need to be in the right order in your configuration:

- name: GET_FOO
  type: property
  property: kong.ctx.shared.foo

# this node won't be executed until after `GET_FOO` _and_ `GET_BAR`
- name: COMBINE
  type: jq
  # jq will be fed an object with fields `foo` and `bar`
  inputs:
    foo: GET_FOO
    bar: GET_BAR
  jq: ".foo * .bar"

- name: GET_BAR
  type: property
  property: kong.ctx.shared.bar
Copied to clipboard!

Order of execution is not strictly defined by your configuration.

Configuration order is a facet in determining execution order, but don’t rely on your configuration to dictate the exact order in which nodes will be executed, as Datakit can and will re-order nodes to optimize its execution plan.

Data types, validation, and connection semantics

A key component of Datakit is its type system. Datakit supports the following types:

  • Primitive, scalar types like strings and numbers
  • Non-scalar container types:
    • object: Statically-defined, struct-like values
    • map: Dynamic string keys and static or dynamically typed values
  • Dynamic types:
    • any: Values whose type may not be known until runtime

Datakit performs validation at “config-time” (when the plugin is created or updated via the admin API) by inspecting the type info on either side of a connection, falling back to runtime checks when necessary:

  • If the input and output have the same type (for example, string -> string, any -> any), the connection is permitted since data compatibility at runtime is guaranteed
  • If the input type can be converted to the output type, runtime compatibility isn’t guaranteed, but the connection is permitted with an additional runtime check to ensure that a node doesn’t receive invalid input data. For example:
    • string -> number
    • number -> string
    • any -> number
    • any -> string
    • any -> object
    • any -> map
  • If the input type and output type are known to be incompatible (for example, number -> object), the connection isn’t permitted

Connection labels can be in the form of {node_name} or {node_name}.{field_name}. Connections without a field name are referred to in this reference as “node-wise” or $self connections.

object -> object

For this type of connection, Datakit iterates over each field that the nodes have in common and connects them. If the nodes have no fields in common, a validation error will be raised.

For example:

# note: don't copy this example. This is a "valid" configuration from Datakit's
# perspective but performs a nonsensical action of copying response headers from
# `api_call` as request headers to `service_request`
- name: api_call
  type: call
  url: "https://example.com/"
  method: POST
  input: request
  output: service_request
Copied to clipboard!

This results in 5 connections:

  • request.body -> api_call.body
  • request.query -> api_call.query
  • request.headers -> api_call.headers
  • api_call.body -> service_request.body
  • api_call.headers -> service_request.headers

All of the request node outputs directly map to api_call node inputs, but in the api_call -> service_request connection, some fields remain unconnected:

  • api_call.status is ignored because service_request has no status input
  • service_request.query is ignored because api_call has no query output

The same intent can be expressed explicitly by setting individual fields on the inputs attribute:

- name: api_call
  type: call
  url: "https://example.com/"
  method: POST
  inputs:
    body: request.body
    query: request.query
    headers: request.headers
  outputs:
    body: service_request.body
    headers: service_request.headers
Copied to clipboard!

Be careful when using this type of connection. Implicit object connections like this one are dynamically expanded after reading the configuration, so a newly-added field in a subsequent Datakit release may be inherited by a configuration from a previous version and lead to unintended behavior changes.

object -> map

This type of connection is not permitted.

- name: invalid
  type: call
  url: https://example.com/
  output: service_request.headers
Copied to clipboard!

Error:

invalid connection ("invalid" -> "service_request.headers"): type mismatch: object -> map
object -> any

This type of connection copies all data from the source object to the target input. In this example, filter will receive a JSON object as input with the keys body, query, and headers:

- name: filter
  type: jq
  input: request
  jq: "."
Copied to clipboard!

Be careful when using this type of connection. Implicit object connections like this one are dynamically expanded after reading the configuration, so a newly-added field in a subsequent Datakit release may be inherited by a configuration from a previous version and lead to unintended behavior changes.

* -> any

Connections of any output type to an any input type are always permitted. At runtime the data is copied as-is.

any -> *

Connections from any output types are permitted under almost all conditions and incur a runtime type conversion check (unless the target type is also any).

A node-wise any -> object or any -> map connection conflicts with any field-level connections:

- name: get-foo
  type: property
  property: kong.ctx.shared.foo
  # connect `get-foo -> response`
  output: response

- name: get-bar
  type: property
  property: kong.ctx.shared.bar
  # Datakit can't validate that this connection will not overlap with
  # `get-foo -> response`
  output: response.body
Copied to clipboard!

Error:

invalid connection ("get-bar" -> "response.body"): conflicts with existing connection ("get-foo" -> "response.body")

Node types

The Datakit plugin provides the following node types:

  • call: Send third-party HTTP calls.
  • jq: Transform data and cast variables with jq to be shared with other nodes.
  • exit: Return directly to the client.
  • property: Get and set Kong Gateway-specific data.
  • static: Configure static input values ahead of time.

Node type

Inputs

Outputs

Attributes

call body, headers, query body, headers, status url, method, timeout, ssl_server_name
jq user-defined user-defined jq
exit body, headers none status
property $self $self property, content_type
static none user-defined values

Implicit nodes

Datakit also defines a number of implicit nodes that can be used without being explicitly declared. These reserved node names can’t be used for user-defined nodes. They include:

Node

Inputs

Outputs

Description

request none body, headers, query Reads incoming client request properties
service_request body, headers, query none Updates properties of the request sent to the service being proxied to
service_response none body, headers Reads response properties from the service being proxied to
response body, headers none Updates properties of the outgoing client response

Headers

The headers type produces and consumes maps from header names to their values:

  • Keys are header names. Original header name casing is preserved for maximum compatibility.
  • Values are strings if there is a single instance of a header or arrays of strings if there are multiple instances of the same header.

Query

The query type produces and consumes maps with key-value pairs representing decoded URL query strings.

Body

The service_request.body and response.body inputs both accept any data type. If the data is an object, it will automatically be JSON-encoded, and the Content-Type header set to application/json (if not already set in the headers input).

The request.body and service_response.body outputs have a similar behavior. If the corresponding Content-Type header matches the JSON mime-type, the body output is automatically JSON-decoded.

call node

Send an HTTP request and retrieve the response.

Inputs:

  • body: Request body
  • headers: Request headers
  • query: Key-value pairs to encode as the request query string

Outputs:

  • body: The response body
  • headers: The response headers
  • status: The HTTP status code of the response

Configuration attributes:

  • url (required): The URL
  • method: The HTTP method (default is GET)
  • timeout: The dispatch timeout, in milliseconds

Examples

Make an external API call:

- name: CALL
  type: call
  url: https://example.com/foo
Copied to clipboard!

Send a POST request with a JSON body:

- name: POST
  type: call
  url: https://example.com/create-entity
  method: POST
  inputs:
    body: ENTITY

- name: ENTITY
  type: static
  values:
    id: 123
    name: Datakit
Copied to clipboard!

Automatic JSON body handling

If the data connected to the body input is an object, it will automatically be encoded as JSON, and the request Content-Type header will be set to application/json unless already present in the headers input.

Similarly, if the response Content-Type header matches the JSON mime-type, the body output will be JSON-decoded automatically.

Async execution

This is an async node. This means that the request will be sent in the background while Datakit executes any other nodes (save for any nodes which depend on it). Multiple call nodes are executed concurrently when no dependency order enforces it.

In this example, both CALL_FOO and CALL_BAR will be started as soon as possible, and then Datakit will block until both have finished to run JOIN:

- name: CALL_FOO
  type: call
  url: https://example.com/foo

- name: CALL_BAR
  type: call
  url: https://example.com/bar

- name: JOIN
  type: jq
  jq: "."
  inputs:
    foo: CALL_FOO.body
    bar: CALL_BAR.body
Copied to clipboard!

Failure conditions

The call node fails execution if a network-level error is encountered or if the endpoint returns a non-2xx status code. It will also fail if the endpoint returns a JSON mime-type in the Content-Type header if the response body is not valid JSON.

Limitations

Due to platform limitations, the call node can’t be executed after proxying a request, so attempting to configure the node using outputs from the upstream service response will yield an error:

- name: CALL
  type: call
  url: https://example.com/
  method: POST
  inputs:
    # dependency error!
    body: service_response.body
Copied to clipboard!

Error:

invalid dependency (node #1 (CALL) -> node service_response): circular dependency

jq node type

The jq node executes a jq script for processing JSON. See the official jq docs for more details.

Inputs

User-defined. For node-wise ($self) connections, jq can handle input of any type:

- name: SERVICE
  type: property
  property: kong.router.service

- name: IP
  type: property
  property: kong.client.ip

- name: FILTER_SERVICE
  type: jq
  input: SERVICE
  # yields: "object"
  jq: ". | type"

- name: FILTER_IP
  type: jq
  input: IP
  # yields: "string"
  jq: ". | type"
Copied to clipboard!

By defining individual inputs, jq’s input will be coerced to an object with string keys and values of any type. Referencing input fields from within the filter is done by using dot (.) notation:

- name: SERVICE
  type: property
  property: kong.router.service

- name: IP
  type: property
  property: kong.client.ip

- name: FILTER_SERVICE_AND_IP
  type: jq
  inputs:
    service: SERVICE
    ip: IP
  # yields: { "$self": "object", "service": "object", "ip": "string" }
  jq: |
    {
      "$self":   (.        | type),
      "service": (.service | type),
      "ip":      (.ip      | type)
    }
Copied to clipboard!

Outputs

User-defined. A jq filter script can produce any type of data:

- name: STRING
  type: jq
  jq: |
    "my string"

- name: NUMBER
  type: jq
  jq: |
    54321

- name: BOOLEAN
  type: jq
  jq: |
    true

- name: OBJECT
  type: jq
  jq: |
    {
      a: 1,
      b: 2
    }
Copied to clipboard!

It’s impossible for Datakit to know ahead of time what kind of data jq will emit, so Datakit uses runtime checks when the output of jq is connected to another node’s input. It’s important to carefully test and validate your Datakit configurations to avoid this case:

- name: HEADERS
  type: jq
  jq: |
    "oops, not an object/map"

- name: EXIT
  type: exit
  inputs:
    # this will cause Datakit to return a 500 error to the client when
    # encountered
    headers: HEADERS
Copied to clipboard!

This is also why the jq node doesn’t allow explicitly referencing individual fields with outputs at config-time:

- name: HEADERS
  type: jq
  jq: |
    "this is completely opaque to Datakit"

  # Datakit will reject this configuration because it can't confirm that the
  # output of HEADERS is an object or has a `body` field
  outputs:
    body: EXIT.body

- name: EXIT
  type: exit
Copied to clipboard!

Configuration attributes

jq: the jq script to execute when the node is triggered.

Handling HTTP headers in jq

To enable a high level of transparency and compatibility when communicating with external services, headers outputs in Datakit always preserve the original case of header names. While HTTP-centric nodes within Datakit are careful to account for this and perform header lookups and transformations in a case-insensitive manner, jq at its core is a library for operating upon JSON data, and JSON object keys are strictly case-sensitive.

Be mindful of this when handling headers in a jq filter to avoid buggy, error-prone behavior. For example:

# adds the `X-Extra` header to the upstream service request if not set by the client
- name: ADD_HEADERS
  type: jq
  input: request.headers
  output: service_request.headers
  jq: |
    {
      "X-Extra": ( .["X-Extra"] // "default value" ),
    }
Copied to clipboard!

This filter will function correctly if the client sets the X-Extra header or omits it entirely, but it won’t have the intended effect if the client sets the header X-EXTRA or x-extra.

jq lets you write a robust filter that handles this condition. The following implementation normalizes header names to lowercase before looking up values from the input:

# adds the `X-Extra` header to the upstream service request if not set by the client
- name: ADD_HEADERS
  type: jq
  input: request.headers
  output: service_request.headers
  jq: |
    with_entries(.key |= ascii_downcase)
    | {
        "X-Extra": ( .["x-extra"] // "default value" ),
    }
Copied to clipboard!
Recipe: merging header objects

These examples take in client request headers and update them from a set of pre-defined values.

The HTTP specification RFC defines header names to be case-insensitive, so in most cases it’s enough to normalize header names to lowercase for ease of merging the two objects:

- name: header_updates
  type: static
  values:
    X-Foo: "123"
    X-Custom: "my header"
    X-Multi:
      - "first"
      - "second"

- name: merged_headers
  type: jq
  inputs:
    original: request.headers
    updates: header_updates
  jq: |
    (.original | with_entries(.key |= ascii_downcase))
    *
    (.updates | with_entries(.key |= ascii_downcase))

- name: api
  type: call
  url: "https://example.com/"
  inputs:
    headers: merged_headers
Copied to clipboard!

However, when dealing with a upstream service or API that isn’t fully compliant with the HTTP spec, it might be necessary to preserve original header name casing. For example:

- name: header_updates
  type: static
  values:
    X-Foo: "123"
    X-Custom: "my header"
    X-Multi:
      - "first"
      - "second"

- name: merged_headers
  type: jq
  inputs:
    original: request.headers
    updates: header_updates
  jq: |
    . as $input

    # store .original key names for lookup
    | $input.original
    | with_entries({ key: .key | ascii_downcase, value: .key })
      as $keys

    # rewrite .updates with .original key names
    | $input.updates
    | with_entries(.key = ($keys[.key | ascii_downcase] // .key))
      as $updates

    | $input.original * $updates

- name: api
  type: call
  url: "https://example.com/"
  inputs:
    headers: merged_headers
Copied to clipboard!

Examples

Coerce the client request body to an object:

- name: BODY
  type: jq
  input: request.body
  jq: |
    if type == "object" then
      .
    else
      { data: . }
    end
Copied to clipboard!

Join the output of two API calls:

- name: FOO
  type: call
  url: https://example.com/foo

- name: BAR
  type: call
  url: https://example.com/bar

- name: JOIN
  type: jq
  inputs:
    foo: FOO.body
    bar: BAR.body
  jq: "."
Copied to clipboard!

exit node

Trigger an early exit that produces a direct response, rather than forwarding a proxied response.

Inputs:

  • body: Body to use in the early-exit response.
  • headers: Headers to use in the early-exit response.

Outputs: None

Configuration attributes:

  • status: The HTTP status code to use in the early-exit response (default is 200).

Examples

Make an HTTP request and send the response directly to the client:

- name: CALL
  type: call
  url: https://example.com/

- name: EXIT
  type: exit
  input: CALL
Copied to clipboard!

property node

Get and set Kong Gateway host and request properties.

Whether a get or set operation is performed depends upon the node inputs:

  • If an input is connected, set the property
  • If no input is connected, get the property and map it to the output

Inputs

This node accepts the $self input:

- name: STORE_REQUEST
  type: property
  property: kong.ctx.shared.my_request
  input: request
Copied to clipboard!

No individual field-level inputs are permitted:

- name: STORE_REQUEST_BY_FIELD
  type: property
  property: kong.ctx.shared.my_request
  # error! property input doesn't allow field access
  inputs:
    body: request.body
Copied to clipboard!

Outputs

This node produces the $self output.

- name: GET_ROUTE
  type: property
  property: kong.router.route
  output: response.body
Copied to clipboard!

Field-level output connections are not supported, even if the output data has known fields:

- name: GET_ROUTE_ID
  type: property
  property: kong.router.route
  # error! property output doesn't allow field access
  outputs:
    id: response.body
Copied to clipboard!

Configuration attributes

  • property (required): The name of the property
  • content_type: The expected mime type of the property value. When set to application/json, set operations will JSON-encode input data before writing it, and get operations will JSON-decode output data after reading it. Otherwise, this setting has no effect.

Supported properties

The following properties support get operations:

Property

Description

Data type

kong.client.consumer kong.client.get_consumer() object
kong.client.consumer_groups kong.client.get_consumer_groups() array
kong.client.credential kong.client.get_credential() object
kong.client.get_identity_realm_source kong.client.get_identity_realm_source() object
kong.client.forwarded_ip kong.client.get_forwarded_ip() string
kong.client.forwarded_port kong.client.get_forwarded_port() number
kong.client.ip kong.client.get_ip() string
kong.client.port kong.client.get_port() number
kong.client.protocol kong.client.get_protocol() string
kong.request.forwarded_host kong.request.get_forwarded_host() string
kong.request.forwarded_port kong.request.get_forwarded_port() number
kong.request.forwarded_scheme kong.request.get_forwarded_scheme() string
kong.request.port kong.request.get_port() number
kong.response.source kong.response.get_source() string
kong.router.route kong.router.get_route() object
kong.route_id Gets the current Route’s ID string
kong.route_name Gets the current Route’s name string
kong.router.service kong.router.get_service() object
kong.service_name Gets the current Service’s name string
kong.service_id Gets the current Service’s ID string
kong.service.response.status kong.service.response.status number
kong.version Gets the Kong version string
kong.node.id kong.node.get_id() string
kong.configuration.{key} Reads {key} from the node configuration any

The following properties support set operations:

Property

Description

Data type

kong.service.target kong.service.set_target({host}, {port}) string ({host}:{port})
kong.service.request_scheme kong.service.set_service_request_scheme({scheme}) string ({scheme})

The following properties support get and set operations:

Property

Description

Data type

kong.ctx.plugin.{key} Gets or sets kong.ctx.plugin.{key} any
kong.ctx.shared.{key} Gets or sets kong.ctx.shared.{key} any

static node

Emits static values to be used as inputs for other nodes. The static node can help you with hardcoding some known value for an input.

Inputs

None.

Outputs

This node produces outputs for each item in its values attribute:

- name: CALL_INPUTS
  type: static
  values:
    headers:
      X-Foo: "123"
      X-Multi:
        - first
        - second
    query:
      a: true
      b: 10
    body:
      data: "my request body data"

- name: CALL
  type: call
  url: https://example.com/
  method: POST
  input: CALL_INPUTS
Copied to clipboard!

The static nature of these values comes in handy, because Datakit can validate them when creating or updating the plugin configuration. Attempting to create a plugin with the following configuration will yield an Admin API validation error instead of bubbling up at runtime:

- name: CALL_INPUTS
  type: static
  values:
    headers: "oops not valid headers"

- name: CALL
  type: call
  url: https://example.com/
  method: POST
  input: CALL_INPUTS
Copied to clipboard!

Configuration attributes

  • values (required): A mapping of string keys to arbitrary values

Examples

Set a property from a static value:

- name: VALUE
  type: static
  values:
    data:
      a: 1
      b: 2

- name: PROPERTY
  type: property
  property: kong.ctx.shared.my_property
  input: VALUE.data
Copied to clipboard!

Set a default value for a jq filter:

- name: VALUE
  type: static
  values:
    default: "my default value"

- name: FILTER
  type: jq
  inputs:
    query: request.query
    default: VALUE.default
  jq: ".query.foo // .default"
Copied to clipboard!

Set common request headers for different API requests:

- name: HEADERS
  type: static
  values:
    X-Common: "we always need this header"

- name: CALL_FOO
  type: call
  url: https://example.com/foo
  inputs:
    headers: HEADERS

- name: CALL_BAR
  type: call
  url: https://example.com/bar
  inputs:
    headers: HEADERS
Copied to clipboard!

Debugging

Datakit includes support for debugging your configuration.

Enabling the debug option in Datakit is considered unsafe for production environments, as it can cause arbitrary information to leak into client responses.

Making node execution errors visible

When a Datakit node encounters an error, the default behavior is to exit immediately with a generic 500 error so as not to leak any information to the client:

{
  "message": "An unexpected error occurred",
  "request_id": "f5e07609d55bd66508c8315b8cf6583a"
}
Copied to clipboard!

You can find the entire error in the Kong Gateway error.log file:

> 2025/06/24 10:55:32 [error] 917449#0: *1292 [lua] runtime.lua:406: handler(): node #1 (API) failed with error: "non-2XX response code: 403", client: 127.0.0.1, server: kong, request: "GET / HTTP/1.1", host: "test-010.datakit.test", request_id: "f5e07609d55bd66508c8315b8cf6583a"
Copied to clipboard!

For quicker feedback during local development and testing, the error can also be passed through to the client response by enabling the debug option in your Datakit plugin configuration. In addition to the full error message, the response contains information about the node which failed, including the node name, type, and index (the position of the node within the nodes array in your plugin configuration):

{
  "message": "node execution error",
  "request_id": "f5e07609d55bd66508c8315b8cf6583a",
  "error": "non-2XX response code: 403",
  "node": {
    "index": 1,
    "name": "API",
    "type": "call"
  }
}
Copied to clipboard!

Execution debug tracing

With debug enabled in the plugin configuration, add the X-DataKit-Debug-Trace header with a value of "true", "yes", "on", "1", or "enabled" to prompt Datakit to perform detailed execution tracing and return a full report in the client response.

Here’s an example where the API node failed due to its endpoint returning a 403 status code. The failure caused all pending/running node tasks to be canceled and resulted in an execution plan error (PLAN_ERROR):

{
  "started_at": 1750789142.703118,
  "status": "PLAN_ERROR",
  "ended_at": 1750789142.705158,
  "nodes": [
    {
      "name": "API",
      "type": "call",
      "status": "NODE_ERROR",
      "error": "non-2XX response code: 403"
    },
    {
      "name": "SLOW_API",
      "status": "NODE_CANCELED",
      "type": "call"
    },
    {
      "name": "FILTER",
      "status": "NODE_CANCELED",
      "type": "jq"
    },
    {
      "name": "request",
      "status": "NODE_COMPLETE",
      "type": "request"
    }
  ],
  "duration": 0.002039670944213867,
  "events": [
    {
      "name": "request",
      "action": "run",
      "values": [],
      "type": "request",
      "at": 0.00001406669616699219
    },
    {
      "name": "request",
      "action": "complete",
      "values": [
        {
          "type": "any",
          "key": "headers"
        },
        {
          "type": "any",
          "key": "body"
        },
        {
          "type": "map",
          "value": {
            "b": "123",
            "a": "true"
          },
          "key": "query"
        }
      ],
      "type": "request",
      "at": 0.0000324249267578125
    },
    {
      "name": "API",
      "action": "run",
      "values": [
        {
          "type": "map",
          "key": "headers"
        },
        {
          "type": "any",
          "key": "body"
        },
        {
          "type": "map",
          "value": {
            "b": "123",
            "a": "true"
          },
          "key": "query"
        }
      ],
      "type": "call",
      "at": 0.00003552436828613281
    },
    {
      "type": "call",
      "action": "run",
      "at": 0.00005054473876953125,
      "name": "API"
    },
    {
      "name": "SLOW_API",
      "action": "run",
      "values": [
        {
          "type": "map",
          "key": "headers"
        },
        {
          "type": "any",
          "key": "body"
        },
        {
          "type": "map",
          "value": {
            "b": "123",
            "a": "true"
          },
          "key": "query"
        }
      ],
      "type": "call",
      "at": 0.00005173683166503906
    },
    {
      "type": "call",
      "action": "run",
      "at": 0.00005674362182617188,
      "name": "SLOW_API"
    },
    {
      "name": "API",
      "action": "resume",
      "type": "call",
      "at": 0.001984357833862305,
      "duration": 0.001933813095092773
    },
    {
      "name": "API",
      "action": "fail",
      "values": [
        {
          "type": "error",
          "value": "non-2XX response code: 403"
        },
        {
          "type": "map",
          "value": {
            "Connection": "keep-alive",
            "X-Powered-By": "mock_upstream",
            "Content-Length": "824",
            "Content-Type": "application/json",
            "Date": "Tue, 24 Jun 2025 18:19:02 GMT",
            "Server": "mock-upstream/1.0.0"
          },
          "key": "headers"
        },
        {
          "type": "any",
          "value": {
            "url": "http://127.0.0.1:15555/status/403?b=123&a=true",
            "post_data": {
              "params": null,
              "text": "",
              "kind": "unknown"
            },
            "code": 403,
            "vars": {
              "host": "127.0.0.1",
              "request_method": "GET",
              "binary_remote_addr": "\u007f\u0000\u0000\u0001",
              "uri": "/status/403",
              "request_time": "0.000",
              "remote_addr": "127.0.0.1",
              "request_length": "119",
              "remote_port": "50902",
              "scheme": "http",
              "ssl_server_name": "no SNI",
              "hostname": "soup",
              "request_uri": "/status/403?b=123&a=true",
              "server_port": "15555",
              "server_name": "mock_upstream",
              "request": "GET /status/403?b=123&a=true HTTP/1.1",
              "server_protocol": "HTTP/1.1",
              "https": "",
              "realip_remote_addr": "127.0.0.1",
              "server_addr": "127.0.0.1",
              "realip_remote_port": "50902",
              "is_args": "?"
            },
            "uri_args": {
              "b": "123",
              "a": "true"
            },
            "headers": {
              "user-agent": "lua-resty-http/0.17.2 (Lua) ngx_lua/10028",
              "host": "127.0.0.1:15555"
            }
          },
          "key": "body"
        },
        {
          "type": "number",
          "value": 403,
          "key": "status"
        }
      ],
      "type": "call",
      "at": 0.00200200080871582
    },
    {
      "name": "SLOW_API",
      "action": "cancel",
      "type": "call",
      "at": 0.002033710479736328,
      "duration": 0.001976966857910156
    },
    {
      "type": "jq",
      "action": "cancel",
      "at": 0.002036333084106445,
      "name": "FILTER"
    }
  ]
}
Copied to clipboard!

The tracing output is emitted instead of any other pending client response body (originating from Datakit or elsewhere), so there are limits to what can be observed in the trace. The response node, for instance, can’t execute fully when tracing is enabled and will appear in the tracing report with a result of NODE_SKIPPED.

The contents of the tracing report are unstable and intended for human consumption to aid development and testing. Backwards-incompatible changes to the report format may be included with any new release of Kong Gateway.

Did this doc help?

Something wrong?

Help us make these docs great!

Kong Developer docs are open source. If you find these useful and want to make them better, contribute today!