You are browsing documentation for an outdated plugin version.
Configuration
This plugin is compatible with DB-less mode.
Compatible protocols
The AI Response Transformer plugin is compatible with the following protocols:
grpc
, grpcs
, http
, https
Parameters
Here's a list of all the parameters which can be used in this plugin's configuration:
-
string required
The name of the plugin, in this case
ai-response-transformer
.- If using the Kong Admin API, Konnect API, declarative configuration, or decK files, the field is
name
. - If using the KongPlugin object in Kubernetes, the field is
plugin
.
- If using the Kong Admin API, Konnect API, declarative configuration, or decK files, the field is
-
string
An optional custom name to identify an instance of the plugin, for example
ai-response-transformer_my-service
.The instance name shows up in Kong Manager and in Konnect, so it's useful when running the same plugin in multiple contexts, for example, on multiple services. You can also use it to access a specific plugin instance via the Kong Admin API.
An instance name must be unique within the following context:
- Within a workspace for Kong Gateway Enterprise
- Within a control plane or control plane group for Konnect
- Globally for Kong Gateway (OSS)
-
string
The name or ID of the service the plugin targets. Set one of these parameters if adding the plugin to a service through the top-level
/plugins
endpoint. Not required if using/services/{serviceName|Id}/plugins
. -
string
The name or ID of the route the plugin targets. Set one of these parameters if adding the plugin to a route through the top-level
/plugins
endpoint. Not required if using/routes/{routeName|Id}/plugins
. -
string
The name or ID of the consumer group the plugin targets. If set, the plugin will activate only for requests where the specified group has been authenticated
/plugins
endpoint. Not required if using/consumer_groups/{consumerGroupName|Id}/plugins
. -
boolean default:
true
Whether this plugin will be applied.
-
record required
-
string required
Use this prompt to tune the LLM system/assistant message for the returning proxy response (from the upstream), adn what response format you are expecting.
-
string
Defines the regular expression that must match to indicate a successful AI transformation at the response phase. The first match will be set as the returning body. If the AI service’s response doesn’t match this pattern, a failure is returned to the client.
-
boolean required default:
false
Set true to read specific response format from the LLM, and accordingly set the status code / body / headers that proxy back to the client. You need to engineer your LLM prompt to return the correct format, see plugin docs ‘Overview’ page for usage instructions.
-
integer required default:
60000
Timeout in milliseconds for the AI upstream service.
-
boolean required default:
true
Verify the TLS certificate of the AI upstream service.
-
string
A string representing a host name, such as example.com.
-
integer between:
0
65535
An integer representing a port number between 0 and 65535, inclusive.
-
string
A string representing a host name, such as example.com.
-
integer between:
0
65535
An integer representing a port number between 0 and 65535, inclusive.
-
record required
-
string required Must be one of:
llm/v1/chat
,llm/v1/completions
The model’s operation implementation, for this provider.
-
record
-
string referenceable
If AI model requires authentication via Authorization or API key header, specify its name here.
-
string referenceable encrypted
Specify the full auth header value for ‘header_name’, for example ‘Bearer key’ or just ‘key’.
-
string referenceable
If AI model requires authentication via query parameter, specify its name here.
-
string referenceable encrypted
Specify the full parameter value for ‘param_name’.
-
string Must be one of:
query
,body
Specify whether the ‘param_name’ and ‘param_value’ options go in a query string, or the POST form/JSON body.
-
-
record required
-
string required Must be one of:
openai
,azure
,anthropic
,cohere
,mistral
,llama2
AI provider request format - Kong translates requests to and from the specified backend compatible formats.
-
string
Model name to execute.
-
record
Key/value settings for the model
-
integer default:
256
Defines the max_tokens, if using chat or completion models.
-
number default:
1
between:0
5
Defines the matching temperature, if using chat or completion models.
-
number default:
1
between:0
1
Defines the top-p probability mass, if supported.
-
integer default:
0
between:0
500
Defines the top-k most likely tokens, if supported.
-
string
Defines the schema/API version, if using Anthropic provider.
-
string
Instance name for Azure OpenAI hosted models.
-
string default:
2023-05-15
‘api-version’ for Azure OpenAI instances.
-
string
Deployment ID for Azure OpenAI instances.
-
string Must be one of:
raw
,openai
,ollama
If using llama2 provider, select the upstream message format.
-
string Must be one of:
openai
,ollama
If using mistral provider, select the upstream message format.
-
string
Manually specify or override the full URL to the AI operation endpoints, when calling (self-)hosted models, or for running via a private endpoint.
-
-
-
record required
-
-