You are browsing documentation for an outdated plugin version.
This guide walks you through setting up the AI Proxy plugin with Gemini.
For all providers, the Kong AI Proxy plugin attaches to route entities.
Prerequisites
- Kong Gateway is installed and running
- Create or retrieve an API key on the Google Cloud API Credentials Page to access Google’s AI services
Configure the AI Proxy plugin
- Create a service in Kong Gateway that will represent the Google Gemini API:
curl -i -X POST http://localhost:8001/services \ --data "name=gemini-service" \ --data "url=https://generativelanguage.googleapis.com"
- Create a route that maps to the service you defined:
curl -i -X POST http://localhost:8001/routes \ --data "paths[]=/gemini" \ --data "service.id=$(curl -s http://localhost:8001/services/gemini-service | jq -r '.id')"
- Use the Kong Admin API to configure the AI Proxy Plugin to route requests to Google Gemini:
curl -i -X POST http://localhost:8001/services/gemini-service/plugins \ --data 'name=ai-proxy' \ --data 'config.auth.param_name=key' \ --data 'config.auth.param_value=<GEMINI_API_TOKEN>' \ --data 'config.auth.param_location=query' \ --data 'config.route_type=llm/v1/chat' \ --data 'config.model.provider=gemini' \ --data 'config.model.name=gemini-1.5-flash'
Be sure to replace GEMINI_API_TOKEN
with your API token.
Test the configuration
Make an llm/v1/chat
type request to test your new endpoint:
curl -X POST http://localhost:8000/gemini \
-H 'Content-Type: application/json' \
--data-raw '{ "messages": [ { "role": "system", "content": "You are a mathematician" }, { "role": "user", "content": "What is 1+1?"} ] }'