Skip to content
Kong Logo | Kong Docs Logo
search
  • Docs
    • Explore the API Specs
      View all API Specs View all API Specs View all API Specs arrow image
    • Documentation
      API Specs
      Kong Gateway
      Lightweight, fast, and flexible cloud-native API gateway
      Kong Konnect
      Single platform for SaaS end-to-end connectivity
      Kong Mesh
      Enterprise service mesh based on Kuma and Envoy
      decK
      Helps manage Kong’s configuration in a declarative fashion
      Kong Ingress Controller
      Works inside a Kubernetes cluster and configures Kong to proxy traffic
      Kong Gateway Operator
      Manage your Kong deployments on Kubernetes using YAML Manifests
      Insomnia
      Collaborative API development platform
      Kuma
      Open-source distributed control plane with a bundled Envoy Proxy integration
  • Plugin Hub
    • Explore the Plugin Hub
      View all plugins View all plugins View all plugins arrow image
    • Functionality View all View all arrow image
      View all plugins
      Authentication's icon
      Authentication
      Protect your services with an authentication layer
      Security's icon
      Security
      Protect your services with additional security layer
      Traffic Control's icon
      Traffic Control
      Manage, throttle and restrict inbound and outbound API traffic
      Serverless's icon
      Serverless
      Invoke serverless functions in combination with other plugins
      Analytics & Monitoring's icon
      Analytics & Monitoring
      Visualize, inspect and monitor APIs and microservices traffic
      Transformations's icon
      Transformations
      Transform request and responses on the fly on Kong
      Logging's icon
      Logging
      Log request and response data using the best transport for your infrastructure
  • Support
  • Community
  • Kong Academy
Get a Demo Start Free Trial
Kong Mesh
2.1.x
  • Home icon
  • Kong Mesh
  • Policies
  • MeshRetry
github-edit-pageEdit this page
report-issueReport an issue
  • Kong Gateway
  • Kong Konnect
  • Kong Mesh
  • Plugin Hub
  • decK
  • Kong Ingress Controller
  • Kong Gateway Operator
  • Insomnia
  • Kuma

  • Docs contribution guidelines
  • dev
  • 2.5.x (latest)
  • 2.4.x
  • 2.3.x
  • 2.2.x
  • 2.1.x
  • 2.0.x
  • 1.9.x
  • 1.8.x
  • 1.7.x
  • 1.6.x
  • 1.5.x
  • 1.4.x
  • 1.3.x
  • 1.2.x
enterprise-switcher-icon Switch to OSS
On this pageOn this page
  • TargetRef support matrix
  • Configuration
    • Retry on
    • Backoff
    • Rate limited backoff
  • Examples
    • HTTP web to backend on 5xx
    • gRPC web to backend on DeadlineExceeded
    • TCP web to backend
  • All policy options
You are browsing documentation for an outdated version. See the latest documentation here.

MeshRetry

This policy uses new policy matching algorithm. Do not combine with Retry.

This policy enables Kong Mesh to know how to behave if there is a failed scenario (i.e. HTTP request) which could be retried.

TargetRef support matrix

targetRef.kind top level to from
Mesh ✅ ✅ ❌
MeshSubset ✅ ❌ ❌
MeshService ✅ ✅ ❌
MeshServiceSubset ✅ ❌ ❌

To learn more about the information in this table, see the matching docs.

Configuration

The policy let you configure retry behaviour for HTTP, GRPC and TCP protocols. The protocol is selected by picking the most specific protocol.

Each protocol has a separate section under default in the policy YAML. Some sections are common between protocols or have similar meaning.

Retry on

The field retryOn is a list of conditions which will cause a retry.

For HTTP these are related to the response status code or method (“5xx”, “429”, “HttpMethodGet”). For gRPC these are status codes in response headers (“canceled”, “deadline-exceeded”, etc.). There is no equivalent for TCP.

One or more conditions can be specified, for example:

retryOn:
  - "429"
  - "503"

means that it the policy will retry on a status code 429 or 503.

Full list of available HTTP conditions:

retryOn:
  - 5XX
  - GatewayError
  - Reset
  - Retriable4xx
  - ConnectFailure
  - EnvoyRatelimited
  - RefusedStream
  - Http3PostConnectFailure
  - HttpMethodConnect
  - HttpMethodDelete
  - HttpMethodGet
  - HttpMethodHead
  - HttpMethodOptions
  - HttpMethodPatch
  - HttpMethodPost
  - HttpMethodPut
  - HttpMethodTrace
  - "429" # any HTTP status code
  - "503"

Full list of available gRPC conditions:

retryOn:
  - Canceled
  - DeadlineExceeded
  - Internal
  - ResourceExhausted
  - Unavailable

Backoff

This parameter is applicable to both HTTP and GRPC.

It consists of BaseInterval (the amount of time between retries) and MaxInterval (the maximal amount of time taken between retries).

We use a fully jittered exponential back-off algorithm for retries. Given a base interval B and retry number N, the back-off for the retry is in the range [0, (2N - 1) × B).

For example, given a 25ms interval, the first retry will be delayed randomly by 0-24ms, the 2nd by 0-74ms, the 3rd by 0-174ms, and so on.

The interval is capped at a MaxInterval, which defaults to 10 times the BaseInterval.

Rate limited backoff

This parameter is applicable to both HTTP and GRPC.

MeshRetry can be configured in such a way that when the upstream server rate limits the request and responds with a header like retry-after or x-ratelimit-reset it uses the value from the header to determine when to send the retry request instead of the backoff algorithm.

Example

Given this configuration:

retryOn:
  - "503"
rateLimitedBackOff:
  resetHeaders:
    - name: retry-after
      format: Seconds
    - name: x-ratelimit-reset
      format: UnixTimestamp

and an HTTP response:

HTTP/1.1 503 Service Unavailable
retry-after: 15

The retry request will be issued after 15 seconds.

If the response is as follows:

HTTP/1.1 503 Service Unavailable
x-ratelimit-reset: 1706096119

The request will be retried at Wed Jan 24 2024 11:35:19 GMT+0000.

If the response does not contain retry-after or x-ratelimit-reset header (with valid integer value) then the amount of time to wait before issuing a request is determined by backoff algorithm.

Examples

HTTP web to backend on 5xx

Kubernetes
Universal
apiVersion: kuma.io/v1alpha1
kind: MeshRetry
metadata:
  name: web-to-backend-retry-http
  namespace: kong-mesh-system
  labels:
    kuma.io/mesh: default # optional, defaults to `default` if unset
spec:
  targetRef:
    kind: MeshService
    name: web
  to:
    - targetRef:
        kind: MeshService
        name: backend
      default:
        http:
          numRetries: 10
          backOff:
            baseInterval: 15s
            maxInterval: 20m
          retryOn:
            - "5xx"

Apply the configuration with kubectl apply -f [..].

type: MeshRetry
name: web-to-backend-retry-http
mesh: default
spec:
  targetRef:
    kind: MeshService
    name: web
  to:
    - targetRef:
        kind: MeshService
        name: backend
      default:
        http:
          numRetries: 10
          backOff:
            baseInterval: 15s
            maxInterval: 20m
          retryOn:
            - "5xx"

Apply the configuration with kumactl apply -f [..] or with the HTTP API.

gRPC web to backend on DeadlineExceeded

Kubernetes
Universal
apiVersion: kuma.io/v1alpha1
kind: MeshRetry
metadata:
  name: web-to-backend-retry-grpc
  namespace: kong-mesh-system
  labels:
    kuma.io/mesh: default # optional, defaults to `default` if unset
spec:
  targetRef:
    kind: MeshService
    name: web
  to:
    - targetRef:
        kind: MeshService
        name: backend
      default:
        grpc:
          numRetries: 5
          backOff:
            baseInterval: 5s
            maxInterval: 1m
          retryOn:
            - "DeadlineExceeded"

Apply the configuration with kubectl apply -f [..].

type: MeshRetry
name: web-to-backend-retry-grpc
mesh: default
spec:
  targetRef:
    kind: MeshService
    name: web
  to:
    - targetRef:
        kind: MeshService
        name: backend
      default:
        grpc:
          numRetries: 5
          backOff:
            baseInterval: 5s
            maxInterval: 1m
          retryOn:
            - "DeadlineExceeded"

Apply the configuration with kumactl apply -f [..] or with the HTTP API.

TCP web to backend

Kubernetes
Universal
apiVersion: kuma.io/v1alpha1
kind: MeshRetry
metadata:
  name: web-to-backend-retry-tcp
  namespace: kong-mesh-system
  labels:
    kuma.io/mesh: default # optional, defaults to `default` if unset
spec:
  targetRef:
    kind: MeshService
    name: web
  to:
    - targetRef:
        kind: MeshService
        name: backend
      default:
        tcp:
          maxConnectAttempt: 5

Apply the configuration with kubectl apply -f [..].

type: MeshRetry
name: web-to-backend-retry-tcp
mesh: default
spec:
  targetRef:
    kind: MeshService
    name: web
  to:
    - targetRef:
        kind: MeshService
        name: backend
      default:
        tcp:
          maxConnectAttempt: 5

Apply the configuration with kumactl apply -f [..] or with the HTTP API.

All policy options

Thank you for your feedback.
Was this page useful?
Too much on your plate? close cta icon
More features, less infrastructure with Kong Konnect. 1M requests per month for free.
Try it for Free
  • Kong
    Powering the API world

    Increase developer productivity, security, and performance at scale with the unified platform for API management, service mesh, and ingress controller.

    • Products
      • Kong Konnect
      • Kong Gateway Enterprise
      • Kong Gateway
      • Kong Mesh
      • Kong Ingress Controller
      • Kong Insomnia
      • Product Updates
      • Get Started
    • Documentation
      • Kong Konnect Docs
      • Kong Gateway Docs
      • Kong Gateway Enterprise Docs
      • Kong Mesh Docs
      • Kong Insomnia Docs
      • Kong Konnect Plugin Hub
    • Open Source
      • Kong Gateway
      • Kuma
      • Insomnia
      • Kong Community
    • Company
      • About Kong
      • Customers
      • Careers
      • Press
      • Events
      • Contact
  • Terms• Privacy• Trust and Compliance
© Kong Inc. 2023