MeshOPA (beta) - OPA Policy Integration

This feature is released as beta and should not be deployed in a production environment.

Warning: This policy uses the new policy matching algorithm and is in a beta state. It should not be mixed with the OPA Policy.

MeshOPA policy plugin

Kong Mesh integrates the Open Policy Agent (OPA) to provide access control for your services.

The agent is included in the data plane proxy sidecar, instead of the more common deployment as a separate sidecar.

When the MeshOPA policy is applied, the control plane configures the following:

• The embedded policy agent, with the specified policy
• Envoy, to use External Authorization that points to the embedded policy agent

TargetRef support matrix

To learn more about the information in this table, see the matching docs.

Configuration

To apply a policy with MeshOPA, you must do the following:

• Specify the group of data plane proxies to apply the policy to with the targetRef property.
• Provide a policy with the appendPolicies property. Policies are defined in the Rego language.

  Note: You cannot currently apply multiple OPA policies. This limitation will be addressed in the future.

• Optionally provide custom configuration for the policy agent.

You must also specify the HTTP protocol in your mesh configuration:

apiVersion: v1
kind: Service
metadata:
  name: web
  namespace: kong-mesh-example
spec:
  selector:
    app: web
  ports:
  - port: 8080
    appProtocol: http

type: Dataplane
mesh: default
name: web
networking:
  address: 192.168.0.1
  inbound:
  - port: 80
    servicePort: 8080
    tags:
      kuma.io/service: web
      kuma.io/protocol: http # required for OPA support

For more information, see the Kong Mesh documentation about protocol support.

Inline

apiVersion: kuma.io/v1alpha1
kind: MeshOPA
metadata:
  name: mopa-1
  namespace: kong-mesh-system
  labels:
    kuma.io/mesh: default # optional, defaults to `default` if unset
spec:
  targetRef:
    kind: Mesh
  default:
    agentConfig: # optional
      inlineString: | # one of: inlineString, secret
        decision_logs:
          console: true
    appendPolicies:
      - inlineString: | # one of: inlineString, secret
          package envoy.authz

          import input.attributes.request.http as http_request

          default allow = false

          token = {"valid": valid, "payload": payload} {
              [_, encoded] := split(http_request.headers.authorization, " ")
              [valid, _, payload] := io.jwt.decode_verify(encoded, {"secret": "secret"})
          }

          allow {
              is_token_valid
              action_allowed
          }

          is_token_valid {
            token.valid
            now := time.now_ns() / 1000000000
            token.payload.nbf <= now
            now < token.payload.exp
          }

          action_allowed {
            http_request.method == "GET"
            token.payload.role == "admin"
          }

type: MeshOPA
mesh: default
name: mopa-1
spec:
  targetRef:
    kind: Mesh
  default:
    agentConfig: # optional
      inlineString: | # one of: inlineString, secret
        decision_logs:
          console: true
    appendPolicies: # optional
      - inlineString: | # one of: inlineString, secret
          package envoy.authz

          import input.attributes.request.http as http_request

          default allow = false

          token = {"valid": valid, "payload": payload} {
              [_, encoded] := split(http_request.headers.authorization, " ")
              [valid, _, payload] := io.jwt.decode_verify(encoded, {"secret": "secret"})
          }

          allow {
              is_token_valid
              action_allowed
          }

          is_token_valid {
            token.valid
            now := time.now_ns() / 1000000000
            token.payload.nbf <= now
            now < token.payload.exp
          }

          action_allowed {
            http_request.method == "GET"
            token.payload.role == "admin"
          }

With secrets

Encoding the policy in a Secret provides some security for policies that contain sensitive data.

Configuration

Kong Mesh defines a default configuration for OPA, but you can adjust the configuration to meet your environment’s requirements.

The following environment variables are available:

apiVersion: v1
kind: ConfigMap
metadata:
  name: kong-mesh-control-plane-config
  namespace: kong-mesh-system
data:
  config.yaml: |
    runtime:
      kubernetes:
        injector:
          sidecarContainer:
            envVars:
              KMESH_OPA_ENABLED: "false"
              KMESH_OPA_ADDR: ":8888"
              KMESH_OPA_CONFIG_OVERRIDES: "config1:x,config2:y"

kuma:
  controlPlane:
    config: |
      runtime:
        kubernetes:
          injector:
            sidecarContainer:
              envVars:
                KMESH_OPA_ENABLED: "false"
                KMESH_OPA_ADDR: ":8888"
                KMESH_OPA_CONFIG_OVERRIDES: "config1:x,config2:y"

--opa-addr
--opa-config-path
--opa-diagnostic-addr
--opa-enabled                    
--opa-ext-authz-addr
--opa-set strings

• Override the config for individual data plane proxies by placing the appropriate annotations on the Pod:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-app
  namespace: kuma-example
spec:
  ...
  template:
    metadata:
      ...
      annotations:
        # indicate to Kong Mesh that this Pod doesn't need a sidecar
        kuma.io/sidecar-env-vars: "KMESH_OPA_ENABLED=false;KMESH_OPA_ADDR=:8888;KMESH_OPA_CONFIG_OVERRIDES=config1:x,config2:y"

Configuring the authorization filter

apiVersion: kuma.io/v1alpha1
kind: MeshOPA
metadata:
  name: mopa-1
  namespace: kong-mesh-system
  labels:
    kuma.io/mesh: default # optional, defaults to `default` if unset
spec:
  targetRef:
    kind: Mesh
  default:
    authConfig: # optional
        statusOnError: 413 # optional: defaults to 403.
        onAgentFailure: allow # optional: one of 'allow' or 'deny', defaults to 'deny' defines the behavior when communication with the agent fails or the policy execution fails.
        requestBody: # optional
            maxSize: 1024 # the max number of bytes to send to the agent, if we exceed this, the request to the agent will have: `x-envoy-auth-partial-body: true`.
            sendRawBody: true # use when the body is not plaintext. The agent request will have `raw_body` instead of `body`
    agentConfig: # optional
      inlineString: | # one of: inlineString, secret
        decision_logs:
          console: true
    appendPolicies:
      - secret: opa-policy

type: MeshOPA
mesh: default
name: mopa-1
spec:
  targetRef:
    kind: Mesh
  default:
    authConfig: # optional
      statusOnError: 413 # optional: defaults to 403. http statusCode to use when the connection to the agent failed.
      onAgentFailure: allow # optional: one of 'allow' or 'deny', defaults to 'deny'. defines the behavior when communication with the agent fails or the policy execution fails.
      requestBody: # optional
        maxSize: 1024 # the maximum number of bytes to send to the agent, if we exceed this, the request to the agent will have: `x-envoy-auth-partial-body: true`.
        sendRawBody: true # use when the body is not plaintext. The agent request will have `raw_body` instead of `body`
    agentConfig: # optional
      inlineString: | # one of: inlineString, secret
        decision_logs:
          console: true
    appendPolicies: # optional
      - secret: opa-policy

By default, the body will not be sent to the agent. To send it, set authConfig.requestBody.maxSize to the maximum size of your body. If the request body is larger than this parameter, it will be truncated and the header x-envoy-auth-partial-body will be set to true.

Support for external API management servers

The agentConfig field lets you define a custom configuration that points to an external management server:

apiVersion: kuma.io/v1alpha1
kind: MeshOPA
metadata:
  name: mopa-1
  namespace: kong-mesh-system
  labels:
    kuma.io/mesh: default
spec:
  targetRef:
    kind: Mesh
  default:
    agentConfig:
      inlineString: |
        services:
          acmecorp:
            url: https://example.com/control-plane-api/v1
            credentials:
              bearer:
                token: "bGFza2RqZmxha3NkamZsa2Fqc2Rsa2ZqYWtsc2RqZmtramRmYWxkc2tm"

        discovery:
          name: example
          resource: /configuration/example/discovery

type: MeshOPA
mesh: default
name: mopa-1
spec:
  targetRef:
    kind: Mesh
  default:
    agentConfig:
      inlineString: | # one of: inlineString, secret
        services:
          acmecorp:
            url: https://example.com/control-plane-api/v1
            credentials:
              bearer:
                token: "bGFza2RqZmxha3NkamZsa2Fqc2Rsa2ZqYWtsc2RqZmtramRmYWxkc2tm"
        discovery:
          name: example
          resource: /configuration/example/discovery

Example

The following example shows how to deploy and test a sample MeshOPA policy on Kubernetes, using the kuma-demo application.

1. Deploy the example application:

   kubectl apply -f https://bit.ly/demokuma
   

2. Make a request from the frontend to the backend:

   kubectl exec -i -t $(kubectl get pod -l "app=kuma-demo-frontend" -o jsonpath='{.items[0].metadata.name}' -n kuma-demo) -n kuma-demo -c kuma-fe -- curl backend:3001 -v
   

   The output looks like:

   Defaulting container name to kuma-fe.
   Use 'kubectl describe pod/kuma-demo-app-6787b4f7f5-m428c -n kuma-demo' to see all of the containers in this pod.
   *   Trying 10.111.108.218:3001...
   * TCP_NODELAY set
   * Connected to backend (10.111.108.218) port 3001 (#0)
   > GET / HTTP/1.1
   > Host: backend:3001
   > User-Agent: curl/7.67.0
   > Accept: */*
   >
   * Mark bundle as not supporting multiuse
   < HTTP/1.1 200 OK
   < x-powered-by: Express
   < cache-control: no-store, no-cache, must-revalidate, private
   < access-control-allow-origin: *
   < access-control-allow-methods: PUT, GET, POST, DELETE, OPTIONS
   < access-control-allow-headers: *
   < host: backend:3001
   < user-agent: curl/7.67.0
   < accept: */*
   < x-forwarded-proto: http
   < x-request-id: 1717af9c-2587-43b9-897f-f8061bba5ad4
   < content-length: 90
   < content-type: text/html; charset=utf-8
   < date: Tue, 16 Mar 2021 15:33:18 GMT
   < x-envoy-upstream-service-time: 1521
   < server: envoy
   <
   * Connection #0 to host backend left intact
   Hello World! Marketplace with sales and reviews made with <3 by the OCTO team at Kong Inc.
   

3. Apply a MeshOPA policy that requires a valid JWT token:

   echo "
   apiVersion: kuma.io/v1alpha1
   kind: MeshOPA
   metadata:
     namespace: kong-mesh-system
     name: mopa-1
     labels:
       kuma.io/mesh: default
   spec:
     targetRef:
       kind: Mesh
     default:
       appendPolicies:
         - inlineString: |
             package envoy.authz
   
             import input.attributes.request.http as http_request
   
             default allow = false
   
             token = {\"valid\": valid, \"payload\": payload} {
                 [_, encoded] := split(http_request.headers.authorization, \" \")
                 [valid, _, payload] := io.jwt.decode_verify(encoded, {\"secret\": \"secret\"})
             }
   
             allow {
                 is_token_valid
                 action_allowed
             }
   
             is_token_valid {
               token.valid
               now := time.now_ns() / 1000000000
               token.payload.nbf <= now
               now < token.payload.exp
             }
   
             action_allowed {
               http_request.method == \"GET\"
               token.payload.role == \"admin\"
             }
   " | kubectl apply -f -
   

4. Make an invalid request from the frontend to the backend:

   kubectl exec -i -t $(kubectl get pod -l "app=kuma-demo-frontend" -o jsonpath='{.items[0].metadata.name}' -n kuma-demo) -n kuma-demo -c kuma-fe -- curl backend:3001 -v
   

   The output looks like:

   Defaulting container name to kuma-fe.
   Use 'kubectl describe pod/kuma-demo-app-6787b4f7f5-bwvnb -n kuma-demo' to see all of the containers in this pod.
   *   Trying 10.105.146.164:3001...
   * TCP_NODELAY set
   * Connected to backend (10.105.146.164) port 3001 (#0)
   > GET / HTTP/1.1
   > Host: backend:3001
   > User-Agent: curl/7.67.0
   > Accept: */*
   >
   * Mark bundle as not supporting multiuse
   < HTTP/1.1 403 Forbidden
   < date: Tue, 09 Mar 2021 16:50:40 GMT
   < server: envoy
   < x-envoy-upstream-service-time: 2
   < content-length: 0
   <
   * Connection #0 to host backend left intact
   

   Note the HTTP/1.1 403 Forbidden message. The application doesn’t allow a request without a valid token.

   The policy can take up to 30 seconds to propagate, so if this request succeeds the first time, wait and then try again.

5. Make a valid request from the frontend to the backend.

   Export the token into an environment variable:

   export ADMIN_TOKEN="eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJyb2xlIjoiYWRtaW4iLCJzdWIiOiJZbTlpIiwibmJmIjoxNTE0ODUxMTM5LCJleHAiOjI1MjQ2MDgwMDB9.H0-42LYzoWyQ_4MXAcED30u6lA5JE087eECV2nxDfXo"
   

   Make the request:

   kubectl exec -i -t $(kubectl get pod -l "app=kuma-demo-frontend" -o jsonpath='{.items[0].metadata.name}' -n kuma-demo) -n kuma-demo -c kuma-fe -- curl -H "Authorization: Bearer $ADMIN_TOKEN" backend:3001
   

   The output looks like:

   Defaulting container name to kuma-fe.
   Use 'kubectl describe pod/kuma-demo-app-6787b4f7f5-m428c -n kuma-demo' to see all of the containers in this pod.
   *   Trying 10.111.108.218:3001...
   * TCP_NODELAY set
   * Connected to backend (10.111.108.218) port 3001 (#0)
   > GET / HTTP/1.1
   > Host: backend:3001
   > User-Agent: curl/7.67.0
   > Accept: */*
   >
   * Mark bundle as not supporting multiuse
   < HTTP/1.1 200 OK
   < x-powered-by: Express
   < cache-control: no-store, no-cache, must-revalidate, private
   < access-control-allow-origin: *
   < access-control-allow-methods: PUT, GET, POST, DELETE, OPTIONS
   < access-control-allow-headers: *
   < host: backend:3001
   < user-agent: curl/7.67.0
   < accept: */*
   < x-forwarded-proto: http
   < x-request-id: 8fd7b398-1ba2-4c2e-b229-5159d04d782e
   < content-length: 90
   < content-type: text/html; charset=utf-8
   < date: Tue, 16 Mar 2021 17:26:00 GMT
   < x-envoy-upstream-service-time: 261
   < server: envoy
   <
   * Connection #0 to host backend left intact
   Hello World! Marketplace with sales and reviews made with <3 by the OCTO team at Kong Inc.
   

   The request is valid again because the token is signed with the secret private key, its payload includes the admin role, and it is not expired.