Kong Mesh - OPA Policy Integration

OPA policy plugin

Kong Mesh integrates the Open Policy Agent (OPA) to provide access control for your services.

The agent is included in the data plane proxy sidecar, instead of the more common deployment as a separate sidecar.

When OPAPolicy is applied, the control plane configures:

• the embedded policy agent, with the specified policy
• Envoy, to use External Authorization that points to the embedded policy agent

Usage

To apply a policy with OPA:

• Specify the group of data plane proxies to apply the policy to with the selectors property.
• Provide the list of policies with the conf property. Policies are defined in the Rego language.
• Optionally provide custom configuration for the policy agent.

Inline

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
apiVersion: kuma.io/v1alpha1
kind: OPAPolicy
mesh: default
metadata:
  name: opa-1
spec:
  selectors:
  - match:
      kuma.io/service: '*'
  conf:
    agentConfig: # optional
      inlineString: | # one of: inlineString, secret
        decision_logs:
          console: true
    policies:
      - inlineString: | # one of: inlineString, secret
          package envoy.authz

          import input.attributes.request.http as http_request

          default allow = false

          token = {"valid": valid, "payload": payload} {
              [_, encoded] := split(http_request.headers.authorization, " ")
              [valid, _, payload] := io.jwt.decode_verify(encoded, {"secret": "secret"})
          }

          allow {
              is_token_valid
              action_allowed
          }

          is_token_valid {
            token.valid
            now := time.now_ns() / 1000000000
            token.payload.nbf <= now
            now < token.payload.exp
          }

          action_allowed {
            http_request.method == "GET"
            token.payload.role == "admin"
          }

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
type: OPAPolicy
mesh: default
name: opa-1
selectors:
- match:
    kuma.io/service: '*'
conf:
  agentConfig: # optional
    inlineString: | # one of: inlineString, secret
      decision_logs:
        console: true
  policies: # optional
    - inlineString: | # one of: inlineString, secret
        package envoy.authz
  
        import input.attributes.request.http as http_request
  
        default allow = false
  
        token = {"valid": valid, "payload": payload} {
            [_, encoded] := split(http_request.headers.authorization, " ")
            [valid, _, payload] := io.jwt.decode_verify(encoded, {"secret": "secret"})
        }
  
        allow {
            is_token_valid
            action_allowed
        }
  
        is_token_valid {
          token.valid
          now := time.now_ns() / 1000000000
          token.payload.nbf <= now
          now < token.payload.exp
        }
  
        action_allowed {
          http_request.method == "GET"
          token.payload.role == "admin"
        }

With Secrets

Encoding the policy in a Secret provides some security for policies that contain sensitive data.

Configuration

Kong Mesh defines a default configuration for OPA, but you can adjust the configuration to meet your environment’s requirements.

The following environment variables are available:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
apiVersion: v1
kind: ConfigMap
metadata:
  name: kong-mesh-control-plane-config
  namespace: kong-mesh-system
data:
  config.yaml: |
    runtime:
      kubernetes:
        injector:
          sidecarContainer:
            envVars:
              KMESH_OPA_ENABLED: "false"
              KMESH_OPA_ADDR: ":8888"
              KMESH_OPA_CONFIG_OVERRIDES: "config1:x,config2:y"

1
2
3
4
5
6
7
8
9
10
11
kuma:
  controlPlane:
    config: |
      runtime:
        kubernetes:
          injector:
            sidecarContainer:
              envVars:
                KMESH_OPA_ENABLED: "false"
                KMESH_OPA_ADDR: ":8888"
                KMESH_OPA_CONFIG_OVERRIDES: "config1:x,config2:y"

1
2
3
4
5
6
--opa-addr 
--opa-config-path 
--opa-diagnostic-addr 
--opa-enabled                    
--opa-ext-authz-addr 
--opa-set strings 

• Override the config for individual data plane proxies by placing the appropriate annotations on the Pod:

1
2
3
4
5
6
7
8
9
10
11
12
13
apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-app
  namespace: kuma-example
spec:
  ...
  template:
    metadata:
      ...
      annotations:
        # indicate to Kuma that this Pod doesn't need a sidecar
        kuma.io/sidecar-env-vars: "KMESH_OPA_ENABLED=false;KMESH_OPA_ADDR=:8888;KMESH_OPA_CONFIG_OVERRIDES=config1:x,config2:y"

Support for external API management servers

The agentConfig field lets you define a custom configuration that points to an external management server:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
apiVersion: kuma.io/v1alpha1
kind: OPAPolicy
mesh: default
metadata:
  name: opa-1
spec:
  selectors:
  - match:
      kuma.io/service: '*'
  conf:
    agentConfig:
      inlineString: |
        services:
          acmecorp:
            url: https://example.com/control-plane-api/v1
            credentials:
              bearer:
                token: "bGFza2RqZmxha3NkamZsa2Fqc2Rsa2ZqYWtsc2RqZmtramRmYWxkc2tm"
        
        discovery:
          name: example
          resource: /configuration/example/discovery

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
type: OPAPolicy
mesh: default
name: opa-1
selectors:
- match:
    kuma.io/service: '*'
conf:
  agentConfig:
    inlineString: | # one of: inlineString, secret
      services:
        acmecorp:
          url: https://example.com/control-plane-api/v1
          credentials:
            bearer:
              token: "bGFza2RqZmxha3NkamZsa2Fqc2Rsa2ZqYWtsc2RqZmtramRmYWxkc2tm"
      discovery:
        name: example
        resource: /configuration/example/discovery

Example

The following example shows how to deploy and test a sample OPA Policy on Kubernetes, using the kuma-demo application.

1. Deploy the example application:

   1
   kubectl apply -f https://bit.ly/demokuma
   

2. Make a request from the frontend to the backend:

   1
   $ kubectl exec -i -t $(kubectl get pod -l "app=kuma-demo-frontend" -o jsonpath='{.items[0].metadata.name}' -n kuma-demo) -n kuma-demo -- curl backend:3001 -v
   

   The output looks like:

   1
   2
   3
   4
   5
   6
   7
   8
   9
   10
   11
   12
   13
   14
   15
   16
   17
   18
   19
   20
   21
   22
   23
   24
   25
   26
   27
   28
   29
   30
   Defaulting container name to kuma-fe.
   Use 'kubectl describe pod/kuma-demo-app-6787b4f7f5-m428c -n kuma-demo' to see all of the containers in this pod.
   *   Trying 10.111.108.218:3001...
   * TCP_NODELAY set
   * Connected to backend (10.111.108.218) port 3001 (#0)
   > GET / HTTP/1.1
   > Host: backend:3001
   > User-Agent: curl/7.67.0
   > Accept: */*
   > 
   * Mark bundle as not supporting multiuse
   < HTTP/1.1 200 OK
   < x-powered-by: Express
   < cache-control: no-store, no-cache, must-revalidate, private
   < access-control-allow-origin: *
   < access-control-allow-methods: PUT, GET, POST, DELETE, OPTIONS
   < access-control-allow-headers: *
   < host: backend:3001
   < user-agent: curl/7.67.0
   < accept: */*
   < x-forwarded-proto: http
   < x-request-id: 1717af9c-2587-43b9-897f-f8061bba5ad4
   < content-length: 90
   < content-type: text/html; charset=utf-8
   < date: Tue, 16 Mar 2021 15:33:18 GMT
   < x-envoy-upstream-service-time: 1521
   < server: envoy
   < 
   * Connection #0 to host backend left intact
   Hello World! Marketplace with sales and reviews made with <3 by the OCTO team at Kong Inc.
   

3. Apply an OPA Policy that requires a valid JWT token:

   1
   2
   3
   4
   5
   6
   7
   8
   9
   10
   11
   12
   13
   14
   15
   16
   17
   18
   19
   20
   21
   22
   23
   24
   25
   26
   27
   28
   29
   30
   31
   32
   33
   34
   35
   36
   37
   38
   39
   40
   41
   echo " 
   apiVersion: kuma.io/v1alpha1
   kind: OPAPolicy
   mesh: default
   metadata:
     name: opa-1
   spec:
     selectors:
     - match:
         kuma.io/service: '*'
     conf:
       policies:
         - inlineString: |
             package envoy.authz
   
             import input.attributes.request.http as http_request
   
             default allow = false
   
             token = {\"valid\": valid, \"payload\": payload} {
                 [_, encoded] := split(http_request.headers.authorization, \" \")
                 [valid, _, payload] := io.jwt.decode_verify(encoded, {\"secret\": \"secret\"})
             }
   
             allow {
                 is_token_valid
                 action_allowed
             }
   
             is_token_valid {
               token.valid
               now := time.now_ns() / 1000000000
               token.payload.nbf <= now
               now < token.payload.exp
             }
   
             action_allowed {
               http_request.method == \"GET\"
               token.payload.role == \"admin\"
             }
   " | kubectl apply -f -
   

4. Make an invalid request from the frontend to the backend:

   1
   $ kubectl exec -i -t $(kubectl get pod -l "app=kuma-demo-frontend" -o jsonpath='{.items[0].metadata.name}' -n kuma-demo) -n kuma-demo -- curl backend:3001 -v
   

   The output looks like:

   1
   2
   3
   4
   5
   6
   7
   8
   9
   10
   11
   12
   13
   14
   15
   16
   17
   18
   Defaulting container name to kuma-fe.
   Use 'kubectl describe pod/kuma-demo-app-6787b4f7f5-bwvnb -n kuma-demo' to see all of the containers in this pod.
   *   Trying 10.105.146.164:3001...
   * TCP_NODELAY set
   * Connected to backend (10.105.146.164) port 3001 (#0)
   > GET / HTTP/1.1
   > Host: backend:3001
   > User-Agent: curl/7.67.0
   > Accept: */*
   >
   * Mark bundle as not supporting multiuse
   < HTTP/1.1 403 Forbidden
   < date: Tue, 09 Mar 2021 16:50:40 GMT
   < server: envoy
   < x-envoy-upstream-service-time: 2
   < content-length: 0
   <
   * Connection #0 to host backend left intact
   

   Note the HTTP/1.1 403 Forbidden message. The application doesn’t allow a request without a valid token.

   The policy can take up to 30 seconds to propagate, so if this request succeeds the first time, wait and then try again.

5. Make a valid request from the frontend to the backend:

   1
   2
   $ export ADMIN_TOKEN="eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJyb2xlIjoiYWRtaW4iLCJzdWIiOiJZbTlpIiwibmJmIjoxNTE0ODUxMTM5LCJleHAiOjE2NDEwODE1Mzl9.WCxNAveAVAdRCmkpIObOTaSd0AJRECY2Ch2Qdic3kU8"
   $ kubectl exec -i -t $(kubectl get pod -l "app=kuma-demo-frontend" -o jsonpath='{.items[0].metadata.name}' -n kuma-demo) -n kuma-demo -- curl -H "Authorization: Bearer $ADMIN_TOKEN" backend:3001
   

   The output looks like:

   1
   2
   3
   4
   5
   6
   7
   8
   9
   10
   11
   12
   13
   14
   15
   16
   17
   18
   19
   20
   21
   22
   23
   24
   25
   26
   27
   28
   29
   30
   Defaulting container name to kuma-fe.
   Use 'kubectl describe pod/kuma-demo-app-6787b4f7f5-m428c -n kuma-demo' to see all of the containers in this pod.
   *   Trying 10.111.108.218:3001...
   * TCP_NODELAY set
   * Connected to backend (10.111.108.218) port 3001 (#0)
   > GET / HTTP/1.1
   > Host: backend:3001
   > User-Agent: curl/7.67.0
   > Accept: */*
   > 
   * Mark bundle as not supporting multiuse
   < HTTP/1.1 200 OK
   < x-powered-by: Express
   < cache-control: no-store, no-cache, must-revalidate, private
   < access-control-allow-origin: *
   < access-control-allow-methods: PUT, GET, POST, DELETE, OPTIONS
   < access-control-allow-headers: *
   < host: backend:3001
   < user-agent: curl/7.67.0
   < accept: */*
   < x-forwarded-proto: http
   < x-request-id: 8fd7b398-1ba2-4c2e-b229-5159d04d782e
   < content-length: 90
   < content-type: text/html; charset=utf-8
   < date: Tue, 16 Mar 2021 17:26:00 GMT
   < x-envoy-upstream-service-time: 261
   < server: envoy
   < 
   * Connection #0 to host backend left intact
   Hello World! Marketplace with sales and reviews made with <3 by the OCTO team at Kong Inc.
   

   The request is valid again because the token is signed with the secret private key, its payload includes the admin role, and it is not expired.