Provisioning a Secret to Your K8s Cluster

Suggest Edits

To provision a secret to your Kubernetes cluster, create the secret in the Akeyless Vault platform, along with an authentication method that has access to the secret. Then, configure K8s to get the secret.

Set Up a Secret in the Akeyless Vault Platform

👍

Prerequisites

  • Ensure that you have a K8s cluster and Helm installed before you begin.

  • A K8s authentication method.

  • Ensure that the K8s cluster is at least as new as v1.16 (to use admissionregistration.k8s.io/v1).

  • Ensure that MutatingAdmissionWebhook admission controllers are enabled. This topic provides a recommended set of admission controllers to enable in general.

  • For Azure AKS, ensure that the Azure AKS cluster is running with the enable-managed-identity flag.

  • For OpenShift, ensure that the OpenShift flag is enabled in the values.yaml file:
    openshiftEnabled: true.

  1. Create a static or dynamic secret in the Akeyless Vault Platform. For example, the following command creates a static secret called my_k8s_secret in the K8s folder.
akeyless create-secret --name K8s/my_k8s_secret --value myPassword
  1. Create an authentication method. For details about the authentication methods that are supported for K8s, see Authentication Methods for K8s.

The following example uses a pre-defined Kubernetes Auth method called K8s_Auth in the K8s folder.

akeyless create-auth-method-k8s --name K8s/K8s_Auth --bound-pod-names mypod1 \
    --bound-pod-names mypod2 --bound-namespaces my-namespace-a --json
  1. Create an access role to provide the authentication method with access to the secret. For example, the following commands create the K8s_Role role in the K8s folder, associate the role with the K8s_Auth authentication method, and grant read and list access to all the secrets in the K8s folder.
akeyless create-role --name /K8s/K8s_Role

akeyless assoc-role-am --role-name /K8s/K8s_Role --am-name K8s/K8s_Auth

akeyless set-role-rule --role-name /K8s/K8s_Role --path /K8s/'*' --capability read --capability list

Configure K8s

  1. Get the Akeyless Helm chart from here.
helm repo add akeyless https://akeylesslabs.github.io/helm-charts
helm repo update

  1. Copy the values.yaml file locally, and modify the access credential values in the file. For example, if you are using a Kubernetes Auth, modify the values as follows:

    • Set AKEYLESS_ACCESS_ID to the access ID of the authentication method that has access to the secret.

    • Set AKEYLESS_ACCESS_TYPE to k8s.

    • If you use a customer fragment, set AKEYLESS_API_GW_URL to the URL of your Akeyless Gateway. If you do not use your own Akeyless Gateway, set AKEYLESS_API_GW_URL to https://rest.akeyless.io, the URL of the public Akeyless Gateway.

AKEYLESS_URL: "https://vault.akeyless.io"
AKEYLESS_ACCESS_ID: "<acc_id>"
AKEYLESS_ACCESS_TYPE: "k8s"
AKEYLESS_K8S_AUTH_CONF_NAME: "K8s_conf_name"
#To use your gw add:
#AKEYLESS_API_GW_URL: "https://Akeyless-gw-url"
  1. On your K8s cluster, create a new namespace for Akeyless.
kubectl create namespace akeyless
kubectl label namespace akeyless name=akeyless

For OpenShift: 

oc create namespace akeyless
oc label namespace akeyless name=akeyless
  1. Deploy the Helm chart to the namespace.
helm install aks akeyless/akeyless-secrets-injection --namespace akeyless -f values.yaml
  1. Validate the deployment state.
kubectl get all -n akeyless

For OpenShift:

oc get all -n akeyless

The following is an example of the output:

Akeyless Annotations

To work with the Akeyless secret injecting webhook, you need to set the following annotations in your deployment YAML files:

  1. Enable the Akeyless webhook
akeyless/enabled: "true"
  1. The default location of the Akeyless secrets folder inside your pod file system is /akeyless/secrets/< Your Secret >. You can modify the default directory in the values.yaml file. You can specify a different location by adding location={path} after your secret name within the annotation. For example, secret 1 and secret 2 will be saved in the /tmp/ folder. 
akeyless/inject_file: "/secret1|location=/tmp/s1,/secret2|location=/tmp/s2"
  1. Set the location of the Akeyless secrets folder inside your pod file system to inject your secret into your pod file system.
AKEYLESS_SECRET_DIR_NAME: "/My/New/Dir" #Path to save secrets inside pods file systems

To inject your secret into your pod environment variable, the value of your env should have the prefix akeyless:/Path/to/secret .

👍

The Akeyless env does not support docker ENTRYPOINT. Use your command inside your YAML file.

The following examples demonstrate Akeyless secret injection for environment variable env.yaml and pod file system Injectfile.yaml 

apiVersion: apps/v1
kind: Deployment
metadata:
  name: test
spec:
  replicas: 1
  selector:
    matchLabels:
      app: hello-secrets
  template:
    metadata:
      labels:
        app: hello-secrets
      annotations:
        akeyless/enabled: "true"
    spec:
      containers:
      - name: alpine
        image: alpine
        command:
          - "sh"
          - "-c"
          - "echo $MY_SECRET && echo going to sleep... && sleep 10000"
        env:
        - name: MY_SECRET
          value: akeyless:/K8s/my_k8s_secret

apiVersion: apps/v1
kind: Deployment
metadata:
  name: test-file
spec:
  replicas: 1
  selector:
    matchLabels:
      app: hello-secrets-2
  template:
    metadata:
      labels:
        app: hello-secrets-2
      annotations:
        akeyless/enabled: "true"
        akeyless/inject_file: "/K8s/my_k8s_secret"
    spec:
      containers:
      - name: alpine
        image: alpine
        command:
          - "sh"
          - "-c"
          - "cat /akeyless/secrets/K8s/my_k8s_secret && echo going to sleep... && sleep 10000"

apiVersion: v1
kind: Service
metadata:
  name: wordpress-mysql
  labels:
    app: wordpress
spec:
  ports:
    - port: 3306
  selector:
    app: wordpress
    tier: mysql
  clusterIP: None
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mysql-pv-claim
  labels:
    app: wordpress
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 20Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: wordpress-mysql
  labels:
    app: wordpress
spec:
  selector:
    matchLabels:
      app: wordpress
      tier: mysql
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: wordpress
        tier: mysql
      annotations:
        akeyless/enabled: "true"
    spec:
      containers:
      - image: bitnami/mysql:5.7.33
        name: mysql
        command: ["/opt/bitnami/scripts/mysql/entrypoint.sh"]
        args: ["/opt/bitnami/scripts/mysql/run.sh"]
        env:
        - name: MYSQL_ROOT_PASSWORD
          value: akeyless:/K8s/my_k8s_secret
        ports:
        - containerPort: 3306
          name: mysql
        volumeMounts:
        - name: mysql-persistent-storage
          mountPath: /var/lib/mysql
      volumes:
      - name: mysql-persistent-storage
        persistentVolumeClaim:
          claimName: mysql-pv-claim

apiVersion: v1
kind: Service
metadata:
  name: wordpress
  labels:
    app: wordpress
spec:
  ports:
    - port: 80
  selector:
    app: wordpress
    tier: frontend
  type: LoadBalancer
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: wp-pv-claim
  labels:
    app: wordpress
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 20Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: wordpress
  labels:
    app: wordpress
spec:
  selector:
    matchLabels:
      app: wordpress
      tier: frontend
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: wordpress
        tier: frontend
      annotations:
        akeyless/enabled: "true"
    spec:
      containers:
      - image: wordpress:4.8-apache
        name: wordpress
        command: ["docker-entrypoint.sh"]
        args: ["apache2-foreground"]
        env:
        - name: WORDPRESS_DB_HOST
          value: wordpress-mysql
        - name: WORDPRESS_DB_PASSWORD
          value: akeyless:/K8s/my_k8s_secret
        ports:
        - containerPort: 80
          name: wordpress
        volumeMounts:
        - name: wordpress-persistent-storage
          mountPath: /var/www/html
      volumes:
      - name: wordpress-persistent-storage
        persistentVolumeClaim:
          claimName: wp-pv-claim

Sidecar Mode

Akeyless plugin for K8s can work as a sidecar, where an additional container will run side by side to your pod in order to track any changes within your secrets values while reflecting those changes into your running pod.

👍

Note:

Sidecar mode can work only with file injection mode.

To enable Akeyless as a sidecar on your K8s cluster add the following annotation to your deployment file: 

akeyless/side_car_enabled: "true"

To set the desired refresh interval for Akeyless sidecar : 

akeyless/side_car_refresh_interval: "300s"

Where supported time units are "s", "m", "h".

To retrieve multiple versions of your secret simultaneously (only for static-secret and rotated-secret)

akeyless/side_car_versions_to_retrieve: "2"

The following example demonstrate the Akeyless sidecar annotation: 

apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: test-file
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: file-secrets
      template:
        metadata:
          labels:
            app: file-secrets
          annotations:
            akeyless/enabled: "true"
            akeyless/inject_file: "/K8s/my_k8s_secret|location=/secrets/secretsVersion.json" 
            akeyless/side_car_enabled: "true"
            akeyless/side_car_refresh_interval: "30s"
            akeyless/side_car_versions_to_retrieve: "2"

        spec:
          containers:
          - name: alpine
            image: alpine
            command:
              - "sh"
              - "-c"
              - "cat /secrets/secretsVersion.json && echo going to sleep... && sleep 10000"

The following table lists the available annotations:

Annotation

Options

Description

akeyless/enabled: "true"

true or false

Enable akeyless K8s plugin.

akeyless/side_car_enabled: "true"

true or false

Set Akeyless K8s plugin to work in side car mode.

akeyless/side_car_refresh_interval:"30s"

Supported time units are "s", "m", "h".

Set the desired refresh interval for Akeyless sidecar

akeyless/side_car_versions_to_retrieve: "2"

'2' or higher

Fetch the last X versions of your secret.

akeyless/inject_file: "/mysecret/ | location=/path/to/save/secret/"

location= /path/to/save/secret

Set the location for your secrets to be saved within your pod file system.
Available for files only.

akeyless/inject_file: "/mysecret | permission=0644"

permission=0777

Set the permission of the file that contains your secret value.

Default permission is 0644.

Available for files only

akeyless/inject_file: "/mysecret | version=1"

version={version of your secret}

Fetch a specific version of your secret.

Default value is set to latest version.

Available for Environment variables as well.

akeyless/inject_file: "/mysecret | decode=base64"

decode=base64

Set default base64 decoding for your secrets values.

Default is set to none

Available for Environment variables as well.

The following examples demonstrate the usage of Akeyless annotations:

Files:

akeyless/inject_file: "/k8s-secrets/secret1|location=/tmp/s1|permission=0777|decode=base64,/k8s-secrets/secret2,/k8s-secrets/secret3|version=3|location=/tmp/s2"

Environment variable: 

- name:  MY_SECRET
    value: 'akeyless:/k8s-secrets/secret1|decode=base64|version=2'

Troubleshooting

When you are working with a GKE cluster, make sure that port 8443 is opened in your firewall rules, This port is needed by the Akeyless Secret Injection mutating webhook. Update your firewall rule as follow:

  1. Review the firewall rule for master access:
gcloud compute firewall-rules list
  1. Replace the existing rule and allow master access:
gcloud compute firewall-rules update <firewall-rule-name> --allow tcp:10250,tcp:443,tcp:8443

Updated 6 days ago

What’s Next