Provisioning a Secret to Your K8s Cluster
To provision a secret to your Kubernetes cluster, create the secret in the Akeyless Vault platform, along with an authentication method that has access to the secret. Then, configure K8s to get the secret.
Set Up a Secret in the Akeyless Vault Platform
Prerequisites
Ensure that you have a K8s cluster and Helm installed before you begin.
A K8s authentication method.
K8s v1.19 and later (to use admissionregistration.k8s.io/v1).
MutatingAdmissionWebhook admission controllers enabled. This topic provides a recommended set of admission controllers to enable in general.
For Azure AKS, managed-identity enabled on your AKS cluster.
- Create a static or dynamic secret in the Akeyless Vault Platform. For example, the following command creates a static secret called my_k8s_secret in the K8s folder.
akeyless create-secret --name K8s/my_k8s_secret --value myPassword
- Create an authentication method. For details about the authentication methods that are supported for K8s, see Authentication Methods for K8s.
The following example uses a pre-defined Kubernetes Auth method called K8s_Auth in K8s folder i.e. K8s/K8s_Auth
.
- Create an access role to provide the authentication method with access to the secret. For example, the following commands create the K8s_Role role in the K8s folder, associate the role with the K8s_Auth authentication method, and grant read and list access to all the secrets in the K8s folder.
akeyless create-role --name /K8s/K8s_Role
akeyless assoc-role-am --role-name /K8s/K8s_Role --am-name K8s/K8s_Auth
akeyless set-role-rule --role-name /K8s/K8s_Role --path /K8s/'*' --capability read --capability list
Configure the Akeyless K8s Plugin
- Get the Akeyless Helm chart from here.
helm repo add akeyless https://akeylesslabs.github.io/helm-charts
helm repo update
- Fetch the values.yaml file locally, and modify the access credential values in the file.
helm show values akeyless/akeyless-secrets-injection > values.yaml
OpenShift
Enable the OpenShift flag in the values.yaml file:
openshiftEnabled: true
.
For example, if you are using a Kubernetes Auth, modify the values as follows:
-
Set AKEYLESS_ACCESS_ID to the access ID of the authentication method that has access to the secret.
-
Set AKEYLESS_ACCESS_TYPE to
k8s
. -
Set AKEYLESS_K8S_AUTH_CONF_NAME with your Gateway Kubernetes Auth name.
-
Set the AKEYLESS_API_GW_URL with the URL of your Akeyless Gateway on port
8080
. -
Optional AKEYLESS_CRASH_POD_ON_ERROR upon any failure, a pod that tries to fetch a secret and fails will crash. By default this option is disabled.
AKEYLESS_URL: "https://vault.akeyless.io"
AKEYLESS_ACCESS_ID: "<acc_id>"
AKEYLESS_ACCESS_TYPE: "k8s"
AKEYLESS_K8S_AUTH_CONF_NAME: "K8s_conf_name"
AKEYLESS_API_GW_URL: "https://gateway.url:8080"
#AKEYLESS_CRASH_POD_ON_ERROR: "enable"
- On your K8s cluster, create a new namespace for Akeyless.
kubectl create namespace akeyless
kubectl label namespace akeyless name=akeyless
For OpenShift:
oc create namespace akeyless
oc label namespace akeyless name=akeyless
- Deploy the Helm chart to the namespace.
helm install aks akeyless/akeyless-secrets-injection --namespace akeyless -f values.yaml
- Validate the deployment state.
kubectl get all -n akeyless
For OpenShift:
oc get all -n akeyless
The following is an example of the output:
kubectl get all -n akeyless
NAME READY STATUS RESTARTS AGE
pod/aks-akeyless-secrets-injection-77c857d496-r5xth 1/1 Running 1 (73s ago) 1d
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/aks-akeyless-secrets-injection ClusterIP 10.97.228.133 <none> 443/TCP 1d
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/aks-akeyless-secrets-injection 1/1 1 1 1d
NAME DESIRED CURRENT READY AGE
replicaset.apps/aks-akeyless-secrets-injection-77c857d496 1 1 1 1d
Akeyless Annotations
To work with the Akeyless secret injecting webhook, you need to set the following annotations in your deployment YAML files:
- Enable the Akeyless webhook
akeyless/enabled: "true"
- The default location of the Akeyless secrets folder inside your pod file system is /akeyless/secrets/. You can modify the default directory in the values.yaml file. You can specify a different location by adding
location=<path>
after your secret name within the annotation. For example, secret 1 and secret 2 will be saved in the /tmp/ folder ass1
ands2
respectively.
akeyless/inject_file: "/secret1|location=/tmp/s1,/secret2|location=/tmp/s2"
Alternatively, you can specify different destinations for files injection for multiple secrets using the following format:
akeyless/inject_file_*
In this example we will inject different secrets to different files:
akeyless/inject_file_s1: "/my-dir/my-json-secret|jq=.object.first|location=/tmp/s1|permission=0777|decode=base64"
akeyless/inject_file_secret2: "/k8s-secrets/secret2|location=/akeyless/secret2"
akeyless/inject_file_secret1: "/k8s-secrets/secret1"
To inject an entire folder, use the following annotation:
akeyless/inject_folder: "/prod/my-secrets-folder/|location=/tmp/secrets/"
In this example, all secrets under /prod/my-secrets-folder
will be injected into your pod fs
at /tmp/secrets/<full-secret-names>
.
- To set a default location of the Akeyless secrets folder inside your pods' file system to inject your secret into your pod file system globally, use this setting on the
values.yaml
file.
AKEYLESS_SECRET_DIR_NAME: "/My/New/Dir" #Path to save secrets inside pods file systems
To inject your secret into your pod environment variable, the value of your env should have the
prefix akeyless:/Path/to/secret
.
The Akeyless
env
does not support docker ENTRYPOINT. Use your command inside your deploymentyaml
The following examples demonstrate Akeyless secret injection for environment variable env.yaml
and pod file system Injectfile.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: test
spec:
replicas: 1
selector:
matchLabels:
app: hello-secrets
template:
metadata:
labels:
app: hello-secrets
annotations:
akeyless/enabled: "true"
spec:
containers:
- name: alpine
image: alpine
command:
- "sh"
- "-c"
- "echo $MY_SECRET && echo going to sleep... && sleep 10000"
env:
- name: MY_SECRET
value: akeyless:/K8s/my_k8s_secret
apiVersion: apps/v1
kind: Deployment
metadata:
name: test-file
spec:
replicas: 1
selector:
matchLabels:
app: hello-secrets-2
template:
metadata:
labels:
app: hello-secrets-2
annotations:
akeyless/enabled: "true"
akeyless/inject_file: "/K8s/my_k8s_secret"
spec:
containers:
- name: alpine
image: alpine
command:
- "sh"
- "-c"
- "cat /akeyless/secrets/K8s/my_k8s_secret && echo going to sleep... && sleep 10000"
apiVersion: v1
kind: Service
metadata:
name: wordpress-mysql
labels:
app: wordpress
spec:
ports:
- port: 3306
selector:
app: wordpress
tier: mysql
clusterIP: None
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: mysql-pv-claim
labels:
app: wordpress
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 20Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: wordpress-mysql
labels:
app: wordpress
spec:
selector:
matchLabels:
app: wordpress
tier: mysql
strategy:
type: Recreate
template:
metadata:
labels:
app: wordpress
tier: mysql
annotations:
akeyless/enabled: "true"
spec:
containers:
- image: bitnami/mysql:5.7.33
name: mysql
command: ["/opt/bitnami/scripts/mysql/entrypoint.sh"]
args: ["/opt/bitnami/scripts/mysql/run.sh"]
env:
- name: MYSQL_ROOT_PASSWORD
value: akeyless:/K8s/my_k8s_secret
ports:
- containerPort: 3306
name: mysql
volumeMounts:
- name: mysql-persistent-storage
mountPath: /var/lib/mysql
volumes:
- name: mysql-persistent-storage
persistentVolumeClaim:
claimName: mysql-pv-claim
apiVersion: v1
kind: Service
metadata:
name: wordpress
labels:
app: wordpress
spec:
ports:
- port: 80
selector:
app: wordpress
tier: frontend
type: LoadBalancer
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: wp-pv-claim
labels:
app: wordpress
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 20Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: wordpress
labels:
app: wordpress
spec:
selector:
matchLabels:
app: wordpress
tier: frontend
strategy:
type: Recreate
template:
metadata:
labels:
app: wordpress
tier: frontend
annotations:
akeyless/enabled: "true"
spec:
containers:
- image: wordpress:4.8-apache
name: wordpress
command: ["docker-entrypoint.sh"]
args: ["apache2-foreground"]
env:
- name: WORDPRESS_DB_HOST
value: wordpress-mysql
- name: WORDPRESS_DB_PASSWORD
value: akeyless:/K8s/my_k8s_secret
ports:
- containerPort: 80
name: wordpress
volumeMounts:
- name: wordpress-persistent-storage
mountPath: /var/www/html
volumes:
- name: wordpress-persistent-storage
persistentVolumeClaim:
claimName: wp-pv-claim
Sidecar Mode
Akeyless plugin for K8s can work as a sidecar, where an additional container will run side by side to your pod in order to track any changes within your secrets values while reflecting those changes into your running pod.
Note:
Sidecar mode can work only with file injection mode.
To enable Akeyless as a sidecar on your K8s cluster add the following annotation to your deployment file:
akeyless/side_car_enabled: "true"
To set the desired refresh interval for Akeyless sidecar :
akeyless/side_car_refresh_interval: "300s"
Where supported time units are "s", "m", "h".
To retrieve multiple versions of your secret simultaneously (only for static-secret and rotated-secret)
akeyless/side_car_versions_to_retrieve: "2"
The following example demonstrate the Akeyless sidecar annotation:
apiVersion: apps/v1
kind: Deployment
metadata:
name: test-file
spec:
replicas: 1
selector:
matchLabels:
app: file-secrets
template:
metadata:
labels:
app: file-secrets
annotations:
akeyless/enabled: "true"
akeyless/inject_file: "/K8s/my_k8s_secret|location=/secrets/secretsVersion.json"
akeyless/side_car_enabled: "true"
akeyless/side_car_refresh_interval: "30s"
akeyless/side_car_versions_to_retrieve: "2"
spec:
containers:
- name: alpine
image: alpine
command:
- "sh"
- "-c"
- "cat /secrets/secretsVersion.json && echo going to sleep... && sleep 10000"
The following table lists the available annotations:
Annotation | Options | Description |
---|---|---|
akeyless/enabled: "true" | true or false | Enable akeyless K8s plugin. |
akeyless/side_car_enabled: "true" | true or false | Set Akeyless K8s plugin to work in side car mode. |
akeyless/side_car_refresh_interval:"30s" | Supported time units are"s", "m", "h". | Set the desired refresh interval for Akeyless sidecar |
akeyless/side_car_versions_to_retrieve: "2" | '2' or higher | Fetch the last X versions of your secret. |
akeyless/inject_file: "/mysecret/|location=/path/to/save/secret/name" | location= /path/to/save/secret | Set the location for your secrets to be saved within your pod file system. Available for files only. |
akeyless/inject_file: "/mysecret|permission=0644" | permission=0777 | Set the permission of the file that contains your secret value. Default is 0644 .Available for files only |
akeyless/inject_file: "/mysecret|version=1" | version={version of your secret} | Fetch a specific version of your secret. Default value is set to latest version. Available for Environment variables as well. |
akeyless/inject_file: "/mysecret|decode=base64" | decode=base64 | Set default base64 decoding for your secrets values. Default is none Available for Environment variables as well. |
akeyless/inject_file: "/mysecret|jq={jq-expresion}" | jq expression to work with conventional JSON data form | jq={jq-expresion} e.g. secret items which contains JSON structure, can be parsed directly. |
akeyless/inject_folder: "/prod/my-secrets-folder/|location=/tmp/secrets/" | permission=0644 | Set the permission of the folder that contains your secrets value. Default is 0644 .Available for files only |
akeyless/inject_folder: "/prod/my-secrets-folder/|location=/tmp/secrets/|track-folder-changes=true" | track-folder-changes=true or false | Track injected folder changes to sync new secrets |
akeyless/post_inject_script: | #!/bin/bash echo "Hello World" | script to execute post fetching the secret | Note: the execution occurs in the init container or/and at the sidecar container, |
akeyless/agent_limits_cpu | Int followed by m units e.g. 600m" | Limit of CPU usage e.g. 600m where the unit suffix m stands for core thousandth. |
akeyless/agent_limits_mem | Int followed by Mi units e.g. 64Mi | Limit of Memory usage. |
akeyless/agent_requests_cpu | Int followed by m units e.g. 250m" | Limit of CPU request e.g. 250m where the unit suffix m stands for core thousandth. |
akeyless/agent_requests_mem | Int followed by Mi units e.g. 64Mi | Limit of Memory requests usage. |
The following examples demonstrate the usage of Akeyless annotations:
Files:
akeyless/inject_file: "/k8s-secrets/secret1|location=/tmp/s1|permission=0777|decode=base64,/k8s-secrets/secret2,/k8s-secrets/secret3|version=3|location=/tmp/s2"
Environment variable:
- name: MY_SECRET
value: 'akeyless:/k8s-secrets/secret1|decode=base64|version=2'
A secret that contains a JSON structured data for example:
akeyless create-secret --name k8s-secrets/secret-json --value '{"aws_access_key":"1234","aws_key_id":"abcd"}'
Can be fetched and parsed directly within the annotation for example to fetch only the aws_access_key
key\value:
- name: MY_JSON_SECRET
value: 'akeyless:/k8s-secrets/secret-json|jq=.aws_access_key'
Troubleshooting
When you are working with a GKE cluster, make sure that port 8443 is opened in your firewall rules, This port is needed by the Akeyless Secret Injection mutating webhook. Update your firewall rule as follow:
- Review the firewall rule for master access:
gcloud compute firewall-rules list
- Replace the existing rule and allow master access:
gcloud compute firewall-rules update <firewall-rule-name> --allow tcp:10250,tcp:443,tcp:8443
Updated 3 months ago