Manage Kubernetes secrets with SOPS

Manage Kubernetes secrets with SOPS, OpenPGP, Age and Cloud KMS.

In order to store secrets safely in a public or private Git repository, you can use SOPS CLI to encrypt Kubernetes secrets with OpenPGP, AWS KMS, GCP KMS and Azure Key Vault.


To follow this guide you’ll need a Kubernetes cluster with the GitOps toolkit controllers installed on it. Please see the get started guide or the installation guide.

Install gnupg and SOPS:

brew install gnupg sops

Generate a GPG key

Generate a GPG/OpenPGP key with no passphrase (%no-protection):

export KEY_NAME=""
export KEY_COMMENT="flux secrets"

gpg --batch --full-generate-key <<EOF
Key-Type: 1
Key-Length: 4096
Subkey-Type: 1
Subkey-Length: 4096
Expire-Date: 0
Name-Comment: ${KEY_COMMENT}
Name-Real: ${KEY_NAME}

The above configuration creates an rsa4096 key that does not expire. For a full list of options to consider for your environment, see Unattended GPG key generation.

Retrieve the GPG key fingerprint (second row of the sec column):

gpg --list-secret-keys "${KEY_NAME}"

sec   rsa4096 2020-09-06 [SC]

Store the key fingerprint as an environment variable:

export KEY_FP=1F3D1CED2F865F5E59CA564553241F147E7C5FA4

Export the public and private keypair from your local GPG keyring and create a Kubernetes secret named sops-gpg in the flux-system namespace:

gpg --export-secret-keys --armor "${KEY_FP}" |
kubectl create secret generic sops-gpg \
--namespace=flux-system \

It’s a good idea to back up this secret-key/K8s-Secret with a password manager or offline storage. Also consider deleting the secret decryption key from your machine:

gpg --delete-secret-keys "${KEY_FP}"

Configure in-cluster secrets decryption

Register the Git repository on your cluster:

flux create source git my-secrets \
--url= \

Create a kustomization for reconciling the secrets on the cluster:

flux create kustomization my-secrets \
--source=my-secrets \
--path=./clusters/cluster0 \
--prune=true \
--interval=10m \
--decryption-provider=sops \

Note that the sops-gpg can contain more than one key, SOPS will try to decrypt the secrets by iterating over all the private keys until it finds one that works.

Optional: Export the public key into the Git directory

Commit the public key to the repository so that team members who clone the repo can encrypt new files:

gpg --export --armor "${KEY_FP}" > ./clusters/cluster0/

Check the file contents to ensure it’s the public key before adding it to the repo and committing.

git add ./clusters/cluster0/
git commit -am 'Share GPG public key for secrets generation'

Team members can then import this key when they pull the Git repository:

gpg --import ./clusters/cluster0/

Configure the Git directory for encryption

Write a SOPS config file to the specific cluster or namespace directory used to store encrypted objects with this particular GPG key’s fingerprint.

cat <<EOF > ./clusters/cluster0/.sops.yaml
  - path_regex: .*.yaml
    encrypted_regex: ^(data|stringData)$
    pgp: ${KEY_FP}

This config applies recursively to all sub-directories. Multiple directories can use separate SOPS configs. Contributors using the sops CLI to create and encrypt files won’t have to worry about specifying the proper key for the target cluster or namespace.

encrypted_regex helps encrypt the data and stringData fields for Secrets. You may wish to add other fields if you are encrypting other types of Objects.

Encrypting secrets using OpenPGP

Generate a Kubernetes secret manifest with kubectl:

kubectl -n default create secret generic basic-auth \
--from-literal=user=admin \
--from-literal=password=change-me \
--dry-run=client \
-o yaml > basic-auth.yaml

Encrypt the secret with SOPS using your GPG key:

sops --encrypt --in-place basic-auth.yaml

You can now commit the encrypted secret to your Git repository.

Encrypting secrets using age

age is a simple, modern alternative to OpenPGP. It’s recommended to use age over OpenPGP, if possible.

Encrypting with age follows the same workflow than PGP.

Generate an age key with age using age-keygen:

$ age-keygen -o age.agekey
Public key: age1helqcqsh9464r8chnwc2fzj8uv7vr5ntnsft0tn45v2xtz0hpfwq98cmsg

Create a secret with the age private key, the key name must end with .agekey to be detected as an age key:

cat age.agekey |
kubectl create secret generic sops-age \
--namespace=flux-system \

Use sops and the age public key to encrypt a Kubernetes secret:

sops --age=age1helqcqsh9464r8chnwc2fzj8uv7vr5ntnsft0tn45v2xtz0hpfwq98cmsg \
--encrypt --encrypted-regex '^(data|stringData)$' --in-place basic-auth.yaml

And finally set the decryption secret in the Flux Kustomization to sops-age.

Encrypting secrets using HashiCorp Vault

HashiCorp Vault is an identity-based secrets and encryption management system.

Encrypting with HashiCorp Vault follows the same workflow as PGP & Age.

Export the VAULT_ADDR and VAULT_TOKEN environment variables to your shell, then use sops to encrypt a Kubernetes Secret (see HashiCorp Vault for more details on enabling the transit backend and sops).

Then use sops to encrypt a Kubernetes Secret:

export VAULT_ADDR=
export VAULT_TOKEN=my-token
sops --hc-vault-transit $VAULT_ADDR/v1/sops/keys/my-encryption-key --encrypt \
--encrypted-regex '^(data|stringData)$' --in-place basic-auth.yaml

Create a secret the vault token, the key name must be sops.vault-token to be detected as a vault token:

kubectl create secret generic sops-hcvault \
--namespace=flux-system \

And finally set the decryption secret in the Flux Kustomization to sops-hcvault.

Encrypting secrets using various cloud providers

When using AWS/GCP KMS, you don’t have to include the gpg secretRef under spec.provider (you can skip the --decryption-secret flag when running flux create kustomization), instead you’ll have to bind an IAM Role with access to the KMS keys to the kustomize-controller service account of the flux-system namespace for kustomize-controller to be able to fetch keys from KMS.


Enabled the IAM OIDC provider on your EKS cluster:

eksctl utils associate-iam-oidc-provider --cluster=<clusterName>

Create an IAM Role with access to AWS KMS e.g.:

    "Version": "2012-10-17",
    "Statement": [
            "Action": [
            "Effect": "Allow",
            "Resource": "arn:aws:kms:eu-west-1:XXXXX209540:key/4f581f5b-7f78-45e9-a543-83a7022e8105"

Bind the IAM role to the kustomize-controller service account:

eksctl create iamserviceaccount \
--role-only \
--name=kustomize-controller \
--namespace=flux-system \
--attach-policy-arn=<policyARN> \

Annotate the kustomize-controller service account with the role ARN:

kubectl -n flux-system annotate serviceaccount kustomize-controller \
--field-manager=flux-client-side-apply \'arn:aws:iam::<ACCOUNT_ID>:role/<KMS-ROLE-NAME>'

Restart kustomize-controller for the binding to take effect:

kubectl -n flux-system rollout restart deployment/kustomize-controller


Workload Identity has to be enabled on the cluster. These are the steps to setup the identity, patch kustomize-controller to authenticate with the federated identity setup with Azure key vault.

Setup the identity:

export IDENTITY_NAME="sops-akv-decryptor"
export FEDERATED_IDENTITY_NAME="sops-akv-decryptor-federated"

# Get the OIDC Issuer URL
export AKS_OIDC_ISSUER="$(az aks show -n ${CLUSTER_NAME} -g ${RESOURCE_GROUP} --query "oidcIssuerProfile.issuerUrl" -otsv)"

# Create the managed identity
az identity create --name "${IDENTITY_NAME}" --resource-group "${RESOURCE_GROUP}"

# Get identity client ID
export USER_ASSIGNED_CLIENT_ID="$(az identity show --resource-group ${RESOURCE_GROUP} --name ${IDENTITY_NAME} --query 'clientId' -o tsv)"

# Federate the identity with the kustomize controller sa in flux-system ns
az identity federated-credential create \
--identity-name "${IDENTITY_NAME}" \
--resource-group "${RESOURCE_GROUP}" \
--issuer "${AKS_OIDC_ISSUER}" \
--subject system:serviceaccount:flux-system:kustomize-controller \
--audience api://AzureADTokenExchange

Create the Azure Key-Vault and give the required permissions to the managed identity. The key id in the last step is used to encrypt secrets with sops client.

export VAULT_NAME="fluxcd-$(uuidgen | tr -d - | head -c 16)"
export KEY_NAME="sops-cluster0"

az keyvault create --name "${VAULT_NAME}" --resource-group "${RESOURCE_GROUP}" --location "${LOCATION}"

az keyvault key create --name "${KEY_NAME}" \
  --vault-name "${VAULT_NAME}" \
  --protection software \
  --ops encrypt decrypt

az keyvault set-policy --name "${VAULT_NAME}" \
 --key-permissions decrypt

az keyvault key show --name "${KEY_NAME}" \
  --vault-name "${VAULT_NAME}" \
  --query key.kid

Setup kustomize-controller to use workload identity adding the following patches to the flux-system kustomization.yaml

kind: Kustomization
  - gotk-components.yaml
  - gotk-sync.yaml
  - patch: |
      apiVersion: v1
      kind: ServiceAccount
        name: controller
          azure.workload.identity/client-id: <AZURE CLIENT ID>
          azure.workload.identity/tenant-id: <AZURE TENANT ID>      
      kind: ServiceAccount
      name: "(kustomize-controller)"
  - patch: |
      apiVersion: apps/v1
      kind: Deployment
        name: controller
          azure.workload.identity/use: "true"
              azure.workload.identity/use: "true"          
      kind: Deployment
      name: "(kustomize-controller)"

At this point, kustomize-controller is now authorized to decrypt values in SOPS encrypted files from your Sources via the related Key Vault.

See the SOPS guide to Encrypting Using Azure Key Vault to get started committing encrypted files to your Git Repository or other Sources.

Google Cloud

Workload Identity has to be enabled on the cluster and on the node pools.

  1. Create a GCP service account with the role Cloud KMS CryptoKey Encrypter/Decrypter.
gcloud iam service-accounts create <SERVICE_ACCOUNT_ID> \
    --description="DESCRIPTION" \
gcloud projects add-iam-policy-binding <PROJECT_ID> \
    --member="serviceAccount:<SERVICE_ACCOUNT_ID>@<PROJECT_ID>" \
  1. Create an IAM policy binding between the GCP service account and the kustomize-controller Kubernetes service account of the flux-system.
gcloud iam service-accounts add-iam-policy-binding \
  --role roles/iam.workloadIdentityUser \
  --member "serviceAccount:<PROJECT_ID>[<K8S_NAMESPACE>/<KSA_NAME>]" \

For a GCP project named total-mayhem-123456 with a configured GCP service account flux-gcp and assuming that Flux runs in the (default) namespace flux-system, this would translate to the following:

gcloud iam service-accounts add-iam-policy-binding \
  --role roles/iam.workloadIdentityUser \
  --member "[flux-system/kustomize-controller]" \
  1. Customize your Flux Manifests and patch the kustomize-controller service account with the proper annotation so that Workload Identity knows the relationship between the gcp service account and the k8s service account.
### add this patch to annotate service account if you are using Workload identity
  - patch: |
      apiVersion: v1
      kind: ServiceAccount
        name: kustomize-controller
        namespace: flux-system
      kind: ServiceAccount
      name: kustomize-controller

If you didn’t bootstrap Flux, you can use this instead

kubectl annotate serviceaccount kustomize-controller \
--field-manager=flux-client-side-apply \
--namespace flux-system \<SERVICE_ACCOUNT_ID>@<PROJECT_ID>

GitOps workflow

A cluster admin should create the Kubernetes secret with the PGP keys on each cluster and add the GitRepository/Kustomization manifests to the fleet repository.

Git repository manifest:

kind: GitRepository
  name: my-secrets
  namespace: flux-system
  interval: 1m

Kustomization manifest:

kind: Kustomization
  name: my-secrets
  namespace: flux-system
  interval: 10m0s
    kind: GitRepository
    name: my-secrets
  path: ./
  prune: true
    provider: sops
      name: sops-gpg

Assuming a team member wants to deploy an application that needs to connect to a database using a username and password, they’ll be doing the following:

  • create a Kubernetes Secret manifest locally with the db credentials e.g. db-auth.yaml
  • encrypt the secret data field with sops
  • create a Kubernetes Deployment manifest for the app e.g. app-deployment.yaml
  • add the Secret to the Deployment manifest as a volume mount or env var
  • commit the manifests db-auth.yaml and app-deployment.yaml to a Git repository that’s being synced by the GitOps toolkit controllers

Once the manifests have been pushed to the Git repository, the following happens:

  • source-controller pulls the changes from Git
  • kustomize-controller loads the GPG keys from the sops-pgp secret
  • kustomize-controller decrypts the Kubernetes secrets with SOPS and applies them on the cluster
  • kubelet creates the pods and mounts the secret as a volume or env variable inside the app container
Last modified 2024-05-02: Remove references to Mozilla (b93c03b)