The problem with secrets in GitOps
GitOps mandates that the desired state of your cluster lives in Git. That's great for manifests, Helm values, and configuration — but it creates an immediate conflict with secrets. You cannot commit plaintext secrets to Git. You shouldn't commit encrypted secrets either, if you can avoid it: key rotation becomes painful, and you now have a second encryption system to manage.
The cleaner pattern: store references to secrets in Git, and let Vault serve the actual values at sync time. ArgoCD doesn't support this natively, but the argocd-vault-plugin (AVP) makes it work transparently.
Architecture overview
The full pipeline looks like this:
Git repo (Helm values + AVP annotations)
↓
ArgoCD Application Controller
↓
argocd-vault-plugin (sidecar)
↓ fetches secrets at render time
HashiCorp Vault (KV v2)
↓
Rendered manifests → applied to cluster
AVP runs as a sidecar in the ArgoCD repo-server Pod. When ArgoCD renders a Helm chart, AVP intercepts the output, finds placeholder annotations like <path:secret/data/myapp#password>, fetches the real values from Vault, and substitutes them before applying to the cluster.
Installing ArgoCD with AVP
Step 1: base ArgoCD installation
kubectl create namespace argocd kubectl apply -n argocd -f \ https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml
Wait for all pods to be running, then patch the repo-server to add the AVP sidecar.
Step 2: configure Vault authentication
AVP authenticates to Vault using Kubernetes Service Account tokens. Enable the Kubernetes auth method in Vault:
# On the Vault server
vault auth enable kubernetes
vault write auth/kubernetes/config \
kubernetes_host="https://$(kubectl get svc kubernetes -o jsonpath='{.spec.clusterIP}'):443" \
kubernetes_ca_cert=@/var/run/secrets/kubernetes.io/serviceaccount/ca.crt \
token_reviewer_jwt="$(kubectl create token argocd-repo-server -n argocd)"
# Create a policy for ArgoCD to read secrets
vault policy write argocd-policy - <
Step 3: patch the repo-server
kubectl patch deployment argocd-repo-server -n argocd --type=json -p='[
{
"op": "add",
"path": "/spec/template/spec/initContainers/-",
"value": {
"name": "download-tools",
"image": "alpine:3.18",
"command": ["sh", "-c"],
"args": ["wget -qO /custom-tools/argocd-vault-plugin https://github.com/argoproj-labs/argocd-vault-plugin/releases/download/v1.17.0/argocd-vault-plugin_1.17.0_linux_amd64 && chmod +x /custom-tools/argocd-vault-plugin"],
"volumeMounts": [{"name": "custom-tools", "mountPath": "/custom-tools"}]
}
}
]'
Writing Helm values with AVP placeholders
In your Helm values.yaml, replace secret values with AVP path annotations:
# values.yaml — committed to Git, safe to read
database:
host: postgres.internal
port: 5432
name: myapp_production
# AVP will replace these at sync time
username: <path:secret/data/myapp/database#username>
password: <path:secret/data/myapp/database#password>
redis:
url: <path:secret/data/myapp/redis#connection_string>
jwt:
secret: <path:secret/data/myapp/auth#jwt_secret>
⚠ Important
AVP placeholders only work in files that AVP processes — Helm values that get templated into manifests, or raw YAML manifests. They do not work in ConfigMap data that isn't templated through Helm.
Creating the ArgoCD Application
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: myapp
namespace: argocd
spec:
project: default
source:
repoURL: https://github.com/your-org/your-repo
targetRevision: HEAD
path: helm/myapp
plugin:
name: argocd-vault-plugin-helm
env:
- name: HELM_ARGS
value: "-f values.yaml -f values-production.yaml"
destination:
server: https://kubernetes.default.svc
namespace: myapp
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true
On each sync, ArgoCD renders the Helm chart, AVP fetches the current secret values from Vault, and the resulting manifests are applied. If a secret rotates in Vault, the next sync picks it up automatically — no manual intervention, no re-committing encrypted values.
Secret rotation without downtime
The real payoff: when you rotate a database password in Vault, trigger an ArgoCD sync, and the new Secret is applied. Pair this with a rolling restart annotation to pick up the new value:
kubectl rollout restart deployment/myapp -n myapp
With the zero-downtime rolling update strategy from my previous post, users never see an interruption.