Use Event Notifications to Update VaultDynamicSecrets Instantly

[jaireddjawed]

Description

This PR updates VaultDynamicSecrets in VSO when the corresponding secret in Vault has a relevant event occur on it.

Manual Testing

This PR was manually validated against the Azure Secrets Enterprise engine using both static and dynamic role configurations. Although event notifications have not yet been implemented across all secrets engines, the changes in this PR are forward-compatible: each plugin exposes the same generic event types (creds-create, static-creds-create, rotate). Once the remaining plugins add support for these notifications, VSO builds that include this PR will automatically work with them without requiring further modifications.

Setup Azure Configuration and K8s Authentication with Vault

#!/usr/bin/env bash
# Copyright (c) HashiCorp, Inc.
# SPDX-License-Identifier: BUSL-1.1

make setup-kind
kind load docker-image --name vault-secrets-operator vault-enterprise:dev
make setup-integration-test

set -e

cat <<EOF | kubectl -n vault exec -i vault-0 -- sh -e
vault secrets disable azure
vault secrets enable azure

vault write azure/config \
    subscription_id=$AZURE_SUBSCRIPTION_ID \
    tenant_id=$AZURE_TENANT_ID \
    client_id=$AZURE_CLIENT_ID \
    client_secret=$AZURE_CLIENT_SECRET

vault write azure/roles/my-role \
    application_object_id=$AZURE_APP_OBJECT_ID

vault write azure/static-roles/my-static-role \
    application_object_id=$AZURE_APP_OBJECT_ID

cat <<EOT > /tmp/policy.hcl
path "azure/data/*" {
   capabilities = ["read", "list", "subscribe"]
   subscribe_event_types = ["azure*"]
}
path "sys/events/subscribe/*" {
    capabilities = ["read"]
}
EOT
vault policy write demo /tmp/policy.hcl

# setup the necessary auth backend
vault auth disable kubernetes
vault auth enable kubernetes
vault write auth/kubernetes/config \
    kubernetes_host=https://kubernetes.default.svc
vault write auth/kubernetes/role/demo \
    bound_service_account_names=default \
    bound_service_account_namespaces=tenant-1,tenant-2 \
    policies=demo \
    ttl=1h
EOF

for ns in tenant-{1,2} ; do
    kubectl delete namespace --wait --timeout=30s "${ns}" &> /dev/null || true
    kubectl create namespace "${ns}"
done

make build docker-build deploy-kind

kubectl apply -f config/samples/secrets_v1beta1_vaultconnection.yaml
kubectl apply -f config/samples/secrets_v1beta1_vaultauth.yaml
kubectl apply -f config/samples/secrets_v1beta1_vaultdynamicsecret.yaml

Create VaultDynamicSecret CRD for static and dynamic secret

# Copyright (c) HashiCorp, Inc.
# SPDX-License-Identifier: BUSL-1.1

apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultDynamicSecret
metadata:
  namespace: tenant-1
  labels:
    app.kubernetes.io/name: vaultdynamicsecret
    app.kubernetes.io/instance: vaultdynamicsecret-sample
    app.kubernetes.io/part-of: vault-secrets-operator
    app.kubernetes.io/managed-by: kustomize
    app.kubernetes.io/created-by: vault-secrets-operator
  name: vaultdynamicsecret-sample
spec:
  # TODO(user): Add fields here
  vaultAuthRef: vaultauth-sample
  mount: azure
  path: creds/my-role
  refreshAfter: 10m
  destination:
    name: my-role-secret
    create: true
  syncConfig:
    instantUpdates: true
---
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultDynamicSecret
metadata:
  namespace: tenant-1
  labels:
    app.kubernetes.io/name: vaultdynamicsecret
    app.kubernetes.io/instance: vaultdynamicsecret-sample
    app.kubernetes.io/part-of: vault-secrets-operator
    app.kubernetes.io/managed-by: kustomize
    app.kubernetes.io/created-by: vault-secrets-operator
  name: vaultdynamicsecret-sample
spec:
  vaultAuthRef: vaultauth-sample
  mount: azure
  path: static-creds/my-static-role
  refreshAfter: 10m
  destination:
    name: my-static-role-secret
    create: true
  syncConfig:
    instantUpdates: true

Get Original Credential details for dynamic role

{
    "apiVersion": "v1",
    "data": {
        "_raw": "eyJjbGllbnRfaWQiOiI1YjgyMWEzZS0zOThmLTRiMzctYjE3My00NTYwMmQyN2QzOTAiLCJjbGllbnRfc2VjcmV0IjoiMzdVOFF+ejlYaXlhYTZ+Y18xblM2SnhFM2J4d2t2MDBhalFfVGI3USJ9",
        "client_id": "NWI4MjFhM2UtMzk4Zi00YjM3LWIxNzMtNDU2MDJkMjdkMzkw",
        "client_secret": "client-secret"
    },
    "kind": "Secret",
    "metadata": {
        "creationTimestamp": "2025-12-05T08:37:25Z",
        "labels": {
            "app.kubernetes.io/component": "secret-sync",
            "app.kubernetes.io/managed-by": "hashicorp-vso",
            "app.kubernetes.io/name": "vault-secrets-operator",
            "secrets.hashicorp.com/vso-ownerRefUID": "967b4e84-f767-4d57-babb-6810902606e1"
        },
        "name": "my-role-secret",
        "namespace": "tenant-1",
        "ownerReferences": [
            {
                "apiVersion": "secrets.hashicorp.com/v1beta1",
                "kind": "VaultDynamicSecret",
                "name": "vaultdynamicsecret-sample",
                "uid": "967b4e84-f767-4d57-babb-6810902606e1"
            }
        ],
        "resourceVersion": "992",
        "uid": "1868c8b0-d32b-4f18-8cc7-55967faebb2f"
    },
    "type": "Opaque"
}

Confirmed that client secret was different after new lease generated

{
    "apiVersion": "v1",
    "data": {
        "_raw": "eyJjbGllbnRfaWQiOiI1YjgyMWEzZS0zOThmLTRiMzctYjE3My00NTYwMmQyN2QzOTAiLCJjbGllbnRfc2VjcmV0IjoiNnZJOFF+b3J3b2ZWRXNzZVI4VXlRU3ZjNE93VlFldmJ0WENVQWN+NyJ9",
        "client_id": "NWI4MjFhM2UtMzk4Zi00YjM3LWIxNzMtNDU2MDJkMjdkMzkw",
        "client_secret": "client_secret_different"
    },
    "kind": "Secret",
    "metadata": {
        "creationTimestamp": "2025-12-05T08:37:25Z",
        "labels": {
            "app.kubernetes.io/component": "secret-sync",
            "app.kubernetes.io/managed-by": "hashicorp-vso",
            "app.kubernetes.io/name": "vault-secrets-operator",
            "secrets.hashicorp.com/vso-ownerRefUID": "967b4e84-f767-4d57-babb-6810902606e1"
        },
        "name": "my-role-secret",
        "namespace": "tenant-1",
        "ownerReferences": [
            {
                "apiVersion": "secrets.hashicorp.com/v1beta1",
                "kind": "VaultDynamicSecret",
                "name": "vaultdynamicsecret-sample",
                "uid": "967b4e84-f767-4d57-babb-6810902606e1"
            }
        ],
        "resourceVersion": "1344",
        "uid": "1868c8b0-d32b-4f18-8cc7-55967faebb2f"
    },
    "type": "Opaque"
}

Setup original dyanmic secret for static role and see original creds

kubectl get secret my-static-role-secret -n tenant-1 -o json
{
    "apiVersion": "v1",
    "data": {
        "client_id": "NWI4MjFhM2UtMzk4Zi00YjM3LWIxNzMtNDU2MDJkMjdkMzkw",
        "client_secret": "client_secret",
        "expiration": "MjAyNy0xMi0wNVQwODozNDoyNy4xNDY1NzEyMjJa",
        "last_vault_rotation": "MjAyNS0xMi0wNVQwODozNDoyNy4xNDY1NzEyMjJa",
        "metadata": "e30=",
        "secret_id": "OTNhM2Y5NmMtMjdjZC00MjA2LTk0YmUtZDc1YTU3NmQ4N2Zk"
    },
    "kind": "Secret",
    "metadata": {
        "creationTimestamp": "2025-12-05T08:44:41Z",
        "labels": {
            "app.kubernetes.io/component": "secret-sync",
            "app.kubernetes.io/managed-by": "hashicorp-vso",
            "app.kubernetes.io/name": "vault-secrets-operator",
            "secrets.hashicorp.com/vso-ownerRefUID": "967b4e84-f767-4d57-babb-6810902606e1"
        },
        "name": "my-static-role-secret",
        "namespace": "tenant-1",
        "ownerReferences": [
            {
                "apiVersion": "secrets.hashicorp.com/v1beta1",
                "kind": "VaultDynamicSecret",
                "name": "vaultdynamicsecret-sample",
                "uid": "967b4e84-f767-4d57-babb-6810902606e1"
            }
        ],
        "resourceVersion": "1757",
        "uid": "e7d6f14f-9cbf-44f6-bf4a-5c133498227f"
    },
    "type": "Opaque"
}

Confirmed that the client secret was updated on an instant basis after rotating the secret

{
    "apiVersion": "v1",
    "data": {
        "client_id": "NWI4MjFhM2UtMzk4Zi00YjM3LWIxNzMtNDU2MDJkMjdkMzkw",
        "client_secret": "client_secret_new",
        "expiration": "MjAyNy0xMi0wNVQwODo1MDo0My4yMjY3OTkyNTRa",
        "last_vault_rotation": "MjAyNS0xMi0wNVQwODo1MDo0My4yMjY3OTkyNTRa",
        "metadata": "e30=",
        "secret_id": "MGJmYmMzY2UtZTEyOS00MGVjLTlmMTUtNDUwNDBhZjNjODJk"
    },
    "kind": "Secret",
    "metadata": {
        "creationTimestamp": "2025-12-05T08:44:41Z",
        "labels": {
            "app.kubernetes.io/component": "secret-sync",
            "app.kubernetes.io/managed-by": "hashicorp-vso",
            "app.kubernetes.io/name": "vault-secrets-operator",
            "secrets.hashicorp.com/vso-ownerRefUID": "967b4e84-f767-4d57-babb-6810902606e1"
        },
        "name": "my-static-role-secret",
        "namespace": "tenant-1",
        "ownerReferences": [
            {
                "apiVersion": "secrets.hashicorp.com/v1beta1",
                "kind": "VaultDynamicSecret",
                "name": "vaultdynamicsecret-sample",
                "uid": "967b4e84-f767-4d57-babb-6810902606e1"
            }
        ],
        "resourceVersion": "2494",
        "uid": "e7d6f14f-9cbf-44f6-bf4a-5c133498227f"
    },
    "type": "Opaque"
}

[digivava]

I see you have these unit tests which is also helpful, but can you also add an integration test for the event-watching functionality like you have in your other PR?

[digivava]

Specifically, I think it'd be good to have an integration test that sets up the database and LDAP secrets engine and validates that events from those specific paths trigger the instant updates that we're expecting.