Age encryption in ArgoCD is not working: the URL scheme 'secrets+age-import' is not allowed

[Michalosu]

Current Behavior

When I'm trying to add an application via ArgoCD UI or via CLI I'm getting error that "File does not exist"

Error log:
[helm-secrets] File does not exist: secrets+age-import:///helm-secrets-private-keys/key.txt?secrets.yaml\\nError: plugin \\\"secrets\\\" exited with error\",\"type\":\"ComparisonError\"}]}}" application=wordpress

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: wordpress
spec:
  destination:
    name: ''
    namespace: wordpress
    server: 'https://kubernetes.default.svc'
  source:
    path: xyz/example-wordpress
    repoURL: '<REPO_URL>'
    targetRevision: init
    helm:
      valueFiles:
        - values.yaml
        - secrets+age-import:///helm-secrets-private-keys/key.txt?secrets.yaml
  project: default

project tree

example-wordpress$ tree .
.
├── Chart.yaml
├── secrets.yaml
└── values.yaml

I double-checked:

• volume with the secret is properly mounted and accessible
• executed helm template . -f secrets+age-import:///helm-secrets-private-keys/key.txty?secrets.yaml locally in repo-server pod and it works as expected, decrypted values are correct
• tried the approach with k8s secret for getting age key

Expected Behavior

When I add the application in ArgoCD with values that are encrypted using age, helm secrets should decrypt it and install app with decrypted values

Steps To Reproduce

No response

Environment

• Helm Version: 3.6.0
• Helm Secrets Version: 3.12.0
• OS: Ubuntu 21.04
• Shell: bash, version 5.1.4(1)-release

Anything else?

No response

I had the exact same issue while trying with gpg and a custom argo docker image that installs secrets plugin and uses a wrapper script around helm. I tried the "install the tools via init containers" approach today and I'm getting a different error:
rpc error: code = Unknown desc = the URL scheme 'secrets+gpg-import' is not allowed

Same as @Michalosu:

• volume with the secret is properly mounted and accessible
• executed helm template . -f secrets+age-import:///helm-secrets-private-keys/key.txty?secrets.yaml locally in repo-server pod and it works as expected, decrypted values are correct
• tried the approach with k8s secret for getting gpg key

[Michalosu]

Yeaah, I forgot to mention that I use custom docker image with wrapper for helm as well. Any ideas why it is working locally but it doesn’t work via ArgoCD?

[jkroepke]

I use custom docker image with wrapper

which wrapper?

In case, it's a wrapper which calls helm secrets internally instead helm. Please remove it.

The tutorials which the wrapper script are outdated.

See https://github.com/jkroepke/helm-secrets/wiki/ArgoCD-Integration#option-1-custom-docker-image for a updated tutorial

[Enrico204]

I have the same issue. I'm using age, and the error is rpc error: code = Unknown desc = the URL scheme 'secrets+age-import' is not allowed

This is my App configuration:

kind: Application
apiVersion: argoproj.io/v1alpha1
metadata:
  name: mariadb
  namespace: argocd
spec:
  project: default
  source:
    path: helm/mariadb/
    repoURL: 'git@gitlab.com:repo'
    targetRevision: HEAD
    helm:
      version: v3
      releaseName: mariadb
      valueFiles:
        - "secrets+age-import://argocd-age-key/keys.txt?secrets.yml"
  destination:
    namespace: default
    server: 'https://kubernetes.default.svc'

I created the custom Docker image as described in your link. No wrapper. ArgoCD is the latest version (2.2.4)

[Enrico204]

Ok apparently this check was introduced in the latest version due to CVE-2022-24348.

As you can see in commit argoproj/argo-cd@78c2084 the allowedHelmRemoteProtocols variable was added with allowed "remote" protocols. I'll open an issue in the argo-cd repo

[jkroepke]

rpc error: code = Unknown desc = the URL scheme 'secrets+age-import' is not allowed

Ok apparently this check was introduced in the latest version due to CVE-2022-24348.

That was my second intension. its something new inside ArgoCD, because I never know this error message from helm-secrets or helm itself.

[jkroepke]

The only workaround that I know it not upgrading ArgoCD to <2.1.9 or <2.2.4

[Michalosu]

which wrapper?

In case, it's a wrapper which calls helm secrets internally instead helm. Please remove it.

The tutorials which the wrapper script are outdated.

See https://github.com/jkroepke/helm-secrets/wiki/ArgoCD-Integration#option-1-custom-docker-image for a updated tutorial

I made several tests:

• my custom image without helm wrapper
• your example Dockerfile with ArgoCD 2.1.8 (to avoid side-effects of fix for [CVE-2022-24348] https://www.tenable.com/cve/CVE-2022-24348) + I had to set user with id 999 -> without that I got error:
  Warning Failed 9s (x4 over 25s) kubelet Error: container has runAsNonRoot and image has non-numeric user (argocd), cannot verify user is non-root

My final Dockerfile:

ARG ARGOCD_VERSION="v2.1.8"
FROM argoproj/argocd:$ARGOCD_VERSION
ARG SOPS_VERSION="3.7.1"
ARG HELM_SECRETS_VERSION="3.12.0"
ARG KUBECTL_VERSION="1.22.0"

ENV HELM_SECRETS_HELM_PATH=/usr/local/bin/helm
ENV HELM_PLUGINS="/home/argocd/.local/share/helm/plugins/"

USER root
RUN apt-get update && \
    apt-get install -y \
      curl && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
RUN curl -fSSL https://github.com/mozilla/sops/releases/download/v${SOPS_VERSION}/sops-v${SOPS_VERSION}.linux \
    -o /usr/local/bin/sops && chmod +x /usr/local/bin/sops
RUN curl -fSSL https://dl.k8s.io/release/v${KUBECTL_VERSION}/bin/linux/amd64/kubectl \
    -o /usr/local/bin/kubectl && chmod +x /usr/local/bin/kubectl

COPY uid_entrypoint.sh /usr/local/bin/uid_entrypoint.sh

USER 999

RUN helm plugin install --version ${HELM_SECRETS_VERSION} https://github.com/jkroepke/helm-secrets

I'm getting such error:

failed exit status 2: Failed to get the data key required to decrypt the SOPS file.\\n\\nGroup 0: FAILED\\n  age1uc47npt8nv9hsrqsyqq8lg56mm7cwe2sqv6l9qup6hruygghnp3qe3e0lr: FAILED\\n    - | failed to open file: open\\n      | /helm-secrets-private-keys/key.txt: no such file or\\n      | directory\\n\\nRecovery failed because no master key was able to decrypt the file. In\\norder for SOPS to recover the file, at least one key has to be successful,\\nbut none were.

File exists in argocd server pod:

lrwxrwxrwx. 1 root root 14 Feb  8 11:14 /helm-secrets-private-keys/key.txt -> ..data/key.txt
        
argocd@argocd-repo-server-5747f6bf4d-mrr56:~$ ls -la /helm-secrets-private-keys/..data/key.txt 
-rw-r--r--. 1 root daemon 189 Feb  8 11:14 /helm-secrets-private-keys/..data/key.txt

argocd@argocd-repo-server-5747f6bf4d-mrr56:~$ grep argocd /etc/passwd
argocd:x:999:999::/home/argocd:/bin/sh

Do you have any other hints about what I'm doing wrong? Tried to be as close as I can with your official delivery

[jkroepke]

Can you run helm template -f secrets+age-import://argocd-age-key/keys.txt?secrets.yml inside container?

Can you verify through cat /helm-secrets-private-keys/key.txt if the key is correct?

[Michalosu]

Can you run helm template -f secrets+age-import://argocd-age-key/keys.txt?secrets.yml inside container?

Can you verify through cat /helm-secrets-private-keys/key.txt if the key is correct?

helm template -f secrets+age-import:///helm-secrets-private-keys/key.txt?secrets.yaml . this command works locally

the key is correct because the output of helm template contains a correct decrypted value. This sounds unbelievable but that's my case. I'm pretty sure that there is some stupid mistake/typo on my side. Is there some way to inject some echo prints/ ls command before argocd+helm secrets+sops are trying to decrypt it?

[jkroepke]

You can enable the shell builtin debug which prints every command.

HELM_SECRETS_DEBUG=true

helm-secrets/scripts/run.sh

Lines 5 to 7 in 4ae8e5f

	if [ -n "${HELM_SECRETS_DEBUG+x}" ]; then
	set -x
	fi

You can also use HELM_SECRETS_DRIVER_ARGS=--verbose. In this case, --verbose is passing to sops.

You can enable the shell builtin debug which prints every command.

HELM_SECRETS_DEBUG=true

helm-secrets/scripts/run.sh

Lines 5 to 7 in 4ae8e5f

	if [ -n "${HELM_SECRETS_DEBUG+x}" ]; then
	set -x
	fi

You can also use HELM_SECRETS_DRIVER_ARGS=--verbose. In this case, --verbose is passing to sops.

Just to be clear:
HELM_SECRETS_DRIVER_ARGS and HELM_SECRETS_DRIVER_ARGS are environment variables in the repo server pod, correct?