Kamal secrets and github actions deployment

[ehutzelman]

Now that I've upgraded to kamal 2, I see the secrets no longer have a separate lifecycle and they are pushed on every deployment. While this simplifies the whole process, it does remove some of the flexibility that kamal 1.x provided with the ability to manage secrets independently of the deployment. I would manage the uploading of secrets (env push) from my local machine, and then leave the grunt work of deployment to github actions. This meant that all my app secrets didn't have to be duplicated in github secrets as well, and github only needed the basics like the KAMAL_REGISTRY_PASSWORD and SSH_KEY to do a deploy.

Curious how others are managing their secrets with github actions and kamal, as maybe I'm missing some path that makes this easier with kamal 2. Kamal 2 has been much easier to get up and going, so maybe this is just a tradeoff of making the whole deployment process more simple and bulletproof.

Thanks!

[dhh]

You can use GH Secrets as ENVs, which you can then refer to in .kamal/secrets. See https://docs.github.com/en/actions/security-for-github-actions/security-guides/using-secrets-in-github-actions.

[gaultierq]

I am having the same issue, deploying an app using a CI/CD service fails unless the CI/CD executor has access to the project's secrets. The process breaks during the secrets upload step.

Uploading my secrets to GH isn’t a viable solution for me, as :

1. it would require constant updates
2. my secrets would be duplicated in my un-versioned .kamal/secrets and the CI/CD configuration
3. It somehow violates the principle of least privilege, as most of these secrets are only needed at runtime, not during build time.

I understand it is important to fail fast if some secret are missing, but maybe this check can happen on the host directly ?

My best option is a dedicated secret manager, or a git-crypt approach (it would still not solve 3.).

[Rjevski]

Would it be possible to reopen this? Support for secrets provisioned out-of-band would be great. We can use IaC to provision secrets on the machine in a specific file, and then Kamal can be told to just use that file as its env. This means the CI infrastructure can deploy to that machine without explicitly having to manage those secrets itself.

[dhh]

Feel free to open a PR. This isn't something we need, so it's not something we'd be working on.

[Rjevski]

I came up with a simple workaround. You deploy your secrets in env-file format out-of-band to the target machines using Ansible or whatever configuration management tool of choice, in some location like /etc/configs/production.

Then in your Kamal roles definition, you use a custom Docker option env-file to reference said file:

servers:
  web:
    hosts:
      - appserver1.example.com
    options:
      env-file: /etc/configs/production
  worker:
    hosts:
      - appserver1.example.com
    options:
      env-file: /etc/configs/production

This will be merged with the clear env vars from Kamal's config (I believe the vars from the file will take precedence as it's the last env-related option in the Docker command line). This allows you to have no sensitive data in the Kamal config nor in the repo.

[ehutzelman]

That looks like a good solution, just following up here with what I ended up doing to manage this. Since I have a fairly large app with a lot of secrets (30+) I didn't want to create/manage an individual secret in github for each one. So I ended up base64 encoding my entire secrets file and storing that as a single secret in github. Then using a step in github actions deploy to rewrite this file out to the target:

      - name: Kamal Build secrets
        id: kamal-build-secrets
        run: |
          echo "${{ secrets.STAGING_ENV }}" | base64 -d > .kamal/secrets.staging
          echo "${{ secrets.PRODUCTION_ENV }}" | base64 -d > .kamal/secrets