container on docker swarm - HOW TO setup TLS correctly

[zihe2-yps]

Hello grindsa,
how is it going?

I tried to set up your image :0.41-nginx-wsgi on our onprem docker swarm cluster and got stuck here:
I wanna put the cert and private key into another folder, because having it under volume and mounted on a CIFS will be accessible for to many people. I changed the paths under '/etc/nginx/sites-available/acme_srv_ssl.conf' and _'/var/www/acme2certifier/examples/nginx/nginx_acme_srv_ssl.conf', but it doesn't work. what am I missing?

Here is also my compose-file:
acme:
image: nexus.corp.itsroot.biz:9880/grindsa/acme2certifier:0.41-nginx-wsgi
ports:
- "22280:80"
- "22443:443"
environment:
- ENV_HOST=${PKI_HOST}
- ENV_USER=${PKI_USER}
- ENV_PASSWORD=${PKI_PASSWORD}
secrets:
- source: autocertcity_pki_chain.pem
target: /etc/ssl/certs/ca_bundle.pem
- source: autocertcity_cert.pem
target: /etc/ssl/certs/acme2certifier_cert.pem
- source: autocertcity_key.pem
target: /etc/ssl/private/acme2certifier_key.pem
volumes:
- type: volume
source: main_volume
target: /var/www/acme2certifier/volume
- type: volume
source: nginx_volume
target: /var/www/acme2certifier/examples/nginx
- type: volume
source: sites_volume
target: /etc/nginx/sites-available
deploy:
mode: global
placement:
constraints:
- node.role == worker

and my acme_srv.cfg:
[DEFAULT]
debug: True

[Nonce]
nonce_check_disable: False

[CAhandler]
handler_file: /var/www/acme2certifier/examples/ca_handler/mscertsrv_ca_handler.py
host_variable: ENV_HOST
user_variable: ENV_USER
password_variable: ENV_PASSWORD
ca_bundle: /etc/ssl/certs/ca_bundle.pem
verify: true
auth_method: "ntlm"
template: "..."

[DBhandler]

[Certificate]
revocation_reason_check_disable: False

[Challenge]
challenge_validation_disable: False

[Order]
tnauthlist_support: False
allowed_domainlist: ["..."]

Thanks for your help and have a good one.

Kind regards

[grindsa]

Hi,

The container checks for the presence of both the certificate and key file in /var/www/acme2certifier/volume and enables TLS only if both exist. You can see the relevant excerpt in the entrypoint-script lines 10-17.

I’ll think about how to make this behavior more flexible, but as a quick workaround I suggest creating two dummy files with the correct names and placing them in /var/www/acme2certifier/volume.

Best,
G

[zihe2-yps]

tanks for your reply, but it is not working at all with TLS.
i put the cert and key into the volume, which is mounted to a CIFS; but the TLS part is not working.
and the logs in my case are not showing anything helpful.

[grindsa]

Ok let me try to replicate your issue. Can i use the above compose-file as guidance or did you change it meanwhile?

[zihe2-yps]

you can use it.

1. in the above code, when using CIFS with credentials, even after putting my cert and key into the volume folder, it doesn't work.
   Maybe a permission error?
   Unfortunately the logs are very sparse.

2. Meanwhile I also tried to set it up without CIFS; but the TLS didn't work either.
   Cause i have to use the cert and key as secret and when mounting it is RO.
   Your code however wants write permission and don't get it.

[grindsa]

I was able to work around the entry‑point script checks by doing the following:

• Mounting the nginx_sites volume directly to /etc/nginx/sites-enabled
• Placing both Nginx configuration files (nginx_acme_srv.conf and nginx_acme_srv_ssl.conf) on that volume-
• Adjusting nginx_acme_srv_ssl.conf:
  • Setting ssl_certificate to /etc/ssl/certs/acme2certifier_cert.pem
  • Setting ssl_certificate_key to /etc/ssl/private/acme2certifier_key.pem
• Mounting the secrets as you described

For reference, here is my configuration:

• docker-compose.yml

services:
  acme-srv:
    image: grindsa/acme2certifier:0.41-nginx-wsgi

    secrets:
       - source: acme2certifier_cert.pem
         target: /etc/ssl/certs/acme2certifier_cert.pem
       - source: acme2certifier_key.pem
         target: /etc/ssl/private/acme2certifier_key.pem

    volumes:
      - type: volume
        source: main_volume
        target: /var/www/acme2certifier/volume/
      - type: volume
        source: sites_volume
        target: /etc/nginx/sites-enabled
    ports:
      - "22280:80"
      - "22443:443"
    restart: always

networks:
  default:
    name: acme
    external: true


secrets:
  acme2certifier_cert.pem:
    file: ./acme2certifier_cert.pem
  acme2certifier_key.pem:
    file: ./acme2certifier_key.pem

volumes: # Define the named volume at the top level
  main_volume:
    driver: local # Optional, default is local
  sites_volume:
    driver: local

• nginx_acme_srv.conf

# zone with 10mb memory which is 160k/s - 5requests per client per second
limit_req_zone $binary_remote_addr zone=ip:10m rate=5r/s;

server {
    listen 80 default_server;
    listen [::]:80 default_server;

    # first 5 requests go trough instantly 5more requests evey 100ms
    limit_req zone=ip burst=10; # delay=5;

    server_name _;
    location = favicon.ico { access_log off; log_not_found off; }
    location / {
        include uwsgi_params;
        uwsgi_pass unix:/run/uwsgi/acme.sock;
        if ($request_method = "HEAD" ) {
           add_header Content-length 0;
           # add_header Transfer-Encoding identity;
        }
   }
}

• nginx_acme_srv_ssl.conf

# zone with 10mb memory which is 160k/s - 5requests per client per second
# limit_req_zone $binary_remote_addr zone=ip:10m rate=5r/s;
server {
    listen 443 ssl default_server;
    listen [::]:443 ssl default_server;

    # first 5 requests go trough instantly 5more requests evey 100ms
    limit_req zone=ip burst=10; # delay=5;

    server_name _;
    location = favicon.ico { access_log off; log_not_found off; }
    location / {
        include uwsgi_params;
        uwsgi_pass unix:/run/uwsgi/acme.sock;
        if ($request_method = "HEAD" ) {
           add_header Content-length 0;
           # add_header Transfer-Encoding identity;
        }
   }
   ssl_certificate "/etc/ssl/certs/acme2certifier_cert.pem";
   ssl_certificate_key "/etc/ssl/private/acme2certifier_key.pem";
   ssl_session_cache shared:SSL:1m;
   ssl_session_timeout  10m;
   ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH";
   ssl_prefer_server_ciphers on;
}

Please give a try on your site and let me know if it works for you.

/G.

[grindsa]

Hi,

I did setup a samba-server and reconfigured the docker-compose.yaml to mount a volume by using cifs.

Unfortunately, I wasn’t able to reproduce your issue. Everything works fine on my side using the following configuration:

services:
  acme-srv:
    image: grindsa/acme2certifier:0.41-nginx-wsgi
    container_name: acme-srv
    secrets:
       - source: acme2certifier_cert.pem
         target: /etc/ssl/certs/acme2certifier_cert.pem
       - source: acme2certifier_key.pem
         target: /etc/ssl/private/acme2certifier_key.pem

    volumes:
      - type: volume
        source: main_volume_cifs
        target: /var/www/acme2certifier/volume/
      - type: volume
        source: sites_volume_cifs
        target: /etc/nginx/sites-enabled
    ports:
      - "22280:80"
      - "22443:443"
    restart: always

networks:
  default:
    name: acme
    external: true

secrets:
  acme2certifier_cert.pem:
    file: ./acme2certifier_cert.pem
  acme2certifier_key.pem:
    file: ./acme2certifier_key.pem

volumes: # Define the named volume at the top level
  main_volume_cifs:
    driver: local # Optional, default is local
    driver_opts:
      type: "cifs"
      o: "username=user,password=password,vers=3.0,rw,uid=70,gid=70,nobrl,noperm"
      device: "//192.168.64.2/Data/volume"
  sites_volume_cifs:
    driver: local
    driver_opts:
      type: "cifs"
      o: "username=user,password=password,vers=3.0,rw,uid=70,gid=70,nobrl,noperm"
      device: "//192.168.64.2/Data/sites"

The sites - volume contains nginx_acme_srv.conf nginx_acme_srv_ssl.conf as described in my earlier post.

I understand that CIFS volumes do not support modifying file permissions or creating symlinks. However, I’m unable to identify the root cause of the mounting error you’re seeing, as I cannot find any code that creates a symlink for /etc/nginx/sites-available/acme_srv.conf. Are you getting the error during the start of the container?

Could you provide a few log messages before and after the error? That would help pinpoint which script is causing the issue.

Another remark:
I do not recommend placing the acme_srv.db on CIFS storage. It may appear to work initially, but performance tests show file‑locking issues that can lead to database corruption and enrollment failures. If storing configuration on CIFS is required (which is understandable), I suggest using the nginx django container and an external database (mariadb or psql). See the support for external database documentation for further information.

The configuration for a containerised environment is pretty straightforward. All you need to do is to

• copy the settings.py to the volume
• generate a new SECRET_KEY and add it to the file as described in the above mentioned guide
• configure the database connection as described in the above mentioned guide

The container will automatically create a migration directory on the volume. Please leave it there, as it will be required for future schema updates.

BTW: I did a quick test of this configuration and it seems to work without issues.

/G

[zihe2-yps]

Ciao,

• cert and key in volume

I am surprised you are not running into the same error, because to me it should always and i tryna explain.
Using docker secrets within a container will always be RO and is not allowed to change; what makes sense when thinking about it.
But in your docker-entrypoint.sh, you tryna set:

chown -R www-data /var/www/acme2certifier/volume
chmod u+s /var/www/acme2certifier/volume/

I could reproduce the exception for you, but as mentioned earlier it is not really specific, only something like "Permission denied".

• symlink

This one I am not sure of, but you creating as well a lot of symlinks in the same file.
I think, while using docker volume as CIFS mount, it is not compatible.

• my opinion (no front)

• cert into /etc/ssl/certs/acme2certifier.pem

• key into /etc/ssl/private/acme2certifier.pem

so they won't be affected by your permission changes

• sperate remaining content of /volume into /volume/configs and /volume/db, so one can only mount /volume/configs and won't run into the errors you mentioned

Anyway, I will try the django version. Thanks for your help!

[zihe2-yps]

Sali,

so after setting up the Django version locally your app is not connecting to the database.
Connection via db plugin works fine for the db.
Here is my database config, where I am surprised that the official documentation states "ENGINE": "django.db.backends.postgresql while you are using "ENGINE": django.db.backends.postgresql_psycopg2 (I tried both):

DATABASES = {
    "default": {
        "ENGINE": "django.db.backends.postgresql",
        "NAME": "acme2certifier",
        "USER": "acme2certifier",
        "PASSWORD": "a2cpasswd",
        "HOST": "127.0.0.1",
        "PORT": "5432",
    },
}

with a new self created SECRET_KEY and following allowed hosts:

SECRET_KEY = '*'
ALLOWED_HOSTS = ["127.0.0.1", "localhost"]

Am I missing something?

[zihe2-yps]

Nevermind, I created a network and then it seems to work.

[zihe2-yps]

I set it up, but still following points (since I am sticking to basic setup and CIFS):

1. What is the SECRET_KEY for?
2. https doesn't work when putting key and cert into CIFS volume-Folder

[zihe2-yps]

I set it up with a local bind.
So far so good.
Since I am passing the password as ENV, do you have any restrictions regarding escaping characters?

[zihe2-yps]

Brudi,
now I got a deeper look into your setup and I see why CIFS are failing, it is all about permissions you tryna force.
Anyway, not I am stuck in communicating with the CA for enrollmen.
What I didn't find yet, is the ca_bundle set correctly and what exactly should be inside?

This is my config:

# https://github.com/grindsa/acme2certifier/blob/master/docs/acme_srv.md
[DEFAULT]
debug: True

[Nonce]
# disable nonce check. THIS IS A SEVERE SECURTIY ISSUE! Please do only for testing/debugging purposes
nonce_check_disable: False

# https://github.com/grindsa/acme2certifier/blob/master/docs/mswcce.md
[CAhandler]
handler_file: examples/ca_handler/mscertsrv_ca_handler.py
host_variable: ENV_HOST
user_variable: ENV_USER
password_variable: ENV_PASSWORD
ca_bundle: /etc/ssl/certs/ca_bundle.pem
auth_method: ntlm
template: whatsgood
enrollment_config_log: True

[Certificate]
revocation_reason_check_disable: False

[Challenge]
# when true disable challenge validation. Challenge will be set to 'valid' without further checking
# THIS IS A SEVERE SECURTIY ISSUE! Please do only for testing/debugging purposes
challenge_validation_disable: False

[grindsa]

The ca_bundle must include the CA certificate bundle in PEM format which are required for validating the certificate of the Microsoft webenrollment server. Easiest way to get them is to run the below openssl command:

openssl s_client -connect <fqdn of ther mscert server>:443 -servername <fqdn of ther mscert server> -showcerts </dev/null

A quick-fix would be to turn of the validation of the server certificate by setting the verify = False parameter in acme_srv.cfg but this is bad security practise and should be avoided outside of test-environments.

[zihe2-yps]

I think it worked for me somehow, but my question was more about what to exactly insert: only the name ca_bundle.pem in what folder, or the entire path? To me it is not clear what to insert, but I used the entire path.

[zihe2-yps]

After setting it up locally with local mount it seemed to work.
Did you ever try it out with traefik and it worked for you?
Because for me it didn't so far; dunno what I'm missing.

Kind regard,

[grindsa]

Tests with traeffik is part of our regular regression. It runs smoothly since months with the this configuration.

Are you getting any error messages in the a2c log-file?

[zihe2-yps]

Yes, it looks like the challenge can't be placed under /.well-known/...
I couldn't figure out why it doesn't work, because my traefik config for the acme part looks similar.
Thank you

[zihe2-yps]

After long research and tests I figured out it won't work on Windows CA.

[grindsa]

Hi,

just out of curiosity. Can you explain why?

/G.

[zihe2-yps]

Moin,

according to research with ChatGPT it is a version mismatch of DCOM OLE.

Kind reagrds,