swag metrics and expiration dates

[eradan]

Hello,

I’m currently using SWAG and I’d like to integrate it into my observability stack.
Specifically, I’m looking for a way to collect metrics in Prometheus format from SWAG, including:

1. general status / usage metrics
2. TLS certificate expiration dates

Is there already a module, exporter, or configuration available for this purpose?
Or alternatively, what would be the recommended way to verify certificate expiration with SWAG?

Thanks for your support!
Ivan

[drizuid]

swag is just nginx, if you can do it in nginx you can do it with swag.
you can also take a look at our dashboard mod which does a lot of what you're asking for. mods are listed in the container readme.

as to recommending a way to verify certificate expiration, letsencrypt certs are good for 90 days, none of us chase this info because swag automatically renews it before expiration, so we just don't care when it expires.

[eradan]

@drizuid thanks for your reply :-)

I understand that SWAG is basically Nginx and that most users don’t track certificate expiration because Let’s Encrypt certs auto-renew.

The issue I’m hitting is that sometimes renewals fail silently, and I end up with expired certificates. That’s why I’m interested in monitoring expiration dates via Prometheus or a similar method, to get alerts before things break, even if SWAG usually handles renewals automatically.

Here’s an example of a failed renewal that occured:

Description: Runs an HTTP server locally which serves the necessary validation files under the /.well-known/acme-challenge/ path. Suitable if no HTTP server is running.
HTTP challenge only (wildcards not supported).
Interfaces: Authenticator, Plugin
Entry point: EntryPoint(name='standalone', value='certbot._internal.plugins.standalone:Authenticator', group='certbot.plugins')
Initialized: <certbot._internal.plugins.standalone.Authenticator object at 0xXXXXXXXXXXXX>
Prep: True
DEBUG: Selected authenticator <certbot._internal.plugins.standalone.Authenticator object at 0xXXXXXXXXXXXX> and installer None
INFO: Plugins selected: Authenticator standalone, Installer None
ERROR: Failed to renew certificate example.com with error: Missing command line flag or config entry for this setting:
Please choose an account
Choices: ['example-account1', 'example-account2']
DEBUG: Traceback was:
Traceback (most recent call last):
  File ".../certbot/_internal/renewal.py", line XXX, in handle_renewal_request
    main.renew_cert(lineage_config, plugins, renewal_candidate)
  File ".../certbot/_internal/main.py", line XXX, in renew_cert
    le_client = _init_le_client(config, auth, installer)
  File ".../certbot/_internal/main.py", line XXX, in _init_le_client
    acc, acme = _determine_account(config)
  File ".../certbot/_internal/main.py", line XXX, in _determine_account
    potential_acc = display_ops.choose_account(accounts)
  File ".../certbot/display/ops.py", line XX, in choose_account
    code, index = display_util.menu("Please choose an account", labels, force_interactive=True)
  File ".../certbot/display/util.py", line XX, in menu
    return obj.get_display().menu(message, choices, default=default, cli_flag=cli_flag,
  File ".../certbot/_internal/display/obj.py", line XXX, in menu
    raise self._interaction_fail(message, cli_flag, "Choices: " + repr(choices))

I’ll take a look at the dashboard mod, thanks for the pointer!

[aptalca]

I realize this is not why you're here, but I'll let you know anyway. That error should not happen in SWAG unless you manually broke things in a major way. SWAG is designed to keep only one certbot/acme account. If you're getting that error, I suggest nuking the /config/etc folder and starting over because that error means something broke so bad that none of the assumptions SWAG's cert renewal logic makes are accurate anymore.

SWAG's certbot logic is pretty well tested at this point. The only times folks have failed renewals are when either they copy/move/restore the config folder incorrectly (invalidating the symlinks certbot uses) or they make a change to their dns.

In any case, prometheus integration is not currently supported and not likely to be added due to the low priority and us being really busy with other tasks. But, you can likely set up a notification for renewal failures via cron by checking for cert expiration date. We actually check on container start whether the cert is to expire within 1 day and if so, it forces a renewal during start. You can use the same logic in a cron script (SWAG has cron built in) to check if the cert expiration is less than 25 days (it should be renewed 30 days prior) and notify yourself via ntfy or any other notification method.

https://github.com/linuxserver/docker-swag/blob/ccd2464a26ef3485f1b24262adcc349d0fcc28af/root/etc/s6-overlay/s6-rc.d/init-renew/run#L4

[eradan]

@aptalca thanks a lot for the detailed explanation!

That actually makes sense — in my case, I had changed the challenge from standalone to DNS using the related DNS provider plugin. Do you think that could be the reason I ran into that error?

Also, I appreciate the tip about using a cron script to check certificate expiration for notifications. I'll take a look at that!