SSL_ERROR_UNRECOGNIZED_NAME_ALERT when accessing SOME of my subdomains locally

[Epuuc]

The issue
I'm trying to set up my subdomains to point to my proxy, but before I can reach the proxy, it throws a SSL_ERROR_UNRECOGNIZED_NAME_ALERT when I try to access them. I have a wildcard certificate using LetsEncrypt through Cloudflare's API, but getting new certs, and even using a certificate for a specific domain doesn't work either. I noticed this happens with more public services than private.

Background
I've been trying out a bunch of different proxies, and I went from Caddy to a smaller one called Zoraxy, to Nginx Proxy Manager, and I really like Nginx Proxy Manager and it seems well maintained, but I'm having an issue when I set it up.

My network setup
I have about 20 subdomains I have pointing to different services I have running in my network. 7 of them are public, and the rest of them are private only. My external services have public DNS records pointing through Cloudflare, then Cloudflare to my proxy. My local network has a couple pi-hole instances which points all the subdomains to my proxy directly.

I've done a bunch of searching for this issue, but most of the articles just has people asking for help with the same error with no solution, or a solution that doesn't solve my issue.

[Epuuc]

I should mention that I have the proxy installed on an LXC container without docker using this script