DockFlare Architecture suggestions / questions

[DennisGaida]

I didn't dig in too much of the code - love your UX / web management, maybe these are some good discussion points:

1. I wouldn't add secrets to environment files. Make use of docker secrets. The usual consensus is to have the same env vars just a _FILE suffix. If time permits I'll just add a PR for this.
2. As for secrets and cloudflared API keys - I don't like giving "something" so many permissions. You could change something upstream, I pull the docker container and you have full access to all of my DNS (with the DNS edit permission). I am perfectly fine with using cloudflared login since years now. This doesn't remediate the upstream problem, but it somehow feels better since this is scoped to only the specific tunnel having permissions, not general API keys.
3. I understand you do a lot via API calls, but at the same time make use of the cloudflared binary. Are you sure you have to do everything manually via API calls - for most of the stuff there are commands with cloudflared.

I have been using cloudflared tunnels since years now with docker containers and I never had to clean up any DNS entries manually via the API or via the web UI, for that one I just use clouflared tunnel delete <name> and everything is gone (there also is cleanup) - see https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/do-more-with-tunnels/local-management/tunnel-useful-commands/.

This doesn't mean to come off as snarky - just wondering why you do everything via API calls and at the same time use the cloudflared binary for other things (same as the option to use the external cloudflared binary).

[ChrispyBacon-dev]

Hey Dennis,

Awesome feedback, man! Seriously, thanks for taking the time to write all this up, great points and glad you're digging the UI! Actually I'm working on a enhanced UI update, I added yesterday API v2 support (unstable branch). This is a needed step to support swarm/k8. The plan is to have one DockFlare "control UI" and than DockFlare agents on hosts deployed with control via API ;) but thats the future

Let's dive in to your valid raised points:

• Docker Secrets for the API Token:
  Spot on about Docker Secrets. Definitely the way to go for better security over plain env vars. DockFlare uses env vars now mostly to keep setup simple for everyone, but yeah, adding _FILE support is a great idea. Its on my list :) DockFlare is about 1 months old ... so still very early in development.

• API Key Permissions & cloudflared login:
  Totally get the concern with broad API key permissions it's a big one, and keeping things secure is top priority.
  The main reason DockFlare needs that Cloudflare API Token (scoped down as much as possible, of course!) is for the features that go beyond what the tunnel's own cloudflared login cert can do. ->

  • Access Policies: This is a big for DockFlare creating and tweaking those Zero Trust rules on the fly based on labels or UI clicks. The tunnel's own creds can't touch that.
  • DNS: It's not just one DNS record per tunnel, DockFlare juggles CNAMEs for all sorts of services you might expose (support manual DNS rules not just Docker labels), all pointing to that one tunnel. Plus, it needs to see what's already there for reconciliation.
  • Finding/Making Tunnels: The initial setup of the tunnel object in Cloudflare if it doesn't exist.
    And yeah, the cloudflared agent itself (whether DockFlare runs it or you bring your own) does use that tunnel-specific token/certificate you get from cloudflared login or cloudflared tunnel token <TUNNEL_ID>. That's purely for the agent to connect and do its job. So, it's like two different sets of keys for two different jobs.
    I'm always hoping Cloudflare gives us even more granular token scopes in the future! For now, the advice is to create a dedicated API Token with just the permissions DockFlare actually needs.

• API Calls vs. cloudflared CLI Commands:
  Good question on why API calls instead of just using cloudflared commands for everything. While cloudflared is great for running the tunnel (and DockFlare uses it for that if it's managing the agent!), I use the API for a these key reasons:

  • Access Policies (again..): The CLI just doesn't have the commands for the detailed Access policy creation and management DockFlare aims for.
  • Reliable State & Control: Getting structured JSON from the API is way more solid for figuring out what's what (Docker labels vs. local state vs. actual Cloudflare setup) than trying to parse CLI text output, which can break easily if Cloudflare changes it.
  • Python-Friendly: Working with APIs directly in Python is just cleaner and more robust when you're building out complex automation logic.
  • Multi-Service DNS/Multi Zones: Juggling lots of CNAMEs for one tunnel, each with its own rules, is easier with the API's flexibility.

• DNS Cleanup (cloudflared tunnel delete / cleanup):
  You're right, cloudflared tunnel delete <name> nukes a tunnel and its main DNS record pretty effectively. DockFlare's approach is a bit different because it's all about managing individual services/rules that are all using that one persistent tunnel.
  So, if you have 10 services running through DockFlare and one Docker container stops, DockFlare just yanks the CNAME for that one service, leaving the tunnel and your other 9 services chugging along.
  Plus, the whole reconciliation thing and the grace period before deleting stuff helps avoid issues if a container just blips offline for a second and comes right back.

So, bottom line: DockFlare uses the cloudflared binary for what it does best -> running the secure tunnel connection. But for all the automatation on top, the dynamic config based on Docker, Access policies, DNS for multiple services, and the UI to tie it all together, the Cloudflare API gives us the power and control we need. My thinking is to have cloudflared be the dedicated workhorse for the connection, and DockFlare as the "brains" managing it and the services it exposes.

Hope that clears things up a bit and explains the "why" behind some of the choices! Seriously, your feedback's gold. This is actually my first public open-source project. Even with over 20 years in IT infrastructure exp. under my belt, diving into building DockFlare has been a unique learning curve, and I'm always trying to improve it and pick up new things along the way.

So, insights like yours are super valuable for that, and I'll definitely keep chewing on these points for future improvements.
Happy to chat more whenever!

Cheers from Switzerland and have a nice weekend,
Chris

[DennisGaida]

Love all of your deep thoughts about this. You should add some of that explanation to the README.

Docker secrets support should be easy enough. In the past I have done most of the implementations in the entryscript which you don't have in this project, hence it should be done in Python in the config.py. Pseudo-could would be: If _FILE environment variable exists, check whether the secret file exists and use that, otherwise use the environment variable.

I totally understand now why you go the API route, the level of control really is a lot more granular and I totally forgot about parsing CLI which of course is a pain.

[DennisGaida]

I'll just add some things here about policies, since I played a bit more with DockFlare:

• You're creating legacy policies, when things are selected via the UI - these seem to be deprecated
• The policies you create are just named e.g. "UI Deny Fallback", "UI Allow Email [email protected]", if you are automating policy creation you should prefix these with DockFlare / DF so I know these are DockFlare policies
• You also seem to be deleting these policies if they are not used anymore - using the names or some UUID (hinting at the above point)?
• I know I can create custom policies with docker labels - I don't understand where your default policies come from, there is e.g. the "default *.tld policy", only by checking the log I figured out that you are searching for a policy that applies to *.domain.tld and you would apply that policy. Problem is if you don't find that policy you actually delete the whole application (at least in my tests).