Kamal Multi Server Deployments Issue

[timwaweru]

Help needed: Load balancing Kamal deployment across multiple servers with SSL termination

I'm deploying an app to multiple EC2 servers using Kamal:

servers:
  web:
    hosts:
      - server-1
      - server-2

Kamal requires SSL to be disabled for multi-server deployments due to the limitation:

SSL is only supported on a single server, found 2 servers for role web

Current setup:

• Main domain: api.example.com (A record)
• Client domain: api.client.com (CNAME pointing to main domain)

Problem:
I need to load balance traffic between the two servers with SSL termination at the load balancer level (AWS ALB/NLB or Cloudflare).

Issues I'm facing:

1. AWS Load Balancer approach:

   • Created target group pointing to port 80 (where Kamal proxy listens)
   • Load balancer listens on port 443 with SSL certificate
   • Problem: Each domain needs its own certificate in ACM since wildcard certs (*.example.com) don't cover the CNAME setup
   • This requires manual certificate creation for each client domain, which isn't scalable

2. Cloudflare approach:

   • Trying to use Cloudflare for TLS termination
   • Unable to properly route traffic to Kamal proxy on port 80

Question: What's the best way to set up load balancing with SSL termination for this multi-tenant architecture while keeping it scalable (avoiding manual certificate management)?

[shawndewet]

It's disappointing that the Kamal team has not responded to this request for guidance (nor many other requests on this repo).

Your request for guidance is a very valid one, and surely a common scenario around which there should be some guidance in the kamal documentation. I can't answer your question in totality, but here's my 2c worth though:

I think your mention of "avoiding manual certificate management" is where your scenario is not supportable by kamal. Note the first sentence of this section in the docs:

So the fact that your app will be hosted on multiple servers means you need to do manual certificate management.

Having said that, I see the release notes of Tag 0.9.0 shows that load balancing support has been added (albeit without any guidance as to how to actually implement this). Perhaps this may be an approach to take to support your scenario?

[kevinmcconnell]

@timwaweru sorry for the slow response (and thanks for the nudge @shawndewet!). Been a bit busy on other projects so have not been replying as quickly as I'd like.

As @shawndewet says, Kamal currently only provides automatic TSL for individual hosts, and it doesn't currently load balance traffic between those hosts. So for a setup like you describe, you'd need to handle load balancing and TLS yourself. Most people doing this now are using either cloud load balancers (like you mentioned) or they use another reverse proxy as the load balancer (something like Caddy, for example) which then routes to the Kamal-deployed app servers.

We will be adding load balancing support to Kamal shortly. Kamal Proxy supports it now (essentially you supply multiple --target options when calling kamal-proxy deploy, and it will load balance traffic between them) but the full integration with Kamal is still being worked on. Once that lands, there will be documentation in Kamal that explains how to use it. I don't have an exact timeframe for when this will be ready, but I'm hoping it'll be in the next Kamal release.

On-demand TLS certs are also being worked on in #63, which will automate the provisioning of new certificates if you need them on a per-tenant basis.

When those both are ready I think it'll provide exactly what you need. But until then I'm afraid you'll need to handle it in a layer in front, and then either use custom certs on the app servers, or route plain HTTP between your load balancer and the app servers. (Most people do the latter).

I'm curious what problem you ran into when routing Cloudflare to port 80, by the way?