Multiple targets per service (load balancing)

[kevinmcconnell]

We should add the ability to register multiple targets for a single service, and distribute requests amongst them round-robin. This will give us the ability to load balance the traffic for a service across multiple hosts. With this setup, Kamal can provide a complete solution for running an app over multiple hosts, by running a proxy instance on a different host.

Adding this also means:

• The semantics of deploying needs to be extended to handle the case where only some of the new targets become healthy. Probably the behaviour here should be that we health-check every host in the list, and if any of them fail to become healthy we return non-zero without having applied any of the changes. But if all become healthy, we apply the new set as an atomic change. This includes the case there there are overlaps between the old and new list.

  For example, if you're deployed to t1, t2, t3 and you attempt a new deployment for t1, t2, t5, t6 but t6 fails, we keep the original set of t1, t2, t3.

• Healthchecks should be ongoing, rather than stopping after success as they do now. We'll want to notice an unhealthy target after the fact and take it out of the routing until it becomes healthy again.

[Jainam-17-18]

Can we have internal load balancer in Kamal-proxy which can handle request by forwarding request to multiple application containers running on Same Server or Single Host. And can we have Kamal deploy config like replicas: 5 (container count), so total 5 container will be created during deploy and Kamal-proxy will handle it.

[mhenrixon]

@kevinmcconnell @djmb have you guys thought more about this? I just ran into a project where I could use load balancing to avoid resorting to traefik.

[kevinmcconnell]

@mhenrixon yes, this is still definitely planned. I recently started work on it, and have a branch that's close to ready. Hope to have something pushed up to share soon.

At the proxy level, it adds the ability to specify multiple target params, and implements the behaviour described above. So if you do something like deploy myservice --target=app01,app02,app03 then it will do round-robin load balancing between those three servers (moving them in and out of rotation according to their healthchecks).

We're still discussing the details of how this will look in Kamal's deploy.yml, but again hopefully will have more to share soon on that!

[mhenrixon]

Oh shoot @kevinmcconnell I kind of got into it and didn't see your message until now.

If you already have something you can close my PR. I have like 3 weeks until I need it but I have a fallback ready with to Traefik (or even a Hetzner load balancer would work I guess).

Let me know if I can help in any way or form.

[kevinmcconnell]

Thanks for your help with this, @mhenrixon! I'm going to close your PR in favor of the one I've had brewing, which I just opened. But I do appreciate you taking a shot at this! Looks like there are quite a few places where we took a similar approach.

I'm planning to merge my PR fairly soon, but if you have a chance to try that branch on your project I'd love to hear how you get on with it, and any feedback you have.

[mhenrixon]

I'm planning to merge my PR fairly soon, but if you have a chance to try that branch on your project I'd love to hear how you get on with it, and any feedback you have.

I'd love to try it out.

Can I take some shortcuts to deploy my app with the binaries built from your branch? If you could help me with how to approach this, I'll be sure to get it done today @kevinmcconnell.

This is too good not to prioritize!

Would the easiest be to build and use my own image based on your branch?

[kevinmcconnell]

@mhenrixon as it's not integrated into Kamal yet, you won't be able to get it through a normal kamal deploy. But I've pushed this version to Docker Hub, tagged as lb, so you could try running it directly with Docker. Something like:

docker run --name load-balancer basecamp/kamal-proxy:lb

And then you can deploy your app's hosts into it by running kamal-proxy commands against that container, like:

docker exec load-balancer kamal-proxy deploy myapp --target=host01,host02,host03

But of course you can also build it locally from the branch too. Just run make to create a plain binary, or make docker to build your own image. Any of these ways should work.

[mhenrixon]

Alright, @kevinmcconnell, I am on it now! So, to deploy, I turned off SSL in the proxy at first because of:

Warning

ERROR: Failed to load configuration: SSL is only supported on a single server, found 2 servers for role

I'm running the setup now. As you suggested, I'll stop the original proxy and run the :lb tag.

Will it work with SSL certificates? It could be a stupid question because I will find out soon enough. 🤣

EDIT:

Ok, now I am confused! The basecamp/kamal-proxy is deployed to each server. Where does that leave the load balancer?

So, I put the load balancer on the database server since the domain was pointed there anyway.

docker exec load-balancer kamal-proxy deploy wm3 --target=IP_1,IP_2

{
    "time": "2025-04-10T10:04:00.222587557Z",
    "level": "INFO",
    "msg": "Healthcheck failed",
    "url": "http://IP_1/up",
    "error": "Unexpected status (404)"
  },
  {
    "time": "2025-04-10T10:04:00.222921545Z",
    "level": "INFO",
    "msg": "Healthcheck failed",
    "url": "http://IP_2/up",
    "error": "Unexpected status (404)"
  },
  {
    "time": "2025-04-10T10:04:01.221310739Z",
    "level": "INFO",
    "msg": "Target failed to become healthy",
    "target": "IP_1"
  },
  {
    "time": "2025-04-10T10:04:01.221384389Z",
    "level": "INFO",
    "msg": "Target failed to become healthy",
    "target": "IP_2"
  }

Do I need to deploy the kamal-proxy load balancer to each server?

[mhenrixon]

Ok, if I use the new proxy with the lb tag on each server it looks like I get connection refused:

{
    "time": "2025-04-10T10:09:05.009555624Z",
    "level": "INFO",
    "msg": "Healthcheck failed",
    "url": "http://IP_1/up",
    "error": "Get \"http://IP_1/up\": dial tcp 65.108.243.92:80: connect: connection refused"
  },
  {
    "time": "2025-04-10T10:09:05.010584576Z",
    "level": "INFO",
    "msg": "Healthcheck failed",
    "url": "http://IP_2/up",
    "error": "Get \"http://IP_2/up\": dial tcp 157.180.40.129:80: connect: connection refused"
  }

[kevinmcconnell]

@mhenrixon yes it will be a bit in-flux and complicated until we add support for setting this up on the Kamal side as well! There are a few different ways to set it up, but for now I think the simplest would probably be:

• Keep the original proxy running as well, on each server. But turn off the SSL option, since you won't be accessing these directly from outside (traffic should go through the load balancer to reach them). This way you can continue to deploy as normal, and Kamal will update the app on each server.
• Run a separate proxy for the load balancer (just one instance, wherever the domain's DNS points to). Then deploy your app servers to it, similar to what you did above. But you can also specify the SSL settings here, so that the load balancer will do the SSL termination. You also won't need to re-deploy anything to that load balancer unless you are adding or removing app servers; but otherwise, normal application deploys shouldn't affect it.

Regarding the connection refused errors, you may just need to ensure that those IPs and ports are accessible from wherever you're running the load balancing proxy. Maybe you need to use an internal IP, or check any firewall rules, to make sure that traffic is allowed. You could try to curl the app server endpoints from the server where the load balancer is, to make sure they are reachable from there.

[kevinmcconnell]

Oh also, if you are setting the host flag on your app deployments, perhaps that's what caused the 404, because the load balancer's healthchecks won't have included the host and so they would have failed to be routed correctly. If this is the only app you're running on those app servers you should be able to remove your host setting from the Kamal side to avoid the 404. (It would have been required for the SSL settings, but if you've moved SSL out to the load balancer you can move the host flag out there too.)

For multiple apps on a server using host-based routing, we may need to add a way to include a Host header in the healthchecks to avoid this. I'll have a think about that.