Horizontally scaling self contained systems #121
Description
One thing that I don't think is addressed, and I can't seem to find an obvious answer to, is how to horizontally scale self contained systems? Currently the main way alot of software manages this is by microservice architecture and heavily relying on things like a message bus or distributed cache. Specifically things like redis, rabbitmq and sharing postgres databases, so they are free to make the core of the microservice stateless (or atleast have checkpoints to pick up from) and thus enabling scaling horizontally through replicas.
The documentation for scs does mention things like light message passing or shared databases with seperate schema's, but even those points call out the risk you are taking with those decisions.
So in truly isolated self contained systems, how do we scale these horizontally (for example in kubernetes just running N replicas with a service definition behind a load balancer) while being able to deal with the state drift problem? The only potential solution I've seen mentioned about this is using sticky sessions to ensure an individual user's state stays correct for the duration of the session. But that is at best a short term solution and is not resilient.
Some larger solutions could be something like leveraging distributed systems patterns and using a consensus algorithm in the core of your scs. However then you also start running into things like service discovery problems between the replicas, where again in the wild it seems this is managed by a shared table with a lock, or more messages with pub/sub. In my mind this breaks the ideals of self contained systems.
What is the official advice on this topic?