Deploy sharded controller as StatefulSet #672
Description
What would you like to be added:
Let's try deploying a sharded controller as a StatefulSet instead of a Deployment.
To evaluate the differences, there should be a simple experiment scenario that performs a rolling update of the webhosting-operator.
If the StatefulSet proves to be more suitable for sharded controllers, it should become the recommended deployment pattern.
Why is this needed:
Performing a rolling update of a sharded controller causes many unnecessary object reassignments because of two problems:
- objects are moved away from terminated old instances, even though there will eventually be a 1:1 replacement for every old instance
- objects might be moved away from a terminated old instance to another old instance, even though it will be terminated shortly afterwards
Both of these problems can be addressed by using stable instance identifiers, e.g., index-based pod names of a StatefulSet.
If terminating instances wouldn't release their shard lease in this scenario, objects would not be moved during a rolling update (given that new instances are available promptly).