Handling sticky sessions to the HTTP bridge

[ppatierno]

Because the HTTP bridge takes trace of the created consumer via an instanceid (after creation), most of the subsequent operations work if the HTTP client connects to the same HTTP bridge instance in a cloud environment with multiple replicas.
We should think about a way for achieving that (i.e. sticky sessions).

[rossbrigoli]

Or at least find a way to propagate consumers to all instances of the Kafka bridge that are connected to the same Kafka cluster. This way the consumer can be created from one bridge endpoint, then the subsequent calls like pulling records can be issued on another bridge endpoint.

[scholzj]

Or at least find a way to propagate consumers to all instances of the Kafka bridge that are connected to the same Kafka cluster. This way the consumer can be created from one bridge endpoint, then the subsequent calls like pulling records can be issued on another bridge endpoint.

I don't think this is desired. Because it would mean that each bridge instance will be assigned some partitions and you will have to make sure in your clients that you are polling for messages on all instances and not just on one.

[echooymxq]

This seems to be a common problem, and it also exists in confluentinc's kafka-rest. Is there a good solution?