Autoscaling KafkaConnectors

[opsocomusr]

We are trying to setup autoscaling for KafkaConnector resources using KEDA. However when we deploy the scaled object, we are getting the following error in the HPA.

KafkaConnector pods:  5 current / 0 desired
Conditions:
  Type           Status  Reason             Message
  ----           ------  ------             -------
  AbleToScale    True    SucceededGetScale  the HPA controller was able to get the target's current scale
  ScalingActive  False   InvalidSelector    the HPA target's scale is missing a selector
Events:
  Type     Reason                        Age                   From                       Message
  ----     ------                        ----                  ----                       -------
  Warning  FailedComputeMetricsReplicas  19m (x12 over 22m)    horizontal-pod-autoscaler  selector is required
  Warning  SelectorRequired              2m39s (x81 over 22m)  horizontal-pod-autoscaler  selector is required

In the KafkaConnector CRD, there is no labelSelectorPath in the scale object.

Is autoscaling not supported yet for KafkaConnector resources? From the other discussion comment, we were under assumption that this is supported for KafkaConnectors.

[scholzj]

The scale subresource is supported by the KafkaConnector resource. But the KafkaConnector is a virtual object which does not represent any pods. So there is nothing the label selector can select.

[hafizmujadidKhalid]

@scholzj!
Can you please point me to some example of how did you manage autoscaling for connect? thanks

[scholzj]

I do not have any examples to share. But IIRC, you simply just set-up the HorizontalPodAutoscaler resource and point it to the KafkaConnect resource.

[hafizmujadidKhalid]

@scholzj! thanks for the hint. So, it will scale connect workers, can we do something about connector tasks as well or is it something not possible? I am not an expert in K8s, excuse me for my ignorance :)

[scholzj]

As discussed earlier here -> the Connector tasks have the scale endpoint, bu they do not have any Pod representation. So I'm not sure if regular HPA resource would work on it.

Also, scaling the connectors is a it more complicated => the tasks.max option does not mean how many tasks will there be. Its just the maximum. Some connector will always use only one task regardless the configuration. Scalability of sink connectors will also depend on on other factors such as how many partitions does the Kafka topic has. Source connectors have often similar limitations imposed from the source system. So it is fairly tricky to have some generic autoscaling for this. You might be able to get something working for some particular connector if you understand how it works.

[hafizmujadidKhalid]

makes sense, Thanks a lot