scs2: Introduce a clusterClass variable controlling the type of service LoadBalancer

[garloff]

/kind feature

Describe the solution you'd like
I would like to control the type of LoadBalancer that the workload cluster creates when creating a service of type LoadBalancer (such as e.g. the LB in front of an ingress).

We already have a variable for the LB type in front of the kubeAPI apiServerLoadBalancer (defaulting to octavia-ovn).
But none for the service. This results in the service LBs to be octavia-amphora on mosts clouds, which is not ideal. (amphorae take a minute to start, eat much more resources, bring more complexity and are less robust. They provide more features, such as L7 LB and TLS termination, but that's not normally needed when used from k8s. Also we lose the originator IP without proxy protocol hacks ...)

So I'd like to have something like serviceLoadBalancer clusterClass variable with the same choices (and defaults) as for apiServerLoadBalancer.

Anything else you would like to add:
On openstack-scs2, this is done by OCCM which is configured by cloud.conf which is automatically created upon deployment from the supplied clouds.yaml. Unfortunately, there is no switch here.

A hack is to patch cloud.conf, see https://github.com/SovereignCloudStack/scs-training-kaas-scripts/blob/main/_11-fixup-ovn-lb.sh. This will not survive the next OCCM update though ...

Environment:

• scs2 cluster stacks (k8s-v1.33.5)