Lack of dataplane service annotation possibility prevents v1 -> v2 migration

[stutommi]

Is your enhancement request related to a problem? Please describe.
TLDR - lack of dataplane service annotation annotations required by Azure AKS makes migration from v1 towards v2 impossible for NGF users who create internally reachable gateways (atleast on Azure).

I ran into an issue while migrating v1 ngf to v2. The transition otherwise went smoothly, but my main problem lies with the way how Azure requires annotations be added for the Loadbalancer type service that would be created for the dataplane. IIUC - the v1 implementation of ngf left it to maintainer of gateway to configure/patch the service with correct annotations (link in TLDR). The v2 on the other hand seems to create these for the user, without any option to add annotations into them, which would be crucial in our context.

I really like the v2 idea and implementation here, but due to how Azure expects internally reachable load balancers to be annotated, it makes us impossible to move forward. We would like to continue using NGF, but might need to migrate to different gateway implementation incase v1 is no longer maintained (which is understandable) and v2 will not provide way for service annotations for the dataplane.

Is there any solution / potential enchancement to allow us using NGF?

What would you like to be added:
A way to provide annotations for the dataplane services created by NGF v2

Why this is needed:
Like said above, not having this feature drops possibilities to use NGF atleast in any environments that need to provide internally reachable loadbalancers.

[nginx-bot]

Hi @stutommi! Welcome to the project! 🎉

Thanks for opening this issue!
Be sure to check out our Contributing Guidelines and the Issue Lifecycle while you wait for someone on the team to take a look at this.

[sjberman]

Hi @stutommi, labels and annotations for provisioned nginx resources (such as the LoadBalancer Service) can be defined in the Gateway resource, in the spec.infrastructure section. See this guide on how to do that. Note that these labels/annotations get applied to every resource we provision (including the Deployment). Functionally, that shouldn't affect anything, but it may look odd.

In our 2.1 release later this week, we're also adding the ability to specify custom patches per resource (in the NginxProxy CRD), which can allow for setting resource-specific labels/annotations on just the Service, instead of setting them in the Gateway's infrastructure and having them propagated onto every resource we provision.

Let me know if this addresses your needs!

[stutommi]

Thanks for the quick reply! Both ways you described sound like valid ways forward, although I'm much happier using the way that the 2.1 release brings.

I'll test the first option tomorrow 👍😊

[stutommi]

Commenting here that I ended up waiting for the 2.1 release - since thinking it will anyway be the preferred solution for my problem. I can comment the outcome once the release lands.