feat(loadbalancer): Add LoadBalancerType Client Side Weighted Round Robin #7407

anuragagarwal561994 · 2025-11-02T03:31:56Z

What type of PR is this?

What this PR does / why we need it:
This PR provides addition of new load balancer type client side weighted round robin. This is a new load balancing extension introduced since envoy 1.32

https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/load_balancing_policies/client_side_weighted_round_robin/v3/client_side_weighted_round_robin.proto

Which issue(s) this PR fixes:

Fixes #7305

Release Notes: Yes/No

anuragagarwal561994 · 2025-11-02T03:34:49Z

@jukie I have added the implementation and also tested it on my local setup

PS: the repo is so easy to contribute everything just works with the docs given on the site :)

anuragagarwal561994 · 2025-11-02T03:37:28Z

Also I wanted to know should I include slow start in client wrr?

So the thing is that I have submitted the proposal in grpc-xds grpc/proposal#498 and also in envoy I have got the proto updated.

It is not implemented yet, but I am trying to pick it up this month if my time allows

anuragagarwal561994 · 2025-11-02T03:40:44Z

Also I am unsure of how to test this e2e, so I have just included an AI generated e2e test suite.

The challenge here is that we need multiple replicas with each server respond with a specific header containing rps and cpu_utilisation and then the traffic is distributed by calculating the weight (rps / cpu)

I don't know what the current e2e tests allow and if this type of test case is feasible to write

api/v1alpha1/loadbalancer_types.go

codecov · 2025-11-04T00:45:46Z

Codecov Report

❌ Patch coverage is 72.85714% with 19 lines in your changes missing coverage. Please review.
✅ Project coverage is 72.38%. Comparing base (aa3ad43) to head (777a1c1).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
internal/xds/translator/cluster.go	58.53%	16 Missing and 1 partial ⚠️
internal/gatewayapi/clustersettings.go	92.00%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #7407      +/-   ##
==========================================
+ Coverage   72.36%   72.38%   +0.02%     
==========================================
  Files         233      233              
  Lines       34343    34410      +67     
==========================================
+ Hits        24851    24908      +57     
- Misses       7712     7723      +11     
+ Partials     1780     1779       -1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

anuragagarwal561994 · 2025-11-04T14:39:41Z

Also I wanted to know should I include slow start in client wrr?

So the thing is that I have submitted the proposal in grpc-xds grpc/proposal#498 and also in envoy I have got the proto updated.

It is not implemented yet, but I am trying to pick it up this month if my time allows

I have started the implementation of slow_start_config and locality lb config with WRR as well, if possible I would also want to include them in the gatway implementation.

envoyproxy/envoy#41841

jukie · 2025-11-13T17:09:33Z

We wait to add features here until they've made it into a full envoy release. The flow would be getting this lb support added for 1.7 and if your envoy changes get merged we can add that support to gateway in 1.8.

Let's keep the scope of this PR to what's currently available and we can always include additional features in a follow-up. Are you able to make the suggested changes or can you join the contributors call next week to discuss?

anuragagarwal561994 · 2025-11-14T06:19:38Z

Sure, I just paused because the other changes were also approved, but I understand I will make the respective changes as suggested. Will try to complete them by today / tomorrow @jukie

anuragagarwal561994 · 2025-11-17T00:15:38Z

@jukie I have made the respective changes

…Gateway CRDs, ensuring configurable parameters and validation rules are integrated. Includes e2e test for validation. Signed-off-by: anurag.ag <[email protected]>

Signed-off-by: anurag.ag <[email protected]>

…eway CRDs and related configurations. Update associated test data and documentation. Signed-off-by: anurag.ag <[email protected]>

…entSideWeightedRoundRobin configuration, update affected tests and CRDs. Signed-off-by: anurag.ag <[email protected]>

…cross Gateway CRDs, configuration files, and related tests. Adjust documentation to reflect percentage-based representation. Signed-off-by: anurag.ag <[email protected]>

internal/ir/xds.go

jukie · 2025-11-27T01:34:11Z

api/v1alpha1/loadbalancer_types.go

+// ClientSideWeightedRoundRobin defines configuration for Envoy's Client-Side Weighted Round Robin policy.
+// See Envoy proto: envoy.extensions.load_balancing_policies.client_side_weighted_round_robin.v3.ClientSideWeightedRoundRobin
+// Note: SlowStart is not supported for this policy in Envoy Gateway at this time.
+type ClientSideWeightedRoundRobin struct {


I saw your pr adding slow_start was merged in as well so feel free to include that now if you'd like and we'll get this into v1.7.

can the existing slow_start field be reused ?

Yep it uses extensions.load_balancing_policies.common.v3.SlowStartConfig which is the same as used in other lb policies.

Should we also support the min_weight_percent and aggression now in this MR or in a separate MR?

internal/ir/xds_test.go

jukie · 2025-11-27T01:38:55Z

Overall looks good, just a few more comments @anuragagarwal561994! Thanks for adding this and make sure to run gen-check before pushing.

Sorry for the delayed review on this. I'll prioritize helping you with this next week.

arkodg · 2025-12-02T01:51:45Z

api/v1alpha1/loadbalancer_types.go

+// ClientSideWeightedRoundRobin defines configuration for Envoy's Client-Side Weighted Round Robin policy.
+// See Envoy proto: envoy.extensions.load_balancing_policies.client_side_weighted_round_robin.v3.ClientSideWeightedRoundRobin
+// Note: SlowStart is not supported for this policy in Envoy Gateway at this time.
+type ClientSideWeightedRoundRobin struct {


in the first iteration, do we need to add all these fields, or can be rely on the defaults that are preset ?
can you share the motivation for adding these fields

So I have removed the other fields which are not currently supported by envoy, but at present I guess the fields being published are important to properly configure the system.

For example black out period can be different for different applications based on how much time they take to get the metrics state. We have some apps which does heavy data loading in the beginning and hence their blackout periods are high.

I think all of these metrics are application specific and we have in our sytems already configured / tuned them for different services. So since the implementation is already done are there any concerns of why we should not include them?

yeah as a project, we want to limit the scope of API fields to the ones majority users will configure, trying to simplify the complexity and only add it when needed

But that compromises with the configurability as it depends lot on the service being served

arkodg · 2025-12-02T02:02:08Z

api/v1alpha1/loadbalancer_types.go

 }

+// ClientSideWeightedRoundRobin defines configuration for Envoy's Client-Side Weighted Round Robin policy.
+// See Envoy proto: envoy.extensions.load_balancing_policies.client_side_weighted_round_robin.v3.ClientSideWeightedRoundRobin


tbh https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/load_balancing_policies/client_side_weighted_round_robin/v3/client_side_weighted_round_robin.proto is not descriptive enough, and doesnt mention how this needs to be instrumented in the upstream, are there other links than can be used

Yeah the docs aren't great... for example out-of-band reporting isn't supported at all yet reading the docs would indicate otherwise.

We could contribute some docs changes upstream but agreed that including a reference to how endpoint-load-metrics and endpoint-load-metrics-bin are instrumented would be useful here.

@arkodg what changes would you suggest me to add here?

This is a bit more elaborative document https://docs.cloud.google.com/load-balancing/docs/https/applb-custom-metrics although this can also be confusing and gcp specific.

do any docs exist that explain how a backend / server should be enhanced to send the appropriate trailors in the response ?
looking for a more end user facing doc for the feature ( which is described in https://www.youtube.com/watch?v=lfv_Oj1BLn0)

There is a google doc I will have to find it though but nothing stated officially as far I am aware of

we should consider adding one if it doesnt exist
cc @AndresGuedez

Ack, this is good feedback on the docs gaps.

I'll work with @efimki on this.

thanks @AndresGuedez ! Happy to be the beta tester

arkodg · 2025-12-02T02:05:36Z

api/v1alpha1/loadbalancer_types.go

 	// RoundRobinLoadBalancerType load balancer policy.
 	RoundRobinLoadBalancerType LoadBalancerType = "RoundRobin"
+	// ClientSideWeightedRoundRobinLoadBalancerType load balancer policy.
+	ClientSideWeightedRoundRobinLoadBalancerType LoadBalancerType = "ClientSideWeightedRoundRobin"


thoughts on other names involving BackendMetrics or Utilization or ORCA ?

If length is your concern ClientSideWRR is an abbreviation I've seen used in a few places.

I agree the name is confusing but it's hard to rename this as the exclusive ORCA or Utilization lb type because similar lb policies exist such as WrrLocality.

also client side doesnt convey any useful meaning to the user, we refer to downstream as clients in EG

We can change the name, also is there a discussion group or slack channel where we can disucss and close this?

…ercentage-based representation in Gateway CRDs, configurations, and tests. Adjust related documentation and ensure backward compatibility. Signed-off-by: anurag.ag <[email protected]>

…r` across gateway configurations and tests for cleaner and consistent duration handling. Remove unused `metav1` imports where applicable. Signed-off-by: anurag.ag <[email protected]>

…dRoundRobin load balancer in Gateway CRDs, configurations, and tests. Adjust validation rules and documentation accordingly. Signed-off-by: anurag.ag <[email protected]>

anuragagarwal561994 requested a review from a team as a code owner November 2, 2025 03:31

anuragagarwal561994 force-pushed the client-wrr branch from 9a814bd to 573a483 Compare November 2, 2025 03:33

jukie reviewed Nov 2, 2025

View reviewed changes

api/v1alpha1/loadbalancer_types.go Outdated Show resolved Hide resolved

jukie reviewed Nov 2, 2025

View reviewed changes

api/v1alpha1/loadbalancer_types.go Outdated Show resolved Hide resolved

zirain reviewed Nov 3, 2025

View reviewed changes

api/v1alpha1/loadbalancer_types.go Outdated Show resolved Hide resolved

anuragagarwal561994 force-pushed the client-wrr branch from 5f3d64b to ff2a353 Compare November 3, 2025 19:23

jukie reviewed Nov 3, 2025

View reviewed changes

api/v1alpha1/loadbalancer_types.go Outdated Show resolved Hide resolved

anuragagarwal561994 force-pushed the client-wrr branch from f926f90 to c7ecca3 Compare November 16, 2025 12:07

anuragagarwal561994 added 6 commits November 17, 2025 05:48

Add support for ClientSideWeightedRoundRobin load balancer policy in …

971c3f1

…Gateway CRDs, ensuring configurable parameters and validation rules are integrated. Includes e2e test for validation. Signed-off-by: anurag.ag <[email protected]>

Adds / Updates test cases for client wrr

a9d01f0

Signed-off-by: anurag.ag <[email protected]>

Fix: gen-check ci, coverage-check

63e8b2f

Signed-off-by: anurag.ag <[email protected]>

Remove enableOOBLoadReport and oobReportingPeriod fields from gat…

522534b

…eway CRDs and related configurations. Update associated test data and documentation. Signed-off-by: anurag.ag <[email protected]>

Remove enableOOBLoadReport and oobReportingPeriod fields from Cli…

4512f9d

…entSideWeightedRoundRobin configuration, update affected tests and CRDs. Signed-off-by: anurag.ag <[email protected]>

Update: Change ErrorUtilizationPenalty type from float to integer a…

dd5d285

…cross Gateway CRDs, configuration files, and related tests. Adjust documentation to reflect percentage-based representation. Signed-off-by: anurag.ag <[email protected]>

anuragagarwal561994 force-pushed the client-wrr branch from a0471e0 to dd5d285 Compare November 17, 2025 00:18

anuragagarwal561994 and others added 2 commits November 26, 2025 11:36

Merge branch 'main' into client-wrr

1101c1d

Merge branch 'main' into client-wrr

14d1d59

jukie reviewed Nov 27, 2025

View reviewed changes

internal/ir/xds.go Outdated Show resolved Hide resolved

jukie reviewed Nov 27, 2025

View reviewed changes

internal/ir/xds_test.go Outdated Show resolved Hide resolved

arkodg reviewed Dec 2, 2025

View reviewed changes

anuragagarwal561994 added 4 commits December 7, 2025 20:41

Update: Refactor ErrorUtilizationPenalty to use integer type with p…

8bce886

…ercentage-based representation in Gateway CRDs, configurations, and tests. Adjust related documentation and ensure backward compatibility. Signed-off-by: anurag.ag <[email protected]>

Merge remote-tracking branch 'upstream/main' into client-wrr

0d65b01

Refactor: Replace ptr.To(metav1.Duration) with `ir.MetaV1DurationPt…

a479e67

…r` across gateway configurations and tests for cleaner and consistent duration handling. Remove unused `metav1` imports where applicable. Signed-off-by: anurag.ag <[email protected]>

Update: Add support for SlowStart configuration for ClientSideWeighte…

777a1c1

…dRoundRobin load balancer in Gateway CRDs, configurations, and tests. Adjust validation rules and documentation accordingly. Signed-off-by: anurag.ag <[email protected]>

anuragagarwal561994 force-pushed the client-wrr branch from 686d984 to 777a1c1 Compare December 7, 2025 22:17

feat(loadbalancer): Add LoadBalancerType Client Side Weighted Round Robin #7407

Are you sure you want to change the base?

feat(loadbalancer): Add LoadBalancerType Client Side Weighted Round Robin #7407

Uh oh!

Conversation

anuragagarwal561994 commented Nov 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

anuragagarwal561994 commented Nov 2, 2025

Uh oh!

anuragagarwal561994 commented Nov 2, 2025

Uh oh!

anuragagarwal561994 commented Nov 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov bot commented Nov 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

anuragagarwal561994 commented Nov 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jukie commented Nov 13, 2025

Uh oh!

anuragagarwal561994 commented Nov 14, 2025

Uh oh!

anuragagarwal561994 commented Nov 17, 2025

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jukie commented Nov 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

anuragagarwal561994 Dec 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

anuragagarwal561994 commented Nov 2, 2025 •

edited

Loading

anuragagarwal561994 commented Nov 2, 2025 •

edited

Loading

codecov bot commented Nov 4, 2025 •

edited

Loading

anuragagarwal561994 commented Nov 4, 2025 •

edited

Loading

jukie commented Nov 27, 2025 •

edited

Loading

anuragagarwal561994 Dec 7, 2025 •

edited

Loading