How to properly use "weighted_target_experimental" load balancing policy?

[davidecaroselli]

Hi gRPC Java community!

I'm trying to configure my gRPC Java application to use weighted_target_experimental load balancing policy, however I cannot find anywhere how to specify the targetName for a given EquivalentAddressGroup. Here's my setup, and what I'm trying to achieve.

I have a custom NameResolver implementation, with the core update policy right now looking like this:

private void refreshInternal() throws Throwable {
    List<EquivalentAddressGroup> addresses = new ArrayList<>();

    for (GrpcNode server : registry.nodes()) {
        EquivalentAddressGroup addressGroup = new EquivalentAddressGroup(server.addr());
        addresses.add(addressGroup);
    }

    ResolutionResult result = ResolutionResult.newBuilder()
        .setAddressesOrError(StatusOr.fromValue(addresses))
        .build();

    this.listener2.onResult(result);
}

So, for what I have understand, the first thing to do is provide a configuration for WeightedTargetLoadBalancerProvider in ResolutionResult like this:

String config = """
    {
        "loadBalancingConfig": [{
            "weighted_target_experimental": {
                "targets": {
                    "target_a": { "weight": 75, "childPolicy": [{ "round_robin": {} }] },
                    "target_b": { "weight": 25, "childPolicy": [{ "round_robin": {} }] }
                }
            }
        }]
    }
    """;

Map<?, ?> parsedConfig = new Gson().fromJson(config, Map.class);

ResolutionResult result = ResolutionResult.newBuilder()
    .setServiceConfig(ConfigOrError.fromConfig(parsedConfig))
    .setAddressesOrError(StatusOr.fromValue(addresses))
    .build();

But now, my question: as far as I understand, I'm supposed to specify the target name (target_a or target_b) as an attribute to EquivalentAddressGroup, so something like this:

EquivalentAddressGroup addressGroup = new EquivalentAddressGroup(server.addr(),
    Attributes.newBuilder()
        .set(/* ... attribute key ... */, "target_a")
        .build());

However, I cannot find any documentation on how to properly do this last bit.

By looking at the source code of WeightedTargetLoadBalancer it looks like the addresses are filtered by an AddressFilter.filter() which, internally, gets the target name from the EquivalentAddressGroup like this:
source code

PathChain pathChain = address.getAttributes().get(PATH_CHAIN_KEY);

However, neither PATH_CHAIN_KEY, PathChain class are accessible as they are private or package-local.
So what's the correct way to attach a "target" to an EquivalentAddressGroup?

[kannanjgithub]

As you are directly constructing the weighted target policy, there is no need to set a target for each EAG, that sort of an info is only needed by the WrrLocalityLoadBalancer when it constructs the weighted target policy from the weights specified for endpoints for a locality.
You observed correctly that the weighted target LB policy code only uses path filter to filter addresses for each target. There is a method AddressFilter.setPathFilter to create an EAG with the path filter specified, but it has package private access (within io.grpc.xds). @ejona86 should we make it public?

[davidecaroselli]

Hi @kannanjgithub

As you are directly constructing the weighted target policy, there is no need to set a target for each EAG

Not sure I'm following you there: if I don't specify the target for each EquivalentAddressGroup, how can the WeightedTargetLoadBalancer know the "bucket" for every EAG? Or am I reading wrong the logic at acceptResolvedAddressesInternal() ?

[davidecaroselli]

FYI I created a simple PR following @kannanjgithub suggestion: #12267

[davidecaroselli]

I'm also facing several additional issues trying to make this work:

ResolutionResult.setServiceConfig() raises error

When I try to do this:

String config = """
    {
        "loadBalancingConfig": [{
            "weighted_target_experimental": {
                "targets": {
                    "target_a": { "weight": 75, "childPolicy": [{ "round_robin": {} }] },
                    "target_b": { "weight": 25, "childPolicy": [{ "round_robin": {} }] }
                }
            }
        }]
    }
    """;

Map<?, ?> parsedConfig = new Gson().fromJson(config, Map.class);

ResolutionResult result = ResolutionResult.newBuilder()
    .setServiceConfig(ConfigOrError.fromConfig(parsedConfig))
    .setAddressesOrError(StatusOr.fromValue(addresses))
    .build();

I get a ClassCastException as class com.google.gson.internal.LinkedTreeMap cannot be cast to class io.grpc.internal.ManagedChannelServiceConfig.

So looks like I'm supposed to provide a "parsed" configuration, but I could not find any documentation online about how to properly pass this. All resources report that it should be possible pass a JSON config (i.e. here: grpc.io/docs), but it fails.

Moreover, ManagedChannelServiceConfig is a package-local class, so no luck in trying to build it directly.

For now, I might have found a workaround which does not solve the issue, but at least allows me to keep on going with the debugging: I pass the configuration via ManagedChannelBuilder.defaultServiceConfig() like this:

String config = """
    {
        "load_balancing_config": [{
            "weighted_target_experimental": {
                "targets": {
                    "target_a": { "weight": 75, "childPolicy": [{ "round_robin": {} }] },
                    "target_b": { "weight": 25, "childPolicy": [{ "round_robin": {} }] }
                }
            }
        }]
    }
    """;

ManagedChannelBuilder
    .defaultLoadBalancingPolicy("weighted_target_experimental")
    .defaultServiceConfig(new Gson().fromJson(config, Map.class))

Missing weighted_target lb config

Now, the configuration seems to pass the first step, however, I face a java.lang.NullPointerException: missing weighted_target lb config raised at this specific line of code: WeightedTargetLoadBalancer.acceptResolvedAddressesInternal().

I'm stuck here, as even if I try to change the JSON key from weighted_target_experimental to weighted_target I keep hitting the same error.

Is there a detailed guide that explains how to properly configure this setup? Could I be overlooking some key documentation that clarifies why everything seems intentionally blocked from outside access?

[kannanjgithub]

Not sure I'm following you there: if I don't specify the target for each EquivalentAddressGroup, how can the WeightedTargetLoadBalancer know the "bucket" for every EAG? Or am I reading wrong the logic at acceptResolvedAddressesInternal() ?

You do need to specify the target a a key in the policy. In the EAG, the grouping under a target is done by prefixing the address with the key and specifying it in the PATH_CHAIN_KEY. I was only saying that there is no separate key in EAG just for the target.

[davidecaroselli]

Update:

I ended up creating my custom policy taking inspiration from WeightedTargetLoadBalancer that works like this:

1. Every EquivalentAddressGroup defines (optionally) a weight via attribute.
2. The policy groups all the addresses with the same weight together.
3. The weight of each group is the "nominal weight" x "number of entries in the group" (so that the global distribution follows the cardinality of each group)
4. Inside a group, the balancing policy is round-robin.

I have tested this policy and we are ready to deploy it in production. The core logic is basically this one:

        // Create targets map based on resolved addresses' weights
        Map<Long, List<EquivalentAddressGroup>> addressesByWeight = new HashMap<>();
        for (EquivalentAddressGroup address : resolvedAddresses.getAddresses()) {
            Integer weight = address.getAttributes().get(WEIGHT);
            if (weight == null)
                weight = config.defaultWeight;

            addressesByWeight.computeIfAbsent(weight.longValue(), k -> new ArrayList<>()).add(address);
        }

        Map<String, Long> newGroups = new HashMap<>();
        Map<String, List<EquivalentAddressGroup>> targets = new HashMap<>();
        for (Map.Entry<Long, List<EquivalentAddressGroup>> entry : addressesByWeight.entrySet()) {
            List<EquivalentAddressGroup> addresses = entry.getValue();
            long weight = entry.getKey();

            String group = "group-" + weight;
            targets.put(group, addresses);
            newGroups.put(group, weight * (long) addresses.size());

            if (!groups.containsKey(group)) {
                ChildHelper childHelper = new ChildHelper(group);
                GracefulSwitchLoadBalancer childBalancer = new GracefulSwitchLoadBalancer(childHelper);
                childHelpers.put(group, childHelper);
                childBalancers.put(group, childBalancer);
            }
        }

        groups = newGroups;

I called this policy WeightedGroupLoadBalancer. Do you think it might worth a PR, or is it too specific to my use case to be generally available to everyone?