Request: a solution that allows us to not have to write 6500 different delegates & overloads

[lucasmeijer]

In Unity's new DOTS C# high performance game engine, we support this syntax to allow users to tell the system they want to run an data transformation on all entities that have a Position component and a Velocity component.

 EntitiesForEach
      .WithBurst(true)
      .WithSupportStructuralChanges(false)
      .WithName("ApplyVelocities")
      .WithCode((ref Position position, in Velocity velocity) =>
        {
           position.Value += velocity.Value;
        });

Our buildsystem recognizes this pattern, and does some IL postprocessing, by virtue of understanding the users intent.

Notice that the lambda expression that WithCode() takes, has one ref and one in parameter.
This makes the jobsystem that runs these transformations realize it can schedule this transormation in parallel with other readers of Velocity component, but not in parallel with other transformations that need to access Position, since that will be written to, as it's marked ref, not in.

We need WithCode() to be able to take any amount of parameters ideally, but defenitely not less than 8. We also would like to give the user full freedom to mark each parameter as ref, in, or by value. In order to let users write code like this, we today have to create custom delegates like this:

public delegate void RRR<T0, T1, T2>(ref T0 t0, ref T1 t1, ref T2 t2);
public delegate void RRI<T0, T1, T2>(ref T0 t0, ref T1 t1, in T2 t2);
public delegate void RRV<T0, T1, T2>(ref T0 t0, ref T1 t1, T2 t2);
public delegate void RIR<T0, T1, T2>(ref T0 t0, in T1 t1, ref T2 t2);
public delegate void RII<T0, T1, T2>(ref T0 t0, in T1 t1, in T2 t2);
public delegate void RIV<T0, T1, T2>(ref T0 t0, in T1 t1, T2 t2);
public delegate void RVR<T0, T1, T2>(ref T0 t0, T1 t1, ref T2 t2);
public delegate void RVI<T0, T1, T2>(ref T0 t0, T1 t1, in T2 t2);
public delegate void RVV<T0, T1, T2>(ref T0 t0, T1 t1, T2 t2);
public delegate void IRR<T0, T1, T2>(in T0 t0, ref T1 t1, ref T2 t2);
public delegate void IRI<T0, T1, T2>(in T0 t0, ref T1 t1, in T2 t2);
public delegate void IRV<T0, T1, T2>(in T0 t0, ref T1 t1, T2 t2);
public delegate void IIR<T0, T1, T2>(in T0 t0, in T1 t1, ref T2 t2);
public delegate void III<T0, T1, T2>(in T0 t0, in T1 t1, in T2 t2);
public delegate void IIV<T0, T1, T2>(in T0 t0, in T1 t1, T2 t2);
public delegate void IVR<T0, T1, T2>(in T0 t0, T1 t1, ref T2 t2);
public delegate void IVI<T0, T1, T2>(in T0 t0, T1 t1, in T2 t2);
public delegate void IVV<T0, T1, T2>(in T0 t0, T1 t1, T2 t2);

there are 6500 more where those came from. I've put them up here for reference: https://gist.github.com/lucasmeijer/e7e7c6ae10ef78adaae9b505ca615470

for each delegate, we need to make a custom overload of the WithCode() function.

It would be great if we wouldn't have to do that. It makes visualstudio slow, resharper slow, horrible for docs, etc.

One possible solution that comes to mind is having some kind of special delegate type that any lambda expression is allowed to be assigned to. It's fine for our purposes if that instances of that delegate type are not invokable. We just need the syntax to be valid.

We'd happily welcome any other ideas, from language features to anything else.

[HaloFour]

What exactly do you propose?

The CLR and C# don't know anything about lambdas, they only know about delegates, and the CLR requires separate nominal types per signature. Those types all have the common System.Delegate base type, but the runtime still requires that a concrete implementation where the Invoke method matches the signature of the target method. I'm not sure what you'd do to work around that.

Also, this proposal seems insanely narrow, in that it only applies to a specific API for a specific runtime that requires IL rewriting to function at all. I'd have to think that there are more general-purpose ways to build this based on what exists in the language.

[lucasmeijer]

The CLR and C# don't know anything about lambdas, they only know about delegates

we don't need a delegate. We just need some type that we're allowed to assign any lambda expression to, and can from IL reason about where the compiler generated function is that was made for the lambda.

[HaloFour]

@lucasmeijer

All lambdas compile to delegates, even in the case of expression trees. How would such a language feature apply to projects outside of this specific use case? Why does it have to use lambdas? This feels like using lambda syntax to accomplish pattern matching instead of using patterns.

[CyrusNajmabadi]

Our buildsystem recognizes this pattern, and does some IL postprocessing, by virtue of understanding the users intent.

How about something like this:

      .WithCode((Position ref_position, Velocity in_velocity) =>
        {
           ref_position.Value += in_velocity.Value;
        });

Your build system would throw if the user didn't properly name things. It could then transform the above however it wanted (making actual 'ref' parameters for ref_xxx named params and whatnot).

Now you'd just need the 8 overloads for the 8 types of actions you would take. You'd still get compile time checking since your tool would verify the 'name' specified how the parameter needed to be passed.

Basically, if you have an IL processing step, you can basically encode things however you want. Just use a pattern that your processing step will look for (and fail on if incorrect), and then rewrite to whatever actual form your "jobsystem" wants.

[snowinmars]

There are some restrictions for C#. You can't mark a generic attribute as 'digitable', you can't mark an operator as 'commutativity', you can't overload function by the count of generic arguments without copypasting.
I think that these restrictions keep the language simple. What could be useful in one case could be a mess in another.

I don't know your business flow or demands, but when I face with stuff like this that means that I should achieve my goal in another way.

[kekekeks]

class DynamicSignatureDelegate
{
      public DynamicSignatureDelegate(object target, MethodInfo methodInfo){}
}

if such class is used instead of a Delegate, compiler should emit

ldloc target
ldtoken Lambda
call MethodBase.GetMethodFromHandle
newobj DynamicSignatureDelegate(object target, MethodInfo method)

instead of

ldloc target
ldftn Lambda
newobj Action(object target, native int function

Being able to convert an arbitrary lambda to a MethodInfo would be also quite useful in non-Unity scenarios.

This could also play nicely with memberof proposal:

Foo(memberof int (int x, int y) => x+y);

[HaloFour]

You can also define your overloads following such a convention that ref parameters must come last, etc., to reduce the number of overall overloads. And you could reconsider the requirement that there be up to 8 arguments. Or you could use interfaces to define a "shape" of the type in question.

DOTS looks like an engine that's already in the wild. How much does your workflow depend on a feature like this being adopted?

[CyrusNajmabadi]

there are 6500 more where those came from. I've put them up here for reference: https://gist.github.com/lucasmeijer/e7e7c6ae10ef78adaae9b505ca615470

Is this actually a problem? I assume you're just generating them. And presumably your IL shaker will just remove teh ones that aren't used (of even remove all of them since you don't actually use delegates at the end of the day). So is there an actual problem that would be solved by a language feature here?

[scottbilas]

@CyrusNajmabadi the problem in the OP is "It makes visualstudio slow, resharper slow, horrible for docs, etc"

It's also insanely confusing for users who have no idea what all this stuff is and will be given a pile of weirdly acronym'd names in their autocomplete.

Anyway from a perf perspective, very few .net tools are expecting a type to have >100 of anything and having 6500 is a good way to discover all the n^2 algorithms that were invisible with low member counts.

Oh and regarding ordering of in/ref etc. to reduce overloads, we've considered this and it's not too bad. It does come with the significant cost of confusing compiler error. Someone changes a ref to an in or vice versa, and then the compiler gives a weird message about some acronym and param types not matching, and what the user is supposed to do is rearrange their parameter list. Not great UX.

What we actually want to do is let people pass in a lambda with whatever they want (lambda because the syntax is so close to an ordinary foreach), and our postprocessor would do the validation and codegen work.

[benaadams]

Something like this to work?

public void WithCode<TAction>(TAction action)
    where TAction : Delegate
{
    // ...
}

Though currently that will give the error

The type arguments for method 'WithCode<TAction>(TAction)' cannot be inferred from the usage. 
Try specifying the type arguments explicitly.	

Specifying them then returns you back to the current situation (lots of delegates); also poor usability in the generic type:

WithCode<RR<Position, Velocity>>((ref Position position, in Velocity velocity) =>
{
    position.Value += velocity.Value;
});

For

WithCode((ref Position position, in Velocity velocity) =>
{
    position.Value += velocity.Value;
});

Also concrete doesn't work

public void WithCode(Delegate action)
{
}

As it gives

Cannot convert lambda expression to type 'Delegate' because it is not a delegate type

[ogxd]

Here is an idea, it make thing much shorter but it has its drawbacks :

public delegate void WithCodeHandler<RefParameters, IntParameters>(ref RefParameters referenceParameters, in IntParameters inParameters);

public struct ParameterSet<T0> {
    public T0 t0;
}

public struct ParameterSet<T0, T1> {
    public T0 t0;
    public T1 t1;
}

public struct ParameterSet<T0, T1, T2> {
    public T0 t0;
    public T1 t1;
    public T2 t2;
}

public struct ParameterSet<T0, T1, T2, T3> {
    public T0 t0;
    public T1 t1;
    public T2 t2;
    public T3 t3;
}

public void WithCode<T0, T1>(WithCodeHandler<ParameterSet<T0>, ParameterSet<T1>> function) { }

public void WithCode<T0, T1, T2>(WithCodeHandler<ParameterSet<T0>, ParameterSet<T1, T2>> function) { }
public void WithCode<T0, T1, T2>(WithCodeHandler<ParameterSet<T0, T1>, ParameterSet<T2>> function) { }

public void WithCode<T0, T1, T2, T3>(WithCodeHandler<ParameterSet<T0>, ParameterSet<T1, T2, T3>> function) { }
public void WithCode<T0, T1, T2, T3>(WithCodeHandler<ParameterSet<T0, T1>, ParameterSet<T2, T3>> function) { }
public void WithCode<T0, T1, T2, T3>(WithCodeHandler<ParameterSet<T0, T1, T2>, ParameterSet<T3>> function) { }

public void WithCode<T0, T1, T2, T3, T4>(WithCodeHandler<ParameterSet<T0>, ParameterSet<T1, T2, T3, T4>> function) { }
public void WithCode<T0, T1, T2, T3, T4>(WithCodeHandler<ParameterSet<T0, T1>, ParameterSet<T2, T3, T4>> function) { }
public void WithCode<T0, T1, T2, T3, T4>(WithCodeHandler<ParameterSet<T0, T1, T2>, ParameterSet<T3, T4>> function) { }
public void WithCode<T0, T1, T2, T3, T4>(WithCodeHandler<ParameterSet<T0, T1, T2, T3>, ParameterSet<T4>> function) { }

public void WithCode<T0, T1, T2, T3, T4, T5>(WithCodeHandler<ParameterSet<T0, T1>, ParameterSet<T2, T3, T4, T5>> function) { }
public void WithCode<T0, T1, T2, T3, T4, T5>(WithCodeHandler<ParameterSet<T0, T1, T2>, ParameterSet<T3, T4, T5>> function) { }
public void WithCode<T0, T1, T2, T3, T4, T5>(WithCodeHandler<ParameterSet<T0, T1, T2, T3>, ParameterSet<T4, T5>> function) { }

public void WithCode<T0, T1, T2, T3, T4, T5, T6>(WithCodeHandler<ParameterSet<T0, T1, T2>, ParameterSet<T3, T4, T5, T6>> function) { }
public void WithCode<T0, T1, T2, T3, T4, T5, T6>(WithCodeHandler<ParameterSet<T0, T1, T2, T3>, ParameterSet<T4, T5, T6>> function) { }

public void WithCode<T0, T1, T2, T3, T4, T5, T6, T7>(WithCodeHandler<ParameterSet<T0, T1, T2, T3>, ParameterSet<T4, T5, T6, T7>> function) { }

And it would be used like this :

.WithCode((
    ref ParameterSet<Position, Rotation> refParameters,
    in ParameterSet<Velocity> inParameters) => {
        refParameters.t0.Value += inParameters.t0.Value;
    return;
 });

Notice I haven't put the value type, only ref and in. Is it required ? That's already a lot less combinations

You could also let the user make his own Parameter containers, that would be recognized by your build system thanks to an attribute or by inheriting from a specific abstract type (RefParameters, InParameters, ...)

[yaakov-h]

@benaadams how about WithCode(delegate (...) { ... })?

[benaadams]

@yaakov-h

WithCode(delegate(ref Position position, in Velocity velocity)
{
    position.Value += velocity.Value;
});

Gives

Cannot convert anonymous method to type 'Delegate' because it is not a delegate type

[scottbilas]

@ogxd cool idea, though "t0" etc. are hard to read. It can be improved just a little..

    .WithCode((ref (Position pos, Rotation rot) refs, in (Velocity vel) ins) =>
        {
            refs.pos.Value += ins.vel.Value;
        });

..via receiving ValueTuples as ref/in. Unfortunately there's no way to do a single-element value tuple (#883). :(

[ogxd]

@scottbilas that is a great usecase for tuples ! It even makes the overloads much cooler :)

For single-element values, I'd say that not making a tuple is fine. The user would have to know that he should use parenthesis to deal with multiple refs or ins but that's all. And there is nothing to change on the API side as the only delegate is already generic enough to handle both tuples and single elements 👍

That would look like this :

public delegate void WithCodeHandler<Refs, Ins>(ref Refs refs, in Ins ins);

public void WithCode<T0, T1>(WithCodeHandler<T0, T1> function) { }
public void WithCode<T0, T1, T2>(WithCodeHandler<T0, (T1, T2)> function) { }
public void WithCode<T0, T1, T2>(WithCodeHandler<(T0, T1), T2> function) { }
public void WithCode<T0, T1, T2, T3>(WithCodeHandler<T0, (T1, T2, T3)> function) { }
public void WithCode<T0, T1, T2, T3>(WithCodeHandler<(T0, T1), (T2, T3)> function) { }
public void WithCode<T0, T1, T2, T3>(WithCodeHandler<(T0, T1, T2), T3> function) { }
public void WithCode<T0, T1, T2, T3, T4>(WithCodeHandler<T0, (T1, T2, T3, T4)> function) { }
public void WithCode<T0, T1, T2, T3, T4>(WithCodeHandler<(T0, T1), (T2, T3, T4)> function) { }
public void WithCode<T0, T1, T2, T3, T4>(WithCodeHandler<(T0, T1, T2), (T3, T4)> function) { }
public void WithCode<T0, T1, T2, T3, T4>(WithCodeHandler<(T0, T1, T2, T3), T4> function) { }
public void WithCode<T0, T1, T2, T3, T4, T5>(WithCodeHandler<(T0, T1), (T2, T3, T4, T5)> function) { }
public void WithCode<T0, T1, T2, T3, T4, T5>(WithCodeHandler<(T0, T1, T2), (T3, T4, T5)> function) { }
public void WithCode<T0, T1, T2, T3, T4, T5>(WithCodeHandler<(T0, T1, T2, T3), (T4, T5)> function) { }
public void WithCode<T0, T1, T2, T3, T4, T5, T6>(WithCodeHandler<(T0, T1, T2), (T3, T4, T5, T6)> function) { }
public void WithCode<T0, T1, T2, T3, T4, T5, T6>(WithCodeHandler<(T0, T1, T2, T3), (T4, T5, T6)> function) { }
public void WithCode<T0, T1, T2, T3, T4, T5, T6, T7>(WithCodeHandler<(T0, T1, T2, T3), (T4, T5, T6, T7)> function) { }

.WithCode((ref Position position, in Velocity velocity) => {
    position.Value += velocity.Value;
});

.WithCode((ref (Position position, Rotation rotation) refs, in Velocity velocity) => {
    refs.position.Value += velocity.Value;
    refs.rotation.Value.y += 0.1f;
});

.WithCode((ref Position position, in (Velocity velocity, Rotation rotation) ins) => {
    position.Value += Vector3.Scale(ins.velocity.Value, ins.rotation.Value);
});

[CyrusNajmabadi]

Currently, i see these options on the table:

1. the status quo. unpalatable because of hte need to generate thousands of delegate types.
2. a system that uses types like Ref<T> and In<T>. Unpalatable because it will seem confusing to users.
3. a system that using naming conventions to inform the IL rewriter. Unpalatable because it seems too magical, and it seems really important that users actually write ref and in as actual keywords.
4. a sketch of a request to allow typed lambdas that are compiled into a form that only an IL rewriter could end up making use of. Unpalatable because of how niche it is and how it basically can't actually be used by anyone except probably unity. Also, would likely take 2 years to even get them out there.
5. source generators. Unpalatable because no clue when we'll ever get them.

Is that a fairly accurate lay of the land?

[lucasmeijer]

Thanks for all the input from everyone.

The comment from @kekekeks is exactly what I'm hoping for. #2668 (comment)

There's a wide range of alternative options kicked around (much appriciated). Most of them I find not particularly appealing:

• the "lets use tuples" approach is hard to make work, because the way the usercode that takes a ref will be invoked is that we pass in a pointer to our actual native datastore for component data. It's important that we do not have to copy a value out, let the user operate on that, and then copy it back in, since that is obviously slow, and nonobviously prevents the burst compiler from generating any SIMD code. (the usercode that is written in the WithCode() lambda gets executed in a way where we have our own forloop written around it. the combination of that forloop + the position and velocity data for all entities being stored linearly in memory, is what makes burst be able to generate such fast code for patterns like this).

• the genre of suggestions that complicates the syntax/way the user has to write the code. (requiring .Value.Value, and friends): while creative, putting burden on our users instead of on ourselves doesn't feel like an amazing tradeoff. I would prefer to deal with 6k lambdas, and maybe make some sacrifices there (like limiting the amount of elements you can use).

One "oh almost so close" scenario is limiting the amount of delegate combinations through restrictions of the "ref arguments always have to come before in arguments" kind. I think it is actually fine to require this, except that I cannot find a way to give the user a great error message when they get it wrong (which they will all the time). If I could find a way to have a overload resolution failure because the delegate the user needs doesn't exist result in an error I can write that clearly explains what the problem is, and how they need to fix it (ideally with an analyzer fixup), then I think we'd totally go for that approach.
I've been unable to find a way to do that, seems like the compiler tries all overloads, and when it doesn't find one it can convert the lambda expression into, it will use the first overload and explain why that one doesn't work.

[CyrusNajmabadi]

If I could find a way to have a overload resolution failure because the delegate the user needs doesn't exist result in an error I can write that clearly explains what the problem is, and how they need to fix it (ideally with an analyzer fixup), then I think we'd totally go for that approach.

You could always have an analyzer that provides this error message. You will likely end up with two error messages for the issue (one from Roslyn and one that you provide) and that will likely take some user education.

I've been unable to find a way to do that, seems like the compiler tries all overloads, and when it doesn't find one it can convert the lambda expression into, it will use the first overload and explain why that one doesn't work.

I don't believe this is accurate. I think the compiler has a bunch of heuristics about picking which overload it thinks is best in error scenairos. If we can well define your error scenarios we could potentially tweak the compiler heuristic such that it did things better here if it would not regress other areas.

[canton7]

(I just want to point out that there are 3279 delegates in the file you linked, not 6500 - the rest of the file is method signatures. That's either a good thing or a bad thing depending on your point of view!)

[canton7]

Part of the combinatorial explosion comes from the fact that you've got ref, in, and plain values.

Do you need both in and plain values? Since you're rewriting this anyway, you can change any ins to plain values, and vice versa. All you're after is a way to distinguish whether a parameter can be written to, and both in and plain values mean the same thing there.

If you can restrict these delegates to just ref and in, or just ref and plain values, that should help somewhat (down from 3279 delegates to 254, in your original file).

[benaadams]

Do you need both in and plain values? Since you're rewriting this anyway, you can change any ins to plain values, and vice versa.

in and by value have different semantics and exchanging them would lose the programmer intent. in has to refer to a memory location; by value can be promoted to registers. by value could involve a large copy; in is just passing a pointer. in makes the fields on a struct readonly; by value they remain mutable.

[canton7]

in and by value have different semantics and exchanging them would lose the programmer intent. in has to refer to a memory location; by value can be promoted to registers. by value could involve a large copy; in is just passing a pointer. in makes the fields on a struct readonly; by value they remain mutable.

Of course, I'm aware of this.

Reading the thread, there's no indication that the OP wants to use ref / in for their pass-by-reference semantics. Rather, they're an indication of whether this particular job modifies a particular components or not, and from that which other jobs it can be scheduled in parallel with.

I don't know what happens to that delegate once it's been rewritten. It could be that it's rewritten such that it's e.g. a method on a class, and the different components (e.g. Position and Velocity) are fields, in which case in has no meaning? It could be that it's ripped apart and becomes part of a larger method, with plain values being rewritten to in? I've got no insight into what DOTS does here.

It could also be that the only parameters to this delegate are components, and it could be that components are always structs which you would never want to pass by value. After all, if your delegate took a primitive like int, how would DOTS decide what int to give it? It might be that nobody in practice would ever use a plain value as an argument to one of these delegates.

Another possibility is that in is being treated purely as an optimization over a plain value. In this case, it might make sense for DOTS to decide what should be passed by value and what should be passed as in, based on which is going to perform better. If this is the case, then plain parameters and in convey the same programmer intent ("I won't modify this component"), and the optimization stuff is left to DOTS. In this case, you don't need both plain parameters and in - pick one or the other.

It could well be that OP has a good reason for wanting both plain parameters and in which answers all of my questions above, but I haven't yet seen it expressed in this thread, so I think it's worth asking the question.

[CodingJar]

Can enforcing a function (as opposed to a lambda) and a simple nameof() be a solution here?

        static void HandlePositionVelocity(ref Position position, in Velocity velocity)
        {
            position.Value += velocity.Value;
        }

        WithFunction( nameof(HandlePositionVelocity) );

This seems pretty clean to me. You could potentially even rewrite the IL to follow this pattern if you're worried about people being confused and using a hard-coded "Handle_Position_Velocity". With this restriction you can even mark-up your functions (HandlePositionVelocity) with attributes in the future to hint at optimizations.

[scottbilas]

@canton7 I can explain why we want n^3 combos rather than n^2

• ref: because the usercode will be directly modifying ECS memory
• in: because the usercode will be directly reading ECS memory
• (undecorated): because we also support reference types in ECS memory, and in/ref there is nonsensical/surprising/misleading.

We need to know programmer intent via in vs ref because it affects all kinds of internal mechanics. We could do this with [ReadOnly] ref foo but lambdas do not support param attrs. It's also nice to reuse existing compiler checks about field modification on in so we don't have to write our own (though obviously we still have the problem with method calls).

I would consider anything we come up with for reducing # combos to be a substandard solution. It will always come with a UX price. Also, if we provide n of something, a user will inevitably want n+1 of those, and then they will be really stuck, probably having to decompile+extend to add the extra param.

What we really want is to take an arbitrary delegate. So long as the compiler can verify that it is well-formed "yes, this is a function", we can handle it. And - crucially - we can give a really nice error message when they do something unsupported, letting them know why it's a problem and what they can do to fix it.

[HaloFour]

@scottbilas

What we really want is to take an arbitrary delegate.

You can do this today by having the method accept delegate. The consumer then has to define their own delegate type, create a lambda and assign it to a delegate variable of that type, and then pass it to the method. Klunky, yes, but that's what the language supports today.

The timeframe for adding any new features to the language is not short. Many of the bigger features slated to be released in C# 8.0 have been in the works for years. And the team is currently heads-down trying to wrap up C# 8.0, so nothing new is going to be considered at the moment. Even if this got championed immediately and slated for the very next C# release you're likely not going to see something until next year. Getting championed will likely require demonstrating use cases outside of DOTS, particularly given what you plan on doing with these delegates is so far removed from what normal developers will be able to do.

[DaZombieKiller]

You can do this today by having the method accept delegate. The consumer then has to define their own delegate type, create a lambda and assign it to a delegate variable of that type, and then pass it to the method.

Honestly I think this may be the best approach thus far. By delegating (haha) the declaration of the delegates to the user instead of the API itself, you won't need 6,500 predefined delegates. It wouldn't be hard at all to clearly document either.

This also means the user can supply attributes via the delegate, which I'm sure would be useful down the line.

[HaloFour]

Similarly you could use a generic type parameter constrained to System.Delegate. The consumer would still have to define their own delegate type and specify it in the generic type argument:

public delegate void ApplyVelocities(ref Position position, in Velocity velocity);

public class C {
    public void M() {
         EntitiesForEach
             .WithBurst(true)
             .WithSupportStructuralChanges(false)
             .WithName("ApplyVelocities") // maybe infer from the delegate name?
             .WithCode<ApplyVelocities>((ref Position position, in Velocity velocity) =>
             {
                 position.Value += velocity.Value;
             });
    }
}

That would give the consumer the flexibility to define literally any combination of parameters that they would want, generically or not, but at the price of having to define the delegate. Maybe you could offer some overloads of those predefined delegates but to a smaller arity and combination to keep the delegate count reasonable but have this overload as an escape hatch for more complicated scenarios, like more than 8 arguments.

[scalablecory]

Variadic generics could be very nice, but not used too often

[DaZombieKiller]

Now that lambdas have a natural type in C# 10, this can be done really nicely without the need to generate any delegates:

public EntitiesForEach WithCode<T>(T code)
    where T : Delegate
{
    // ...
    return this;
}

// ...
EntitiesForEach
    .WithBurst(true)
    .WithSupportStructuralChanges(false)
    .WithName("ApplyVelocities")
    .WithCode((ref Position position, in Velocity velocity) =>
    {
        position.Value += velocity.Value;
    });

The C# compiler will generate a delegate type automatically, and the Burst compiler can then query it from the IL.

[Florian-Voss]

Excuse me if this suggestion is bad, I don't have much experience. correct me if I get anything wrong.

As far as I understand you have a lot of overloads of different delegates because you wanna accept different arguments and you wanna reduce the amount of overloads.

Couldn't you create a Delegate that accepts any arguments? Then you only have to overload it for the different return types but no overloads for anything param/argument related anymore:

delegate void MyVoidMethod(params object[] args);
delegate TReturn1 MyReturn1Method(params object[] args);
delegate TReturn2 MyReturn2Method(params object[] args);

or make this thing return a Generic type, potentially one that all the returntypes have in common as their base?
I never tried generic constraints on a delegate i hope its possible and i get the syntax right:

delegate TReturn MyMethod(params object[] args) where TReturn : ReturnType;

if you don't want all to have ReturnType as their base you could use the object type as return too, every type ultimatively inherits from object in c#.

Another idea that came to my mind is constructing the lambda at runtime, there are different types that represent lambdas, for example LambdaExpression and there is a Compile() method that returns a Delegate. I think compiling it at runtime might be a good idea. This way you have the highest possible flexibility and the client of your stuff is completely free what to do with it.

I think the moment that you decide for a Func<> / Action<> / Predicate<> type you are already too limited and narrowed in what you can do with it regarding the method signature of the delegate or lambda for your usecase which is how you kind of forced yourself unnecessarily to create these 6500 thingies.

I dont know if any of those ideas is useful but I am confident there is a clever workaround for those 6500 overloads within the language without any new language feature.

[yaakov-h]

This doesn't work for argument modifiers such as ref or in, and currently requires additional memory allocations (the array plus boxing of any value types).

[Florian-Voss]

what about my last idea of using the Compile() method? If not that then there has to be an other way to work around creating thousands of Action<> Func<> and Predicate<> delegates.

[yaakov-h]

I don't see how that solves anything, runtime is far too late to create type definitions. One could write a source generator, but otherwise I don't believe there is any workaround for this use case that has not been already specified above.