Language Proposal: params dictionary

[hez2010]

params dictionary

Motivation

While working with TorchSharp, I found I have to use params object[] vars to support variadic arguments as input. For example:

public override torch.Tensor forward(torch.autograd.AutogradContext ctx, params object?[] vars)
{
    var (A, B) = (vars[0] as torch.Tensor, vars[1] as torch.Tensor);
    ArgumentNullException.ThrowIfNull(A, nameof(A));
    ArgumentNullException.ThrowIfNull(B, nameof(B));
    var output = (vars.Length > 2 ? vars[2] : null) as torch.Tensor?;
    var bias = (vars.Length > 3 ? vars[3] : null) as torch.Tensor?;
    var quant_state= (vars.Length > 4 ? vars[4] : null) as F.QuantState?;
    // use A, B, output, bias, quant_state here
}

var f = ...;
f.forward(ctx, A, B, null, bias, null); // User has to insert redundant null before bias as `bias` is forced to be placed at `vars[3]` in the method

The C# way has several issues:

• You have to enforce the order and the ordinal in both the definition and the call site
• You cannot have nominal optional arguments
• It's extremely hard to use as you need to check the length everywhere

While in Python you can just write it like:

def forward(ctx, A, B, output=None, bias=None, quant_state: Optional[F.QuantState] = None):
    # use A, B, output, bias and quant_state here

Apparently params collection is really hard to work with in AL/ML scenarios, but we need params here so that user can pass variadic arguments to the method.

Proposal

To address this, I proposed C# to support params dictionary, like params Dictionary<string, T>, params ConcurrentDictionary<string, T> and params IDictionary<string, T>..., where the key stands for the argument name, and the value is the value of argument. And this also perfectly falls into the category of params collection.

With params dictionary, the above code will become:

public override torch.Tensor forward(torch.autograd.AutogradContext ctx, params Dictionary<string, object?> vars)
{
    var A = vars["A"] as torch.Tensor; // this will throw immediately if there's no argument named 'A', which is expected
    var B = vars["B"] as torch.Tensor;
    vars.TryGetValue("output", out var output); // we can use TryGetValue to fetch an optional argument
    vars.TryGetValue("bias", out var bias);
    vars.TryGetValue("quant_state", out var quant_state);
    // use A, B, output, bias, quant_state here
}

var f = ...;
f.forward(ctx, A: A, B: B, bias: bias);

This eliminates the necessity of inserting null just to keeping the ordinal of arguments, and allows you to pass arguments in a optional nominal arguments way.

Alternative

Dictionary with dictionary collection expression

It will still be better to have params dictionary as you don't have to type the redundant [ ] or the quote symbols around the key

Params tuple

We use a params (string Key, T Value)[] args, which is the existing concept in today's C#, but we can allow omitting the braces and using like named arguments at call sites:

public override torch.Tensor forward(torch.autograd.AutogradContext ctx, params (string Key, object? Value)[] args)
{
    var vars = args.ToDictionary(a => a.Key, a => a.Value);
    var A = vars["A"] as torch.Tensor; // this will throw immediately if there's no argument named 'A', which is expected
    var B = vars["B"] as torch.Tensor;
    vars.TryGetValue("output", out var output); // we can use TryGetValue to fetch an optional argument
    vars.TryGetValue("bias", out var bias);
    vars.TryGetValue("quant_state", out var quant_state);
    // use A, B, output, bias, quant_state here
}

var f = ...;
f.forward(ctx, A: A, B: B, bias: bias); // instead of f.forward(ctx, ("A", A), ("B", B), ("bias", bias)); 

[333fred]

Apparently params collection is really hard to work with in AL/ML scenarios, but we need params here so that user can pass variadic arguments to the method.

I'm not understanding why you can't just use a regular signature. You have an expected order, and expected types. Seems like normal optional parameters would be just fine.

[hez2010]

I'm not understanding why you can't just use a regular signature.

Because in this case you need to define the forward method in an interface to allow different implementations, each of those implementations comes with different parameters.

I updated the proposal to include a new params tuple alternative.

[333fred]

This feels like trying to force an exact replica of the python api into C#. In interop scenarios, we encourage wrapping the native api into something that's more native to C#. Consider taking that approach, rather than making a broad, unspecific interface that can only take objects.

[colejohnson66]

Why can't you use optional parameters?

public torch.Tensor forward(
    torch.autograd.AutogradContext ctx,
    torch.Tensor a,
    torch.Tensor b,
    torch.Tensor? output = null,
    torch.Tensor? bias = null,
    F.QuantState? quant_state = null)

forward(ctx, a, b, bias: bias);

[hez2010]

The only known parameter of forward you know when you design the forward method for the abstract class/interface is torch.autograd.AutogradContext ctx, and all other parameters are up to the implementations, so you cannot just write those parameters as optional parameters in the method signature.

[AndersKehlet]

I think it could be helpful with a full (though simplified) example. Why is an interface required?

[colejohnson66]

Wait. What's the purpose of an interface if downstream implementers have their own, extra, parameters? That defeats the entire purpose of programming against an interface — to abstract away the implementation. If you necessarily have to know the implementation, why program against the interface at all?

[hez2010]

In ML all networks are set of components that implementing nn.Module and it has a forward and backward fucntions. With this you can pipe several networks easily. Say you have a network that has three modules A, B and C:

A(x, y, z) -> (u, v)
B(u, v) -> (a, b, c, d, e, f)
C(a, b, c, d, e, f) -> output

Then you can pipe those modules into a network network = pipe(A, B, C), then you can get the output by passing params calling A, without need of handling any intermediate result: network([1,2,3], [4,5,6], [7,8,9]) // returns the final output

It also supports backward to propagate gradients, which behaves like an inverse operation of network.

That's why all those modules are implementing a common interface.

[NN---]

Can you explain where it is better than creating dictionary ?

forward(new(){ // new Dictionary<string, object>
{“a”, 1},
{“b”, 2},
})

[CyrusNajmabadi]

f.forward(ctx, A: A, B: B, bias: bias);

You will be able to write this as the following in C#14:

f.forward(ctx, ["A": A, "B": B, "bias": bias]);

This is succinct and clear.