[Proposal] Make tensors a first class feature.

[elephantpanda]

Currently whenever you want to do some sort of GPU programming such as dealing with neural networks etc. you have to use a tensor library. e.g. this one. The library is not always the same or compatible with other libraries.

My proposal is to make tensor a first class feature. Since the core idea is to do operations on giant multi-dimensional arrays on the GPU.

If it is a first class feature, then different AI languages or graphics libraries will all be compatible. And the C# language would make sure to deal with all the error handling to better make it work on different GPUs.

It would make it easier to define shaders too as operations on tensors instead of using a different shader language.

I am not sure on the details but basically something that can do the basics of Tensor etc. Then we would put things like Conv2D in a standard library built on top of this first class feature.

I don't know if it would benefit from some new notation like tensor float x

[HaloFour]

Do you have any specific proposals as to how this would look in C#? IIRC it's currently implemented entirely in the runtime and the C# language is entirely agnostic. Would there actually be new syntax or just analysis to ensure that you're using the libraries correctly?

[elephantpanda]

I'm not 100% sure but I would imagine you could maybe extend the notation of multi-dimensional arrays:

tensor float[_,640,640,3] x

But these are symbolic tensors in the sense that they aren't created in memory straight away but you could build up expressions with them like:

tensor y = tanh(5*Conv2D(x,[3,3]))

Pretty much how you make LINQ expressions.

Then just like LINQ you have a choice of how to execute the expressions. e.g. on GPU, CPU or something else. For example a tensor expression could be interpreted as a shader or a neural network.

These are just my initial thoughts.

Alternatively there might be an immediate mode where you might declare

[GPU]
tensor float16[1024,1024,64];

which creates the tensor on the device straight away, perhaps.

[CyrusNajmabadi]

Why would a language construct (tensor) be better than a library construct Tensor?

[elephantpanda]

Because it would be integrated into the language. And then not every 3rd party making incompatible Tensor libraries.

Also there's probably optimisations the compiler can do knowing things are tensors that it can't do with a library construct.

It's just an idea

[333fred]

None of that is true though. There are tensors built into .NET, which is what such a syntax would have to leverage anyways. Roslyn is also not an optimizing compiler, and wouldn't do anything there: it's the JIT that would do optimizations, which it already does today, particularly for the built-in System.Numerics.Tensor types. Without a concrete proposal here, there's nothing really more to say.

[elephantpanda]

OK. I don't really care. It's just an idea. You can delete this proposal if you like. I don't know how to.