Why are the shapes produced by dr.cuda.ArrayXf and dr.cuda.ad.ArrayXf sometimes different?

[maxfrei750]

Hello again!

This code

import drjit as dr
import mitsuba as mi

mi.set_variant("cuda_ad_rgb")

a = [1, 2, 3]

print(dr.shape(dr.cuda.ArrayXf(a)))
print(dr.shape(dr.cuda.ad.ArrayXf(a)))
print(dr.shape(dr.cuda.ArrayXf(mi.Float(a))))
print(dr.shape(dr.cuda.ad.ArrayXf(mi.Float(a))))

produces

[3, 1]
[3, 1]
[3, 1]
[3, 3]

Why is that? Thanks!

[njroussel]

Hi again :)

This looks like a bug. I'm confused as to why it could happen.
dr.cuda.ad.ArrayXf(mi.Float(a)) seems to broadcast a 3 times.

---

Maybe it would help to define what ArrayXf is/should be used for.

A dr.*.Float can be an array of any size, it's dynamic. A dr.*.Array3f is meant to be an array of exactly 3 dr.*.Float values. So, it's a static array (size 3) of dynamic arrays.

So you can write something like this:

a = dr.cuda.Float([1])
b = dr.cuda.Float([1, 2])
c = dr.cuda.Float([1, 2, 3])

arr = dr.cuda.Array3f(a, b, c)

This is entirely legal, although you're most likely looking for trouble.

Now, dr.*.ArrayXf is the same as a dr.*.Array3f except that it is not static but dynamic. It's a dynamic array of dynamic arrays. It's a container for however many dr.*.Float objects you want. You can think of it as a list[dr.*.Float] (Python list). I would say that this is rarely something you need, and can often be substituted for other types.
One option is to just use a dr.*.Float because you essentially want a dynamic container. There's no need for the extra level.
The other option is to use a dr.*.TensorXf because it's also a dynamic array with some metadata about the tensor's shape.

[maxfrei750]

Before I forget: the LLVM backend seems to be affected by the bug as well.

Thanks for the explanation, which makes perfect sense. Luckily, that's close to what I was thinking.

I would say that this is rarely something you need, and can often be substituted for other types.

My initial motivation was to make this code run, without having to rely on drjit data types, so that users may choose their preferred backend.

import drjit as dr
import mitsuba as mi

mi.set_variant("cuda_ad_rgb")

def test(a, b):
    print(dr.shape(b))
    print(dr.dot(a, b))

# I receive something similar by calling `mi.Vector3f(...).sh_eval_()`.
# So there is not much I can do about the format. There are more entries, and they are
# longer.
a = [mi.Float(0, 0, 0), mi.Float([1, 2, 3]), mi.Float([4, 5, 6])]

# Works, since ArrayXf (erroneously?) broadcasts to 3x3
b = dr.cuda.ad.ArrayXf(mi.Float([1, 2, 3]))
test(a, b)

# Doesn't work, although the shape is 3x3.
test(a, mi.TensorXf([3] * 9, shape=(3, 3)))

I hope that I'm not missing something obvious. 😅