Referenced BSDF parameters get merged with shape in a hard-to-predict way

[leroyvn]

Summary

Referenced BSDF parameters seem to be accessed as nested parameters held by the first shape (in alphabetical order) that references them.

System configuration

System information:

  OS: 13.1
  CPU: b'Intel(R) Core(TM) i7-8700B CPU @ 3.20GHz'
  Python: 3.8.13 | packaged by conda-forge | (default, Mar 25 2022, 06:05:47) [Clang 12.0.1 ]
  LLVM: 15.0.7

  Dr.Jit: 0.4.0
  Mitsuba: 3.2.0
     Is custom build? True
     Compiled with: AppleClang 14.0.0.14000029
     Variants:
        scalar_mono
        scalar_mono_double
        scalar_rgb
        scalar_spectral
        llvm_mono_double
        llvm_ad_rgb

Description

Consider the following script:

import mitsuba as mi

mi_scene = mi.load_dict(
    {
        "type": "scene",
        "some_bsdf": {"type": "diffuse"},
        "rectangle": {
            "type": "rectangle",
            "bsdf": {
                "type": "ref",
                "id": "some_bsdf",
            },
        },
        "disk": {
            "type": "disk",
            "bsdf": {
                "type": "ref",
                "id": "some_bsdf",
            },
        },
    }
)
print(mi.traverse(mi_scene))

I get the following output (I'm using a monochrome variant):

SceneParameters[
  ----------------------------------------------------------------------------------
  Name                           Flags    Type  Parent
  ----------------------------------------------------------------------------------
  disk.bsdf.reflectance.value    ∂        float UniformSpectrum
  disk.to_world                  ∂, D     Transform4f Disk
  rectangle.to_world             ∂, D     Transform4f Rectangle
]

The BSDF's reflectance value is accessed by disk.bsdf.reflectance.value. This wouldn't really be a problem if parameter paths would be always visually inferred by a human; but in my case, the dictionary and associated parameter paths are generated together by an algorithm. I could maybe get away with a workaround consisting in first generating the dictionary, detecting the shape specifications in it, sorting their keys and picking the first, but this seems overcomplicated and fragile.

Things become even more confusing when the referencing shape is nested in an instanced shapegroup:

import mitsuba as mi

mi_scene = mi.load_dict(
    {
        "type": "scene",
        "some_bsdf": {"type": "diffuse"},
        "disk": {
            "type": "disk",
            "bsdf": {
                "type": "ref",
                "id": "some_bsdf",
            },
        },
        "a_group": {
            "type": "shapegroup",
            "rectangle": {
                "type": "rectangle",
                "bsdf": {
                    "type": "ref",
                    "id": "some_bsdf",
                },
            },
            "sphere": {
                "type": "sphere",
                "bsdf": {
                    "type": "ref",
                    "id": "some_bsdf",
                },
            },
        },
        "an_instance": {
            "type": "instance",
            "the_group": {"type": "ref", "id": "a_group"},
        },
    }
)
print(mi.traverse(mi_scene))

which produces the output

SceneParameters[
  -----------------------------------------------------------------------------------------------
  Name                                        Flags    Type  Parent
  -----------------------------------------------------------------------------------------------
  a_group.rectangle.bsdf.reflectance.value    ∂        float UniformSpectrum
  a_group.rectangle.to_world                  ∂, D     Transform4f Rectangle
  a_group.sphere.to_world                     ∂, D     Transform4f Sphere
  an_instance.to_world                        ∂, D     Transform4f Instance
  disk.to_world                               ∂, D     Transform4f Disk
]

Would it be possible to reference top-level BSDF parameters by their ID? For instance, in this case: some_bsdf.reflectance.value.

[njroussel]

This is an interesting problem. We've had a few similar questions already, I'll give a longer explanation here.

There is one main problem currently: the referencing with the ref type is only used during the parsing phase of the object construction. Once the object is constructed the "top-level"-nature of the object is lost. For example, in your shapegroup example, the some_bsdf object is obviously not a child of the scene once the scene is built. The traverse mechanism only woks on instantiated objects, so the whole referencing mechanism/information is lost at that point.
In short, references are only used in the object description and are almost completely lost once the object is built which is why it cannot be shown in mi.traverse().

Fundamentally, this boils down to how we want to structure any exposed parameters:

• Like the object description, which is what the user inputs
• Like the object's internal representation, which is how Mitsuba "sees" it internally.
  These are two different things and there are arguments for both.

---

Now, as you mentioned in your thread there are workarounds. I think the more robust way of programmatically "mapping" parameters to the description is to mi.traverse() individual objects rather than the whole scene. For example, if you're looking for the BSDF of a specific shape you could do this:

import mitsuba as mi
mi.set_variant('scalar_rgb')

scene = mi.load_dict(
    {
        "type": "scene",
        "some_bsdf": {"type": "diffuse"},
        "rectangle": {
            "type": "rectangle",
            "bsdf": {
                "type": "ref",
                "id": "some_bsdf",
            },
        },
        "disk": {
            "type": "disk",
            "bsdf": {
                "type": "ref",
                "id": "some_bsdf",
            },
        },
    }
)
print(mi.traverse(scene)) # Rectangle does not have `bsdf`

key = ['rectangle', 'some_bsdf'] # The keys in the scene description
index = 0
for shape in scene.shapes():
    if shape.id() == key[index]: # The id() matches the object's key in the description
        index += 1
        bsdf = shape.bsdf()
        if bsdf.id() == key[index]: # Not necessary (only 1 BSDF per shape) but can serve as an assert
            print(mi.traverse(bsdf)) # Will only show `relfectance.value`

        index -= 1

This is not super generic, it still requires you to know that the scene structure is scene->shape->bsdf. But it's fairly programmatic and robust w.r.t. to the ordering in your scene description. I've used variations of this successfully a few times.

[leroyvn]

Thanks @njroussel for this explanation. I think I can work out a lookup protocol to recover parameter IDs thanks to your suggestion.

By the way, I'm now realising that this thread should rather have been a discussion than an issue, sorry for posting it on the wrong channel.

[leroyvn]

I have another question now: I'd like my lookup to be able to go through shape groups as well, is that possible?

[leroyvn]

I think I found a way: I modified mi.traverse() and the associated infrastructure to inject additional node search actions during scene tree traversal. This way, the traversal works exactly as for the original mi.traverse(): node order and names are identical.

[njroussel]

Great!

FWIW, mi.traverse() is just making use of the TraversalCallback mechanism used by all traverse() function definitions in plugins. So you can totally do custom things by implementing your own mi.traverse().