Expose a particular shape type to the integrator

[mandyxmq]

Hi, I'm trying to create a small scene inside my new integrator and do tracing against it when necessary. I have looked at the suggestion in this issue #1337. I'm trying to do something similar to what is suggested there.

In my case, I would like to create a small scene using bspline curves instead of mesh. I created include/mitsuba/render/curve.h and src/render/curve.cpp, mimicking the mesh class. In this way, I can define Curve as a base type and import in my integrator.

What I didn't completely figure out is the modification for the Optix part. Right now I have a dummy bsplinecurve.cpp inheriting from the curve class. In render/optix/shapes.h, I keep BSplineCurve as a OptixShapeType. I can render scenes with bspline curves fine in this way. However, in my new integrator, if I create a Curve object and try to include it in the scene, it will complain "Unexpected shape: Curve. Couldn't be found in the 'OPTIX_SHAPE_TYPE_NAMES' table." if I want to use the cuda variant. It looks like mesh and curves are handled slightly differently in shapes.h. It wouldn't let me include Curve as a OptixShapeType. Could you please provide some suggestions on the correct way of handling this? I might be misunderstanding the structure of things here.

Thank you!!

[njroussel]

Hi @mandyxmq 😄

In retrospect, I answered that thread maybe too rapidly. The way in which the code functions is a bit different from what I thought. I've had longer look at it again now.
(@EgeCiklabakkal this might still be relevant to you).

My original point still holds: you can ray-intersect a different scene in the middle of your current megakernel. I was simply wrong about the mechanics of how this is supported.

Let me use the following terminology:

• "main scene" for the original scene
• "sub-scene" for your supplementary objects that you want to trace rays against. Note that the object in this scene do not necessarily have to match the original scene.

When creating the sub-scene, it must reference the original main scene in its Properties for it to run in the same megakernel. Or else, you will run into the original more than one OptiX pipeline was used within a single kernel issue. (In the source code, the Mesh does this here, and the Scene initialization checks for other Scene objects here)

In addition, the original scene must hold at least one instance of each "type" of geometry that the sub-scene will use. Why? Well, the sub-scene will re-use the original scene's pipeline and therefore it must already be able to handle all possible geometry types that the sub-scene contains. (This part of the source code re-uses the original pipeline).

I've created an example below, exclusively in Python. It's kind of a non-sensical example, but hopefully it illustrates how to use these nested scenes. In the example, we define a special integrator that wraps another and only returns the wrapped integrator's result if the original ray intersects some other geometry:

import mitsuba as mi
import drjit as dr

mi.set_variant('cuda_ad_rgb')

class IntegratorWrapper(mi.SamplingIntegrator):
    def __init__(self, props):
        super().__init__(props)
        self.sub_integrator = props.get('integrator')

    @dr.syntax
    def sample(self,
               scene: mi.Scene,
               sampler: mi.Sampler,
               ray: mi.Ray3f,
               medium: mi.Medium,
               active: mi.Bool
    ):
        # Get the underlying intergrator's result
        spec, mask, aovs = self.sub_integrator.sample(scene, sampler, ray, medium, active)
        # Intersect the original ray against our sub-scene
        si = self.sub_scene.ray_intersect(ray, active)
        # Only return the underlying intergrator's result if the intersection exists
        return dr.select(si.is_valid(), spec, mi.Color3f(0)), mask, aovs

mi.register_integrator("wrapper", lambda props: IntegratorWrapper(props))

integrator = mi.load_dict({
    'type': 'wrapper',
    'integrator': {
        'type': 'path',
        'max_depth': 8
    }
})

main_scene = mi.load_dict(mi.cornell_box())
sub_scene = mi.load_dict({
    'type': 'scene',
    'shape': {
        'type': 'obj',
        'filename': 'mitsuba3/resources/data/common/meshes/sphere.obj',
    },
    # Important! Reference the main scene in the sub-scene
    'original_scene': main_scene
})

# Pass the sub-scene to the integrator so we can use it in `sample`
integrator.sub_scene = sub_scene

# Render and save
mi.Bitmap(mi.render(main_scene, integrator=integrator)).write('out.exr')

Does something like this work for both of you?

(Sorry for the delay 🙇 ...)

[mandyxmq]

Thank you Nicolas for your help, as always! : )

I got an implementation in cpp to work with the LLVM variant and achieved the two-stage rendering that I wanted, but this definitely helps me understand how it should be done for the CUDA variant and run things in the same megakernel! I think this should work for my problem and I will give it a try.

[mandyxmq]

issue.zip

I tried to use it with the CUDA variant. I ran into the following error when I used 128 samples (not 64spp or less). In line 97 of integrator_wrapper.py, I perform the intersection with the sub_scene, and return the interaction si_patch. Then it is later used to evaluate the BSDF in line 130. It seems that this caused the problem when having more samples but I haven't figured out the reason yet. I attached the files to reproduce this. Please ignore the hard-coded tiling parameters for this test in the integrator :). Also, I removed the code for non-primal mode to streamline the integrator.

Critical Dr.Jit compiler failure: cuda_check(): API error 0700 (CUDA_ERROR_ILLEGAL_ADDRESS): "an illegal memory access was encountered" in /project/ext/drjit-core/src/eval.cpp:395.
Aborted (core dumped)

---

EDIT: This seems less an issue using the newest Mitsuba version. Although it still segfault at the very end, but it produces the correct image before segfault!