Extracting Albedo of Occluded Mesh via AOV Integrator

[vinlo]

Hello,

I created a simple toy project to demonstrate the issue.

The scene contains two quads, each with a different texture. One of the quads is slightly offset toward the camera. In the final render, the front quad uses a Mask BSDF to control its alpha.

When extracting the albedo using the aov integrator, I only get the texture from the front-most quad, regardless of its transparency.

However, I would like to pass the albedo of the quad behind to Optix for denoising (not included in the toy project).

Question
Is there a way to extract the albedo of a specific mesh (e.g., the quad behind) using the AOV integrator, or to retrieve an albedo pass that respects transparency?

Additional info

• os: ubuntu
• mitsuba version: v3.6.4
• mitsuba variant: cuda_spectral

[njroussel]

Hi @vinlo

I don't think there's a way to do this out-of-the box. But it's a fairly simple change in aov: check if it the BSDF has a BSDFGFlags::Null flag, and if it does continue tracing the same ray until you hit a non-Null surface. Another approach would be to write a tiny aov like integrator in Python that only does that.

[vinlo]

@njroussel thanks for the answer!

I followed your suggestion and used aov.cpp as a base to experiment with continuing the ray traversal:

• Method 1: spawn a new ray if the BSDF has the BSDF::Null flag.
• Method 2: spawn a new ray If the BSDF has an opacity attribute.

With both approaches, after compiling and running the toy project (with a small tweak so the front quad doesn’t fully cover the one behind), the albedo pass only shows the front quad’s texture, which is the opposite of what I was aiming for.

I'm still fairly new to the codebase and the concepts involved—could you confirm if I'm on the right track with the code below?

// modifying aov.cpp / std::pair<Spectrum, Mask> sample(...) function

std::pair<Spectrum, Mask> result{ 0.f, false };

// SurfaceInteraction3f si = scene->ray_intersect(ray, (uint32_t)cur_ray RayFlags::All, true,  active);
// dr::masked(si, !si.is_valid()) = dr::zeros<SurfaceInteraction3f>();

RayDifferential3f cur_ray = ray;
SurfaceInteraction3f si;
BSDFPtr m_bsdf;

for (int iter = 0; iter < 5; ++iter) {
    si = scene->ray_intersect(cur_ray, (uint32_t) RayFlags::All, true, active);
    dr::masked(si, !si.is_valid()) = dr::zeros<SurfaceInteraction3f>();
    m_bsdf = si.bsdf(cur_ray);

    // 1st method
    Mask is_transparent = has_flag(m_bsdf->flags(), BSDFFlags::Null);
    if (dr::any(is_transparent)) {

        cur_ray             = si.spawn_ray(cur_ray.d);
        cur_ray.wavelengths = ray.wavelengths;
        cur_ray.time        = ray.time;
         continue;
    }

    // 2nd method
    // std::string attr = "opacity";
    // if (dr::any(m_bsdf->has_attribute(attr)) ) {
         // cur_ray             = si.spawn_ray(cur_ray.d);
         // cur_ray.wavelengths = ray.wavelengths;
         // cur_ray.time        = ray.time;
        // continue;
    // }

    break;
}
// m_bsdf and the cur_ray variables are then used in the `switch` case
// where the RGB values for the albedo are written.

[vinlo]

Following up on the discussion, in case it’s useful to someone, the final piece of code I ended up with doesn’t fully answer the original question, but it goes a step further for my specific use case. It handles transparency by accumulating the albedo color at each intersection.

// modified version of aov.cpp
SurfaceInteraction3f si;
BSDFPtr bsdf;
Color3f final_albedo(0.f);
Float throughput          = 1.f;
RayDifferential3f cur_ray = ray;
int max_iter              = 5;
std::string attr          = "opacity";

// If the intersected surface has a bsdf with an opacity attribute:
//      - Accumulate the albedo contribution based on transparency
//      - Spawn a new ray to continue sampling deeper layers
for (int iter = 0; iter < max_iter; ++iter) {
    si = scene->ray_intersect(cur_ray, (uint32_t) RayFlags::All, true,
                              active);
    dr::masked(si, !si.is_valid()) = dr::zeros<SurfaceInteraction3f>();

    if (!dr::any(si.is_valid())) {
        break;
    }

    bsdf                   = si.bsdf(cur_ray);
    Spectrum spec          = bsdf->eval_diffuse_reflectance(si, active);
    Color3f current_albedo = spectrum_to_color3f(spec, cur_ray, active);

    auto attr_mask = bsdf->has_attribute(attr, active);
    Float opacity  = dr::select(
        attr_mask, bsdf->eval_attribute_1(attr, si, active), 1.f);

    final_albedo += throughput * opacity * current_albedo;
    throughput *= (1.f - opacity);

    if (dr::all(throughput < 1e-3f)) {
        break;
    }

    cur_ray             = si.spawn_ray(cur_ray.d);
    cur_ray.wavelengths = ray.wavelengths;
    cur_ray.time        = ray.time;
}