Optimizing height of rectangular emitter light

[gioarma]

Hello, I'm trying to do a test optimization which I thought would be rather simple, but I'm having some troubles.
I define a simple scene with the suzanne monkey ply, a rectangle shape with area Emitter property, and a perspective camera looking at the monkey. I'm using the prb integrator. In the details at the bottom you can find the full scene dictionary.

What I want to achieve

• Take a reference image with the emitter at a certain height above the monkey
• Modify the height of the emitter (Initial image)
• Optimize for the emitter height to obtain a final image that is as close as possible to the reference image

My current code

Reference image

import mitsuba as mi
import drjit as dr
import numpy as np
mi.set_variant('cuda_ad_rgb')
from mitsuba.scalar_rgb import Transform4f as T
import matplotlib.pyplot as plt

ref_scene = mi.load_dict(ref_dict)  # ref_dict is the dictionary in the details below
ref_params = mi.traverse(ref_scene)
# I set the emitter at an height of 50
ref_params['area_emitter.to_world'] = mi.Transform4f.translate([0,0,50]).rotate([1,0,0], 180).scale(50)
ref_params['area_emitter.emitter.radiance.value'] = 2
ref_params.update()

ref_img = mi.render(ref_scene, spp=50)

Initial image

# I initialize the scene as the reference one
scene = mi.load_dict(ref_dict)
params = mi.traverse(scene)
# I modify the height
init_height = 200.
params['area_emitter.to_world'] = mi.Transform4f.translate([0,0,init_height]).rotate([1,0,0], 180).scale(50)
params.update();

Define the optimizer, the update function, and the objective function

For this I followed the Object pose estimation tutorial. Since I just want to optimize the height I'm redefining the to_world parameter with a Transform4f.translate function that takes the optimized height as the "z" parameter.

opt = mi.ad.Adam(lr=0.05)
opt['emitter_height'] = mi.Float(init_height)

def update_params(params, opt):
    params['area_emitter.to_world'] =  mi.Transform4f.translate([0.,0., opt['emitter_height']]).rotate([1,0,0], 180).scale(50)
    params.update()
def loss(image, image_ref):
    return dr.sum(dr.sqr(image - image_ref)) / len(image)

Optimization loop

iterations = 10
spp = 10
loss_hist = []
for it in range(iterations):
    # Apply the mesh transformation
    update_params(params, opt)
    # Perform a differentiable rendering
    img = mi.render(scene,  params=params, spp=spp)
    # Evaluate the objective function
    loss_ = loss(img, ref_img)
    # Backpropagate through the rendering process
    dr.backward(loss_)
    # Optimizer: take a gradient descent step
    opt.step()

    loss_hist.append(loss_)
    print(f"Iteration {it:02d}: error={loss_[0]:6f}, height={opt['emitter_height'][0]}", end='\r')

The loop starts without any error, but there is no optimization occuring. The error remains constant at all iterations, and the final image is identical to the initial one.

Can anyone tell me what I'm doing wrong? Thank you!

Scene dict

ref_dict = {
    'type': 'scene',
    'integrator': {
        'type': 'prb'
    },
    'suzanne':{
        'type': 'ply',
        'filename': 'Scenes/suzanne/suzanne.ply',
        'bsdf':{
            'type':'roughplastic',
            'diffuse_reflectance': {
                'type': 'rgb',
                'value': [0.9, 0.2,0.1]
            },
            'alpha': 0.2
        }
    },
    'top_camera': {
        'type': 'perspective',
        'focal_length': '350',
        'to_world': T.look_at(target=[0,0,0], origin=[0,0,50], up=[0,1,0]),
        'film': {
            'type': 'hdrfilm',
            'width': 2000,
            'height':2000
        }
    },
    'area_emitter' : {
        'type': 'rectangle',
        'emitter': {
            'type': 'area',
            'radiance': {
                'type': 'rgb',
                'value': 2
            }
        }
    }
}

<\details>

[njroussel]

Hi @gioarma

This all seems correct and fine, with the exception of your choice of integrator. As stated in the tutorial you referenced, you should either use direct_reparam or prb_reparam to handle moving objects in your scene.

[gioarma]

Hi, thank you for your help, it indeed converges using direct_reparam.
I thought it was not necessary because the object whose pose I was estimating (the emitter) was outside the field of view of the camera, and so its silhouette was not causing discontinuity issues.