Gradient of mesh vertices

[supertan0204]

Hi,

If I have a mesh with vertices $\mathbf{x}$ and normals $\mathbf{n}$, and I use Mitsuba 3 to render this mesh into a rgb image $\mathbf{I}=\mathcal{R}(\mathbf{x},\mathbf{n}, ...)$. Here $\mathcal{R}$ stands for rendering function, and "..." means other parameters (lighting, materials, etc.). Is there any way that I could get explicitly the tensor $\partial \mathbf{I}/\partial \mathbf{x}$ without specifying a loss? (I do not have a explicit loss value in my optimization pipeline.)

Here is my attemping:

scene_dict = {......some scene parameters.......}
mesh = mi.Mesh(...........created a mesh.........)

mesh_params = mi.traverse(mesh)
mesh_params["vertex_positions"] = dr.ravel(v_data)
mesh_params["faces"] = dr.ravel(f_dat)
mesh_params["vertex_normals"] = dr.ravel(n_data)
mesh_params.update()

scene_dict["my_mesh"] = mesh         
scene = mi.load_dict(scene_dict)

dr.enable_grad(mesh_params["vertex_positions"])
        
img = mi.render(scene)
dr.backward(img) # ERROR HERE

The dr.backward(img) throws the error message:

Exception has occurred: TypeError
backward_from(): the argument does not depend on the input variable(s) being differentiated. Raising an exception since this is usually indicative of a bug (for example, you may have forgotten to call dr.enable_grad(..)). If this is expected behavior, skip the call to backward_from(..) if dr.grad_enabled(..) returns False.

[njroussel]

Hi @supertan0204

Yes, this should work. You need to pass the scene parameters to the mi.render() function:

params = mi.traverse(scene)
img = mi.render(scene, params=params)
dr.backward(img)