Optimize vertex color

[luca-morreale]

Hello,
is there any way to optimize a mesh vertex_color attribute?
I have tried to follow the pose estimate tutorial and change to my case but failed. Similarly, I looked into other discussions, but ultimately nothing seems to work. It feels like I am missing something very basic.

I have a very simple scene (see below), and I want to optimize the vertex_color through rendering:

params   = mi.traverse(scene)
key      = 'mesh.vertex_color'
opt      = mi.ad.Adam(lr=0.1)    
opt[key] = params[key]

for _ in range(50):
    params[key] = dr.ravel(dr.clamp(opt[key], 0.0, 1.0))
    params.update()

    image = mi.render(scene, params)

    loss  = dr.mean(dr.sqr(image - reference_image))
    # even the loss below does not work
    # loss  = dr.mean(dr.sqr(image))

    dr.backward(loss) # Critical Dr.Jit compiler failure: jitc_vcall_upload(): could not find callable
    opt.step()

Scene parameters:

SceneParameters[
  -------------------------------------------------------------------------
  Name                           Flags    Type  Parent
  -------------------------------------------------------------------------
  light.radiance.value           ∂        Float UniformSpectrum
  mesh.bsdf.reflectance.scale             float MeshAttribute
  mesh.vertex_count                       int   PLYMesh
  mesh.face_count                         int   PLYMesh
  mesh.faces                              UInt  PLYMesh
  mesh.vertex_positions          ∂, D     Float PLYMesh
  mesh.vertex_normals            ∂, D     Float PLYMesh
  mesh.vertex_texcoords          ∂        Float PLYMesh
  mesh.face_color                ∂        Float PLYMesh
  mesh.vertex_color              ∂        Float PLYMesh
  sensor.near_clip                        float PerspectiveCamera
  sensor.far_clip                         float PerspectiveCamera
  sensor.shutter_open                     float PerspectiveCamera
  sensor.shutter_open_time                float PerspectiveCamera
  sensor.film.size                        ScalarVector2u HDRFilm
  sensor.film.crop_size                   ScalarVector2u HDRFilm
  sensor.film.crop_offset                 ScalarPoint2u HDRFilm
  sensor.x_fov                   ∂, D     Float PerspectiveCamera
  sensor.to_world                ∂, D     Transform4f PerspectiveCamera
]

However, the following snippet successfully optimizes the vertex color:

params   = mi.traverse(scene)
key      = 'mesh.vertex_color'
opt      = mi.ad.Adam(lr=0.1)    
opt[key] = params[key]

for _ in range(50):
    params[key] = dr.ravel(dr.clamp(opt[key], 0.0, 1.0))
    params.update()

    loss  = dr.mean(dr.sqr(opt[key]))

    dr.backward(loss)
    opt.step()

Thank you for your help.

[njroussel]

Hi @luca-morreale

I don't think we've ever actually tried to optimize mesh attributes, although it seems theoretically possible. The first optimization snippets looks good.
The error message is quite worrying, it might be have some bug specifically with mesh_attribute. Could you please provide us with a full reproducer?