Shape optimization with highly reflective material

[teddy8193]

Hi there!
I was wondering how difficult it is to do the shape optimization with highly reflective material.
For instance, when I do this tutorial with more metallic BRDF, the loss never converges and the shape stays just a bumpy blob.
Is it technologically harder to do it with shinier material than diffused material, and if not, could anyone give me some advice to achieve it?
Thanks!

[njroussel]

Hi @teddy8193

Specular/reflective materials are indeed typically a bit harder to handle. Think of it like this: by modifying the surface normal at any point, you can have a much larger range of possible appearances compared to a diffuse material. This in turn means that you're much more likely to hit a local minimum during the optimization. The derivatives will very often push you in a part of the loss landscape that does indeed look better "locally", but it will also mean that you'll be stuck in that area.

The number of points of views and the envmap obviously play a big role here. You want to leave as little shading-lighting ambiguity as possible. From my personal experience, I've also had good success by doing coarse-to-fine schemes on the mesh and film resolution.

[ziyi-zhang]

Hi,
Shiny materials are very hard to reconstruct. Consider the extreme case of smooth conductors (mirror-like), observing a reference color of blue contains very little useful information -- it could be the reflected color from any part of the sky, and in many regions light would reflect multiple times.
A related field is transparent object reconstruction, where strong regularization, shape prior/guess and special background is often needed. See these papers:
https://ivrl.github.io/NEMTO/
https://arxiv.org/pdf/2004.10904
https://arxiv.org/pdf/2202.00948v1
https://arxiv.org/pdf/2009.09144