Dialing in an optimization: strategies?

[Polyrhythm]

Target render (Blender Cycles)

Starting render (Mitsuba)

Optimizing for BRDF roughness value.

Optimized render (Mitsuba, 25 iterations, 8spp)

The optimized render is way too rough still. The Blender material is about 0.1 roughness, Mitsuba is only at about 0.4 or so after optimization (not accounting for minor differences in shading models...).

If I increase the spp more, my optimize render loop takes way too long (I don't even have enough time to let it finish).
If I increase the iteration count, I don't seem to make much forward progress...the mse just kind of hovers around the same range.

What are my options to get closer to the ground truth?
The code and approach is essentially the same as the gradient optimization example: https://mitsuba.readthedocs.io/en/stable/src/inverse_rendering/gradient_based_opt.html

I found that I could get much closer when optimizing for "reflectance value" (albedo), but roughness is noticeably off.

[njroussel]

This seems fishy, 25 iterations at 8spp should be relatively quick, even when bumping the spp it should be relatively quick.
Do you see a big performance difference when compared to the tutorial you linked?

There also seems to be some sort of shading artifacts on the sphere itself. Could you share the mitsuba scene?

Another debugging step would be to generate and then use a reference with Mitsuba (using a lower roughness). This would at least get rid of model differences Between Blender and Mitsuba and should help you figure out any bugs in your optimization if the result does not converge to the reference rapidly.

[Polyrhythm]

Hey @njroussel, I had some time to follow up on this. Maybe you can provide some further guidance...
First off, I'd like to clarify that I am primarily interested in testing if Mitsuba can be used for extracting values from real-world photos and applying them to scanned assets, so if it can't even handle the very minimal differences in Blender/Mitsuba renders on 3d geometry, I may as well stop the testing there.

https://colab.research.google.com/drive/1-A7Q0R_DmrRP7GrnV07VYpUZtbp-_5ag?usp=sharing
There is a link to my current Notebook for this test.

Unfortunately, the files are local, but I also am sharing the files here if you want to take the notebook offline and try it out: https://drive.google.com/drive/folders/1H90Q55oDPnPlw-OU2dtQmvVUVPl05-ld?usp=share_link

The big thing I noticed is that the MSE just does not seem to go down even with a huge amount of iterations, as seen here:

This leads me to believe that either I have something incorrectly configured, or that there is something I am not thinking about correctly with how optimization works. I would expect a different curve shape, like as seen in the tutorial:

[njroussel]

Thank you for the reproducer, I've noticed a few things.

In your notebook you use the path integrator. You should preferably use an AD integrator like prb, it will be more efficient. Secondly, the roughness parameter can introduce discontinuities, which need to be explicitly handled by the integrator. This is mentionned in the plugin documentation or you can also print(mi.traverse(scene)) to see a similar table. For that reason, you should actually use the prb_reparam integrator (BSDF sampling is not attached in PRB, so reparam is not actually needed here).
If this seems unclear, I'd highly recommend making your way through all "inverse rendering" tutorials and also spending some time looking at some of the papers that are referenced there.

I haven't actually ran your code with these changes. So please keep this thread updated 😄

[Polyrhythm]

I did try setting up a comparison between a Mitsuba scene and another Mitsuba scene with just roughness changed, and the optimization performs vastly better.

So the question becomes:
Why does Mitsuba struggle so much in optimizing a single variable when the reference render is very close but not exactly the same as the Mitsuba render?

How can I help make the optimization more adaptable to imperfect reference target images?