About gradient loss in the parameter space

[monlabiss]

Hi :)
The below image is gradient loss image with respect to scene parameter(painting texture) in Veach door scene using the dr.forward() function.
I would like to understand how the gradient loss in the parameter space is actually calculated to obtain a gradient image.

Since the actual resolution of the painting texture is larger than the resolution of the painting in the image, a many-to-one relationship is created. Therefore, I would like to know how the calculation is done when obtaining such a gradient image or gradient values. Because it is hard to calculate because of correlation.

Please let me know both the theoretical explanation and how the code implementation is carried out or which parts of the code to look at.

[rtabbara]

Hi @monlabiss,

A texture lookup that for instance uses bilinear interpolation is expressed as a weighted sum of neighbouring texels and hence the derivative of that lookup with respect to some parameter is similarly a weighted sum of derivatives of each texel with respect to the parameter. For a given pixel in the scene image, the incoming radiance could have contributions from multiple texture lookups due to light paths that hit the painting (potentially multiple times due to multiple bounces). Hence, regardless of the size of the painting in the scene, you can still have a "many-to-one" relationship.

If the question is more, from an optimisation point of view, is it difficult to recover for instance a high-resolution albedo texture when it's larger than the rendered view, the answer is yes however there are techniques such as regularization or Large Steps that can help with convergence.

I'm sorry, I don't think I can provide a single piece of code that encompasses all of this but I think the suite of Inverse rendering tutorials can help to shed further light.

[monlabiss]

I understand! Many thanks.