Gradient Dimension?

[coult099]

In forward-mode AD, is the gradient computed with respect to each parameter which has the enable_grad flag and then added/averaged together?

For 1D parameters such as color of an object, it seems the gradient should have the same dimension as the output image (one gradient per color channel). However, if differentiating with respect to an object's vertex positions (when there are multiple), it seems we should have a gradient for each color channel with respect to each vertex individually. Do these gradients get added or averaged together in the final image? Or, is there a different process I am not understanding? Thank you.

[njroussel]

Hi @coult099

They are added together.

I particularly like Dr.Jit because setting up tiny experiments like this is quite easy, and usually helps to reason about some larger problems.

import drjit as dr

from drjit.llvm.ad import Float as Float
from drjit.llvm.ad import Array2f as Array2f
from drjit.llvm.ad import Array3f as Array3f

a0 = Float([0])
a1 = Float([1])
a = Array2f(a0, a1)
dr.enable_grad(a) # 2D parameter to differentiate w.r.t.

b = Float([2])
vec = Array3f(a.x, a.y, b)
out = dr.sum(vec)

dr.forward(a, dr.ADFlag.ClearNone) # Keep gradients on inputs

print(f"{dr.grad(a.x)=}") # [1.0]
print(f"{dr.grad(a.y)=}") # [1.0]
print(f"{dr.grad(out)=}") # [2.0] only 1D
print(f"{out=}") # [3.0]

[coult099]

Thank you for the clarification and example!