Error: the argument does not depend on the input variable(s) being differentiated

[Galaxeaaa]

I am trying a very simple gradient-based optimization procedure, the code is shown as follows. However I got the error TypeError: backward_from(): the argument does not depend on the input variable(s) being differentiated. Raising an exception since this is usually indicative of a bug (for example, you may have forgotten to call dr.enable_grad(..)). If this is expected behavior, skip the call to backward_from(..) if dr.grad_enabled(..) returns False. I understand this might be caused by break of dependency between the rendering result and the variable being optimized. But I just simply called the mi.render() function, and the result definitely depends on envmap.scale. I checked a previous discussion, but didn't find a working solution. Could anyone point out where might be wrong in the code? Thanks!

def loadSensor(origin, lookat, up):
    sensor = mi.sensors.PerspectiveSensor(origin=origin, target=lookat, up=up, fov=45)
    return sensor


def display(image):
    plt.axis("off")
    plt.imshow(image)
    plt.show()


def loss_fn(img, gt):
    return dr.mean(dr.sqr(img - gt))


if __name__ == "__main__":
    dataset = loadDataset()
    cam_lookat_mat = dataset[0]["camera_lookat_mat"]
    scene = mi.load_file("convert_xml_test/xml_test.xml")
    params = mi.traverse(scene)
    params["sensor.to_world"] = mi.Transform4f.look_at(
        origin=cam_lookat_mat[0], target=cam_lookat_mat[1], up=cam_lookat_mat[2]
    )
    params["envmap.scale"] = mi.Float(1.0)
    params.update()

    params.keep(r"envmap\.(scale)")
    print(f"Optimizing parameters {list(params.keys())}.")

    gt_image = mi.Bitmap("test_im0.exr")
    display(gt_image)

    dr.enable_grad(params)
    print(dr.grad_enabled(params))

    optimizer = mi.ad.Adam(lr=0.05, params=params)
    params.update(optimizer)

    images = []

    for i in trange(10):
        image = mi.render(scene, spp=1024)
        display(image)
        images.append(image)

        loss = loss_fn(image, gt_image)
        dr.backward(loss)
        optimizer.step()
        params.update(optimizer)

[merlinND]

Hello @Galaxeaaa,

Could you try passing params=params to mi.render(), as is done in this tutorial: https://mitsuba.readthedocs.io/en/stable/src/inverse_rendering/gradient_based_opt.html#Optimization

[Galaxeaaa]

Hello @merlinND,

This works! Thank you so much!