Depth image based rendering depth warping

[vgrau98]

Hi!

My use case is simple, i want to generate a synthetic view (from new extrinsic parameters) given a src rgbd frame associated with extrinsic and intrinsic parameters. I use the DepthWarper class but results seems stange:

pinhole_dst = PinholeCamera(
    torch.tensor([
        [525, 0, 512/2., 0],
        [0, 525, 512/2., 0],
        [0, 0, 1, 0],
        [0, 0, 0, 1]  
    ], dtype=torch.float32)[None],
    torch.inverse(c2ws[0][None, :, :]),
    torch.tensor([512]),
    torch.tensor([512])
)

pinhole_src = PinholeCamera(
    torch.tensor([
        [525, 0, 512/2., 0],
        [0, 525, 512/2., 0],
        [0, 0, 1, 0],
        [0, 0, 0, 1]
    ], dtype = torch.float32)[None],
    torch.inverse(c2ws[-1][None, :, :]),
    torch.tensor([512]),
    torch.tensor([512])
)

warper = DepthWarper(pinhole_dst, 512, 512)
warper.compute_projection_matrix(pinhole_src)

depth_src = ((Pil_to_T(Image.open(depth_path))[None])/255.)*0.56
src_image = Pil_to_T(Image.open(img_path))[None, ...].to(torch.float32)/255.
dst_image = warper(depth_src, src_image)

I expect to have missing pixels in the middle of the image (not just on the sides) because of the occlusion, but I don't have any on the result, and the image is distorted.

Does DepthWarper fit with my use case?

Thanks for your help!

[rusty1s]

I think this is the wrong repository :)