In our tests, we noticed that the ground truth object to world transformation is incorrect. If we fix the following line to use @ to perform actual matrix multiplication instead of elementwise multiplication, it will give the correct result.
Fixed in PR #20