Questions about PointNet++'s inference accuracy

[crea397]

Hi,

I implemented the program with reference to examples/pointnet2_classification.py and used Google Colablatory's GPU / Ubuntu PC's GPU to learn the model.

When I save a model trained on a Colab / Ubuntu PC and load that model into Jetson Xavier NX for inference, the inference accuracy is much lower than when I do it on a Colab / Ubuntu PC.

There is no particular loss of accuracy between devices other than Jetson, for example, when saving a model trained on Colab, loading it on an Ubuntu PC, and performing inference.
There is also no loss of accuracy when saving models trained on Ubuntu PC and loading them in Colab.

When I save a model trained on another device (Colab, Ubuntu PC), load it into Jetson Xavier NX, and perform inference, I can't figure out why the accuracy drops.

How can I solve this problem?
Thanks!

---

Enviroment

$ uname -a
Linux JestonXavierNX 4.9.140-tegra #1 SMP PREEMPT Tue Oct 27 21:02:46 PDT 2020 aarch64 aarch64 aarch64 GNU/Linux

Device	Xavier NX	Colab	Ubuntu PC
OS	Ubuntu 18.04	Ubuntu 18.04	Ubuntu 20.04
Python	3.6.9	3.7.10	3.6.13
torch	1.7.0	1.7.0	1.7.0
torch.version.cuda	10.2	10.2	10.2
torch-scatter	2.0.5	2.0.5	2.0.7
torch-sparse	0.6.8	0.6.8	0.6.10
torch-cluster	1.5.8	1.5.8	1.5.9
torch-spline-conv	1.2.0	1.2.0	1.2.1
torch-geometric	1.6.1	1.6.1	1.7.1

---

• The same test data is used.
• Saving the models trained by Jetson Xavier NX and loading them into Jetson Xavier NX for inference works fine.
• The following code is used to save and load the model.

# save
torch.save(model.state_dict(), './xxx.pth')

# load
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
model = Net()
model.load_state_dict(torch.load('./xxx.pth'))
model.to(device)

[rusty1s]

Do you know if this is only the case for the PointNet++ model? Do you know which model operation causes the divergence?

[rusty1s]

Thank you very much. It is really weird that both radius and knn perform differently on Jetson. Are you sure that the input point cloud is the same in both cases? Is there a possibility that points are scaled differently?
I'm also wondering if you can confirm whether radius yields the correct result on Jetson and Ubuntu. What happens if you input a simple toy example with say 100 points?

[crea397]

Are you sure that the input point cloud is the same in both cases? Is there a possibility that points are scaled differently?

I think the same input point cloud is used.
For the input point cloud, I am applying T.NormalizeScale() on both machines.
Can this be a possibility that the input point cloud is different or the point scaling is different?
Just to be sure, I also tried not applying T.NormalizeScale() on both machines. However, the accuracy was reduced by Jetson as well.

What happens if you input a simple toy example with say 100 points?

I used the data of a rabbit (113 points) and a chair (111 points) as input point clouds and checked the output of radius.
In this case, I did not apply T.NormalizeScale() on both machines. Also, I used a value of 64 for max_num_neighbors.

The result is shown in the screenshot below.
The output results of radius on both machines did not match.

rabbit

chair

[rusty1s]

Ok, thanks a lot for digging into this so closely. At this point, we should probably move this discussion over to pytorch-cluster. Feel free to create an issue there. Can you run the test-suite of pytorch-cluster in Jetson and post the output log?

[crea397]

Thank you very much for your suggestion. I will create a new issue in the pytorch-cluster repository and post the output log.

Sorry, but please let me know before I do that. What is the test-suite for pytorch-cluster?

Do I just run python setup.py test on https://github.com/rusty1s/pytorch_cluster and post the output?
For example, or do I just run pytorch_cluster/test/test_radius.py and post the output for each variable as before?

[rusty1s]

You can run python setup.py test and let me know the error log of tests that fail.