Point cloud classification/regression with 400.0-400K points per input

[davodogster]

Hi I successfully wrote the PyG dataloader to iteratively read in points clouds ranging from 400 points to 400,000 points (but most have >100K points!). I have X,Y,Z coords + ReturnIntensity. number of points clouds = 1000-10000. I have multiple parameters (multi-task regression) to estimate per point cloud. I will start with just single paramater regression. Which architecture/network would you recommend???
Any advice is much appreciated. And something that can learn both global and local features. The network needs to be quite computationally efficient and able to train on my 3090 GPU :)

I will probably just use the KNNGraph function to convert the PCDs into graphs (on the fly?).

Thanks everyone!! Sam

[rusty1s]

I would recommend to start small and then improve gradually. Importantly, I think you should pre-process your dataset to have a uniform size, e.g., to slice big point clouds into smaller chunks and save them as individual examples. You can then start by applying PointConv operators on a pre-defined kNN-graph, and apply some global pooling at the end. One can then extend the network to incorporate features such as hierarchical pooling, etc (similar to our examples/pointnet2_*.py examples).

[davodogster]

@rusty1s Do you think it will it improve performance if I use 4 features = X,Y,Z,I, pos= X,Y,Z vs 1x features = I, pos = X,Y,Z ?

[rusty1s]

This depends, best to try out. Moving the pos to the input features means that not only the relations between points, but also their absolute values will be used within the model. That means that your model is no longer translation-invariant.

[davodogster]

Thanks @rusty1s I'll try that out!

I now have three parameters to estimate in multitask learning. One of them has 4 states but they are continuous values 3,4,5,6. The other params range from 0.2,1.0 (10 states) and 30,90. Do you know how I can adjust or weight the loss in multitask learning to account for these different scales and parameter ranges?

Here is my current code for the losses. It assumes equal weighting for all params.

[davodogster]

It has the same functionality, it is just more efficient.

Thanks, I have made the change and hopefully I am doing it correctly:
I also changed:

to:

[rusty1s]

I think you would want to normalize the labels individually, e.g., via standard normalization. In this case, you do not need to perform any weighting of the loss.

[davodogster]

Thanks, so if I perform min-max scale all labels (3) to 0-100 will that suffice? MinMaxScaler

[rusty1s]

Yes, that works as well.

[davodogster]

@rusty1s Is it possible in PyG to perform GNN Explainability for my MLP Pointnet GNN for graph regression? I would really like to know what features the model is learning. https://pytorch-geometric.readthedocs.io/en/latest/tutorial/explain.html

Thanks, Sam

[rusty1s]

Yes, that's possible as long as your edge_index is part of the input.

[davodogster]

Hi @rusty1s

This is the top down view of the Tree Point Cloud with KNNGraph used to create the edges. Do you think it's useful for learning tree params or too complex and a terrible representation?

[rusty1s]

This is hard to tell from the image, but it looks reasonable, doesn't it?

[jediofgever]

@davodogster Hi!,
Could you please share some of your insights and results? were you able to get decent results for multitask regression based on the clouds?

[davodogster]

Hi @jediofgever yes, so far the results have been quite promising for the PCD-KNN Graph Multitask Regression. Try different values of K for the KNN Graph like 5, 10, 20. Also try different architectures like PointNet vs GCN. Next we will try a more challenging dataset.

[jediofgever]

Thanks @davodogster, happy to hear that! I am at the very beginning, so I do not have the code yet, could you please share your repo for my reference?
Also, are there any papers describing your work?