Why torch-geom doesnt need fixed number of points for PointNetConv ?

[QuanticDisaster]

I was wondering, in the PointNet paper as well as in tutorials the convolution part of PointNet needs a fixed number of points thus the subsampling or upsampling in somes cases.

How come PointNetConv in torch-geometric doesnt need any of this and can train and work when i give him samples with different number of points ?

[rusty1s]

Most GNN operators such as PointNet++ share weights across the node/point dimension, so in theory there is no limitation in applying GNNs to a varying number of nodes. PyG makes use of sparse tensor arithmetics and can therefore handle point clouds of varying size. In particular, we do this by concatenating points from different examples in the point dimension, and keep track of which point belongs to which example via an additional batch vector. As a result, we do not have a dedicated batch dimension, which is in contrast to most other implementations that batch point clouds via a [batch_size, num_points, 3] representation. This explains why alternative implementations can only work on fixed-size point clouds.

[QuanticDisaster]

Sorry but I don't get it. Well I think I understand the idea of sharing weights a bit, but on the other hand I can't get to represent it with perceptrons.

Most GNN operators such as PointNet++ share weights across the node/point dimension

If I understand well, we are learning a continuous convolution 3D operator which explains why whether there are five points or a hundred (like in the second and third images, from PointConv paper), the operation is the same.

But while convolutions on 2D images is done in a fixed structure of pixels which allows to use a MLP equivalent to a 3x3 matrix over and over for instance, points clouds are unstructured, so how is it handled by an MLP ?

You keep track of which point belongs to which example via an additional batch vector, but other implementations do the same, just not the same way, so why the structure [batch_size, num_points, 3] itself seals this functionnality ?

[rusty1s]

Instead of having a fixed structure (e.g., 3x3 kernel size), graph convolution is performed via continuous convolution with a fixed set of parameters. In PointNet++, this is done via an MLP that aggregates the result of MLP(p_j - p_i) for each neighboring point. This formulation does not require a fixed number of neighbors, as we can aggregate the set of neighbors independently of size.

The shape [batch_size, num_points, 3] requires a fixed number of points per example in a batch, as otherwise you wouldn't be able to stack your examples into such a tensor. Representing batches via [batch_size * num_points, 3] does not require that, as we can concatenate point clouds in the first dimension independent of size.

Hope this clarifies your doubts :)

[QuanticDisaster]

The shape [batch_size, num_points, 3] requires a fixed number of points per example in a batch, as otherwise you wouldn't be able to stack your examples into such a tensor. Representing batches via [batch_size * num_points, 3] does not require that, as we can concatenate point clouds in the first dimension independent of size.

Oh indeed, it was quite simple in fact, the tensor can't have variable size along the second dimension

Instead of having a fixed structure (e.g., 3x3 kernel size), graph convolution is performed via continuous convolution with a fixed set of parameters. In PointNet++, this is done via an MLP that aggregates the result of MLP(p_j - p_i) for each neighboring point. This formulation does not require a fixed number of neighbors, as we can aggregate the set of neighbors independently of size.

Ok I think I'm beginning to get the hang of it, MLP(p_j - p_i) is applied on each neighbor, and then we agregate the results before going further

Thanks for you explanation !