Voxel_grid clustering value issue on gpu

[ericyj-lee]

This is a snippet of my code using voxel_grid clustering with max_pool_x, for simplicity, I just let the data run through train_loader in the model without training the data at all.

The data.pos[x,y,z] of my data has maximum value of [127,127,1.5]. I aimed to cluster the nodes only based on [x,y] and I have tried the size parameters in voxel_grid of [32,32], [32,32,0] and [32,32,a large value(ex:100000)],

1. First, want to check which form is correct since these 3 work good for me when running my code on “cpu”, (issue happen when run on “gpu”, will explain later)
2. I group my data using PyG Dataloader of batch_size=64, and ideally the cluster number of each batch is (127/32)*(127/32) = 16, and through batches should be 16 * 64(batch_size) = 1024 as shown.

However, sometimes when I run my code on “cuda:0”, it shows the following results.

I analyze this issue by printing out more detailed relations of batch, data.pos ,and cluster(1st column in the picture)

It seems that when assigning clusters in different batches, it does not start correctly (correct one batch 0: cluster 0-15, batch1:16-31,…) though it has correct cluster numbers in the above case(batch1: 485301~485310 has 16 clusters). Moreover, sometimes I even got negative numbers for clusters. This issue only happens when I run my code on cuda:0, and it could run perfectly on cpu. The error I got is shown below. These results would disable the max_pool_x function(global_max_pool will be fine since it only gets one node from each batch) from working properly when having the size parameter and further applying it to the Fully-connected layer. Thanks!

3. Using the same code as above, I have 5 voxel_grid clustering step in my model

Sometimes I got the results of decreased batch size through these 5 steps,

This also never happens when run on cpu. Want to ask why would this happen since the batch size should not change after voxel_grid clustering. Thanks!

[rusty1s]

Thanks for the issue and sorry for my late reply. This is definitely unexpected. Is this reproducible for the same data across several runs? If yes, do you mind sharing the point cloud in order to reproduce?