Using Pointnet for direct Prediction

[WizKhalista]

Hey everyone,

first of all Thank you for your great work and good documentation!
Im currently working on a Task where i need to predict a semantic segmentation of a scene. So far I have used the pointnet_segmentation Model from here:
https://github.com/pyg-team/pytorch_geometric/blob/master/examples/pointnet2_segmentation.py
and trained it sucessfully on my own Dataset.
I am running a gazebo simulation where i have a realsense camera in it to capture my pointcloud data like this:

Since I am getting the coordinates of each point and can calculate the normals which are neccessary for the Model i thought i was good to go. I Was building my Data Objects like this:
data = Data(x=normals, pos=pos)
However for the model to predict something i need a DataLoader which as far as i can see only accepts a whole Dataset.

Is there any chance i can kind of do a "live prediction" for my simulation, where i take my point coordinates and normals, construct a Data Object and somehow let the Model predict that?

Thank you in advance!

[Padarn]

Am I right in understanding that you want to predict only on a subset of the nodes of your graph?

[WizKhalista]

I want so capture a Pointcloud from my Simulation and use it as input for my pointnet segmentation model.
But the model expects an input in form of (data.x (normals), data.pos (positions), data.batch).
def forward(self, data):
sa0_out = (data.x, data.pos, data.batch)
sa1_out = self.sa1_module(*sa0_out)
I am only getting normals and positions from the points of the pointcloud and dont know how to fill in the data.batch value. When loading data with a DataLoader the data-batch gets filled by defining the batch_size automatically i think. But i can´t use the DataLoader, since i want to construct my Data Objects directly and not from a Dataset.
Basically as far as i understand it i need to build a Graph out of the Pointcloud i receive from the Camera Sensor in my Simulation.

[Padarn]

Oh I see - most of the models taking batch allow you to use None which is treated as no batching. But if this is not the case here you can just fill it with a tensor of ones I believe

[rusty1s]

Yes, either try to pass in None or use a vector of zeros: torch.zeros(data.x.size(0), dtype=torch.long, device=data.x.device)

[WizKhalista]

Thank you both, looks like it is working now 👍