How to run code on other dataset

[vedantpatel41]

hello there, i am interested in applying this model on my dataset. i have point cloud data of an building floor. how can i run this model on it?
can you elaborate how can i do this.

[vedantpatel41]

well i have experimented with my point cloud data, i have first created the density maps and ran the inference on them. here are the density images and the results.

result:

image:

results:

i have tried to create the density image using the method in the code itself:

the results i am getting are not good. can you tell me what i can do to improve this results. also if i want to train this model on my dataset then how should i create the training data ? i mean how can i annotate it ? it will be great help if you can answer this.

[ywyue]

Hi @vedantpatel41,

RoomFormer was trained on Structured3D, a synthetic dataset of residential apartment scenes. We also had a variant of model trained on SceneCAD, a dataset of single-room scenes. Due to domain gap with your dataset, they may not work well, just as you show. Regarding the density map generation, did you follow the code for Structured3D or SceneCAD? Please observe which dataset is most similar to yours and use the corresponding model and density map preprocessing procedure.

To train the model on your dataset, you need to prepare the training pairs: density map + annotation. The density map can be obtained by projecting 3D point clouds to 2D using our code. The annotation is a list of polygons (which in turn are a list of ordered vertices), which needs to be collected by hand. I had no experience for annotating this. But one potential way to do that is to load point clouds to CouldCompare (or other PC viewer), check the coordinates of room corners and read them out sequentially.

[zxbzhineng]

@ywyue @vedantpatel41 Sorry to bother you, I want to make inferences on my own data, how should I proceed? Thank you

[ywyue]

Hi @zxbzhineng, to make inferences on your own data using the pretrained network, you need to first convert your 3D scan to density map. Here is the code for preprocessing: https://github.com/ywyue/RoomFormer/tree/main/data_preprocess

If your scan looks more similar with Structured3D, please follow the preprocessing step of Structured3D, otherwise SceneCAD.

The inference code is here:

RoomFormer/engine.py

Line 205 in 33ad087

def evaluate_floor(model, dataset_name, data_loader, device, output_dir, plot_pred=True, plot_density=True, plot_gt=True, semantic_rich=False):

You need to modify the code a bit to adapt it to your dataset.