Multi-view image reconstruction failure

[surpoloyang]

I used a set of multi-view images of plants, hoping to reconstruct a 3D model of the entire plant.
I used the default parameters of the demo, loaded the provided pre-trained weights, but the resulting glb file only shows the camera position.

[rover-xingyu]

Hi @surpoloyang, the camera poses look good. Maybe the point clouds are filtered out with conf_other_tosave > 1

We recommend visualizing the results in Viser. There you can adjust the “Visibility Threshold” slider to pick a confidence level that suits you. Please give it a try, and let me know how it goes—I’m looking forward to seeing your results!

[surpoloyang]

Not as good as expected, as shown below:

[surpoloyang]

[rover-xingyu]

Oh, this visualization config is for long sequences (e.g., >1000 frames).

For short sequences, adjust these visualization settings in Viser:

• Train Step → 102
• Show pointclouds → 3D (All Frames)
• Downsample Factor → 1

Try again with these settings!

[surpoloyang]

You can see the point cloud of the object, but the effect is off, and the point cloud didn't match. The images from training steps 1->102 are as follows:

[rover-xingyu]

I see. The black background is the devil. Unfortunately, TTT3R can't handle this case yet.
Please try with the original images instead of the masked ones if available.

[surpoloyang]

Okay, thank you! Do not use a mask mean separating the main plant from the image and saving the main plant as a new image, then inferring?

[rover-xingyu]

Yes, CUT3R is trained on images with natural backgrounds, so both CUT3R and TTT3R cannot handle masked images with black/white backgrounds.

[surpoloyang]

Thank you for your replies!