Skip to content

worldbench/awesome-3d-in-the-wild

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

23 Commits
LICENSE
LICENSE
 
 
README.md
README.md
 
 
index.html
index.html
 
 

Repository files navigation

Awesome Logo Visitors PR's Welcome

😎 Awesome 3D Scene Understanding in the Wild

Table of Contents

1. LiDAR Semantic Segmentation

1️⃣ Raw Points

⏲️ In chronological order, from the earliest to the latest.

Model Paper Venue Website Github
PointNet arXiv
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation		 CVPR 2017 - GitHub
PointNet++ arXiv
PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space		 NeurIPS 2017 - GitHub
TangentConv arXiv
Tangent Convolutions for Dense Prediction in 3D		 CVPR 2018 Website GitHub
KPConv arXiv
KPConv: Flexible and Deformable Convolution for Point Clouds		 ICCV 2019 - GitHub
RandLA-Net arXiv
RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds		 CVPR 2020 Website GitHub
PointASNL arXiv
PointASNL: Robust Point Clouds Processing using Nonlocal Neural Networks with Adaptive Sampling		 CVPR 2020 - GitHub
PTv1 arXiv
Point Transformer		 CVPR 2021 - GitHub
RandLA-Net+ arXiv
Learning Semantic Segmentation of Large-Scale Point Clouds with Random Sampling		 TPAMI 2021 Website GitHub
BAF-LAC arXiv
Backward Attentive Fusing Network With Local Aggregation Classifier for 3D Point Cloud Semantic Segmentation		 TIP 2021 - GitHub
PTv2 arXiv
Point Transformer V2: Grouped Vector Attention and Partition-based Pooling		 NeurIPS 2022 - GitHub
WaffleIron arXiv
Using a Waffle Iron for Automotive Point Cloud Semantic Segmentation		 ICCV 2023 - GitHub
PCB-RandNet arXiv
PCB-RandNet: Rethinking Random Sampling for LiDAR Semantic Segmentation in Autonomous Driving Scene		 ICRA 2024 - GitHub
PTv3 arXiv
Point Transformer V3: Simpler Faster Stronger		 CVPR 2024 - GitHub

2️⃣ Pseudo Images

⏲️ In chronological order, from the earliest to the latest.

Model Paper Venue Website Github
SqueezeSeg arXiv
SqueezeSeg: Convolutional Neural Nets with Recurrent CRF for Real-Time Road-Object Segmentation from 3D LiDAR Point Cloud		 ICRA 2018 - GitHub
SqueezeSegV2 arXiv
SqueezeSegV2: Improved Model Structure and Unsupervised Domain Adaptation for Road-Object Segmentation from a LiDAR Point Cloud		 ICRA 2019 - GitHub
RangeNet++ arXiv
Rangenet++: Fast and accurate lidar semantic segmentation		 IROS 2019 - GitHub
PolarNet arXiv
PolarNet: An Improved Grid Representation for Online LiDAR Point Clouds Semantic Segmentation		 CVPR 2020 - GitHub
SqueezeSegV3 arXiv
SqueezeSegV3: Spatially-Adaptive Convolution for Efficient Point-Cloud Segmentation		 ECCV 2020 - GitHub
SalsaNet arXiv
SalsaNet: Fast Road and Vehicle Segmentation in LiDAR Point Clouds for Autonomous Driving		 IV 2020 - GitHub
SalsaNext arXiv
SalsaNext: Fast, Uncertainty-aware Semantic Segmentation of LiDAR Point Clouds for Autonomous Driving		 ISVC 2020 - GitHub
3D-MiniNet arXiv
3D-MiniNet: Learning a 2D Representation from Point Clouds for Fast and Efficient 3D LIDAR Semantic Segmentation		 RA-L 2020 - GitHub
KPRNet arXiv
KPRNet: Improving projection-based LiDAR semantic segmentation		 arXiv 2020 - GitHub
Lite-HDSeg arXiv
Lite-HDSeg: LiDAR Semantic Segmentation Using Lite Harmonic Dense Convolutions		 ICRA 2021 - -
FIDNet arXiv
FIDNet: LiDAR Point Cloud Semantic Segmentation with Fully Interpolation Decoding		 IROS 2021 - GitHub
MINet arXiv
Multi-Scale Interaction for Real-Time LiDAR Data Segmentation on an Embedded Platform		 RA-L 2021 - GitHub
CENet arXiv
CENet: Toward Concise and Efficient LiDAR Semantic Segmentation for Autonomous Driving		 ICME 2022 - GitHub
RangeViT arXiv
RangeViT: Towards Vision Transformers for 3D Semantic Segmentation in Autonomous Driving		 CVPR 2023 - GitHub
RangeFormer arXiv
Rethinking Range View Representation for LiDAR Segmentation		 ICCV 2023 - -
FRNet arXiv
FRNet: Frustum-Range Networks for Scalable LiDAR Segmentation		 TIP 2025 Website GitHub
RangeSAM arXiv
RangeSAM: Leveraging Visual Foundation Models for Range-View repesented LiDAR segmentation		 arXiv 2025 - -

3️⃣ Sparse Voxels

⏲️ In chronological order, from the earliest to the latest.

Model Paper Venue Website Github
SSCN arXiv
3D Semantic Segmentation with Submanifold Sparse Convolutional Networks		 CVPR 2018 - GitHub
MinkUNet arXiv
4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks		 CVPR 2019 Website GitHub
JS3C-Net arXiv
Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene Completion		 AAAI 2021 - GitHub
Cylinder3D arXiv
Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation		 CVPR 2021 - GitHub
(AF)2-S3Net arXiv
Attentive Feature Fusion with Adaptive Feature Selection for Sparse Semantic Segmentation Network		 CVPR 2021 - -
Cylinder3D+ arXiv
Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR-based Perception		 TPAMI 2021 - GitHub
PVKD arXiv
Point-to-Voxel Knowledge Distillation for LiDAR Semantic Segmentation		 CVPR 2022 - GitHub
SDSeg3D arXiv
Self-Distillation for Robust LiDAR Semantic Segmentation in Autonomous Driving		 ECCV 2022 - GitHub
GASN arXiv
Efficient Point Cloud Segmentation with Geometry-Aware Sparse Networks		 ECCV 2022 - -
MSSNet arXiv
Point Cloud Semantic Segmentation using Multi Scale Sparse Convolution Neural Network		 arXiv 2022 - -
SphereFormer arXiv
Spherical Transformer for LiDAR-based 3D Recognition		 CVPR 2023 - GitHub
LinK arXiv
LinK: Linear Kernel for LiDAR-based 3D Perception		 CVPR 2023 - GitHub
SFPNet arXiv
SFPNet: Sparse Focal Point Network for Semantic Segmentation on General LiDAR Point Clouds		 ECCV 2024 - GitHub
NUC-Net arXiv
NUC-Net: Non-uniform Cylindrical Partition Network for Efficient LiDAR Semantic Segmentation		 TCSVT 2025 - GitHub

4️⃣ Multi-Representation

⏲️ In chronological order, from the earliest to the latest.

Model Paper Venue Website Github
SPVCNN arXiv
Searching Efficient 3D Architectures with Sparse Point-Voxel Convolution		 ECCV 2020 Website GitHub
FusionNet arXiv
Deep FusionNet for Point Cloud Semantic Segmentation		 ECCV 2020 - GitHub
AMVNet arXiv
AMVNet: Assertion-based Multi-View Fusion Network for LiDAR Semantic Segmentation		 arXiv 2020 - -
MPF arXiv
Multi Projection Fusion for Real-time Semantic Segmentation of 3D LiDAR Point Clouds		 WACV 2021 - -
RPVNet arXiv
RPVNet: A Deep and Efficient Range-Point-Voxel Fusion Network for LiDAR Point Cloud Segmentation		 ICCV 2021 - -
PMF arXiv
Perception-Aware Multi-Sensor Fusion for 3D LiDAR Semantic Segmentation		 ICCV 2021 - GitHub
CPGNet arXiv
CPGNet: Cascade Point-Grid Fusion Network for Real-Time LiDAR Semantic Segmentation		 ICRA 2022 - GitHub
2DPASS arXiv
2DPASS: 2D Priors Assisted Semantic Segmentation on LiDAR Point Clouds		 ECCV 2022 - GitHub
GFNet arXiv
GFNet: Geometric Flow Network for 3D Point Cloud Semantic Segmentation		 TMLR 2022 Website GitHub
LidarMultiNet arXiv
LidarMultiNet: Towards a Unified Multi-Task Network for LiDAR Perception		 AAAI 2023 - -
MSeg3D arXiv
MSeg3D: Multi-Modal 3D Semantic Segmentation for Autonomous Driving		 CVPR 2023 - GitHub
UniSeg arXiv
UniSeg: A Unified Multi-Modal LiDAR Segmentation Network and the OpenPCSeg Codebase		 ICCV 2023 - -
M3Net arXiv
Multi-Space Alignments Towards Universal LiDAR Segmentation		 CVPR 2024 - GitHub
TASeg arXiv
TASeg: Temporal Aggregation Network for LiDAR Semantic Segmentation		 CVPR 2024 - GitHub
EPMF arXiv
EPMF: Efficient Perception-aware Multi-sensor Fusion for 3D Semantic Segmentation		 TPAMI 2024 - GitHub
PC-BEV arXiv
PC-BEV: An Efficient Polar-Cartesian BEV Fusion Framework for LiDAR Semantic Segmentation		 AAAI 2025 - GitHub

2. LiDAR Panoptic Segmentation

1️⃣ Proposal-based

⏲️ In chronological order, from the earliest to the latest.

Model Paper Venue Website Github
Panoptic-TrackNet arXiv
MOPT: Multi-Object Panoptic Tracking		 arXiv 2020 - -
EfficientLPS arXiv
EfficientLPS: Efficient LiDAR Panoptic Segmentation		 TRO 2021 Website GitHub

2️⃣ Proposal-free

⏲️ In chronological order, from the earliest to the latest.

Model Paper Venue Website Github
LPSAD arXiv
LiDAR Panoptic Segmentation for Autonomous Driving		 IROS 2020 - -
Panoptic-PolarNet arXiv
Panoptic-PolarNet: Proposal-free LiDAR Point Cloud Panoptic Segmentation		 CVPR 2021 - GitHub
DS-Net arXiv
LiDAR-based Panoptic Segmentation via Dynamic Shifting Network		 CVPR 2021 - GitHub
4D-PLS arXiv
4D Panoptic LiDAR Segmentation		 CVPR 2021 Website GitHub
GP-S3Net arXiv
GP-S3Net: Graph-Based Panoptic Sparse Semantic Segmentation Network		 ICCV 2021 - -
PanosterK arXiv
Panoster: End-to-end Panoptic Segmentation of LiDAR Point Clouds		 RA-L 2021 - -
CPSeg arXiv
CPSeg: Cluster-free Panoptic Segmentation of 3D LiDAR Point Clouds		 arXiv 2021 - -
SCAN arXiv
Sparse Cross-scale Attention Network for Efficient LiDAR Panoptic Segmentation		 AAAI 2022 - -
PC-Cluster arXiv
A Divide-and-Merge Point Cloud Clustering Algorithm for LiDAR Panoptic Segmentation		 ICRA 2022 - -
SMAC-Seg arXiv
SMAC-Seg: LiDAR Panoptic Segmentation via Sparse Multi-directional Attention Clustering		 ICRA 2022 - -
PVCL arXiv
Prototype-Voxel Contrastive Learning for LiDAR Point Cloud Panoptic Segmentation		 ICRA 2022 - -
Panoptic-PHNet arXiv
Panoptic-PHNet: Towards Real-Time and High-Precision LiDAR Panoptic Segmentation via Clustering Pseudo Heatmap		 CVPR 2022 - -
MaskRange arXiv
MaskRange: A Mask-classification Model for Range-view based LiDAR Segmentation		 arXiv 2022 - -
PUPS arXiv
PUPS: Point Cloud Unified Panoptic Segmentation		 AAAI 2023 - -
LCPS arXiv
LiDAR-Camera Panoptic Segmentation via Geometry-Consistent and Semantic-Aware Alignment		 ICCV 2023 - GitHub
MaskPLS arXiv
Mask-Based Panoptic LiDAR Segmentation for Autonomous Driving		 RA-L 2023 - GitHub
Mask4D arXiv
Mask4D: End-to-End Mask-Based 4D Panoptic Segmentation for LiDAR Sequences		 RA-L 2023 - GitHub
Mask4Former arXiv
Mask4Former: Mask Transformer for 4D Panoptic Segmentation		 ICRA 2024 Website GitHub
4D-DS-Net arXiv
Unified 3D and 4D Panoptic Segmentation via Dynamic Shifting Networks		 TPAMI 2024 - GitHub
P3Former arXiv
Position-Guided Point Cloud Panoptic Segmentation Transformer		 IJCV 2024 - GitHub

3. Occupancy Prediction

1️⃣ Camera

⏲️ In chronological order, from the earliest to the latest.

Model Paper Venue Website Github
3DSketch arXiv
3D Sketch-aware Semantic Scene Completion via Semi-supervised Structure Prior		 CVPR 2020 - GitHub
AIC-Net arXiv
Anisotropic Convolutional Networks for 3D Semantic Scene Completion		 CVPR 2020 Website GitHub
MonoScene arXiv
MonoScene: Monocular 3D Semantic Scene Completion		 CVPR 2022 Website GitHub
TPVFormer arXiv
Tri-Perspective View for Vision-Based 3D Semantic Occupancy Prediction		 CVPR 2023 Website GitHub
VoxFormer arXiv
VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completion		 CVPR 2023 - GitHub
OccFormer arXiv
OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Prediction		 ICCV 2023 - GitHub
SurroundOcc arXiv
SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Driving		 ICCV 2023 Website GitHub
FB-Occ arXiv
FB-OCC: 3D Occupancy Prediction based on Forward-Backward View Transformation		 arXiv 2023 - GitHub
MonoOcc arXiv
MonoOcc: Digging into Monocular Semantic Occupancy Prediction		 ICRA 2024 - GitHub
SparseOcc arXiv
SparseOcc: Rethinking Sparse Latent Representation for Vision-Based Semantic Occupancy Prediction		 CVPR 2024 Website GitHub
Symphonies arXiv
Symphonize 3D Semantic Scene Completion with Contextual Instance Queries		 CVPR 2024 - GitHub
HASSC arXiv
Not All Voxels Are Equal: Hardness-Aware Semantic Scene Completion with Self-Distillation		 CVPR 2024 - GitHub
COTR arXiv
COTR: Compact Occupancy TRansformer for Vision-based 3D Occupancy Prediction		 CVPR 2024 - GitHub
GaussianFormer arXiv
GaussianFormer: Scene as Gaussians for Vision-Based 3D Semantic Occupancy Prediction		 ECCV 2024 Website GitHub
CGFormer arXiv
Context and Geometry Aware Voxel Transformer for Semantic Scene Completion		 NeurIPS 2024 - GitHub
ReliOcc arXiv
RELIOCC: Towards Reliable Semantic Occupancy Prediction via Uncertainty Learning		 arXiv 2024 - -
VLScene arXiv
VLScene: Vision-Language Guidance Distillation for Camera-Based 3D Semantic Scene Completion		 AAAI 2025 - GitHub
TrackOcc arXiv
TrackOcc: Camera-based 4D Panoptic Occupancy Tracking		 ICRA 2025 - GitHub
GaussianFormer-2 arXiv
GaussianFormer-2: Probabilistic Gaussian Superposition for Efficient 3D Occupancy Prediction		 CVPR 2025 - GitHub
SceneDINO arXiv
Feed-Forward SceneDINO for Unsupervised Semantic Scene Completion		 ICCV 2025 Website GitHub
DISC arXiv
Disentangling Instance and Scene Contexts for 3D Semantic Scene Completion		 ICCV 2025 - GitHub
ALOcc arXiv
ALOcc: Adaptive Lifting-Based 3D Semantic Occupancy and Cost Volume-Based Flow Predictions		 ICCV 2025 - GitHub
CausalOcc arXiv
Semantic Causality-Aware Vision-Based 3D Occupancy Prediction		 ICCV 2025 - GitHub
VoxDet arXiv
VoxDet: Rethinking 3D Semantic Occupancy Prediction as Dense Object Detection		 NeurIPS 2025 Website GitHub
QuadricFormer arXiv
QuadricFormer: Scene as Superquadrics for 3D Semantic Occupancy Prediction		 arXiv 2025 Website GitHub
FMOcc arXiv
FMOcc: TPV-Driven Flow Matching for 3D Occupancy Prediction with Selective State Space Model		 arXiv 2025 - -

2️⃣ LiDAR

⏲️ In chronological order, from the earliest to the latest.

Model Paper Venue Website Github
LMSCNet arXiv
LMSCNet: Lightweight Multiscale 3D Semantic Completion		 3DV 2020 - GitHub
JS3C-Net arXiv
Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene Completion		 AAAI 2021 - GitHub
S3CNet arXiv
S3CNet: A Sparse Semantic Scene Completion Network for LiDAR Point Clouds		 CoRL 2021 - -
SSA-SC arXiv
Semantic Segmentation-assisted Scene Completion for LiDAR Point Clouds		 IROS 2021 - GitHub
Local-DIFs arXiv
Semantic Scene Completion using Local Deep Implicit Functions on LiDAR Data		 TPAMI 2021 - -
SCPNet arXiv
SCPNet: Semantic Scene Completion on Point Cloud		 CVPR 2023 - GitHub
SSC-RS arXiv
SSC-RS: Elevate LiDAR semantic scene completion with representation separation and BEV fusion		 IROS 2023 - GitHub
PointOcc arXiv
PointOcc: Cylindrical Tri-Perspective View for Point-based 3D Semantic Occupancy Prediction		 arXiv 2023 - GitHub

3️⃣ Multi-Modality

⏲️ In chronological order, from the earliest to the latest.

Model Paper Venue Website Github
OpenOccupancy arXiv
OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perception		 ICCV 2023 - GitHub
OccGen arXiv
OccGen: Generative Multi-modal 3D Occupancy Prediction for Autonomous Driving		 ECCV 2024 Website GitHub
TEOcc arXiv
TEOcc: Radar-camera Multi-modal Occupancy Prediction via Temporal Enhancement		 ECAI 2024 - GitHub
FusionOcc arXiv
FusionOcc: Multi-Modal Fusion for 3D Occupancy Prediction		 MM 2024 - GitHub
AFOcc arXiv
AFOcc: Multimodal Semantic Occupancy Prediction With Accurate Fusion		 JSEN 2024 - -
EFFOcc arXiv
EFFOcc: Learning Efficient Occupancy Networks from Minimal Labels for Autonomous Driving		 arXiv 2024 - GitHub
MR-Occ arXiv
MR-Occ: Efficient Camera-LiDAR 3D Semantic Occupancy Prediction Using Hierarchical Multi-Resolution Voxel Representation		 arXiv 2024 - -
L2COcc arXiv
L2COcc: Lightweight Camera-Centric Semantic Scene Completion via Distillation of LiDAR Model		 arXiv 2025 Website GitHub

4. Label-Efficient Learning

1️⃣ Weakly-Supervised Learning

⏲️ In chronological order, from the earliest to the latest.

Model Paper Venue Website Github
W4DTS arXiv
Weakly Supervised Segmentation on Outdoor 4D point clouds with Temporal Matching and Spatial Graph Propagation		 CVPR 2022 - -
SQN arXiv
SQN: Weakly-Supervised Semantic Segmentation of Large-Scale 3D Point Clouds		 ECCV 2022 - GitHub
IGNet arXiv
2D Feature Distillation for Weakly- and Semi-Supervised 3D Semantic Segmentation		 WACV 2024 - -
P4G arXiv
Weakly Supervised Segmentation on Outdoor 4D Point Clouds With Progressive 4D Grouping		 TPAMI 2025 - -

2️⃣ Semi-Supervised Learning

⏲️ In chronological order, from the earliest to the latest.

Model Paper Venue Website Github
GPC arXiv
Guided Point Contrastive Learning for Semi-supervised Point Cloud Semantic Segmentation		 ICCV 2021 Website GitHub
LaserMix arXiv
LaserMix for Semi-Supervised LiDAR Semantic Segmentation		 CVPR 2023 Website GitHub
Lim3D arXiv
Less is More: Reducing Task and Model Complexity for 3D Point Cloud Semantic Segmentation		 CVPR 2023 Website GitHub
ImageTo360 arXiv
360deg from a Single Camera: A Few-Shot Approach for LiDAR Segmentation		 ICCVW 2023 - -
IGNet arXiv
2D Feature Distillation for Weakly- and Semi-Supervised 3D Semantic Segmentation		 WACV 2024 - -
SSMP arXiv
Semi-Supervised Class-Agnostic Motion Prediction with Pseudo Label Regeneration and BEVMix		 AAAI 2024 - GitHub
DDSemi arXiv
Density-Guided Semi-Supervised 3D Semantic Segmentation with Dual-Space Hardness Sampling		 CVPR 2024 - -
IT2 arXiv
ItTakesTwo: Leveraging Peer Representations for Semi-supervised LiDAR Semantic Segmentation		 ECCV 2024 - GitHub
BST arXiv
Bayesian Self-Training for Semi-Supervised 3D Segmentation		 ECCV 2024 Website -
LASS3D arXiv
LASS3D: Language-Assisted Semi-Supervised 3D Semantic Segmentation with Progressive Unreliable Data Exploitation		 ECCV 2024 - -
PLE arXiv
Learning from Spatio-temporal Correlation for Semi-Supervised LiDAR Semantic Segmentation		 IROS 2024 - GitHub
AIScene arXiv
Exploring Scene Affinity for Semi-Supervised LiDAR Semantic Segmentation		 CVPR 2025 - GitHub
HiLoTs arXiv
HiLoTs: High-Low Temporal Sensitive Representation Learning for Semi-Supervised LiDAR Segmentation in Autonomous Driving		 CVPR 2025 - GitHub
LaserMix++ arXiv
Multi-Modal Data-Efficient 3D Scene Understanding for Autonomous Driving		 TPAMI 2025 Website GitHub

3️⃣ Unsupervised Learning

⏲️ In chronological order, from the earliest to the latest.

Model Paper Venue Website Github
xMUDA arXiv
xMUDA: Cross-Modal Unsupervised Domain Adaptation for 3D Semantic Segmentation		 CVPR 2020 - GitHub
SF-UDA^{3D} arXiv
SF-UDA^{3D}: Source-Free Unsupervised Domain Adaptation for LiDAR-Based 3D Object Detection		 3DV 2020 - GitHub
AUDA arXiv
Adversarial unsupervised domain adaptation for 3D semantic segmentation with multi-modal learning		 ISPRS 2021 - GitHub
CoSMix arXiv
Unsupervised Domain Adaptation for 3D LiDAR Semantic Segmentation Using Contrastive Learning and Multi-Model Pseudo Labeling		 ECCV 2022 - GitHub
GIPSO arXiv
GIPSO: Geometrically Informed Propagation for Online Adaptation in 3D LiDAR Segmentation		 ECCV 2022 - GitHub
OGC arXiv
OGC: Unsupervised 3D Object Segmentation from Rigid Dynamics of Point Clouds		 NeurIPS 2022 - GitHub
GrowSP arXiv
GrowSP: Unsupervised Semantic Segmentation of 3D Point Clouds		 CVPR 2023 - GitHub
U3DS^3 arXiv
U3DS^3: Unsupervised 3D Semantic Scene Segmentation		 WACV 2024 - -
LiOn-XA arXiv
LiOn-XA: Unsupervised Domain Adaptation via LiDAR-Only Cross-Modal Adversarial Training		 IROS 2024 - GitHub
OGC+ arXiv
Unsupervised 3D Object Segmentation of Point Clouds by Geometry Consistency		 TPAMI 2024 - GitHub
DAKD arXiv
Domain Adaptation-Based Crossmodal Knowledge Distillation for 3D Semantic Segmentation		 ICRA 2025 - GitHub
LogoSP arXiv
LogoSP: Local-global Grouping of Superpoints for Unsupervised Semantic Segmentation of 3D Point Clouds		 CVPR 2025 - GitHub
VFMSeg arXiv
Visual foundation models boost cross-modal unsupervised domain adaptation for 3d semantic segmentation		 T-ITS 2025 - GitHub
- arXiv
Unsupervised Domain Adaptation for 3D LiDAR Semantic Segmentation Using Contrastive Learning and Multi-Model Pseudo Labeling		 arXiv 2025 - -

4️⃣ Self-Supervised Learning

⏲️ In chronological order, from the earliest to the latest.

Model Paper Venue Website Github
PointContrast arXiv
PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding		 ECCV 2020 - GitHub
Info3D arXiv
Info3D: Representation Learning on 3D Objects using Mutual Information Maximization and Contrastive Learning		 ECCV 2020 - -
DepthContrast arXiv
Self-Supervised Pretraining of 3D Features on any Point-Cloud		 ICCV 2021 - GitHub
OcCo arXiv
Unsupervised Point Cloud Pre-training via Occlusion Completion		 ICCV 2021 - GitHub
STRL arXiv
Spatio-temporal Self-Supervised Representation Learning for 3D Point Clouds		 ICCV 2021 - GitHub
PPKT arXiv
Learning from 2D: Contrastive Pixel-to-Point Knowledge Transfer for 3D Pretraining		 arXiv 2021 - -
SLidR arXiv
Image-to-Lidar Self-Supervised Distillation for Autonomous Driving Data		 CVPR 2022 - GitHub
Point-BERT arXiv
Point-BERT: Pre-Training 3D Point Cloud Transformers with Masked Point Modeling		 CVPR 2022 Website GitHub
MaskPoint arXiv
Masked Discrimination for Self-Supervised Learning on Point Clouds		 ECCV 2022 - GitHub
Point-MAE arXiv
Masked Autoencoders for Point Cloud Self-supervised Learning		 ECCV 2022 - GitHub
Also arXiv
ALSO: Automotive Lidar Self-Supervision by Occupancy Estimation		 CVPR 2023 - GitHub
ST-SLidR arXiv
Self-Supervised Image-to-Point Distillation via Semantically Tolerant Contrastive Loss		 CVPR 2023 - -
TriCC arXiv
Unsupervised 3D Point Cloud Representation Learning by Triangle Constrained Contrast for Autonomous Driving		 CVPR 2023 - -
Seal arXiv
Segment Any Point Cloud Sequences by Distilling Vision Foundation Models		 NeurIPS 2023 Website GitHub
BEVContrast arXiv
BEVContrast: Self-Supervision in BEV Space for Automotive Lidar Point Clouds		 3DV 2024 - GitHub
ScaLR arXiv
Three Pillars improving Vision Foundation Model Distillation for Lidar		 CVPR 2024 - GitHub
CSC arXiv
Building a Strong Pre-Training Baseline for Universal 3D Large-Scale Perception		 CVPR 2024 - GitHub
SuperFlow arXiv
4D Contrastive Superflows are Dense 3D Representation Learners		 ECCV 2024 - GitHub
HVDistill arXiv
HVDistill: Transferring Knowledge from Images to Point Clouds via Unsupervised Hybrid-View Distillation		 IJCV 2024 - GitHub
CMCR arXiv
Is Contrastive Distillation Enough for Learning Comprehensive 3D Representations?		 arXiv 2024 - -
LiMoE arXiv
LiMoE: Mixture of LiDAR Representation Learners from Automotive Scenes		 CVPR 2025 Website GitHub
LiMA arXiv
Beyond One Shot, Beyond One Perspective: Cross-View and Long-Horizon Distillation for Better LiDAR Representations		 ICCV 2025 Website GitHub
LargeAD arXiv
LargeAD: Large-Scale Cross-Sensor Data Pretraining for Autonomous Driving		 TPAMI 2025 Website GitHub
SuperFlow++ arXiv
Enhanced Spatiotemporal Consistency for Image-to-LiDAR Data Pretraining		 TPAMI 2025 - GitHub
CleverDistiller arXiv
CleverDistiller: Simple and Spatially Consistent Cross-modal Distillation		 arXiv 2025 - -

5️⃣ Open Vocabulary Segmentation

⏲️ In chronological order, from the earliest to the latest.

Model Paper Venue Website Github
OpenScene arXiv
OpenScene: 3D Scene Understanding with Open Vocabularies		 CVPR 2023 Website GitHub
CLIP2Scene arXiv
CLIP2Scene: Towards Label-efficient 3D Scene Understanding by CLIP		 CVPR 2023 - GitHub
PLA arXiv
PLA: Language-Driven Open-Vocabulary 3D Scene Understanding		 CVPR 2023 Website GitHub
CLIP-FO3D arXiv
CLIP-FO3D: Learning Free Open-world 3D Scene Representations from 2D Dense CLIP		 ICCV 2023 - -
LERF arXiv
LERF: Language Embedded Radiance Fields		 ICCV 2023 Website GitHub
CNS arXiv
Towards Label-free Scene Understanding by Vision Foundation Models		 NeurIPS 2023 - GitHub
OpenMask3D arXiv
OpenMask3D: Open-Vocabulary 3D Instance Segmentation		 NeurIPS 2023 Website GitHub
OpenNeRF arXiv
OpenNeRF: Open Set 3D Neural Scene Segmentation with Pixel-Wise Features and Rendered Novel Views		 ICLR 2024 Website GitHub
OV3D arXiv
Open-Vocabulary 3D Semantic Segmentation with Foundation Models		 CVPR 2024 - -
RegionPLC arXiv
RegionPLC: Regional Point-Language Contrastive Learning for Open-World 3D Scene Understanding		 CVPR 2024 Website GitHub
LEGaussians arXiv
Language Embedded 3D Gaussians for Open-Vocabulary Scene Understanding		 CVPR 2024 Website GitHub
LangSplat arXiv
LangSplat: 3D Language Gaussian Splatting		 CVPR 2024 Website GitHub
Feature 3DGS arXiv
Feature 3DGS: Supercharging 3D Gaussian Splatting to Enable Distilled Feature Fields		 CVPR 2024 Website GitHub
Open3DIS arXiv
Open3DIS: Open-Vocabulary 3D Instance Segmentation with 2D Mask Guidance		 CVPR 2024 Website GitHub
GGSD arXiv
Open Vocabulary 3D Scene Understanding via Geometry Guided Self-Distillation		 ECCV 2024 - GitHub
Gaussian Grouping arXiv
Gaussian Grouping: Segment and Edit Anything in 3D Scenes		 ECCV 2024 Website GitHub
EgoLifter arXiv
EgoLifter: Open-world 3D Segmentation for Egocentric Perception		 ECCV 2024 Website GitHub
OpenIns3D arXiv
OpenIns3D: Snap and Lookup for 3D Open-vocabulary Instance Segmentation		 ECCV 2024 - GitHub
OpenGaussian arXiv
OpenGaussian: Towards Point-Level 3D Gaussian-based Open Vocabulary Understanding		 NeurIPS 2024 Website GitHub
OWL arXiv
Lidar Panoptic Segmentation in an Open World		 IJCV 2024 - GitHub
SAL arXiv
Zero-Shot 4D Lidar Panoptic Segmentation		 CVPR 2025 - -
ULOPS arXiv
Open-Set LiDAR Panoptic Segmentation Guided by Uncertainty-Aware Learning		 IROS 2025 Website -
OVGaussian arXiv
OVGaussian: Generalizable 3D Gaussian Segmentation with Open Vocabularies		 arXiv 2025 - GitHub
LOSC arXiv
LOSC: LiDAR Open-voc Segmentation Consolidator		 arXiv 2025 - -

5. Datasets

⏲️ In chronological order, from the earliest to the latest.

Datasets Paper Venue Website Github
SemanticKITTI arXiv
SemanticKITTI: A Dataset for Semantic Scene Understanding of LiDAR Sequences		 ICCV 2019 Website GitHub
Waymo Open arXiv
Scalability in Perception for Autonomous Driving: Waymo Open Dataset		 CVPR 2020 Website GitHub
SemanticPOSS arXiv
SemanticPOSS: A Point Cloud Dataset with Large Quantity of Dynamic Instances		 IV 2020 Website GitHub
A2D2 arXiv
A2D2: Audi Autonomous Driving Dataset		 arXiv 2020 Website -
RELLIS-3D arXiv
RELLIS-3D Dataset: Data, Benchmarks and Analysis		 ICRA 2021 Website GitHub
PandaSet arXiv
PandaSet: Advanced Sensor Suite Dataset for Autonomous Driving		 ITSC 2021 Website GitHub
SynLiDAR arXiv
Transfer Learning from Synthetic to Real LiDAR Point Cloud for Semantic Segmentation		 AAAI 2022 - GitHub
ScribbleKITTI arXiv
Scribble-Supervised LiDAR Semantic Segmentation		 CVPR 2022 Website GitHub
Synth4D arXiv
GIPSO: Geometrically Informed Propagation for Online Adaptation in 3D LiDAR Segmentation		 ECCV 2022 - GitHub
Panoptic nuScenes arXiv
Panoptic nuScenes: A Large-Scale Benchmark for LiDAR Panoptic Segmentation and Tracking		 RAL 2022 Website GitHub
SemanticSTF arXiv
3D Semantic Segmentation in the Wild: Learning Generalized Models for Adverse-Condition Point Clouds		 CVPR 2023 - GitHub
nuScenes-Occupancy arXiv
OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perception		 ICCV 2023 - GitHub
Robo3D arXiv
Robo3D: Towards Robust and Reliable 3D Perception against Corruptions		 ICCV 2023 Website GitHub
Occ3D arXiv
Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Driving		 NeurIPS 2023 Website GitHub
DAPS3D arXiv
DAPS3D: Domain Adaptive Projective Segmentation of 3D LiDAR Point Clouds		 Access 2023 - GitHub
SSCBench arXiv
SSCBench: A Large-Scale 3D Semantic Scene Completion Benchmark for Autonomous Driving		 IROS 2024 - GitHub

About

🌐 A Roadmap for 3D Scene Understanding in the Wild

worldbench.github.io/3d-in-the-wild

Topics

robotics awesome-list autonomous-driving semantic-segmentation 3d 4d occupancy-prediction lidar-point-cloud panoptic-segmentation semantic-scene-completion embodied-ai 3d-scene-understanding spatial-intelligence

Resources

Readme

License

MIT license
Activity
Custom properties

Stars

23 stars

Watchers

1 watching

Forks

1 fork
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages