Skip to content

Commit e3eb18b

Browse files
JokerJohnJokerJohn
committed
add related work
1 parent cb75f4b commit e3eb18b

File tree

1 file changed

+29
-20
lines changed

1 file changed

+29
-20
lines changed

‎README.md‎

Lines changed: 29 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88

99
<sup>1</sup>**HKUST&emsp;&emsp;&emsp;<sup>2</sup>CityU&emsp;&emsp;&emsp;<sup>3</sup>USTB**
1010
<br>
11-
&dagger;project lead&emsp;*corresponding author
11+
&dagger;Project lead&emsp;*Corresponding Author
1212

1313
<a href="https://ieeexplore.ieee.org/document/10910156"><img src='https://img.shields.io/badge/RAL 2025- MapEval -red' alt='Paper PDF'></a><a ><img alt="PRs-Welcome" src="https://img.shields.io/badge/PRs-Welcome-white" /></a>[![GitHub Stars](https://img.shields.io/github/stars/JokerJohn/Cloud_Map_Evaluation.svg)](https://github.com/JokerJohn/Cloud_Map_Evaluation/stargazers)<a href="https://github.com/JokerJohn/PALoc/network/members">
1414
<img alt="FORK" src="https://img.shields.io/github/forks/JokerJohn/Cloud_Map_Evaluation?color=white" />
@@ -51,7 +51,6 @@ These complementary aspects require different evaluation approaches, as global d
5151
| ![image-20250304121753995](./README/image-20250304121753995.png) |
5252
| ------------------------------------------------------------ |
5353

54-
5554
## Results
5655

5756
### Simulated experiments
@@ -71,15 +70,10 @@ These complementary aspects require different evaluation approaches, as global d
7170
| ![image-20241129092052634](./README/image-20241129092052634.png) |
7271
| ------------------------------------------------------------ |
7372

74-
## Computational Efficiency
75-
76-
| ![image-20250322192323830](./README/image-20250322192323830.png) |
77-
| ------------------------------------------------------------ |
78-
79-
## Parameter Sensitivity Analysis
73+
## Efficiency and Parameter Analysis
8074

81-
| ![image-20250322192349614](./README/image-20250322192349614.png) |
82-
| ------------------------------------------------------------ |
75+
| ![image-20250322192323830](./README/image-20250322192323830.png) | ![image-20250322192349614](./README/image-20250322192349614.png) |
76+
| ------------------------------------------------------------ | ------------------------------------------------------------ |
8377

8478
## Datasets
8579

@@ -152,11 +146,13 @@ we have a point cloud map generated by a pose-slam system, and we have a ground
152146
153147
![image-20250214100110872](./README/image-20250214100110872.png)
154148
149+
### Visulization
150+
155151
We can also get a rendered raw distance-error map(10cm) and inlier distance-error map(2cm) in this process, the color R->G->B represent for the distance error at a level of 0-10cm.
156152
157153
![image (4)](./README/image%20(4).png)
158154
159-
if we do not have gt map, we can evaluate the **Mean Map Entropy (MME)**, smaller means better consistency. just set `evaluate_mme: false` in **[config.yaml](map_eval/config/config.yaml)**.
155+
**if we do not have gt ma**p, we can only evaluate the **Mean Map Entropy (MME)**, smaller means better consistency. just set `evaluate_mme: false` in **[config.yaml](map_eval/config/config.yaml)**.
160156

161157
![image (5)](./README/image%20(5).png)
162158

@@ -176,8 +172,9 @@ we can also get a simpe mesh reconstructed from point cloud map.
176172
python3 error-visualization.py
177173
```
178174

179-
| ![image-20250212202920950](./README/image-20250212202920950.png) | ![image-20250212202933255](./README/image-20250212202933255.png) | ![image-20250212203009074](./README/image-20250212203009074.png) | ![image-20250212203025149](./README/image-20250212203025149.png) |
180-
| ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ |
175+
| ![image-20250212202920950](./README/image-20250212202920950.png) | ![image-20250212202933255](./README/image-20250212202933255.png) |
176+
| ------------------------------------------------------------ | ------------------------------------------------------------ |
177+
| ![image-20250212203009074](./README/image-20250212203009074.png) | ![image-20250212203025149](./README/image-20250212203025149.png) |
181178

182179
## Issues
183180

@@ -199,6 +196,18 @@ The primary function of the r**aw rendered map** (left) is to color-code the err
199196

200197
![image (6)](./README/image%20(6).png)
201198

199+
### **Applicable Scenarios:**
200+
201+
1. **With a ground truth map:** All metrics are applicable.
202+
203+
2. **Without a ground truth map**:
204+
205+
Only **MME** can be used for evaluation. It is crucial to remember that the maps being evaluated must be on the same scale.
206+
207+
- For example, **you cannot compare a LIO map with a LIO SLAM map** that has performed loop closure optimization. This is because loop closure adjusts the local point cloud structure, leading to inaccurate MME evaluation. You can compare the MME of different LIO maps.
208+
209+
210+
202211
## Publications
203212

204213
We kindly recommend to cite [our paper](https://arxiv.org/abs/2411.17928) if you find this library useful:
@@ -227,16 +236,16 @@ We kindly recommend to cite [our paper](https://arxiv.org/abs/2411.17928) if you
227236

228237
## Related Package
229238

230-
The folloing work using MapEval for map evalution.
239+
The folloing works use MapEval for map evalution.
231240

232-
| Work | Task | Date | Metrics | Demo |
241+
| Work | Tasks | Date | Metrics | Demo |
233242
| ------------------------------------------------------------ | ---------------------------------------- | ---------- | --------- | ------------------------------------------------------------ |
234243
| **[CompSLAM](https://arxiv.org/abs/2505.06483)** | Multi-Modal Localization<br/>and Mapping | Arxiv'2025 | AWD/SCS | ![image-20250513202215918](./README/image-20250513202215918.png) |
235-
| [ELite](https://github.com/dongjae0107/ELite) | LiDAR-based Lifelong Mapping | ICRA'2025 | AC/CD | ![image-20250513202654237](./README/image-20250513202654237.png) |
236-
| [PALoc](https://github.com/JokerJohn/PALoc) | Prior-Assisted 6-DoF Localization | TMECH'2024 | AC/CD | ![image-20250513205043100](./README/image-20250513205043100.png) |
237-
| [MS-Mapping](https://github.com/JokerJohn/MS-Mapping) | Multi-session LiDAR mapping | Arxiv'2024 | AC/CD/MME | ![image-20240730152951528](./README/image-20240730152951528.png) |
238-
| [FusionPortableV2](https://journals.sagepub.com/doi/full/10.1177/02783649241303525) | SLAM Dataset | IJRR'2024 | COM/CD | ![img](./README/10.1177_02783649241303525-fig15.jpg) |
239-
| [GEODE](https://github.com/PengYu-Team/GEODE_dataset) | SLAM Dataset | IJRR'2025 | | ![image-20250513204209752](./README/image-20250513204209752.png) |
244+
| [**ELite**](https://github.com/dongjae0107/ELite) | LiDAR-based Lifelong Mapping | ICRA'2025 | AC/CD | ![image-20250513202654237](./README/image-20250513202654237.png) |
245+
| [**PALoc**](https://github.com/JokerJohn/PALoc) | Prior-Assisted 6-DoF Localization | TMECH'2024 | AC/CD | ![image-20250513205043100](./README/image-20250513205043100.png) |
246+
| [**MS-Mapping**](https://github.com/JokerJohn/MS-Mapping) | Multi-session LiDAR mapping | Arxiv'2024 | AC/CD/MME | ![image-20240730152951528](./README/image-20240730152951528.png) |
247+
| [**FusionPortableV2**](https://journals.sagepub.com/doi/full/10.1177/02783649241303525) | SLAM Dataset | IJRR'2024 | COM/CD | ![img](./README/10.1177_02783649241303525-fig15.jpg) |
248+
| [**GEODE**](https://github.com/PengYu-Team/GEODE_dataset) | SLAM Dataset | IJRR'2025 | | ![image-20250513204209752](./README/image-20250513204209752.png) |
240249

241250
## Contributors
242251

0 commit comments

Comments
 (0)