Skip to content

Commit 3e46f63

Browse files
mh0797mh0797
committed
devkit-v0.2
1 parent 526e9ac commit 3e46f63

25 files changed

+283
-84
lines changed

‎README.md‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,10 @@
4747

4848

4949
## Changelog <a name="changelog"></a>
50-
50+
- **`[2024/03/11]`** NAVSIM v0.2 release
51+
- Easier installation and download
52+
- mini and test split integration
53+
- Privileged `Human` agent
5154
- **`[2024/02/20]`** NAVSIM v0.1 release (initial demo)
5255
- OpenScene-mini sensor blobs and annotation logs
5356
- Naive `ConstantVelocity` agent

‎docs/agents.md‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,4 @@ Let’s dig deeper into this class.
1717
Given this input, you will need to override the `compute_trajectory()` method and output a `Trajectory`. This is an array of BEV poses (with x, y and heading in local coordinates), as well as a `TrajectorySampling` config object that indicates the duration and frequency of the trajectory. The PDM score is evaluated for a horizon of 4 seconds at a frequency of 10Hz. The `TrajectorySampling` config facilitates interpolation when the output frequency is different from the one used during evaluation.
1818

1919
We provide a naive constant velocity agent as part of our demo, for reference:
20-
2120
https://github.com/autonomousvision/navsim/blob/51cecd51aa70b0e6bcfb3541b91ae88f2a78a25e/navsim/agents/constant_velocity_agent.py#L9
22-

‎docs/cache.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,4 +9,5 @@ OpenScene is a compact redistribution of the large-scale [nuPlan dataset](https:
99
cd $NAVSIM_DEVKIT_ROOT/scripts/
1010
./run_metric_caching.sh
1111
```
12-
Note that you have to set the `LOG_PATH` and `METRIC_CACHE_PATH` first. `LOG_PATH` has to point to the [OpenScene annotations](https://github.com/autonomousvision/navsim/blob/main/docs/install.md#1-download-the-demo-data). The cache will be saved under the `METRIC_CACHE_PATH` which you can chose freely.
12+
13+
This will create the meric cache under `$NUPLAN_EXP_ROOT/metric_cache`, where `$NUPLAN_EXP_ROOT` is defined by the environment variable set during installation.

‎docs/install.md‎

Lines changed: 44 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -2,36 +2,59 @@
22

33
To get started with NAVSIM:
44

5-
### 1. Download the demo data
6-
First, you need to download the OpenScene mini logs and sensor blobs, as well as the nuPlan maps.
5+
### 1. Clone the navsim-devkit
6+
Clone the repository
7+
```
8+
git clone https://github.com/autonomousvision/navsim.git
9+
cd navsim
10+
```
11+
### 2. Download the demo data
12+
You need to download the OpenScene logs and sensor blobs, as well as the nuPlan maps.
13+
We provide scripts to download the nuplan maps, the mini split and the test split.
14+
Navigate to the download directory and download the maps
715

816
**NOTE: Please check the [LICENSE file](https://motional-nuplan.s3-ap-northeast-1.amazonaws.com/LICENSE) before downloading the data.**
917

1018
```
11-
wget https://motional-nuplan.s3-ap-northeast-1.amazonaws.com/public/nuplan-v1.1/nuplan-maps-v1.1.zip && unzip nuplan-maps-v1.1.zip
12-
wget https://s3.eu-central-1.amazonaws.com/avg-projects-2/navsim/navsim_logs.zip && unzip navsim_logs.zip
13-
wget https://s3.eu-central-1.amazonaws.com/avg-projects-2/navsim/sensor_blobs.zip && unzip sensor_blobs.zip
19+
cd download && ./download_maps
1420
```
15-
The `sensor_blobs` file is fairly large (90 GB). For understanding the metrics and testing the naive baselines in the demo, this is not strictly necessary.
1621

17-
### 2. Install the navsim-devkit
18-
Next, setup the environment and install navsim.
19-
Clone the repository
22+
Next download the mini split and the test split
2023
```
21-
git clone https://github.com/kashyap7x/navsim.git
22-
cd navsim
23-
```
24-
Then create a new environment and install the required dependencies:
25-
```
26-
conda env create --name navsim -f environment.yml
27-
conda activate navsim
28-
pip install -e .
24+
./download_mini
25+
./download_test
2926
```
3027

28+
**The mini split and the test split take around ~160GB and ~220GB of memory respectively**
29+
30+
This will download the splits into the download directory. From there, move it to create the following structure.
31+
```angular2html
32+
~/navsim_workspace
33+
├── navsim (containing the devkit)
34+
├── exp
35+
└── dataset
36+
   ├── maps
37+
   ├── navsim_logs
38+
| ├── test
39+
   │ └── mini
40+
   └── sensor_blobs
41+
├── test
42+
   └── mini
43+
```
3144
Set the required environment variables, by adding the following to your `~/.bashrc` file
45+
Based on the structure above, the environment variables need to be defined as:
3246
```
33-
export NAVSIM_DEVKIT_ROOT=/path/to/navsim/devkit
34-
export NUPLAN_EXP_ROOT=/path/to/navsim/exp
35-
export NUPLAN_MAPS_ROOT=/path/to/nuplan/maps
36-
export OPENSCENE_DATA_ROOT=/path/to/openscene
47+
export NUPLAN_MAPS_ROOT="$HOME/navsim_workspace/dataset/maps"
48+
export NUPLAN_EXP_ROOT="$HOME/navsim_workspace/exp"
49+
export NAVSIM_DEVKIT_ROOT="$HOME/navsim_workspace/navsim"
50+
export OPENSCENE_DATA_ROOT="$HOME/navsim_workspace/dataset"
3751
```
52+
53+
### 3. Install the navsim-devkit
54+
Finally, install navsim.
55+
To this end, create a new environment and install the required dependencies:
56+
```
57+
conda env create --name navsim -f environment.yml
58+
conda activate navsim
59+
pip install -e .
60+
```

‎docs/metrics.md‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16,10 +16,11 @@ i.e., `PDM Score = NC * DAC * DDC * (5*TTC + 2*C + 5*EP) / 12`
1616
To evaluate the PDM score for an agent you can run:
1717
```
1818
cd $NAVSIM_DEVKIT_ROOT/scripts/
19-
./run_pdm_score_evaluation.sh
19+
./run_cv_pdm_score_evaluation.sh
2020
```
21-
**Note: You have to adapt the variables `LOG_PATH` so that it points to the [logs (annotations)](https://github.com/autonomousvision/navsim/blob/main/docs/install.md#1-download-the-demo-data), `METRIC_CACHE_PATH` so that it points to the [metric cache](https://github.com/autonomousvision/navsim/blob/main/docs/cache.md#understanding-the-data-format-and-classes) and `OUTPUT_DIR` so that it points to a directory where the evaluation csv will be stored**
2221

2322
By default, this will generate an evaluation csv for a simple constant velocity [planning baseline](https://github.com/autonomousvision/navsim/blob/main/docs/agents.md#output). You can modify the script to evaluate your own planning agent.
23+
2424
For instance, you can add a new config for your agent under `$NAVSIM_DEVKIT_ROOT/navsim/navsim/planning/script/config/pdm_scoring/agent/my_new_agent.yaml`.
2525
Then, running your own agent is as simple as adding an override `agent=my_new_agent` to the script.
26+
You can find an example in `run_human_agent_pdm_score_evaluation.sh`

‎download/download_maps.sh‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
wget https://motional-nuplan.s3-ap-northeast-1.amazonaws.com/public/nuplan-v1.1/nuplan-maps-v1.1.zip
2+
unzip nuplan-maps-v1.1.zip
3+
rm nuplan-maps-v1.1.zip
4+
mv nuplan-maps-v1.0 maps

‎download/download_mini.sh‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
wget https://huggingface.co/datasets/OpenDriveLab/OpenScene/resolve/main/openscene-v1.1/openscene_metadata_mini.tgz
2+
tar -xzf openscene_metadata_mini.tgz
3+
rm openscene_metadata_mini.tgz
4+
5+
for split in {0..31}; do
6+
wget https://huggingface.co/datasets/OpenDriveLab/OpenScene/resolve/main/openscene-v1.1/openscene_sensor_mini_camera/openscene_sensor_mini_camera_${split}.tgz
7+
echo "Extracting file openscene_sensor_mini_camera_${split}.tgz"
8+
tar -xzf openscene_sensor_mini_camera_${split}.tgz
9+
rm openscene_sensor_mini_camera_${split}.tgz
10+
done
11+
12+
for split in {0..31}; do
13+
wget https://huggingface.co/datasets/OpenDriveLab/OpenScene/resolve/main/openscene-v1.1/openscene_sensor_mini_lidar/openscene_sensor_mini_lidar_${split}.tgz
14+
echo "Extracting file openscene_sensor_mini_lidar_${split}.tgz"
15+
tar -xzf openscene_sensor_mini_lidar_${split}.tgz
16+
rm openscene_sensor_mini_lidar_${split}.tgz
17+
done
18+
19+
mv openscene_v1.1/meta_datas mini_navsim_logs
20+
rm -r openscene_v1.1
21+
22+
mv openscene-v1.1/sensor_blobs mini_sensor_blobs
23+
rm -r openscene-v1.1

‎download/download_test.sh‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
wget https://huggingface.co/datasets/OpenDriveLab/OpenScene/resolve/main/openscene-v1.1/openscene_metadata_test.tgz
2+
tar -xzf openscene_metadata_test.tgz
3+
rm openscene_metadata_test.tgz
4+
5+
for split in {0..31}; do
6+
wget https://huggingface.co/datasets/OpenDriveLab/OpenScene/resolve/main/openscene-v1.1/openscene_sensor_test_camera/openscene_sensor_test_camera_${split}.tgz
7+
echo "Extracting file openscene_sensor_test_camera_${split}.tgz"
8+
tar -xzf openscene_sensor_test_camera_${split}.tgz
9+
rm openscene_sensor_test_camera_${split}.tgz
10+
done
11+
12+
for split in {0..31}; do
13+
wget https://huggingface.co/datasets/OpenDriveLab/OpenScene/resolve/main/openscene-v1.1/openscene_sensor_test_lidar/openscene_sensor_test_lidar_${split}.tgz
14+
echo "Extracting file openscene_sensor_test_lidar_${split}.tgz"
15+
tar -xzf openscene_sensor_test_lidar_${split}.tgz
16+
rm openscene_sensor_test_lidar_${split}.tgz
17+
done
18+
19+
mv openscene_v1.1/meta_datas test_navsim_logs
20+
rm -r openscene_v1.1
21+
mkdir
22+
mv openscene-v1.1/sensor_blobs test_sensor_blobs
23+
rm -r openscene-v1.1

‎navsim/agents/abstract_agent.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ class AbstractAgent(abc.ABC):
1212
"""
1313
Interface for a generic end-to-end agent.
1414
"""
15+
requires_scene = False
1516

1617
def __new__(cls, *args: Any, **kwargs: Any) -> AbstractAgent:
1718
"""

‎navsim/agents/constant_velocity_agent.py‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,8 @@
88

99
class ConstantVelocityAgent(AbstractAgent):
1010

11+
requires_scene = False
12+
1113
def __init__(
1214
self,
1315
trajectory_sampling: TrajectorySampling = TrajectorySampling(

0 commit comments

Comments
 (0)