Skip to content

Commit a09ef11

Browse files
authored
Add MLC Automation commands (#2115)
1 parent 7bf5997 commit a09ef11

File tree

1 file changed

+30
-0
lines changed

1 file changed

+30
-0
lines changed

automotive/3d-object-detection/README.md

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,30 @@ This is the reference implementation for the MLPerf automotive 3D detection benc
66
| ---- | ---- | ---- | ---- | ---- | ---- |
77
| PointPainting | 0.5425 mAP | Waymo Open Dataset | https://github.com/rod409/pp | fp32 | Single-Stream 99.9 percentile |
88

9+
## Automated command to run the benchmark via MLCFlow
10+
11+
Please see the [new docs site](https://docs.mlcommons.org/inference/benchmarks/automotive/3d_object_detection/pointpainting/) for an automated way to run this benchmark across different available implementations and do an end-to-end submission with or without docker.
12+
13+
You can also do `pip install mlc-scripts` and then use `mlcr` commands for downloading the model and datasets using the commands given in the later sections.
14+
15+
### Download model through MLCFlow Automation
16+
17+
> [!Note]
18+
> By default, the waymo dataset is downloaded from the mlcommons official drive. One has to accept the [MLCommons Waymo Open Dataset EULA](https://waymo.mlcommons.org/) to access the dataset files.
19+
20+
```
21+
mlcr get,ml-model,pointpainting --outdirname=<path_to_download> -j
22+
```
23+
24+
### Download dataset through MLCFlow Automation
25+
26+
> [!Note]
27+
> By default, the waymo dataset is downloaded from the mlcommons official drive. One has to accept the [MLCommons Waymo Open Dataset EULA](https://waymo.mlcommons.org/) to access the dataset files.
28+
29+
```
30+
mlcr get,dataset,waymo --outdirname=<path_to_download> -j
31+
```
32+
933
## Downloading the dataset and model checkpoints
1034
Contact [MLCommons](https://waymo.mlcommons.org/) support for accessing the Waymo Open Dataset along with the model checkpoints for the reference implementation. You will need to accept a license agreement and will be given directions to download the data with rclone. You will need to place the kitti_format folder under a directory named waymo. There are four total checkpoints 2 for pytorch and 2 for onnx.
1135

@@ -66,6 +90,12 @@ Add the accuracy flag
6690
cd /inference/automotive/3d-object-detection
6791
python main.py --dataset waymo --dataset-path /waymo/kitti_format/ --lidar-path <checkpoint_path>/pp_ep36.pth --segmentor-path <checkpoint_path>/best_deeplabv3plus_resnet50_waymo_os16.pth --accuracy
6892
```
93+
### Evaluate the accuracy through MLCFlow Automation
94+
```bash
95+
mlcr process,mlperf,accuracy,_waymo --result_dir=<Path to directory where files are generated after the benchmark run>
96+
```
97+
98+
Please click [here](https://github.com/mlcommons/inference/blob/master/automotive/3d-object-detection/accuracy_waymo.py) to view the Python script for evaluating accuracy for the Waymo dataset.
6999

70100
### Run the accuracy checker on the accuracy run log
71101
```

0 commit comments

Comments
 (0)