You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
## Automated command to run the benchmark via MLCFlow
10
+
11
+
Please see the [new docs site](https://docs.mlcommons.org/inference/benchmarks/automotive/3d_object_detection/pointpainting/) for an automated way to run this benchmark across different available implementations and do an end-to-end submission with or without docker.
12
+
13
+
You can also do `pip install mlc-scripts` and then use `mlcr` commands for downloading the model and datasets using the commands given in the later sections.
14
+
15
+
### Download model through MLCFlow Automation
16
+
17
+
> [!Note]
18
+
> By default, the waymo dataset is downloaded from the mlcommons official drive. One has to accept the [MLCommons Waymo Open Dataset EULA](https://waymo.mlcommons.org/) to access the dataset files.
> By default, the waymo dataset is downloaded from the mlcommons official drive. One has to accept the [MLCommons Waymo Open Dataset EULA](https://waymo.mlcommons.org/) to access the dataset files.
Contact [MLCommons](https://waymo.mlcommons.org/) support for accessing the Waymo Open Dataset along with the model checkpoints for the reference implementation. You will need to accept a license agreement and will be given directions to download the data with rclone. You will need to place the kitti_format folder under a directory named waymo. There are four total checkpoints 2 for pytorch and 2 for onnx.
### Evaluate the accuracy through MLCFlow Automation
94
+
```bash
95
+
mlcr process,mlperf,accuracy,_waymo --result_dir=<Path to directory where files are generated after the benchmark run>
96
+
```
97
+
98
+
Please click [here](https://github.com/mlcommons/inference/blob/master/automotive/3d-object-detection/accuracy_waymo.py) to view the Python script for evaluating accuracy for the Waymo dataset.
69
99
70
100
### Run the accuracy checker on the accuracy run log
0 commit comments