Skip to content

Commit e537f5e

Browse files
authored
Create tests.md
1 parent cdc1eaf commit e537f5e

File tree

1 file changed

+37
-0
lines changed

1 file changed

+37
-0
lines changed

tests.md

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
The below tables lists the actively tests being carried out by MLCommons for the currently active MLPerf inference benchmarks. Please contact [Support](support@mlcommons.org) if you would like to add any new tests here. The tests are run using GitHub actions and the test results go to the below two repositories.
2+
1. Short Runs: https://github.com/mlcommons/mlperf_inference_test_submissions_v5.0/
3+
2. Full Runs: https://github.com/mlcommons/mlperf_inference_unofficial_submissions_v5.0/
4+
5+
These reporitories are per inference round.
6+
7+
## Reference Implementation
8+
| Benchmark | CPU/GPU | Test Type |
9+
|----------------------|---------|-----------|
10+
| resnet50 | CPU | Short |
11+
| retinanet | CPU | Short |
12+
| 3d-unet | - | TBD |
13+
| bert | CPU | Short |
14+
| gptj | Both | Short |
15+
| llama2-70b | CPU | Short |
16+
| mixtral-8x7b | CPU | Short |
17+
| llama3.1-405b | - | - |
18+
| rgat | CPU | Short |
19+
| pointpainting | GPU | - |
20+
| stable-diffusion-xl | GPU | Short |
21+
| dlrm_v2 | CPU | Short |
22+
23+
## Nvidia Implementation
24+
| Benchmark | Test Type |
25+
|----------------------|-----------|
26+
| resnet50 | Full |
27+
| retinanet | Full |
28+
| 3d-unet | Full |
29+
| bert | Full |
30+
| gptj | Full |
31+
| llama2-70b | - |
32+
| mixtral-8x7b | - |
33+
| llama3.1-405b | - |
34+
| rgat | NA |
35+
| pointpainting | NA |
36+
| stable-diffusion-xl | TBD |
37+
| dlrm_v2 | TBD |

0 commit comments

Comments
 (0)