Skip to content

Commit d7a6652

Browse files
committed
fix error test
Signed-off-by: Yang Wang <[email protected]>
1 parent ffe6839 commit d7a6652

File tree

3 files changed

+249
-204
lines changed

3 files changed

+249
-204
lines changed

.ci/scripts/benchmark_tooling/README.md

Lines changed: 11 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -43,13 +43,21 @@ python3 .ci/scripts/benchmark_tooling/get_benchmark_analysis_data.py \
4343
- `--outputDir`: Directory to save output files (default: current directory)
4444

4545
##### Filtering Options:
46-
Notice, the filter needs full name matchings with correct format, to see all the options of the filter choices, please run the script with `--print-all-table-info`, and pay attention to section `Full list of table info from HUD API` with the field 'info', which contains normalized data we use to filter records from the original metadata 'groupInfo'.
47-
The filter block any record if it does not in any of the filter keywords.
4846

49-
- `--devices`: Filter by specific device names (e.g., "samsung-galaxy-s22-5g", "samsung-galaxy-s22plus-5g")
47+
- `--private-device-pools`: Filter by private device pool names (e.g., "samsung-galaxy-s22-5g", "samsung-galaxy-s22plus-5g")
5048
- `--backends`: Filter by specific backend names (e.g., "qnn-q8" , ""llama3-spinquan)
5149
- `--models`: Filter by specific model names (e.g "mv3" "meta-llama-llama-3.2-1b-instruct-qlora-int4-eo8")
5250

51+
#### Example Usage
52+
call multiple private device pools and models
53+
```bash
54+
python3 get_benchmark_analysis_data.py \
55+
--startTime "2025-06-01T00:00:00" \
56+
--endTime "2025-06-11T00:00:00" \
57+
--private-device-pools 'apple_iphone_15_private' 'samsung_s22_private'
58+
--models 'mv3' 'allenai/OLMo-1B-hf'
59+
```
60+
5361
#### Working with Output Files CSV and Excel
5462

5563
You can use methods in `common.py` to convert the file data back to DataFrame format, those methods read the first row in csv/excel file, and return result with format list of {"groupInfo":DICT, "df":df.Dataframe{}} format.

0 commit comments

Comments
 (0)