Skip to content

Commit 615ef69

Browse files
authored
Update README.md
1 parent c7ca227 commit 615ef69

File tree

1 file changed

+28
-1
lines changed

1 file changed

+28
-1
lines changed

README.md

Lines changed: 28 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -87,13 +87,40 @@ We define two key metrics here: **rectified speedup** and **GraphNet Score**. Re
8787

8888
**Demo: How to benchmark your compiler on the model:**
8989

90+
1. Benchmark
91+
92+
We use ```graph_net/benchmark_demo.sh``` to benchmark GraphNet computation graph samples:
93+
94+
```
95+
bash graph_net/benchmark_demo.sh &
96+
```
97+
98+
The script will run ```graph_net.torch.test_compiler``` with specific batch and log configurations.
99+
100+
Or you can customize and use ```graph_net.torch.test_compiler``` yourself:
101+
90102
```
91103
python3 -m graph_net.torch.test_compiler \
92104
--model-path $GRAPH_NET_EXTRACT_WORKSPACE/model_name/ \
93-
--compiler /path/to/custom/compiler
105+
--compiler /path/to/custom/compiler/ \
106+
--output-dir /path/to/save/JSON/result/file/
94107
# Note: if --compiler is omitted, PyTorch’s built-in compiler is used by default
95108
```
96109

110+
2. Analysis
111+
112+
After processing, we provide ```graph_net/analysis.py``` to generate [violin plot](https://en.m.wikipedia.org/wiki/Violin_plot) based on the JSON results.
113+
114+
```
115+
python3 graph_net/analysis.py \
116+
--benchmark-path /path/to/read/JSON/result/file/ \
117+
--output-dir /path/to/save/output/figures/
118+
```
119+
120+
After executing, one summary plot of results on all compilers (as shown below in "Evaluation Results Example"), as well as multiple sub-plots of results in categories (model tasks, Library...) on a single compiler.
121+
122+
The script was design to process a file structure as /benchmark_path/compiler_name/category_name/, for example /benchmark_logs/paddle/nlp/. And items on x-axis are identified by name of the folders. So you can modify ```read_all_speedups``` function to fit the benchmark settings on your demand.
123+
97124
### Evaluation Results Example
98125

99126
<div align="center">

0 commit comments

Comments
 (0)