[Autotuner] Plotting utility + test#2394
[Autotuner] Plotting utility + test#2394vvbandeira merged 15 commits intoThe-OpenROAD-Project:masterfrom
Conversation
luarss
commented
Sep 29, 2024
luarss
commented
- Plotting utility
- README
vvbandeira
left a comment
vvbandeira
left a comment
There was a problem hiding this comment.
Please add a call to this script in the test_helper.py so we test this script.
luarss
changed the title
| except Exception as e: | ||
| print(f"Error in {fname}: {e}") | ||
| continue |
There was a problem hiding this comment.
Should we keep a list of the files that were not processed so we can print them at the very end in order for the error not to be buried in the logs? And should the script fail silently in this case?
There was a problem hiding this comment.
And should the script fail silently in this case?
Just to clarify, this exception happens when any Autotuner run does not proceed to the "finish" stage, it does not fail silently
Should we keep a list of the files that were not processed so we can print them at the very end in order for the error not to be buried in the logs?
Sure I can add that.
| z = np.polyfit(df["timestamp"], df[key], 1) | ||
| p = np.poly1d(z) |
There was a problem hiding this comment.
z I assume is the z-axis; what is p?
There was a problem hiding this comment.
z is the np array of polynomial (ax+b) coefficients, and p is the actual polynomial function.
There was a problem hiding this comment.
Then let's use more descriptive names, maybe:
z --> coeff
p --> poly_func
| plt.boxplot(df[key]) | ||
| plt.ylabel(key) | ||
| plt.title(f"{key} Boxplot") | ||
| plt.savefig(f"images/{key}-boxplot.png") |
There was a problem hiding this comment.
Let's also parametrize the output folder to save images from CI into the reports folder for the design. This way it will be automatically saved to the build artifacts.
| python3 -m unittest tools.AutoTuner.test.smoke_test_sample_iteration.${PLATFORM}SampleIterationSmokeTest.test_sample_iteration | ||
|
|
||
| if [ "$PLATFORM" == "asap7" ] && [ "$DESIGN" == "gcd" ]; then | ||
| if [ "$PLATFORM" == "asap7" ] && [ "$DESIGN_NAME" == "gcd" ]; then |
There was a problem hiding this comment.
The first check will never be true because you are casting to uppercase on line 14.
Would it be simpler/easier if we just use the platform name everywhere and just remove the dashes if that is not supported by the unittest Python lib?
There was a problem hiding this comment.
The main concern is that I prefer name of smoke tests to be capitalized. We could also do something like asap7SampleIterationSmokeTest, but might be less readable
Perhaps a UPPERCASE_PLATFORM would make the distinction clearer?
There was a problem hiding this comment.
Modifying PLATFORM or another variable already in use somewhere else in the code can easily lead to confusion. Other options could be PLATFORM_UT where UT is 'under test' or TEST_PLATFORM. That said, I don't see any issue with asap7SampleIterationSmokeTest.
flow/test/test_autotuner.sh
Outdated
| echo "No experiments found for plotting" | ||
| exit 0 |
There was a problem hiding this comment.
This is still exiting with a success code.
There was a problem hiding this comment.
This is returning 0 because I didn't want to error out for smoke tests that are not suitable for plotting. I will modify it to only run plot.py for valid smoke tests then.
There was a problem hiding this comment.
We could then change line 31 to list either manually the tests we want or use a better regex in the experiment names to differentiate whether the plot is available or not.
Signed-off-by: Jack Luar <jluar@precisioninno.com>
Signed-off-by: luarss <jluar@precisioninno.com>
Signed-off-by: Vitor Bandeira <vvbandeira@users.noreply.github.com>
Signed-off-by: luarss <jluar@precisioninno.com>
Signed-off-by: Jack Luar <jluar@precisioninno.com>
Signed-off-by: Jack Luar <jluar@precisioninno.com>
Signed-off-by: Jack Luar <jluar@precisioninno.com>
- run plot.py for all log files in the autotuner supported design/platforms - parameterised img_dir, so we can store all AT image runs in build artifacts - argparse interface now takes in platform, design, experiment for easier usage - chdir before running plot.py - add docstrings - more error handling Signed-off-by: Jack Luar <jluar@precisioninno.com>
Signed-off-by: luarss <jluar@precisioninno.com>
Signed-off-by: Jack Luar <jluar@precisioninno.com>
Signed-off-by: Jack Luar <jluar@precisioninno.com>
Signed-off-by: Jack Luar <jluar@precisioninno.com>
Signed-off-by: Jack Luar <jluar@precisioninno.com>
Signed-off-by: Jack Luar <jluar@precisioninno.com>
Signed-off-by: Jack Luar <jluar@precisioninno.com>
2 participants
