Add migx bert squad quant example #441
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Added additional pieces with argparse to select which version of squad we want to test, batch size, sequence length and some other useful things.
…runs Seeing stall on larger batch sizes. Adding flags for debugging.
- Allow for mixed precsion via fp16 flag -Allow for variable batch -Allow for variable calibrate data size
Running into an issue with shapes with the calibration tools if I break up this calibration read. This needs a large amount of memory to create the histogram.
Contributor
Author
|
ping @cloudhan @PeixuanZuo @ytaous |
cloudhan
previously requested changes
Jun 11, 2024
cloudhan
left a comment
cloudhan
left a comment
There was a problem hiding this comment.
There are some unresolved conflicts leak into your commit.
quantization/image_classification/migraphx/resnet50/e2e_migraphx_resnet_example.py
Outdated
Show resolved
Hide resolved
This comment was marked as resolved.
This comment was marked as resolved.
gyulaz-htec
reviewed
Jun 14, 2024
gyulaz-htec
reviewed
Jun 14, 2024
gyulaz-htec
reviewed
Jun 14, 2024
Useful if we want to use another falvor of bert for now. TODO need to handle/fix some of the input/output arg maps vs the input data vs model input/outputs
Another knob to tune/play with in perf runs. Right now just allow this to be default.
…o handle features in each example Our MIGraphX EP requires a recompile of the model if we constantly change the input dimensions or batch size of the parameters. Without this we actually cause a slowdown with the larger batch size runs as we tend to go above the feature index. A workaround is to ensure that batch size stays constant as we feed data into the model we're testing to get inference timing and accuracy results via repeating the same sample until we have enough data for a proper batch size.
TedThemistokleous
force-pushed
the
add_migx_bert_squad_quant_example
branch
from
feec95d to
a258158
Compare
tianleiwu
reviewed
Aug 17, 2024
tianleiwu
reviewed
Aug 17, 2024
tianleiwu
reviewed
Aug 19, 2024
|
@cloudhan, please help run the example end-2-end to see whether it works. |
|
@tianleiwu We don't work on AMD related EPs anymore. |
tianleiwu
dismissed
cloudhan’s stale review
cloudhan don't work on the EP now.
Contributor
Author
That's news to me. When did that occur. Is this something we need to also bring up with our other Devs? |
tianleiwu
approved these changes
Oct 25, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add an MIGraphX based quantization test the inference examples repo