8 bit quantization
#514
Replies: 3 comments
-
|
Yeah, int8 quantization would be of interest for me as well! |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
Any update on this? |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
I just tried it with the int4 phi-3 onnx model and it worked fine if that's any help. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Hello,
Are 8 bit quantization methods planned to be supported by ONNXRuntime-genai in the future?
Thanks,
Beta Was this translation helpful? Give feedback.
All reactions