-
Notifications
You must be signed in to change notification settings - Fork 741
Quant doc updates #12240
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Quant doc updates #12240
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/12240
Note: Links to docs will display an error until the docs builds have been completed. ❌ 3 New Failures, 15 PendingAs of commit dd47664 with merge base 9d599c9 ( NEW FAILURES - The following jobs have failed:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
This PR needs a
|
GregoryComer
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great. Thanks for updating this.
|
@jerryzh168 can you give the changes a look over too? |
docs/source/quantization-overview.md
Outdated
| 3. Lower the model to the target backend | ||
|
|
||
| In addition to export based quantization (described above), ExecuTorch wants to highlight source based quantizations, accomplished via [torchao](https://github.com/pytorch/ao). Unlike export based quantization, source based quantization directly modifies the model prior to export. One specific example is `Int8DynActInt4WeightQuantizer`. | ||
| ## 1. Create a Backend-Specific Quantizer |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: I think Configure would be more accurate
docs/source/quantization-overview.md
Outdated
|
|
||
| ## 3. Lower the model | ||
|
|
||
| The final step is to lower the quantized_model to the desired backend, as you would an unquantized one. See backend-specific pages for lowering information. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: link to docs as well?
jerryzh168
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
makes sense, wondering if we want to connect with transformer (or optimum-executorch?) as well, like what we did in https://docs.pytorch.org/ao/main/serving.html
The LLM doc overhaul is separate |
|
@pytorchbot cherry-pick --onto release/0.7 -c docs |
Cherry picking #12240The cherry pick PR is at #12260 The following tracker issues are updated: Details for Dev Infra teamRaised by workflow job |
|
@pytorchbot cherry-pick --onto release/0.7 -c “docs” |
|
❌ 🤖 pytorchbot command failed: Try |
Initial draft of quantization doc updates (pytorch#10603)
Initial draft of quantization doc updates (#10603)