Skip to content

Conversation

@metascroy
Copy link
Contributor

Initial draft of quantization doc updates (#10603)

@pytorch-bot
Copy link

pytorch-bot bot commented Jul 4, 2025

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/12240

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures, 15 Pending

As of commit dd47664 with merge base 9d599c9 (image):

NEW FAILURES - The following jobs have failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 4, 2025
@metascroy metascroy changed the title Quant docs Quant doc updates Jul 4, 2025
@github-actions
Copy link

github-actions bot commented Jul 4, 2025

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

@metascroy metascroy marked this pull request as ready for review July 7, 2025 19:44
@metascroy metascroy requested a review from mergennachin as a code owner July 7, 2025 19:44
Copy link
Member

@GregoryComer GregoryComer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great. Thanks for updating this.

@metascroy
Copy link
Contributor Author

@jerryzh168 can you give the changes a look over too?

3. Lower the model to the target backend

In addition to export based quantization (described above), ExecuTorch wants to highlight source based quantizations, accomplished via [torchao](https://github.com/pytorch/ao). Unlike export based quantization, source based quantization directly modifies the model prior to export. One specific example is `Int8DynActInt4WeightQuantizer`.
## 1. Create a Backend-Specific Quantizer
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I think Configure would be more accurate


## 3. Lower the model

The final step is to lower the quantized_model to the desired backend, as you would an unquantized one. See backend-specific pages for lowering information.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: link to docs as well?

Copy link
Contributor

@jerryzh168 jerryzh168 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

makes sense, wondering if we want to connect with transformer (or optimum-executorch?) as well, like what we did in https://docs.pytorch.org/ao/main/serving.html

@metascroy
Copy link
Contributor Author

makes sense, wondering if we want to connect with transformer (or optimum-executorch?) as well, like what we did in https://docs.pytorch.org/ao/main/serving.html

The LLM doc overhaul is separate

@metascroy metascroy merged commit 8497ea7 into main Jul 8, 2025
91 of 96 checks passed
@metascroy metascroy deleted the quant-docs branch July 8, 2025 00:40
@metascroy
Copy link
Contributor Author

@pytorchbot cherry-pick --onto release/0.7 -c docs

pytorchbot pushed a commit that referenced this pull request Jul 8, 2025
Initial draft of quantization doc updates
(#10603)

(cherry picked from commit 8497ea7)
@pytorchbot
Copy link
Collaborator

Cherry picking #12240

The cherry pick PR is at #12260 The following tracker issues are updated:

Details for Dev Infra team Raised by workflow job

@metascroy
Copy link
Contributor Author

@pytorchbot cherry-pick --onto release/0.7 -c “docs”

@pytorch-bot
Copy link

pytorch-bot bot commented Jul 8, 2025

❌ 🤖 pytorchbot command failed:

@pytorchbot cherry-pick: error: argument -c/--classification: invalid choice: '“docs”' (choose from 'regression', 'critical', 'fixnewfeature', 'docs', 'release')

usage: @pytorchbot cherry-pick --onto ONTO [--fixes FIXES] -c
                               {regression,critical,fixnewfeature,docs,release}

Try @pytorchbot --help for more info.

Tanish2101 pushed a commit to Tanish2101/executorch that referenced this pull request Jul 9, 2025
Initial draft of quantization doc updates
(pytorch#10603)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants