-
Notifications
You must be signed in to change notification settings - Fork 794
Fix DecomposeLayerNormPass to handle 6-arg layer_norm #16516
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/16516
Note: Links to docs will display an error until the docs builds have been completed. ❌ 1 New Failure, 1 Cancelled Job, 1 Unrelated FailureAs of commit 49be1d5 with merge base 10f72fc ( NEW FAILURE - The following job has failed:
CANCELLED JOB - The following job was cancelled. Please retry:
UNSTABLE - The following job is marked as unstable, possibly due to flakiness on trunk:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
This PR needs a
|
cc93676 to
0b027d2
Compare
Summary: ## Problem When using `nn.LayerNorm` in models that go through the ARM backend's quantization flow, the `DecomposeLayerNormPass` fails with: ``` ValueError: DecomposeLayerNormPass: too many values to unpack (expected 2) ``` This happens because `torch.ops.aten.layer_norm.default` has **6 arguments**: ``` layer_norm(input, normalized_shape, weight, bias, eps, cudnn_enable) ``` But `DecomposeLayerNormPass` only handled up to 5 arguments (for `native_layer_norm`). The error occurs during `transform_for_annotation_pipeline` in the ARM quantizer, which runs before edge transformation when the op is still `aten.layer_norm.default`. ## Solution Add `case 6:` to the `match len(args)` block in `DecomposeLayerNormPass.call()` to handle the 6th argument (`cudnn_enable`). This argument is simply ignored during decomposition since it's only relevant for cuDNN GPU optimization. --- > Generated by [Confucius Code Assist (CCA)](https://www.internalfb.com/wiki/Confucius/Analect/Shared_Analects/Confucius_Code_Assist_(CCA)/) [Confucius Session](https://www.internalfb.com/confucius?host=92481.od.fbinfra.net&port=8086&tab=Chat&session_id=eace3d92-ed78-11f0-b67c-c7843469b0d5&entry_name=Code+Assist), [Trace](https://www.internalfb.com/confucius?session_id=eace3d92-ed78-11f0-b67c-c7843469b0d5&tab=Trace) Reviewed By: JacobSzwejbka Differential Revision: D90395786
0b027d2 to
862df9c
Compare
Summary: ## Problem When using `nn.LayerNorm` in models that go through the ARM backend's quantization flow, the `DecomposeLayerNormPass` fails with: ``` ValueError: DecomposeLayerNormPass: too many values to unpack (expected 2) ``` This happens because `torch.ops.aten.layer_norm.default` has **6 arguments**: ``` layer_norm(input, normalized_shape, weight, bias, eps, cudnn_enable) ``` But `DecomposeLayerNormPass` only handled up to 5 arguments (for `native_layer_norm`). The error occurs during `transform_for_annotation_pipeline` in the ARM quantizer, which runs before edge transformation when the op is still `aten.layer_norm.default`. ## Solution Add `case 6:` to the `match len(args)` block in `DecomposeLayerNormPass.call()` to handle the 6th argument (`cudnn_enable`). This argument is simply ignored during decomposition since it's only relevant for cuDNN GPU optimization. --- > Generated by [Confucius Code Assist (CCA)](https://www.internalfb.com/wiki/Confucius/Analect/Shared_Analects/Confucius_Code_Assist_(CCA)/) [Confucius Session](https://www.internalfb.com/confucius?host=92481.od.fbinfra.net&port=8086&tab=Chat&session_id=eace3d92-ed78-11f0-b67c-c7843469b0d5&entry_name=Code+Assist), [Trace](https://www.internalfb.com/confucius?session_id=eace3d92-ed78-11f0-b67c-c7843469b0d5&tab=Trace) Reviewed By: JacobSzwejbka Differential Revision: D90395786
862df9c to
49be1d5
Compare
Summary:
Problem
When using
nn.LayerNormin models that go through the ARM backend's quantization flow, theDecomposeLayerNormPassfails with:This happens because
torch.ops.aten.layer_norm.defaulthas 6 arguments:But
DecomposeLayerNormPassonly handled up to 5 arguments (fornative_layer_norm).The error occurs during
transform_for_annotation_pipelinein the ARM quantizer, which runs before edge transformation when the op is stillaten.layer_norm.default.Solution
Add
case 6:to thematch len(args)block inDecomposeLayerNormPass.call()to handle the 6th argument (cudnn_enable). This argument is simply ignored during decomposition since it's only relevant for cuDNN GPU optimization.Testing
Added a new test file
test_layernorm_modai_compat.pythat:torch.exporttransform_for_annotation_pipeline(the exact path that was failing)