Fix 8w8a qat qconfig setting activations

[navsud]

Summary: 8-bit activation qconfig should not use reduce_range=True which limits the range to 0,127. This diff fixes that issue.

Differential Revision: D80007226

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/13284

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 5 Unrelated Failures

As of commit 9f528a0 with merge base 310a05d ():

NEW FAILURES - The following jobs have failed:

• Build documentation / build (buck2) / Build doc (gh)
  At least one of the pre-conditions you specified did not hold
• pull / unittest-arm-backend-with-no-fvp (test_pytest_ops) / linux-job (gh)
  RuntimeError: Command docker exec -t 58ecf5de365a4c5de353f497e903017f170598e77bc657fb2baf1d1e42263a6c /exec failed with exit code 1

FLAKY - The following job failed but was likely due to flakiness present on trunk:

• pull / unittest / macos / macos-job (gh) (similar failure)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1

BROKEN TRUNK - The following jobs failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / unittest / linux / linux-job (gh) (trunk failure)
  examples/models/llama/tests/test_ring_attention.py::TestRingAttention::test_single_token_processing_quantized
• pull / unittest-arm-backend-with-no-fvp (test_pytest_models) / linux-job (gh) (trunk failure)
  backends/arm/test/models/stable_diffusion/test_vae_AutoencoderKL.py::TestAutoencoderKL::test_AutoencoderKL_tosa_MI
• pull / unittest-editable / linux / linux-job (gh) (trunk failure)
  examples/models/llama/tests/test_ring_attention.py::TestRingAttention::test_single_token_processing_quantized
• pull / unittest-editable / macos / macos-job (gh) (trunk failure)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[facebook-github-bot]

This pull request was exported from Phabricator. Differential Revision: D80007226

[facebook-github-bot]

This pull request was exported from Phabricator. Differential Revision: D80007226

[cccclai]

8-bit activation qconfig should not use reduce_range=True which limits the range to 0,127

What range should it be instead?

[navsud]

8-bit activation qconfig should not use reduce_range=True which limits the range to 0,127

What range should it be instead?

Without reduce_range=True, it would use the default of 0,255 ranges, which is the correct ranges for uint8. With this change, I was able to recover the training loss during QAT back to that of the fp32 training.

[cccclai]

Thanks for the fix!

[facebook-github-bot]

This pull request was exported from Phabricator. Differential Revision: D80007226

[Novelfor]

I found the weight use reduce_range too..

    weight_fake_quant_ctr = FusedMovingAvgObsFakeQuantize.with_args(
        dtype=torch.int8,
        quant_min=torch.iinfo(torch.int8).min + 1,
        quant_max=torch.iinfo(torch.int8).max,
        qscheme=torch.per_tensor_symmetric,
        reduce_range=True,
        observer=MovingAverageMinMaxObserver,
    )

I think if modify it to False will improve performance too...

[cccclai]

I found the weight use reduce_range too..

    weight_fake_quant_ctr = FusedMovingAvgObsFakeQuantize.with_args(
        dtype=torch.int8,
        quant_min=torch.iinfo(torch.int8).min + 1,
        quant_max=torch.iinfo(torch.int8).max,
        qscheme=torch.per_tensor_symmetric,
        reduce_range=True,
        observer=MovingAverageMinMaxObserver,
    )

I think if modify it to False will improve performance too...

cc: @haowhsu-quic @winskuo-quic @shewu-quic @DannyYuyang-quic

[navsud]

@haowhsu-quic
I see reduce_range=True at many places in this file. Do we really need to reduce the range by 1-bit for qualcomm backends? If not, we should remove them to get back the 1-bit range, which would improve the model accuracy.