Fix FP8 torchao default config with padding and FSDP2 all-gather support

[shimizust]

What does this PR do?

• If you use accelerate with transformers-based Trainers, using config-file based approach to launching the training job, fp8 using torchao doesn't work properly
• This PR sets reasonable defaults if use of torchao is specified via accelerate configs, specifically enable_fsdp_float8_all_gather=True and pad_inner_dim=True
• Added ability to set these params by CLI or accelerate configs

mixed_precision: fp8
fp8_config:
  backend: AO

or

fp8_config:
  backend: AO
  pad_inner_dim: true
  enable_fsdp_float8_all_gather: true

Fixes #3830

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

• Did you read the contributor guideline,
  Pull Request section?

• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.

• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.

• Did you write any new necessary tests?

• Ran pytest tests/test_fp8.py -v successfully

Who can review?

• Command Line Interface: @SunMarc @zach-huggingface

[SunMarc]

Thanks a lot for the changes, really appreciate it ! Left a few minor comments to make it better !

[SunMarc]

Nice, maybe we can also allow users to easily change that with env var + update the cluster.py file which is responsible of the behavior of accelerate config ? Here's a PR that should help with the changes: #2983

env_prefix = "ACCELERATE_FP8_"
enable_fsdp_float8_all_gather = os.environ.get(env_prefix + "ENABLE_FSDP_FLOAT_ALL_GATHER", True)
pad_inner_dim = os.environ.get(env_prefix + "PAD_INNER_DIM", True)

[shimizust]

Makes sense, thanks for the reference. Will add that

[shimizust]

Updated @SunMarc

[shimizust]

Hi @SunMarc let me know if there's anything else needed

[SunMarc]

nope everything looks good !

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[SunMarc]

That's really nice, thanks for fixing this !

[shimizust]

@SunMarc Sorry, had to fix style issue. Can you re-approve?