Update dependency accelerate to v1.10.0

[renovate]

This PR contains the following updates:

Package	Change	Age	Confidence
accelerate	==1.8.1 -> ==1.10.0

---

Release Notes

huggingface/accelerate (accelerate)

v1.10.0: : N-D Parallelism

Compare Source

N-D Parallelism

Training large models across multiple GPUs can be complex, especially when combining different parallelism strategies (e.g TP, CP, DP). To simplify this process, we've collaborated with Axolotl to introduce an easy-to-use integration that allows you to apply any combination of parallelism strategies directly in your training script. Just pass a ParallelismConfig specifying the size of each parallelism type—it's that simple.
Learn more about how it works in our latest blogpost.

parallelism_config = ParallelismConfig(
    dp_shard_size=2,
    dp_replicate_size=2,
    cp_size=2,
    tp_size=2,
)
accelerator = Accelerator(
    parallelism_config=parallelism_config,
   ...
)
model = AutoModelForCausalLM.from_pretrained("your-model-name", device_mesh=accelerator.torch_device_mesh)
model = accelerator.prepare(model)

• Parallelism config + TP + HSDP + BYODM (Bring Your Own Device Mesh) by @​SalmanMohammadi in https://github.com/huggingface/accelerate/pull/3682
• Feat: context parallel v2.0 by @​S1ro1 in https://github.com/huggingface/accelerate/pull/3700
• set default submesh_tp_size to prevent unset local variable error by @​winglian in https://github.com/huggingface/accelerate/pull/3687
• Add Parallelism getter property to Accelerator class by @​WoosungMyung in https://github.com/huggingface/accelerate/pull/3703
• Fix: prepare works even if nothing except tp specified (rare) by @​S1ro1 in https://github.com/huggingface/accelerate/pull/3707
• Set parallelism_config in constructor due to Trainer reset of State by @​winglian in https://github.com/huggingface/accelerate/pull/3713
• Fix: tp size wouldn't read from env by @​S1ro1 in https://github.com/huggingface/accelerate/pull/3716
• Remove ParallelismConfig from PartialState by @​SunMarc in https://github.com/huggingface/accelerate/pull/3720

FSDP improvements

We've fixed ignored modules attribute. With this, it is now possible to train PEFT model that moe layers that contrains q_proj and v_proj parameters. This is especially important for fine-tuning gpt-oss model.

• ENH: Allow FSDP ignored modules to be regex by @​BenjaminBossan in https://github.com/huggingface/accelerate/pull/3698
• TST Add test for FSDP ignored_modules as str by @​BenjaminBossan in https://github.com/huggingface/accelerate/pull/3719

Minor improvements

• feature: CpuOffload pre_forward don't attempt to move if already on device by @​JoeGaffney in https://github.com/huggingface/accelerate/pull/3695
• Fix: Ensure environment variable values are case-insensitive in Accelerate by @​jp1924 in https://github.com/huggingface/accelerate/pull/3712
• remove use_ipex by @​SunMarc in https://github.com/huggingface/accelerate/pull/3721

New Contributors

• @​SalmanMohammadi made their first contribution in https://github.com/huggingface/accelerate/pull/3682
• @​WoosungMyung made their first contribution in https://github.com/huggingface/accelerate/pull/3703
• @​jp1924 made their first contribution in https://github.com/huggingface/accelerate/pull/3712
• @​JoeGaffney made their first contribution in https://github.com/huggingface/accelerate/pull/3695

Full Changelog: huggingface/accelerate@v1.9.0...v1.10.0

v1.9.0: : Trackio support, Model loading speedup, Minor distributed improvements

Compare Source

Trackio tracker support

We've added support for a trackio, lightweight, 💯 free experiment tracking Python library built on top of 🤗 Datasets and Spaces.

Main features are:

• Local-first design: dashboard runs locally by default. You can also host it on Spaces by specifying a space_id.
• Persists logs locally (or in a private Hugging Face Dataset)
• Visualize experiments with a Gradio dashboard locally (or on Hugging Face Spaces)
• Everything here, including hosting on Hugging Faces, is free!

To use it with accelerate, you need to set log_with and initialize the trackers

accelerator = Accelerator(log_with="trackio")
config={"learning_rate": 0.001, "batch_size": 32}

### init_kwargs in order to host the dashboard on spaces
init_kwargs = {"trackio": {"space_id": "hf_username/space_name"}
accelerator.init_trackers("example_project", config=config, init_kwargs=init_kwargs})

Thanks @​pcuenca for the integration !

• trackio by @​pcuenca in https://github.com/huggingface/accelerate/pull/3669

Model loading speedup when relying set_module_tensor_to_device

Setting tensor while clearing cache is very slow, so we added clear_device option to disable it.
Another small optimization is using non_blocking everywhere and syncing just before returning control to the user. This makes the loading slightly faster.

• Speedup model loading by 4-5x in Diffusers ⚡ by @​a-r-r-o-w in https://github.com/huggingface/accelerate/pull/3674

FDSP, Deepspeed, FP8 minor improvements

• Add support for e5e2 and default to hybrid when launcher is used by @​IlyasMoutawwakil in https://github.com/huggingface/accelerate/pull/3640
• Fix FP8 tests, enable FP8 to be used without direct Accelerator() configuring by @​pstjohn in https://github.com/huggingface/accelerate/pull/3677
• Bunch of FSDP improvements by @​S1ro1 in https://github.com/huggingface/accelerate/pull/3671
• Fix: properly error when DDP + Dtensor model by @​S1ro1 in https://github.com/huggingface/accelerate/pull/3629
• Fix fsdp2 example typo by @​shimizust in https://github.com/huggingface/accelerate/pull/3657
• Added a check in no_sync() to avoid errors when using deepspeed zero2/3 by @​xliu0105 in https://github.com/huggingface/accelerate/pull/3656

🚨🚨🚨 Breaking changes 🚨🚨🚨

find_executable_batch_size() will no longer halves the batch after every OOM. Instead, we will multiply the batch size by 0.9. This should help user not waste gpu capacity.

• “Stop Halving My Batch!” · Default back-off 0.5 → 0.9 by @​SunMarc in https://github.com/huggingface/accelerate/pull/3684

What's Changed

• [typo] shards instead of shard by @​SunMarc in https://github.com/huggingface/accelerate/pull/3645
• Docs: Fix typos in gradient accumulation guide by @​kilavvy in https://github.com/huggingface/accelerate/pull/3649
• xpu enablement on left cases by @​yao-matrix in https://github.com/huggingface/accelerate/pull/3654
• unpin datasets in examples requirements by @​SunMarc in https://github.com/huggingface/accelerate/pull/3681
• fix: wandb config not saved in offline mode by @​ved1beta in https://github.com/huggingface/accelerate/pull/3648
• accelerate/data_loader.py: do not yield if the base_dataloader is empty by @​0xnightwind in https://github.com/huggingface/accelerate/pull/3659
• warn for invalid keys by @​ved1beta in https://github.com/huggingface/accelerate/pull/3613
• Update Gaudi runner image to latest SynapseAI and enable previously disabled tests by @​IlyasMoutawwakil in https://github.com/huggingface/accelerate/pull/3653

New Contributors

• @​kilavvy made their first contribution in https://github.com/huggingface/accelerate/pull/3649
• @​shimizust made their first contribution in https://github.com/huggingface/accelerate/pull/3657
• @​xliu0105 made their first contribution in https://github.com/huggingface/accelerate/pull/3656
• @​0xnightwind made their first contribution in https://github.com/huggingface/accelerate/pull/3659

Full Changelog: huggingface/accelerate@v1.8.1...v1.9.0

---

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

---

• If you want to rebase/retry this PR, check this box

---

This PR was generated by Mend Renovate. View the repository job log.