Update dependency accelerate to v1.12.0

[konflux-internal-p02]

This PR contains the following updates:

Package	Change	Age	Confidence
accelerate	==1.11.0 -> ==1.12.0

---

Release Notes

huggingface/accelerate (accelerate)

v1.12.0: : Deepspeed Ulysses/ALST

Compare Source

Deepspeed Ulysses/ALST integration

Deepspeed Ulysses/ALST is an efficient way of training on long sequences by employing sequence parallelism and attention head parallelism. You can learn more about this technology in this paper https://arxiv.org/abs/2506.13996 or this deepspeed tutorial https://www.deepspeed.ai/tutorials/ulysses-alst-sequence-parallelism/.

To enable Deepspeed Ulysses, you first need to create ParallelismConfig and setting sp related args:

parallelism_config = ParallelismConfig(
    sp_backend="deepspeed",
    sp_size=2,
    sp_handler=DeepSpeedSequenceParallelConfig(...),
)

Then, you need to make sure to compute the correct loss as described on our docs

        ...
        losses_per_rank = torch.distributed.nn.functional.all_gather(loss, group=sp_group)
        good_tokens = (shift_labels != -100).view(-1).sum()
        good_tokens_per_rank = torch.distributed.nn.functional.all_gather(good_tokens, group=sp_group)
        total_loss = sum(
            losses_per_rank[rank] * good_tokens_per_rank[rank]
            for rank in range(sp_world_size)
            if good_tokens_per_rank[rank] > 0
        )
        total_good_tokens = sum(good_tokens_per_rank)
        loss = total_loss / max(total_good_tokens, 1)

Thanks @​S1ro1 for starting this work and for @​stas00 for finishing this work. Also thanks @​kashif for adding docs and reviewing/testing this PR !

This feature will also be available in HF Trainer thanks for this PR from @​stas00: huggingface/transformers#41832

Minor changes

• Remove warning for cpu_ram_efficient_loading by @​SunMarc in #​3816
• update typo in bnb quantisation 4bit flag docstring by @​hbraith in #​3828
• ArXiv -> HF Papers by @​qgallouedec in #​3834
• Fix typo in broadcast_object_list docstring by @​wsntxxn in #​3823
• [Bug] Update torch.optim.Optimizer parameter states after tensor parallelism by @​naomili0924 in #​3835
• use self hosted runner by @​SunMarc in #​3841
• device type helper by @​kashif in #​3843

New Contributors

• @​hbraith made their first contribution in #​3828
• @​wsntxxn made their first contribution in #​3823
• @​naomili0924 made their first contribution in #​3835

Full Changelog: huggingface/accelerate@v1.11.0...v1.12.0

---

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

---

• If you want to rebase/retry this PR, check this box

---

To execute skipped test pipelines write comment /ok-to-test.

---

Documentation

Find out how to configure dependency updates in MintMaker documentation or see all available configuration options in Renovate documentation.

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: konflux-internal-p02[bot]
Once this PR has been reviewed and has the lgtm label, please assign raghul-m for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci]

Hi @konflux-internal-p02[bot]. Thanks for your PR.

I'm waiting for a red-hat-data-services member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.