Change default vllm_mode to "colocate" and add v0→v1 migration guide

[qgallouedec]

Summary

• Default vllm_mode changed from "server" to "colocate" across all configs (GRPOConfig, RLOOConfig, GOLDConfig, OnlineDPOConfig). Colocate is a better default — it works out of the box without spinning up a separate vLLM server, which is the most common friction point for new users. Users who need server mode can still set vllm_mode="server" explicitly.
• Updated docs and tests that implicitly assumed the old "server" default to explicitly set vllm_mode="server" where needed (vllm_integration.md, grpo_trainer.md, rloo_trainer.md, speeding_up_training.md, test_online_dpo_trainer.py).
• Added a minimal MIGRATION.md at the repo root for v0 → v1. It should stay small — we have already done most of the breaking work in v0.29 (trainers moved to experimental, model classes removed, etc.), so the guide just highlights what is genuinely new and keeps the v0.29 changes in collapsible sections for anyone migrating from an older version.

---

Note

Medium Risk
Changes a widely-used default that can affect training/inference GPU memory usage and runtime behavior for users relying on implicit vLLM server mode; code changes are small but behaviorally impactful.

Overview
Switches the default vLLM integration mode from server to colocate across configs (GRPOConfig, RLOOConfig, OnlineDPOConfig, GOLDConfig) and updates VLLMGeneration’s default accordingly, so use_vllm=True now runs vLLM in-process unless vllm_mode="server" is explicitly set.

Documentation and examples are updated to reflect the new default and to explicitly opt into server mode where a separate vLLM server is assumed (grpo_trainer.md, rloo_trainer.md, speeding_up_training.md, vllm_integration.md), and the Online DPO vLLM test is adjusted to run in server mode explicitly and to assert the new default. Adds a small root MIGRATION.md highlighting the breaking default change for v0→v1.

^{Written by Cursor Bugbot for commit 40beb9d. This will update automatically on new commits. Configure here.}

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 424fc21035

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[albertvillanova]

Thanks. A couple of questions below.

[albertvillanova]

Shouldn't we follow a deprecation cycle instead? This would give users time to explicitly set the mode until v1 is released.

[qgallouedec]

I considered a deprecation cycle, but since v1 is right ahead, I’d recommend we skip it. Best practice is usually a deprecation cycle when you’ve got more pre-releases coming, but here, v0.29 was the last one, and v1 should be our stable baseline.
Let’s just set the final defaults at v1, communicate it clearly with a migration guide.

[albertvillanova]

Not sure of fully understanding this: are we creating a migration guide for changes that were already released in the previous minor version?

[qgallouedec]

Yeah, I was a bit hesitant about including those pre-0.29 changes. I realize now it might be confusing, since they’re already effective. I see how that could mislead, so I’ll take them out and keep the guide focused on what’s truly new.

[albertvillanova]

Thanks for your clarifying explanation: that makes sense to me.

That said, I also understand the original motivation. I think having a place summarizing the recent breaking changes can be useful for users upgrading after skipping a few versions.

What about this compromise?

• Keep MIGRATION.md focused only on the v0 to v1 migration
• Document the v0.29 breaking changes elsewhere
  • For now, we document these in the release notes: is this sufficient?
  • What about introducing a CHANGELOG.md to track changes across versions?
  • Even though for v1.x series we will try to avoid breaking changes, maybe we could have a clear place to document them in case any become unavoidable
• We could optionally add a short note in the migration guide pointing users to the v0.29 release notes in case they are upgrading from an earlier version

[AmineDiro]

+1 on the CHANGELOG.md idea.

I usually think about migration from a major to another if they were breaking changes. So I think this file is more of "nice to have" for users and mandatory for now ?

[albertvillanova]

Thanks for your explanation. Just a comment below.

[albertvillanova]

Thanks for your clarifying explanation: that makes sense to me.

That said, I also understand the original motivation. I think having a place summarizing the recent breaking changes can be useful for users upgrading after skipping a few versions.

What about this compromise?

• Keep MIGRATION.md focused only on the v0 to v1 migration
• Document the v0.29 breaking changes elsewhere
  • For now, we document these in the release notes: is this sufficient?
  • What about introducing a CHANGELOG.md to track changes across versions?
  • Even though for v1.x series we will try to avoid breaking changes, maybe we could have a clear place to document them in case any become unavoidable
• We could optionally add a short note in the migration guide pointing users to the v0.29 release notes in case they are upgrading from an earlier version