[vllm] feat: retires vllm spmd mode in the codebase (verl-project#4411)

### What does this PR do?

Retires the legacy SPMD rollout path and standardizes the codebase on
async-only rollout for vLLM (SGLang in the next PR). All Python modules,
docs, workflows, and examples now reference the async server mode
exclusively; the sync/SPMD runners, helpers, and CI jobs have been
removed.

### Checklist Before Starting

- [ ] Search for similar PRs. Paste at least one query link here: _N/A
(internal task to delete SPMD support)._
- [ ] Format the PR title as `[vllm, sglang, rollout, trainer, recipe,
ci, doc] refactor: remove SPMD rollout`

### Test

Not run (SPMD suites deleted; async flow already covered by existing
CI).

### API and Usage Example

All configs/scripts must now use `actor_rollout_ref.rollout.mode=async`.
Example:

```bash
python -m verl.trainer.main_ppo \
  ... \
  actor_rollout_ref.rollout.name=vllm \
  actor_rollout_ref.rollout.mode=async \
  ...
```

### Design & Code Changes

- Deleted `verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py` and
the entire SGLang SPMD engine, leaving only async implementations.
Updated `BaseRollout` registry, `RolloutConfig`, and `main_ppo` to error
on `mode=sync`.
- Removed SPMD-specific docs, tests
(`tests/workers/rollout/test_sglang_*`, `test_vllm_spmd`,
`test_vllm_model_rope_scaling`), and CI steps
(`.github/workflows/vllm.yml`, `sgl.yml`). Simplified lint exclusions
and helper scripts accordingly.
- Cleaned recipes/examples to default `rollout_mode=async` and
eliminated conditional sync branches (`examples/**`, `recipe/**`, e2e
scripts). Added explicit validation in agent-loop utilities and SFT
runner to reject non-async requests.
- Updated documentation (FS- DP/Megatron worker guides, hybrid flow,
r1_ascend notes, FP8 guide) to describe async-only rollout and mention
removal of the old SPMD pathway.

### Checklist Before Submitting

- [ ] Read the [Contribute
Guide](https://github.com/volcengine/verl/blob/main/CONTRIBUTING.md).
- [ ] Apply [pre-commit
checks](https://github.com/volcengine/verl/blob/main/CONTRIBUTING.md#code-linting-and-formatting).
- [x] Add / Update [the
documentation](https://github.com/volcengine/verl/tree/main/docs).
- [x] Add unit or end-to-end test(s) to [the CI
workflow](https://github.com/volcengine/verl/tree/main/.github/workflows)
to cover all the code. If not feasible, explain why: _Removed obsolete
SPMD jobs; async coverage already exists._
- [ ] Once your PR is ready for CI, notify the `ci-request` channel (or
Feishu group).

Author: PeterSH6

.github/workflows/vllm.yml

@@ -124,12 +124,6 @@ jobs:
       - name: Test the latest vLLM Rollout async with agent loop
         run: |
           ROLLOUT_NAME=vllm pytest -svvv tests/experimental/agent_loop
-      - name: Test the latest vLLM
-        run: |
-          torchrun --standalone --nnodes=1 --nproc_per_node=4 $(which pytest) -s tests/workers/rollout/rollout_vllm/test_vllm_spmd.py
-      - name: Test the latest vLLM on model with rope scaling
-        run: |
-          torchrun --standalone --nnodes=1 --nproc_per_node=4 $(which pytest) -s tests/workers/rollout/rollout_vllm/test_vllm_model_rope_scaling.py
       # Note(haibin.lin): for any new test, please update gpu_unit_tests.yaml to avoid repeated tests
 
   cleanup:

examples/grpo_trainer/run_qwen2-7b_math_megatron.sh

@@ -2,11 +2,9 @@ set -x
 
 export CUDA_DEVICE_MAX_CONNECTIONS=1 # For megatron communication/computation overlapping
 
-rollout_mode="sync"
-if [ "$rollout_mode" = "async" ]; then
-    export VLLM_USE_V1=1
-    return_raw_chat="True"
-fi
+rollout_mode="async"
+export VLLM_USE_V1=1
+return_raw_chat="True"
 
 gsm8k_train_path=$HOME/data/gsm8k/train.parquet
 gsm8k_test_path=$HOME/data/gsm8k/test.parquet

examples/grpo_trainer/run_qwen2-7b_seq_balance.sh

@@ -4,9 +4,9 @@ set -x
 # For async rollout mode, dataset should return raw chat.
 rollout_mode="async"
 rollout_name="sglang" # sglang or vllm
-if [ "$rollout_mode" = "async" ]; then
+return_raw_chat="True"
+if [ "$rollout_name" = "vllm" ]; then
     export VLLM_USE_V1=1
-    return_raw_chat="True"
 fi
 
 python3 -m verl.trainer.main_ppo \

examples/grpo_trainer/run_qwen2_5_vl-7b-sglang.sh

@@ -40,7 +40,7 @@ python3 -m verl.trainer.main_ppo \
     actor_rollout_ref.rollout.n=5 \
     actor_rollout_ref.ref.log_prob_micro_batch_size_per_gpu=20 \
     actor_rollout_ref.ref.fsdp_config.param_offload=True \
-    actor_rollout_ref.rollout.mode=sync \
+    actor_rollout_ref.rollout.mode=async \
     algorithm.use_kl_in_reward=False \
     trainer.critic_warmup=0 \
     trainer.logger='["console","wandb"]' \

examples/gspo_trainer/run_qwen30b_gspo.sh

@@ -131,6 +131,9 @@ fi
 
 # ===================================== Inference =====================================
 rollout_name=vllm
+if [ "$rollout_name" = "vllm" ]; then
+    export VLLM_USE_V1=1
+fi
 infer_tp=4
 infer_dp=1
 infer_ep=1

examples/gspo_trainer/test_gspo_3b_math.sh

@@ -47,7 +47,11 @@ loss_agg_mode="seq-mean-token-mean"
 MODEL_PATH=Qwen/Qwen2.5-3B-Instruct
 offload=false # it's a small model, offloading will just slow-down training
 rollout_engine=vllm
-rollout_mode=sync # can be async to speedup large scale xps
+rollout_mode=async
+return_raw_chat="True"
+if [ "$rollout_engine" = "vllm" ]; then
+    export VLLM_USE_V1=1
+fi
 gpu_memory_utilization=0.8
 reward_manager=dapo
 adv_estimator=grpo
@@ -121,6 +125,7 @@ python3 -m verl.trainer.main_ppo \
     data.prompt_key=prompt \
     data.truncation='error' \
     data.filter_overlong_prompts=true \
+    data.return_raw_chat=${return_raw_chat} \
     data.train_batch_size=${train_batch_size} \
     data.max_prompt_length=${max_prompt_length} \
     data.max_response_length=${max_response_length} \
@@ -138,7 +143,6 @@ python3 -m verl.trainer.main_ppo \
     actor_rollout_ref.actor.ppo_max_token_len_per_gpu=${actor_ppo_max_token_len} \
     actor_rollout_ref.ref.log_prob_max_token_len_per_gpu=${infer_ppo_max_token_len} \
     actor_rollout_ref.rollout.log_prob_max_token_len_per_gpu=${infer_ppo_max_token_len} \
-    actor_rollout_ref.rollout.name=vllm \
     actor_rollout_ref.rollout.name=${rollout_engine} \
     actor_rollout_ref.rollout.mode=${rollout_mode} \
     actor_rollout_ref.model.path="${MODEL_PATH}" \

examples/gspo_trainer/test_gspo_3b_math_slurm.sh

@@ -51,7 +51,11 @@ loss_agg_mode="seq-mean-token-mean"
 MODEL_PATH=Qwen/Qwen2.5-3B-Instruct
 offload=false # it's a small model, offloading will just slow-down training
 rollout_engine=vllm
-rollout_mode=sync # can be async to speedup large scale xps
+rollout_mode=async
+return_raw_chat="True"
+if [ "$rollout_engine" = "vllm" ]; then
+    export VLLM_USE_V1=1
+fi
 gpu_memory_utilization=0.8
 reward_manager=dapo
 adv_estimator=grpo
@@ -125,6 +129,7 @@ python3 -m verl.trainer.main_ppo \
     data.prompt_key=prompt \
     data.truncation='error' \
     data.filter_overlong_prompts=true \
+    data.return_raw_chat=${return_raw_chat} \
     data.train_batch_size=${train_batch_size} \
     data.max_prompt_length=${max_prompt_length} \
     data.max_response_length=${max_response_length} \
@@ -142,7 +147,6 @@ python3 -m verl.trainer.main_ppo \
     actor_rollout_ref.actor.ppo_max_token_len_per_gpu=${actor_ppo_max_token_len} \
     actor_rollout_ref.ref.log_prob_max_token_len_per_gpu=${infer_ppo_max_token_len} \
     actor_rollout_ref.rollout.log_prob_max_token_len_per_gpu=${infer_ppo_max_token_len} \
-    actor_rollout_ref.rollout.name=vllm \
     actor_rollout_ref.rollout.name=${rollout_engine} \
     actor_rollout_ref.rollout.mode=${rollout_mode} \
     actor_rollout_ref.model.path="${MODEL_PATH}" \

examples/gspo_trainer/test_gspo_qwen30b_a3b_ep.sh

@@ -64,6 +64,9 @@ offload=True
 
 # gen
 rollout_name=vllm # vllm or sglang
+if [ "$rollout_name" = "vllm" ]; then
+    export VLLM_USE_V1=1
+fi
 gen_tp=1
 gen_dp=4
 gen_ep=4

examples/ppo_trainer/run_qwen2-7b_seq_balance.sh

@@ -9,10 +9,8 @@ train_files="['$gsm8k_train_path', '$math_train_path']"
 test_files="['$gsm8k_test_path', '$math_test_path']"
 
 # For async rollout mode, dataset should return raw chat.
-rollout_mode="sync"
-if [ "$rollout_mode" = "async" ]; then
-    return_raw_chat="True"
-fi
+rollout_mode="async"
+return_raw_chat="True"
 
 python3 -m verl.trainer.main_ppo \
     algorithm.adv_estimator=gae \

recipe/dapo/test_dapo_gptoss_20b_megatron.sh

@@ -58,13 +58,10 @@ use_dynamic_bsz=False # recommended but not necessary
 
 ################################################### quick config ###################################################
 
-rollout_mode="sync"
+rollout_mode="async"
 rollout_name="vllm" # sglang or vllm
-return_raw_chat="False"
-if [ "$rollout_mode" = "async" ]; then
-    export VLLM_USE_V1=1
-    return_raw_chat="True"
-fi
+export VLLM_USE_V1=1
+return_raw_chat="True"
 dtype="bfloat16" # ["bfloat16", "float16"]
 
 project_name='DAPO'