[skyrl-train] Add multi-node slurm training script

[BounharAbdelaziz]

This PR adds a full multi-node Slurm training script to run SkyRL on HPC clusters. It addresses the issue #709. The script is optimized for NVIDIA H200 GPUs.

The script:

• Automatically sets up the environment and repository paths.
• Launches a Ray head node and Ray worker nodes across all allocated Slurm nodes.
• Detects GPUs and configures NCCL/UCX for fast multi-node communication.
• Verifies the Ray cluster is correctly formed before training.
• Starts SkyRL SAPO training through srun, with support for checkpointing and automatic resume.

[gemini-code-assist]

Code Review

This pull request introduces a comprehensive Slurm script for multi-node training, which is a great addition for HPC users. The script is well-structured and covers the key steps from environment setup to launching Ray and the training job. My review focuses on improving the script's robustness and maintainability. I've identified a critical bug in the GPU availability check for worker nodes and suggest replacing brittle sleep-based synchronization with more reliable polling mechanisms. Additionally, I've provided suggestions to make IP address detection more robust and to improve the script's portability and configuration management.

[gemini-code-assist]

This until nvidia-smi loop is intended to wait for GPUs to become visible on worker nodes, but it's executed on the launching node (the head node), not on the target worker node ($node_i). This is a critical bug, as it doesn't actually check the GPU status on the workers. To fix this, the check must be executed on the worker node itself, for example by wrapping both the check and the ray start command within the srun command's payload.

Suggested change

	until nvidia-smi > /dev/null 2>&1; do
	echo "Waiting for GPU visibility..."
	sleep 2
	done
	srun --nodes=1 --ntasks=1 -w "$node_i" \
	uv run --python "$PYTHON" --active -- ray start --address "$ip_head" \
	--num-cpus "$SLURM_CPUS_PER_TASK" --num-gpus "$SLURM_GPUS_PER_NODE" --block &
	srun --nodes=1 --ntasks=1 -w "$node_i" bash -c "
	until nvidia-smi > /dev/null 2>&1; do
	echo \"Waiting for GPU visibility on worker \$HOSTNAME...\"
	sleep 2
	done
	uv run --python \"$PYTHON\" --active -- ray start --address \"$ip_head\" \
	--num-cpus \"$SLURM_CPUS_PER_TASK\" --num-gpus \"$SLURM_GPUS_PER_NODE\" --block
	" &

[gemini-code-assist]

The current logic for extracting an IPv4 address from hostname --ip-address is complex and potentially brittle, as it relies on string length heuristics to differentiate between IPv4 and IPv6. A more robust and simpler approach is to use hostname -I, which is designed to print the host's network addresses, and then select the first one.

Suggested change

	head_node_ip=$(srun --nodes=1 --ntasks=1 -w "$head_node" hostname --ip-address)
	# Convert to IPv4 if needed
	if [[ "$head_node_ip" == *" "* ]]; then
	IFS=' ' read -ra ADDR <<<"$head_node_ip"
	if [[ ${#ADDR[0]} -gt 16 ]]; then
	head_node_ip=${ADDR[1]}
	else
	head_node_ip=${ADDR[0]}
	fi
	echo "IPV6 address detected. Using IPV4: $head_node_ip"
	fi
	head_node_ip=$(srun --nodes=1 --ntasks=1 -w "$head_node" hostname -I | awk '{print $1}')

[gemini-code-assist]

Using a fixed sleep to wait for the Ray head node to start is unreliable, as the startup time can vary depending on system load. This can lead to race conditions where subsequent steps fail because the head node is not yet ready. A more robust approach is to actively poll for the head node's availability, for example by using ray health-check in a loop. The same principle applies to the sleeps on lines 220 and 224 while waiting for workers.

[gemini-code-assist]

The path to the Python executable is hardcoded, which ties the script to a specific virtual environment layout (skyrl/bin/python). This reduces portability. To make the script more flexible, you can allow this path to be overridden via an environment variable, while keeping the current path as a default.

Suggested change

	PYTHON="$REPO_ROOT/skyrl/bin/python"
	PYTHON=${PYTHON:-"$REPO_ROOT/skyrl/bin/python"}

[gemini-code-assist]

This srun command has a very long list of command-line arguments, which makes the script difficult to read and maintain. Since the training script uses Hydra, a better practice would be to define these parameters in a dedicated Hydra configuration file (e.g., conf/my_experiment.yaml). The shell script would then only need to override a few dynamic values (like file paths or resource counts), making it much cleaner and the experiment configuration more reusable and version-controlled.

[gemini-code-assist]

The file is missing a newline character at the end. It's a POSIX standard and a common convention to end text files with a newline. This prevents issues with certain tools like cat and can avoid unexpected behavior in shell scripts.