[Context Parallel] Add support for custom device mesh (#13064) #5379

Job	Run time
PyTorch Compile CUDA tests	21m 32s
Examples PyTorch CUDA tests on Ubuntu	39m 20s
PyTorch xformers CUDA tests	6m 53s
Setup Torch Pipelines CUDA Slow Tests Matrix	1m 4s
Torch CUDA Tests (single_file)	1m 15s
Torch CUDA Tests (schedulers)	6m 32s
Torch CUDA Tests (models)	37m 8s
Torch CUDA Tests (lora)	26m 44s
Torch CUDA Tests (others)	1m 25s
Torch Pipelines CUDA Tests (cogvideo)	5m 21s
Torch Pipelines CUDA Tests (ddpm)	3m 39s
Torch Pipelines CUDA Tests (ip_adapters)	1m 8s
Torch Pipelines CUDA Tests (flux)	5m 52s
Torch Pipelines CUDA Tests (controlnet_flux)	4m 23s
Torch Pipelines CUDA Tests (controlnet)	14m 49s
Torch Pipelines CUDA Tests (latent_consistency_models)	2m 16s
Torch Pipelines CUDA Tests (controlnet_sd3)	4m 23s
Torch Pipelines CUDA Tests (audioldm2)	4m 15s
Torch Pipelines CUDA Tests (flux2)	6m 36s
Torch Pipelines CUDA Tests (hidream_image)	1m 58s
Torch Pipelines CUDA Tests (hunyuandit)	1m 40s
Torch Pipelines CUDA Tests (ltx)	2m 28s
Torch Pipelines CUDA Tests (marigold)	2m 21s
Torch Pipelines CUDA Tests (qwenimage)	15m 4s
Torch Pipelines CUDA Tests (sana)	12m 32s
Torch Pipelines CUDA Tests (ltx2)	6m 19s
Torch Pipelines CUDA Tests (stable_diffusion)	4m 54s
Torch Pipelines CUDA Tests (stable_diffusion_2)	6m 26s
Torch Pipelines CUDA Tests (stable_diffusion_3)	1m 57s
Torch Pipelines CUDA Tests (stable_diffusion_xl)	6m 28s
Torch Pipelines CUDA Tests (stable_video_diffusion)	1m 33s
Torch Pipelines CUDA Tests (wan)	5m 13s
Torch Pipelines CUDA Tests (z_image)	4m 47s
	4h 28m 15s

Provide feedback