Hybrid Context Parallel Feature#2282

Merged

parthmannan merged 98 commits intoNVIDIA:mainfrom

parthmannan:pmannan/hybrid_dp_cp_main

Jan 17, 2026

Commits on Jul 15, 2025

Start heterogeneous CP idea prototyping
parthmannan
committed

Commits on Jul 16, 2025

Commits on Jul 17, 2025

Return sub-sample per CP rank count to schedules
parthmannan
committed

Commits on Jul 18, 2025

Commits on Jul 19, 2025

Fix implementation issues [WIP]
parthmannan
committed

Commits on Jul 21, 2025

Add sharding logic and packed_seq_params attribute for TE support
parthmannan
committed
Protect attention shape change logic with THD assumption
parthmannan
committed

Commits on Jul 22, 2025

Add sharding logic and TODO for sync across multiple microbatches
parthmannan
committed

Commits on Aug 13, 2025

Merge branch 'main' of https://gitlab-master.nvidia.com/ADLR/megatron-lm into pmannan/hetero_cp_support
parthmannan
committed

Commits on Aug 14, 2025

Updates to working milestone 1 Hybrid CP scheduling
parthmannan
committed

Commits on Aug 21, 2025

Add SFT dataset support
parthmannan
committed

Commits on Aug 23, 2025

Remove new SFT loss calculation in favor of regular loss calc
parthmannan
committed

Commits on Aug 29, 2025

Milestone 2: Add global batch scheduling, data movement and training
parthmannan
committed

Commits on Sep 15, 2025

Fix for 0 seq len in data
parthmannan
committed

Commits on Sep 16, 2025

Commits on Sep 17, 2025

Commits on Sep 23, 2025

Commits on Sep 24, 2025

little adjustment according to comments
xiaoyao0115
committed

Commits on Sep 25, 2025

little fix to loop parameter
xiaoyao0115
committed

Commits on Sep 26, 2025

disable cudagraph anb fsdp when using hybrid cp
xiaoyao0115
committed

Commits on Sep 29, 2025

Commits on Oct 3, 2025

Commits on Oct 16, 2025

Commits on Nov 17, 2025

Commits on Jan 12, 2026

Commits on Jan 15, 2026

Commits on Jan 16, 2026

Commits on Jan 17, 2026

Merge branch 'main' of github.com:NVIDIA/Megatron-LM into pmannan/hybrid_dp_cp_main
parthmannan
committed