You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Since packing is usually combined with sequence parallel (SP). Is is feasible to add SP for your dpo packing repo?
Additionally, I am not sure if FlexAttention would conflict with SP-attention like ring-attention.