Skip to content

What are ones options for manually defining the parallelization? #9881

Discussion options

You must be logged in to vote

Dear @roman955b,

1 ) Currently, Lightning automatically implement distributed data parallelism. However, we are currently working on making manual parallelization for users who want deeper control of the parallelisation schema.

2 ) Lightning supports only (S, P) with DeepSpeed, FSDP integrations.

3 ) Yes, we are currently working on this. Here is an issue to track the conversation #9375

Best,
T.C

Replies: 1 comment

Comment options

You must be logged in to vote
0 replies
Answer selected by roman955b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
2 participants