Skip to content

Commit 6cf9d67

Browse files
lkhphucwwwjn
andauthored
Apply suggestion from @wwwjn
Co-authored-by: Jiani Wang <[email protected]>
1 parent 12027be commit 6cf9d67

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

torchtitan/experiments/vlm/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,4 +16,4 @@ Distributed training usually does not play nice with input of varying shapes. To
1616
Then we scatter the patch embeddings to their actual positions in the LLM input tokens.
1717
This result in a very simple and general interface to train modern VLM with interleaved data and native resolution & aspect ratio.
1818
By setting the appropriate dataloader hyperparameters, we can easily reduce the amount of padding tokens.
19-
We leverage Flex Attention to efficiently handle varying number of patches per image.
19+
We leverage FlexAttention to efficiently handle varying number of patches per image.

0 commit comments

Comments
 (0)