Skip to content

README of FSDP app says I can use "--dataset_path" argument to use local dataset but it doesn't exist #889

@shimomut

Description

@shimomut

The README of FSDP app (https://github.com/aws-samples/awsome-distributed-training/blob/main/3.test_cases/pytorch/FSDP/kubernetes/README.md and https://github.com/aws-samples/awsome-distributed-training/blob/main/3.test_cases/pytorch/FSDP/slurm/README.md) say I can use local dataset by using --dataset_path,

If you'd like to instead use your own dataset, you can do so by formatting it as a HuggingFace dataset, and passing its location to the --dataset_path argument.

but this argument is not implemented.
https://github.com/aws-samples/awsome-distributed-training/blob/main/3.test_cases/pytorch/FSDP/src/model_utils/arguments.py

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions