train qwen3-vl-moe on ShareGPT4V-small with quick-start #194
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds support for resolving relative image/video/audio paths relative to the
train_pathdirectory, making it easier to use simple datasets like ShareGPT4V-small without requiring absolute paths in data files.Changes
Bug Fixes
AttributeError: 'DataArguments' object has no attribute 'mm_configs'intrain_qwen_vl.pyby usingMyDataArgumentsinstead ofDataArgumentsin theArgumentsclassNew Features
resolve_relative_path()utility function inveomni/data/multimodal/file_utils.pyto handle unified relative path resolutionimage_utils.py,video_utils.py, andaudio_utils.pyto use the new path resolvertrain_paththrough kwargs intrain_qwen_vl.pyto enable relative path resolution for Qwen2.5-VL and Qwen3-VL modelsDocumentation
docs/examples/qwen3vl_moe.mdlink for Qwen3-VL MoE modelBehavior
When image/video/audio paths in dataset files are relative (e.g.,
coco/train2017/image.jpg), they are automatically resolved relative to the directory containingtrain_path. This allows datasets like ShareGPT4V-small to work without requiring absolute paths.Example:
train_path:/path/to/ShareGPT4V-small-coco-128.jsonlcoco/train2017/image.jpg/path/to/coco/train2017/image.jpgTesting
Tested with ShareGPT4V-small dataset using relative image paths.
Related Issues
Fixes issues related to multimodal data file path resolution for easier dataset setup. #186 #191