Multimodal Input: from_webdataset()#560

Open

valerie-cal wants to merge 50 commits intoray-project:2.0from

nsaadhvi:wds-schema

Commits on Mar 10, 2025

webdataset schema finished, test cases not working
valerie-cal
committed

Commits on Mar 13, 2025

Merge branch '2.0' of https://github.com/nsaadhvi/codebase-deltacat into wds-schema
valerie-cal
committed
wds schema pytest and wds schema logic
valerie-cal
committed

Commits on Mar 15, 2025

edits to building wds schema
valerie-cal
committed

Commits on Mar 31, 2025

Initial implementation of from_webdataset
valerie-cal
committed

Commits on Apr 3, 2025

Commits on Apr 7, 2025

Commits on Apr 24, 2025

Commits on May 1, 2025

webdataset schema finished, test cases not working

valerie-cal
authored and
025rhu
committed
wds schema pytest and wds schema logic

valerie-cal
authored and
025rhu
committed
edits to building wds schema

valerie-cal
authored and
025rhu
committed
Initial implementation of from_webdataset

valerie-cal
authored and
025rhu
committed
fixed error with reading tar files and added sample tar files

valerie-cal
authored and
025rhu
committed
finished pytests

valerie-cal
authored and
025rhu
committed
inconsistent jsons pytest added

valerie-cal
authored and
025rhu
committed
comment out failing test

valerie-cal
authored and
025rhu
committed
restored failing test case

valerie-cal
authored and
025rhu
committed
create datasets in tmp_path

valerie-cal
authored and
025rhu
committed
add image binary column to dataset and schema
valerie-cal
committed
resolved merge conflicts with 2.0 branch, wds tests passing
nsaadhvi
committed
Throw error for image binary and batch row mismatch
nsaadhvi
committed

Commits on May 2, 2025

edit demo to process images with binary column
nsaadhvi
committed

Commits on Jun 15, 2025

Commits on Jun 16, 2025

Update wds_demo.py with comments
NeeralBhalgat
authored

Commits on Aug 21, 2025

Using WDS reader for tests, added intra-batch schema merge handling logic and fixed media_binary logic in from_webdataset
025rhu
committed