You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/developer_guides/pipelines/datamanagers.md
+16-4Lines changed: 16 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -95,9 +95,9 @@ See the code!
95
95
96
96
We currently don't have other implementations because most papers follow the VanillaDataManager implementation. However, it should be straightforward to add a VanillaDataManager with logic that progressively adds cameras, for instance, by relying on the step and modifying RayBundle and RayGT generation logic.
97
97
98
-
## Migrating Your Datamanager to the New Datamanager
98
+
## Migrating Your DataManager to the New DataManager
99
99
100
-
As of January 2025, the FullImageDatamanager and ParallelImageDatamanager implementation now supports parallelized dataloading and dataloading from disk to preserve CPU RAM. If you would like your custom datamanager to also support these new features, you can migrate any custom dataloading logic to the `custom_view_processor` API. Let's take a look at an example for the LERF method, which was built on Nerfstudio's VanillaDataManager.
100
+
As of January 2025, the FullImageDatamanager and ParallelImageDatamanager implementations now support parallelized dataloading and dataloading from disk to avoid Out-Of-Memory errors. If you would like your custom datamanager to also support these new features, you can migrate any custom dataloading logic to the new `custom_view_processor()` API. Let's take a look at an example for the LERF method, which was built on Nerfstudio's VanillaDataManager.
@@ -175,7 +175,7 @@ class LERFDataManager(VanillaDataManager): # pylint: disable=abstract-method
175
175
return ray_bundle, batch
176
176
```
177
177
178
-
To migrate this custom datamanager to the new datamanager, we can shift the data customization process in`next_train()` to `custom_view_processor()`.
178
+
To migrate this custom datamanager to the new datamanager, we'll subclass the new ParallelDataManager and shift the data customization process from`next_train()` to `custom_view_processor()`.
To train a NeRF-based method with a large dataset that's unable to fit in memory, please add the `load_from_disk` flag to your `ns-train` command. For example with nerfacto:
To train a Gaussian Splatting method with a large dataset that's unable to fit in memory, please set the device of `cache_images` to disk. For example with splatfacto:
209
+
```bash
210
+
ns-train splatfacto --data {PROCESSED_DATA_DIR} --pipeline.datamanager.cache_images disk
0 commit comments