memory and GPU requirements #2
Description
My process is being killed during SATVideoDiffusionEngineV2V see below
What are the memory ad GPU requirements?
I am running pytorch 2.6 with cuda 12.2 but it seems to be a CPU memory kill.
Global alignement - optimizing for:
['pw_poses', 'im_depthmaps', 'im_poses', 'im_focals']
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 300/300 [00:03<00:00, 94.36it/s, lr=3.433e-05 loss=0.00256797]
[2025-03-19 08:25:40,404] [INFO] [real_accelerator.py:222:get_accelerator] Setting ds_accelerator to cuda (auto detect)
no module 'xformers'. Processing without...
no module 'xformers'. Processing without...
[2025-03-19 08:25:42,304] [WARNING] No training data specified
WARNING:sat:No training data specified
[2025-03-19 08:25:42,305] [WARNING] No train_iters (recommended) or epochs specified, use default 10k iters.
WARNING:sat:No train_iters (recommended) or epochs specified, use default 10k iters.
[2025-03-19 08:25:42,305] [INFO] using world size: 1
INFO:sat:using world size: 1
[2025-03-19 08:25:42,306] [INFO] [RANK 0] > initializing model parallel with size 1
INFO:sat:[RANK 0] > initializing model parallel with size 1
[2025-03-19 08:25:42,306] [INFO] [RANK 0] You didn't pass in LOCAL_WORLD_SIZE environment variable. We use the guessed LOCAL_WORLD_SIZE=1. If this is wrong, please pass the LOCAL_WORLD_SIZE manually.
INFO:sat:[RANK 0] You didn't pass in LOCAL_WORLD_SIZE environment variable. We use the guessed LOCAL_WORLD_SIZE=1. If this is wrong, please pass the LOCAL_WORLD_SIZE manually.
[2025-03-19 08:25:42,307] [INFO] [RANK 0] building SATVideoDiffusionEngineV2V model ...
INFO:sat:[RANK 0] building SATVideoDiffusionEngineV2V model ...
Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s]Killed