-
Notifications
You must be signed in to change notification settings - Fork 7
Description
Error executing job with overrides: ['dset=cifar', 'model=nft', 'compile=false']
Traceback (most recent call last):
File "/nfs/hpc/share/gokulea/research/nfn/experiments/launch_inr2array.py", line 9, in main
train_and_eval(cfg)
File "/nfs/hpc/share/gokulea/research/nfn/experiments/inr2array.py", line 290, in train_and_eval
spec = network_spec_from_wsfeat(WeightSpaceFeatures(*next(iter(trainloader))[0]).to("cpu"), set_all_dims=True)
File "/nfs/hpc/share/gokulea/research/nfn/env/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 633, in next
data = self._next_data()
File "/nfs/hpc/share/gokulea/research/nfn/env/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1345, in _next_data
return self._process_data(data)
File "/nfs/hpc/share/gokulea/research/nfn/env/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1371, in _process_data
data.reraise()
File "/nfs/hpc/share/gokulea/research/nfn/env/lib/python3.8/site-packages/torch/_utils.py", line 644, in reraise
raise exception
IndexError: Caught IndexError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/nfs/hpc/share/gokulea/research/nfn/env/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 308, in _worker_loop
data = fetcher.fetch(index)
File "/nfs/hpc/share/gokulea/research/nfn/env/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 51, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/nfs/hpc/share/gokulea/research/nfn/env/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 51, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/nfs/hpc/share/gokulea/research/nfn/env/lib/python3.8/site-packages/torch/utils/data/dataset.py", line 243, in getitem
return self.datasets[dataset_idx][sample_idx]
File "/nfs/hpc/share/gokulea/research/nfn/experiments/data_utils.py", line 208, in getitem
params, siren_label = self.siren_dset[idx]
File "/nfs/hpc/share/gokulea/research/nfn/env/lib/python3.8/site-packages/torch/utils/data/dataset.py", line 298, in getitem
return self.dataset[self.indices[idx]]
File "/nfs/hpc/share/gokulea/research/nfn/experiments/data_utils.py", line 153, in getitem
data_idx = self.idcs[idx]
IndexError: list index out of range