Unable to access targets when building a custom Avalanche Classification Dataset

[niniack]

Hi, I have a custom dataset, for which I have written a simple class extending the Torch Dataset class

class CustomDataset(Dataset):
    """Dataset generator for the Custom dataset
    """

    def __init__(self,
                 root: str,
                 split: str,
                 transform: Optional[Callable] = None,
                 ):
        self.root = root
        self.split = split

        self.transform = transform
        self.targets = [
            int(np.load(str(self.files[x]))['label']) for x in range(len(self.files))
        ]

    def __len__(self):
        """
        Returns the size of the dataset
        """
        return len(self.targets)

    def __getitem__(self, idx):
        """
        Returns a batch of image, heatmap, labels as Torch tensors
        """
        np_image = np.load(str(self.files[idx]))['image']
        processed_np_image = preprocess_input(np_image)
        image = torch.from_numpy(np.transpose(processed_np_image, (2, 0, 1)))

        label = self.targets[idx]

        if self.transform is not None:
            image = self.transform(image)

        return image, label

Now, I want to use the AvalancheDataset class, so I do the following

def initialize_dataset(root:str, split:str):
    base_ds = CustomDataset(root=root, split=split)
    return make_classification_dataset(base_ds)

Finally, I want to access the targets

train_ds = initialize_dataset(root=root, split="train")
print(train_ds.targets)

I run into:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[8], line 1
----> 1 print(train_ds.targets)

File ~/miniconda3/envs/map-transform/lib/python3.9/site-packages/avalanche/benchmarks/utils/data_attribute.py:64, in DataAttribute.__str__(self)
     63 def __str__(self):
---> 64     return str(self.data[:])

File ~/miniconda3/envs/map-transform/lib/python3.9/site-packages/avalanche/benchmarks/utils/flat_data.py:191, in FlatData.__getitem__(self, idx)
    190 def __getitem__(self, idx):
--> 191     dataset_idx, idx = self._get_idx(idx)
    192     return self._datasets[dataset_idx][idx]

File ~/miniconda3/envs/map-transform/lib/python3.9/site-packages/avalanche/benchmarks/utils/flat_data.py:188, in FlatData._get_idx(self, idx)
    186     else:
    187         idx = idx - self._cumulative_sizes[dataset_idx - 1]
--> 188 return dataset_idx, int(idx)

TypeError: int() argument must be a string, a bytes-like object or a number, not 'slice'

Interestingly, if I try train_ds.targets.count or train_ds.targets.unique, I get the expected output.

My end goal is to build a custom nc benchmark with this dataset. I recognize that doing so internally handles the creation of an avalanche classification dataset, but I run into a different error there, which, after some debugging, I am fairly confident has to do with the error I have shared above.

[AntonioCarta]

Thanks for reporting the problem. This looks like a bug in the __str__ method. I opened an issue to track it.
btw, if you do train_ds.targets it should work, only print(train_ds.targets) fails.

[niniack]

I get the same error for both print(train_ds.targets) and train_ds.targets. Maybe it matters: I'm doing this in a notebook

[AntonioCarta]

It makes sense. When train_ds.targets is the last line in a cell, the notebook calls __str__ to print the output, causing the error.

[niniack]

Ah, yes that does make sense. Thank you.

[AntonioCarta]

@niniack update: the issue seems fixed on the master branch.