Cannot access underlying data for custom in memory dataset

[mxv001]

Hello experts!

I am trying to use k-fold cross validation for a custom dataset I created, similar to this post:
#3663

The solution in the post references the code here:
https://github.com/pyg-team/pytorch_geometric/blob/master/benchmark/kernel/train_eval.py

And the specific code snippet that is the heart of the solution is this:

def k_fold(dataset, folds):
    skf = StratifiedKFold(folds, shuffle=True, random_state=12345)

    test_indices, train_indices = [], []
    for _, idx in skf.split(torch.zeros(len(dataset)), dataset.data.y):
        test_indices.append(torch.from_numpy(idx).to(torch.long))

    val_indices = [test_indices[i - 1] for i in range(folds)]

    for i in range(folds):
        train_mask = torch.ones(len(dataset), dtype=torch.bool)
        train_mask[test_indices[i]] = 0
        train_mask[val_indices[i]] = 0
        train_indices.append(train_mask.nonzero(as_tuple=False).view(-1))

    return train_indices, test_indices, val_indices

def train_with_cross_validation(args):
    for fold, (train_idx, test_idx,
               val_idx) in enumerate(zip(*k_fold(self.dataset, self.folds))):
                     ###TRAIN YOUR MODEL

But when I run this code on a custom InMemoryDataset, I get an error:
AttributeError: 'NoneType' object has no attribute 'y'

So I need to implement a setter/getter for the data attribute, but do not see how, or where I can guide myself. Simplified code for my custom data set is below. Can anyone guide me on how to set the data for a dataset so that the k-fold code will work?

class CustomDataset(InMemoryDataset):
    def __init__(self, df):
        super().__init__(df)
        # PANDAS DATAFRAME
        self.df=df 
        
    def len(self):
        return len(self.df.index)
    
    @property
    def num_classes(self):
        return 1

    #@property
    #def data(self):
    #    return WHAT TO PUT HERE

    def get(self, idx):
        if torch.is_tensor(idx):
            idx = idx.tolist()

        data=()

        x=self.df['custom_graph'].iloc[idx].x.clone().detach()
        edge_index=self.df['custom_graph'].iloc[idx].edge_index.clone().detach()
        edge_attr=self.df['custom_graph'].iloc[idx].edge_attr.clone().detach()

        y=torch.tensor(self.df['target'].iloc[idx],dtype=self.target_type).unsqueeze(dim=0)

        data = Data(
                x=x,
                edge_index=edge_index,
                edge_attr=edge_attr,
                y=y
                )

        return data

Thank you very much!

[anthonysirico]

Take a look at this here, ensure that you keep to the structure. The one thing I see missing the def process(self):. The process function will upload your data to a list, so at the end of your code, it can look like:

def process(self, idx):
        if torch.is_tensor(idx):
            idx = idx.tolist()

        dataset=[]

        x=self.df['custom_graph'].iloc[idx].x.clone().detach()
        edge_index=self.df['custom_graph'].iloc[idx].edge_index.clone().detach()
        edge_attr=self.df['custom_graph'].iloc[idx].edge_attr.clone().detach()

        y=torch.tensor(self.df['target'].iloc[idx],dtype=self.target_type).unsqueeze(dim=0)

        data = Data(
                x=x,
                edge_index=edge_index,
                edge_attr=edge_attr,
                y=y
                )

        dataset.append(data)

[mxv001]

I was able to resolve my difficulty using the referenced methods.