A question about "transform" function in my own "InMemoryDataset"

[wwma]

I customized a "transform" function in the dataset.
But when I load this dataset, runing “print (dataset [0])”, it automatically calls the "transform" function I defined, but the number of variables changed in it is more than the number of variables defined to be changed in the function. However, when I use the "transform" function alone, this does not happen.

[akihironitta]

@wwma I am not too sure what issue you're experiencing. Could I see your code by chance?

[wwma]

import torch
import random
class MaskAtom:
    def __init__(self, num_atom_type, mask_rate):
        self.num_atom_type = num_atom_type
        self.mask_rate = mask_rate
    def __call__(self, data, masked_atom_indices=None):
        if masked_atom_indices == None:
            # sample x distinct atoms to be masked, based on mask rate. But
            # will sample at least 1 atom
            num_atoms = data.x.size()[0]
            sample_size = int(num_atoms * self.mask_rate + 1)
            masked_atom_indices = random.sample(range(num_atoms), sample_size)
        # create mask node label by copying atom feature of mask atom
        mask_node_labels_list = []
        for atom_idx in masked_atom_indices:
            mask_node_labels_list.append(data.x[atom_idx].view(1, -1))
        data.mask_node_label = torch.cat(mask_node_labels_list, dim=0)
        data.masked_atom_indices = torch.tensor(masked_atom_indices)
        # modify the original node feature of the masked node
        for atom_idx in masked_atom_indices:
            data.x[atom_idx] = torch.tensor([self.num_atom_type, 0])
        return data

    def __repr__(self):
        return '{}(num_atom_type={}, num_edge_type={}, mask_rate={}, mask_edge={})'.format(
            self.__class__.__name__, self.num_atom_type, self.num_edge_type,
            self.mask_rate, self.mask_edge)
from loader import MoleculeDataset
dataset = MoleculeDataset("chem/dataset/bace", dataset='bace')
print(dataset[0])
print(dataset[0].x)

as you can see ,this is the original data, and this is the print_info.

however,when I add the "transform" function ,this is the code and the print_info

dataset = MoleculeDataset("chem/dataset/bace", dataset='bace',transform=MaskAtom(119,mask_rate=0.15))
print(dataset[0])
print(dataset[0].x)
print(dataset[0].masked_atom_indices)

It should have changed only five, but from the data it looks like it changed more than five

[akihironitta]

Can you check how many times MaskAtom.__call__() is being called in your script? I think it's calling the transform three times because you are taking the 0-th data from the dataset three times, leading to transforming the 0-th data three times:

print(dataset[0])
print(dataset[0].x)
print(dataset[0].masked_atom_indices)

[wwma]

Yes, you're right! So if I use the transform, everytime I take the data it will be called.
Thanks a lot!

[wwma]

Now I have another problem.
As the code shows,the "transformer function" changes the label of the data, and save the original label as a tensor.
During the training ,the first epoch is okay,however the data in the second epoch I loaded are transformered data.
Its original labels are changed because the first epoch used "transformer function" , so the "transformer function" is used again on the data , and the original labels are wrong .

[akihironitta]

@wwma Sorry, I wasn't clear in my previous comment.

If your transform doesn't change at all, I'd apply it via pre_transform so that it only runs once when creating your dataset.

(If you'd like to apply transform every time, you have to make sure that your transform doesn't modify the original data. In this case, the changes in #7429 might be helpful for you since the PR made built-in transforms non-in-place.)