Batching Dicts

[vkovac2]

Hello everyone.

I am trying to make a transformation dataset class that will be used in “pre_transform” of QM9 dataset, and I have came up with one issue when dealing with dictionaries. When having nodes of two different node types we can use the “inc” method as described in “Advanced Mini-Batching” post. However, is there a way to deal with dictionaries in a similar manner? So if I have a dictionary that matches an id with an edge {0:[0,1], 1:[2,3]...}, is it possible to make it work for batches later somehow? The problem that arises is that the values in the dictionary won’t get incremented when constructing a batch, thus leading to confusions. Is there a way to overcome this issue when constructing custom dataset transformation class?

Thank you very much for your time in advance

[rusty1s]

Yes, that is possible. You can see that __inc__ gets called here for every key in the dictionary:

import torch

from torch_geometric.data import Batch, Data


class MyData(Data):
    def __inc__(self, key, value, *args, **kwargs):
        print(key, value)
        return 0


data = MyData(x_dict={'1': torch.tensor([0]), '2': torch.tensor([1])})

batch = Batch.from_data_list([data, data])
print(batch)

[vkovac2]

That is amazing. That is exactly what I was looking for. Thanks a lot. After experimenting a bit however, I am getting the following error:

Could not run 'aten::cat' with arguments from the 'NestedTensorCPU' backend. This could be because the operator doesn't exist for this backend, or was omitted during the selective/custom build process (if using custom build). If you are a Facebook employee using PyTorch on mobile, please visit https://fburl.com/ptmfixes for possible resolutions. 'aten::cat' is only available for these backends: [CPU, CUDA, HIP, MPS, IPU, XPU, HPU, VE, Meta, MTIA, PrivateUse1, PrivateUse2, PrivateUse3, FPGA, ORT, Vulkan, Metal, QuantizedCPU, QuantizedCUDA, QuantizedHIP, QuantizedMPS, QuantizedIPU, QuantizedXPU, QuantizedHPU, QuantizedVE, QuantizedMeta, QuantizedMTIA, QuantizedPrivateUse1, QuantizedPrivateUse2, QuantizedPrivateUse3, CustomRNGKeyId, MkldnnCPU, SparseCPU, SparseCsrCPU, SparseCsrCUDA, BackendSelect, Python, FuncTorchDynamicLayerBackMode, Functionalize, Named, Conjugate, Negative, ZeroTensor, ADInplaceOrView, AutogradOther, AutogradCPU, AutogradCUDA, AutogradHIP, AutogradXLA, AutogradMPS, AutogradIPU, AutogradXPU, AutogradHPU, AutogradVE, AutogradLazy, AutogradMeta, AutogradMTIA, AutogradPrivateUse1, AutogradPrivateUse2, AutogradPrivateUse3, AutogradNestedTensor, Tracer, AutocastCPU, AutocastCUDA, FuncTorchBatched, FuncTorchVmapMode, Batched, VmapMode, FuncTorchGradWrapper, PythonTLSSnapshot, FuncTorchDynamicLayerFrontMode, PythonDispatcher].

Or when I put the increment that I want

    def __inc__(self, key, value, *args, **kwargs):
        if 'adj' in key: 
            return torch.tensor([[getattr(self, 'x').size(0)], [getattr(self, 'x').size(0)]])
        elif "list" in key:
            return torch.tensor([getattr(self, 'x').size(0)])
        else:
            return super().__inc__(key, value, *args, **kwargs)

I get the following error:

"
Exception has occurred: RuntimeError
Expected both self and other to be nested, but got a nested self and non-nested other.
File "/Users/try.py", line 38, in
batch = Batch.from_data_list([data, data])
"

I found out that PyG has its own function for that called "torch_geometric.utils.to_nested_tensor()", but it doesn't function for my code as well with the following error although the input is list of tensors as expected:

"Traceback (most recent call last):
File "", line 1, in
File "/Users/opt/anaconda3/envs/py3.9/lib/python3.9/site-packages/torch_geometric/utils/nested.py", line 9, in to_nested_tensor
def to_nested_tensor(
File "/Users/opt/anaconda3/envs/py3.9/lib/python3.9/site-packages/torch/nested/init.py", line 18, in as_nested_tensor
def as_nested_tensor(
TypeError: nested_tensor(): Expected first argument to be a list of tensors "

Do you by any chance know how can I solve the above issues?

Thanks a lot again.

[rusty1s]

How do you call to_nested_tensor from PyG? I would need to investigate a bit how we can enable nested tensors in Batch.from_data_list. Will take a look.

[vkovac2]

Actually I do not use "Batch.from_data_list" in my actual code and it still doesn't work. This was just to explain my problem better. In the actual code I just use it similarly with a dataloader, but it never reaches that point as it throws an error earlier.

A potential solution is the following:

list_relations = torch.nested.to_padded_tensor(torch.nested.nested_tensor(list(relations.values())), padding = -9999)

, but then I would have to remove the negative values from my batch in order to be able to index the respective positions.

The way I use the PyG "to_nested_tensor" is the following:

import torch

from torch_geometric.data import Batch, Data
from typing import Any
from torch_geometric.utils import to_nested_tensor

class MyData(Data):
    def __inc__(self, key, value, *args, **kwargs):
        if 'adj' in key: 
            return torch.tensor([[getattr(self, 'x').size(0)], [getattr(self, 'x').size(0)]])
        elif "list" in key:
            return torch.tensor([getattr(self, 'x').size(0)])
        else:
            return super().__inc__(key, value, *args, **kwargs)
    def __cat_dim__(self, key: str, value: Any, *args, **kwargs) -> Any:
        if 'adj' in key:
            return 1
        else:
            return 0
            


adj = torch.tensor([[0,  1,  1,  2,  2,  2,  3,  3,  4,  4,  5,  5,  5,  6,  6,  7,  7,  8,
          8,  8,  9, 10, 10, 10, 11, 11, 12, 12, 12, 13, 13, 14, 14, 15, 15, 15,
         16, 16, 16, 16, 17, 18, 19, 19, 19, 20, 20, 21, 21, 21, 22, 23, 23, 24,
         24, 25, 25, 26, 26, 27, 27, 27, 28, 28], 
         [ 1,  0,  2,  1,  3, 28,  2,  4,  3,  5,  4,  6, 27,  5,  7,  6,  8,  7,
          9, 10,  8,  8, 11, 27, 10, 12, 11, 13, 26, 12, 14, 13, 15, 14, 16, 25,
         15, 17, 18, 19, 16, 16, 16, 20, 24, 19, 21, 20, 22, 23, 21, 21, 24, 19,
         23, 15, 26, 12, 25,  5, 10, 28,  2, 27]])

relations = {0:torch.tensor([2, 28, 27, 10, 8, 7, 6, 5, 4, 3]), 1:torch.tensor([2, 28, 27, 5, 4, 3]), 2:torch.tensor([2, 3, 4, 5, 27, 28])}
list_relations = to_nested_tensor(list(relations.values()))

data = MyData(x = torch.randn([29,5]), adj = adj, list_relations = list_relations)

batch = Batch.from_data_list([data, data])
print(batch)

Thanks

[rusty1s]

Added this via #7643

[vkovac2]

Amazing. Thank you very much. It is not possible still to increment the values in the nested tensor though right? Is there any way to overcome that issue?

[rusty1s]

That should work just fine by incrementing via __inc__.

[rusty1s]

Confirmed here: #7647

[vkovac2]

Amazing. Thank you very much. Increment was not working last night bcs I had a tensor for its value, but I saw from your test code that it should be an integer when we have nested tensors. Thanks!