GraphSAINT heterogeneous loader

[cctony1]

Somewhat related to #2587

I'm wondering if there is a timeline for enabling the conversion of the graphsaint samplers from homo to hetero graphs (or hetero-to-homo and back again conversions before and after sampling).

Thanks for the awesomeness of PyG!

[rusty1s]

We currently do not have such plans, it's definitely lower priority :( I see what I can do. Note that you can probably always right your own custom logic to go from homogeneous sampled subgraph back to HeteroData.

[cctony1]

Oh, thanks. I'll look into it for reference.

[cctony1]

HI again!

We're really close to running NARS - actually a colleague had it working and then left.

I'm trying to recreate it and have an error. Would it be possible to post something here and have you look at it? Or is that bad protocol?

---

TypeError Traceback (most recent call last)
Input In [12], in <cell line: 17>()
16 n_epochs = 50
17 for epoch in range(1, n_epochs+1):
---> 18 loss = train(x_train, y_train, train_idx,wts_tns)
19 #print(f'Epoch: {epoch:03d}, Loss: {loss:.4f}')
20 if epoch % 10 == 0:

Input In [10], in train(x, y, idx, wts)
12 optimizer.zero_grad()
13 x_mini = nars.weighted_aggregation(x_mini)
---> 14 out = sign(x_mini.unbind(0))
15 loss = F.cross_entropy(out, y_mini,wts)
16 loss.backward()

File ~\OneDrive - Nationwide\Documents.Envs.venv_neo4j_01\lib\site-packages\torch\nn\modules\module.py:1130, in Module._call_impl(self, *input, **kwargs)
1126 # If we don't have any hooks, we want to skip the rest of the logic in
1127 # this function, and just call forward.
1128 if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
1129 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1130 return forward_call(*input, **kwargs)
1131 # Do not call functions when jit is used
1132 full_backward_hooks, non_full_backward_hooks = [], []

Input In [6], in SIGN.forward(self, xs)
42 def forward(self, xs):
43 outs = [mlp(x) for x, mlp in zip(xs, self.mlps)]
---> 44 return self.mlp(torch.cat(outs, dim=-1))

File ~\OneDrive - Nationwide\Documents.Envs.venv_neo4j_01\lib\site-packages\torch\nn\modules\module.py:1130, in Module._call_impl(self, *input, **kwargs)
1126 # If we don't have any hooks, we want to skip the rest of the logic in
1127 # this function, and just call forward.
1128 if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
1129 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1130 return forward_call(*input, **kwargs)
1131 # Do not call functions when jit is used
1132 full_backward_hooks, non_full_backward_hooks = [], []

File ~\AppData\Local\Temp\clara32_pyg\tmp4t0oh9jt.py:22, in Sequential_0fbc96.forward(self, x)
20 x = self.module_2(x)
21 x = self.module_3(x)
---> 22 x = self.module_4(x)
23 return x

File ~\OneDrive - Nationwide\Documents.Envs.venv_neo4j_01\lib\site-packages\torch\nn\modules\module.py:1137, in Module._call_impl(self, input, **kwargs)
1135 if _global_forward_pre_hooks or self._forward_pre_hooks:
1136 for hook in (_global_forward_pre_hooks.values(), *self._forward_pre_hooks.values()):
-> 1137 result = hook(self, input)
1138 if result is not None:
1139 if not isinstance(result, tuple):

File ~\OneDrive - Nationwide\Documents.Envs.venv_neo4j_01\lib\site-packages\torch\autograd\grad_mode.py:27, in _DecoratorContextManager.call..decorate_context(*args, **kwargs)
24 @functools.wraps(func)
25 def decorate_context(*args, **kwargs):
26 with self.clone():
---> 27 return func(*args, **kwargs)

File ~\OneDrive - Nationwide\Documents.Envs.venv_neo4j_01\lib\site-packages\torch_geometric\nn\dense\linear.py:124, in Linear.initialize_parameters(self, module, input)
122 if is_uninitialized_parameter(self.weight):
123 self.in_channels = input[0].size(-1)
--> 124 self.weight.materialize((self.out_channels, self.in_channels))
125 self.reset_parameters()
126 self._hook.remove()

File ~\OneDrive - Nationwide\Documents.Envs.venv_neo4j_01\lib\site-packages\torch\nn\parameter.py:110, in UninitializedTensorMixin.materialize(self, shape, device, dtype)
108 if dtype is None:
109 dtype = self.data.dtype
--> 110 self.data = torch.empty(shape, device=device, dtype=dtype)
111 self.class = self.cls_to_become

TypeError: empty() received an invalid combination of arguments - got (tuple, dtype=torch.dtype, device=torch.device), but expected one of:

• (tuple of ints size, *, tuple of names names, torch.memory_format memory_format, torch.dtype dtype, torch.layout layout, torch.device device, bool pin_memory, bool requires_grad)
• (tuple of ints size, *, torch.memory_format memory_format, Tensor out, torch.dtype dtype, torch.layout layout, torch.device device, bool pin_memory, bool requires_grad)

where our SIGN class is modified from yours:

class SIGN(torch.nn.Module):
def init(self, in_channels, hidden_channels, out_channels, num_hops,
dropout=0.5):
super(SIGN, self).init()

    self.mlps = ModuleList()
      
    for _ in range(num_hops):
        mlp = Sequential('x', [
            (Linear(-1, hidden_channels), 'x -> x'),
            ReLU(),
            (Linear(-1, hidden_channels), 'x -> x'),
            ReLU()])

        self.mlps.append(mlp)

    self.mlp = Sequential('x', [
            (Dropout(p=dropout), 'x -> x'),
            (Linear(-1, hidden_channels), 'x -> x'),
            ReLU(),
            (Dropout(p=dropout), 'x -> x'),
            (Linear(-1, out_channels), 'x -> x'),
            ])

def forward(self, xs):
    outs = [mlp(x) for x, mlp in zip(xs, self.mlps)]
    return self.mlp(torch.cat(outs, dim=-1))

    
def reset_parameters(self):
    for layer in self.children():
        if hasattr(layer, 'reset_parameters'):
            layer.reset_parameters()     

[rusty1s]

Can you create a separate issue for this? I am happy to look into this. If you have a reproducible example, even better :)

[cctony1]

Ok. I'll try to put something together this week. Much appreciated.

[cctony1]

Oh my goodness. I found a typo. A stray comma messed up the code, turning the number of output channels into a tuple. It's working now!