Problems when tried to use NELL dataset

[jianchaoji]

Thank you for your updata of NELL dataset! I met some problems when I tried to use this dataset on graph_sage_unsup.py:

The first problem is, the dataset is too large. Since we should input a tensor dataset instead of sparseTensor dataset for the model,I used the x.to_dense() function to transform it. But when I tried to do so, it exceeded the memory of the GPU.Can I have any potential solution ? Thank you!

x, edge_index = data.x.to_dense().to(device), data.edge_index.to(device)
RuntimeError: CUDA out of memory. Tried to allocate 15.01 GiB (GPU 0; 11.75 GiB total capacity; 29.95 MiB already allocated; 3.89 GiB free; 34.00 MiB reserved in total by PyTorch)

I met the sencond problem when I tried to use data.num_nodes to assign other variable. I fixed it by changing the data.num_nodes to 65755 which is the number of nodes of NELL dataset. Is it a correct using or I made some mistakes here?

  File "my_graph_sage.py", line 72, in <module>
    shuffle=True, num_nodes=data.num_nodes)#changed the batch size
  File "/home/jj635/anaconda3/lib/python3.7/site-packages/torch_geometric/data/data.py", line 212, in num_nodes
    return item.size(self.__cat_dim__(key, item))
  File "/home/jj635/anaconda3/lib/python3.7/site-packages/torch_sparse/tensor.py", line 214, in size
    return self.sizes()[dim]
TypeError: list indices must be integers or slices, not tuple

[rusty1s]

The data.num_nodes issue is now fixed in master.

Indeed, data.x is way to large to convert it into a dense tensor. I suggest to use a linear layer to reduce its dimensionality, e.g.:

x = data.x.to_torch_sparse_coo_tensor().to(device)
...
def forward(self, x, adjs):
    x = self.lin(x)  # Reduce to dense `hidden_channels` representation.
    ...

[jianchaoji]

I tried to solve this problem by transfer the data to sparsetensor by x = data.x.to_torch_sparse_coo_tensor().to(device). In the next step, I should select some nodes and transfer them to a tensor type. And put them into the model,like what I do in a tensor :model(x[n_id], adjs).
I think the sparsetensor can't support such operation. Do I miss anything?

And when I tried to use data.x.to_scipy(), there is an error raised :

/home/jj635/anaconda3/lib/python3.7/site-packages/torch_sparse/storage.py:14: UserWarning: `layout` argument unset, using default layout "coo". This may lead to unexpected behaviour.
  warnings.warn('`layout` argument unset, using default layout '
Traceback (most recent call last):
  File "my_graph_sage.py", line 109, in <module>
    x, edge_index = data.x.to_scipy().to(device), data.edge_index.to(device)
  File "/home/jj635/anaconda3/lib/python3.7/site-packages/scipy/sparse/base.py", line 691, in __getattr__
    raise AttributeError(attr + " not found")
AttributeError: to not found 

Can I get any help? Thank you!

[rusty1s]

Yes, you are right. I missed that one:
I suggest you convert to torch_sparse_coo_tensor after indexing the specific rows, e.g.:

x = data.x.to(device)  # This will be a `SparseTensor` from `torch-sparse`
...
model(x[n_id], adjs)  # This works on `SparseTensor`
...
# Before calling `self.lin`, you need to convert to `torch_sparse_coo_tensor`:
x = self.lin(x.to_torch_sparse_coo_tensor())

[jianchaoji]

Thank you so much for your help!

When I indexing the specific rows:
model(x[n_id], adjs)

Can we just use: x = x.to_dense() instead of x = self.lin(x.to_torch_sparse_coo_tensor()) to get a tensor.

And another question is,for the full_forward() in the test():

def test():
    model.eval()

    out = model.full_forward(x, edge_index).cpu()

    
    clf = LogisticRegression(max_iter = 1000)
    clf.fit(out[data.train_mask], data.y[data.train_mask])

    val_acc = clf.score(out[data.val_mask], data.y[data.val_mask])
    test_acc = clf.score(out[data.test_mask], data.y[data.test_mask])

    return val_acc, test_acc

 def full_forward(self, x, edge_index):
        for i, conv in enumerate(self.convs):
            x = conv(x, edge_index)
            if i != self.num_layers - 1:
                x = x.relu()
                x = F.dropout(x, p=0.5, training=self.training)
        return x

Since the data is too large,in the test() shall we split the whole data into batches like what we do in train(), before we do the LogisticRegression(). Such as:

for batch_size, n_id, adjs in test_loader:

     adjs = [adj.to(device) for adj in adjs]
     out = model.full_forward(x[n_id], edge_index).cpu()


     clf = LogisticRegression(max_iter = 1000)
     clf.fit(out[data.train_mask], data.y[data.train_mask])


     val_acc = clf.score(out[data.val_mask], data.y[data.val_mask])
     test_acc = clf.score(out[data.test_mask], data.y[data.test_mask])

     return val_acc, test_acc

[rusty1s]

You may also be able to convert it to a dense matrix after indexing, as long as it fits into GPU memory.

For inference, you might be able to fit the data into GPU when using a sparse representation. Otherwise, you can either just run it on CPU, or use mini-batching on GPU.

[jianchaoji]

Well reveived. Thank you!