How to do mini-batching on the fly without the PyG dataloader?

[JosephGatto]

Suppose I am not using the PyG datasets/dataloader class and I just have a regular dataloader with graphs stored as adjacency matrices.

What I would like to accomplish is the following. Suppose I have these two batched sets of features/matrices

node_feats # (32,100,300)
dense_adj_mats # (32,100,100)

Where I am training with batch_size=32.

Can I, on the fly, batch these so I can then simply pass to GCNConv? I tried simply converting the dense_adj_mats with dense_to_sparse() but this doesnt seem to work properly. I feel something must also be done with node_feats.

Note: The reason I don't want to use the PyG datasets class is this is largely an NLP problem with lots of other features I need to batch in a certain way. I obtain node_feats after lots of text transformations. Thus, the pipeline is already in place -- hoping to just drop in GCNConv without restructuring everything.

Thanks!

[rusty1s]

Yes, you can use dense_to_sparse for the adjacency matrix. For node features, simply reshaping to (batch_size * num_nodes, num_features) is sufficient.

[JosephGatto]

Worked! Thanks!