Tensor of data.y printed to be of [128] size but why when reshaping it warns me RuntimeError: shape '[1, 128]' is invalid for input of size 28?

[Kevin-shihello-world]

I was using GFN from<> on PYG. And this model is quite weird. It seems composed of MLP instead of layers of graphconv and passes messages by using global_mean_pool, global_add_pool, and batch normalization. And my experience needs to use its data.y, which prints to be a Tensor with [128] size, the same as its batch size. But it warns me that 127 is too large for its size which is 28. So I tried to figure out what was going on and in the train_eval code I wrote the following code to see if it means in this PyTorch version I can't get items on a one dimension Tensor by using '[]' tag:

python
label = data.y#.view(-1)
#label = torch.randint(0,3,(radidx.shape[0],)).cuda()
label__ = torch.reshape(label, [1,128])
print(label[100].item())

And it warned me this

python
 File "/GFN/train_eval.py", line 241, in train
    label__ = torch.reshape(label, [1,128])
RuntimeError: shape '[1, 128]' is invalid for input of size 28

It's so weird, what could be the problem?
I would appreciate it if anyone could find me an answer and I think it would also help others who meet the same problem.
Thank you

[rusty1s]

Can you claridy what you mean by GFN? How does that model look like?

[Kevin-shihello-world]

The more interesting thing is I tried and found that batch normalization also helps with HGP-sl's training, and to lessen contingency, I use stored model parameters to initialize the model in the two experiences instead of using rand-seed for once before I found with rand-seed on the same model with the same training set it get very different output for me.

[rusty1s]

Interesting findings :) Yes, batch_size can actually have huge effects on model performance. Is your initial question resolved? If you have problems running their code, you might have success asking in https://github.com/chentingpc/gfn directly.

[Kevin-shihello-world]

The reason for this problem is really more about PYG, as I finally found it (covering my face.jpg). I was thought it more about the tensor arrangement and PYG workflow and my last question seems to haven't been mentioned on their website so sorry I haven't posted an issue about this on their page.
I found the key to this question is while training, the PYG.dataloader do not make every data contain the same number of graphs, instead if the num of graphs in a training set is divided batch_size not be 0 it will let the last data to contain fewer graphs. I thought in this case it will resent some of the graphs being trained in this epoch, but it turned out that I was wrong.
So yes, my initial question has been resolved.

[rusty1s]

You can also use drop_last in the loader to ensure that every batch holds the same number of graphs.

[Kevin-shihello-world]

Thanks. And I tried to set test_acc divided (10 + len(train_loader) ) as what shows on the PYGs' examples and HGP-sl, and only to find the test_acc be much less than the HGP-sl and even GIN with code from an example on PYG. I do not know why the GFN still ranks so high on the ENZYME dataset on paperwithcode and gets its paper to ICLR2020. I thought its writers, as from some judged earthy and hard-working universities won't do things like change the results