Backpropagating losses for two different outputs

[enter802]

I have the network below with two different outputs. To train the network, I tried adding the two losses and backpropagate the total, however, doing that, the performance of y1 only improves - y2 remains pretty much stale, in fact, worsening in some cases (different initialization, different hyperparameters). I was wondering if there is any other way(s) the network could be trained. For example, I was wondering if it is possible to train the weights of MLP1 and MLP2 with L1 and L2 respectively, while training the weights of the GNN with L1+L2. How do you code in pytorch to do that exactly?

Thanks for your inputs in advance and appreciate any guidance.

[rusty1s]

How does your loss function looks like? It may well be that you need to tune this quite a bit, e.g., by increasing the L2` loss:

loss = loss_1 + factor * loss_2
loss.backward()

[enter802]

My loss function looks the same as you have suggested except that I used factor=1. I will try increasing the factor. However, I was hoping if there is a way to implement what I asked earlier:

is it possible to train the weights of MLP1 and MLP2 with L1 and L2 respectively, while training the weights of the GNN with L1+L2?

your response is much appreciated. thanks.

[rusty1s]

That is what will automatically happen since any gradients from L1 will only go back through MLP1, not MLP2.

[enter802]

oh. that makes sense. Thanks a lot.