The computation of L_MCE in DIFO

[yxsysu]

Hi, thanks for your great work.
I have a question after reading the paper and the code.
It seems that the loss L_MCE is computed at lines 180-184 in src/methods/net/difo.py,
but it does not exactly match the form in the paper.
Could you give me some hints? Thanks for your reply.

[HazelSu]

This is a question of the calculation method for contrastive learning. The traditional version of Moco uses cross entropy to implement the calculation. Its core is to construct a label (target). In our method, we also follow the manner of Moco. The label uses two one-hot, sum of them satisfies 1. Then bi is equal to 1 and ai is less than 1, achieving a similar regularization effect.

[chendianshuiyin]

Thank you very much for sharing your work. However, I have some questions regarding the loss function part in the DIFO paper and code. Specifically, the calculation method of the L_MCE loss function proposed in the paper seems to differ from the code implementation. Could you please provide a more detailed explanation?

[HazelSu]

Thank you very much for sharing your work. However, I have some questions regarding the loss function part in the DIFO paper and code. Specifically, the calculation method of the L_MCE loss function proposed in the paper seems to differ from the code implementation. Could you please provide a more detailed explanation?

Please refer to our previous answer:
This is a question of the calculation method for contrastive learning. The traditional version of Moco uses cross entropy to implement the calculation. Its core is to construct a label (target). In our method, we also follow the manner of Moco. The label uses two one-hot, sum of them satisfies 1. Then bi is equal to 1 and ai is less than 1, achieving a similar regularization effect.

[chendianshuiyin]

Thank you for your patient reply. I understand. I am looking at Moco's calculation method.

[029-stack]

Could you please tell me on which line of code this MCE loss is？ I only find the IID loss which is about a joint mutual information maximization.

source-free-domain-adaptation/src/methods/oh/difo.py

Line 243 in deb4b1f

iic_loss = IID_losses.IID_loss(softmax_out, clip_all_output[tar_idx])

[HazelSu]

Could you please tell me on which line of code this MCE loss is？ I only find the IID loss which is about a joint mutual information maximization.

source-free-domain-adaptation/src/methods/oh/difo.py

Line 243 in deb4b1f

iic_loss = IID_losses.IID_loss(softmax_out, clip_all_output[tar_idx])

line 235-239

[RSMung]

It seems that the formula 5 lacks a minus sign: '-'. Is that right?
Our objective is to increase the ai.
However, the value of formula 5 in your paper will increase after the increase of ai because the log and exp function are all monotonically increasing functions.

[tntek]

It seems that the formula 5 lacks a minus sign: '-'. Is that right? Our objective is to increase the ai. However, the value of formula 5 in your paper will increase after the increase of ai because the log and exp function are all monotonically increasing functions.

Seem indeed lack '-'. Thanks for your comment. We will fix the typo.