What is class supervision?!

[AdityaNikhil]

Hi there,

I just completed reading your paper, and I see that in the paper you were stressing on class supervision a lot. Also in sec 3.3, you mentioned that, notable performance of Omni-GAN comes from making full use of class supervision.

Can you explain what is class supervision here?!

[PeterouZh]

Hi. Thanks for your interest in our work!

"Class supervision" refers to the class label of an image. In fact, both projection-based loss and omni-loss employ class labels. But projection-based loss is a weak implicit classification loss. Omni-loss is a classification loss that makes full use of the class label's supervision. Thanks for asking this. We will clarify it in an updated version of the paper.

We have done classification experiments to show that the classifier's classification ability trained with projection-based loss is relatively weak. One way to enhance the classification ability of projection-based loss is to increase the batch size. This also explains why using a large batch size for BigGAN can improve the performance. We will update the manuscript as soon as possible to detail this experiment.

On the other hand, a strong classification loss will cause the discriminator to overfit the training data, which will cause the training of cGANs to collapse earlier. As mentioned in the paper, using weight decay on the discriminator can effectively solve this problem.

The conclusion is that for cGANs, using a strong classification loss function combined with an appropriate regularization method (e.g., weight decay employed in the paper) for the discriminator can help achieve significant performance improvements.

Recently I have access to large-scale GPU resources. Experiments show that Omni-GAN has completely surpassed BigGAN in the ImageNet 128x128 experiment. These experiments will be added to our manuscript.

[AdityaNikhil]

Thank you for the explanation.!
I just have one more question, i searched all over internet but couldn't get proper answer.
Can you explain the major differences between projection based and classification based loss functions in GANs?!

[PeterouZh]

Suppose there is a real image (x_real, y_real) and a generated image (x_fake, y_fake). y is the corresponding class label of x.

Note that the original projection-based discriminator is implemented in the manner of class embedding (refer to the code). This can be transformed into a discriminator whose last layer is a multi-dimensional fully connected layer. As shown above, the discriminator takes as input an image and outputs a (C+1)-dimensional vector, where C is the number of classes.

The main difference between projection-based loss and classification-based loss is that projection-based loss is a weaker implicit classification loss (In one iteration, many logits are not optimized, e.g., s_0, s_1, ..., s_{c-1}). However, for classification-based loss, all logits will be optimized.

Note that for the sake of simplicity in the above figure, the output dimension of the discriminator is the C+1 dimension, and its function is similar to the C+2 dimension used in our paper. In our paper, in order to decouple the real and fake categories of GAN, one more dimension is used.

I hope this helps!