How to use CircleLoss like ArcfaceLoss

[fancyerii]

my training data has label, And I want to learning embeddings for fast search. I have used ArcFaceLoss like:

def __init__:
        if loss_name == 'ArcFaceLoss':
            self.loss_func = losses.ArcFaceLoss(
                num_classes=num_classes, 
                embedding_size=self.fusion.get_out_size(),
                **loss_kwargs_dict
            )

def forward():
        if self.loss_name == 'ArcFaceLoss':
            combined_emb = F.normalize(combined_emb, p=2, dim=1)
            results = {}
            if labels is not None:
                loss = self.loss_func(combined_emb, labels)
                results["loss"] = loss

it works fine. And I want to try CircleLoss. But according to the documentation, it's for pairwise labels only.

class CircleLoss(GenericPairLoss):
    """
    Circle loss for pairwise labels only.

    Args:
    m:  The relaxation factor that controls the radious of the decision boundary.
    gamma: The scale factor that determines the largest scale of each similarity score.

    According to the paper, the suggested default values of m and gamma are:

    Face Recognition: m = 0.25, gamma = 256
    Person Reidentification: m = 0.25, gamma = 128
    Fine-grained Image Retrieval: m = 0.4, gamma = 80

    By default, we set m = 0.4 and gamma = 80
    """

How can I use it for my data?

[KevinMusgrave]

Hi, I think you should be able to use CircleLoss with your data, as long as each batch contains multiple samples per label.

So you can still use CircleLoss like this:

self.loss_func(combined_emb, labels)

And it'll form positive/negative pairs internally.

To guarantee that there are multiple samples per label in each batch, you can use MPerClassSampler. See #611 for an example (and a common problem) when using this sampler.

[fancyerii]

I have about 8000+ classes, and each class has about 2~3 samples. So If I use MPerClassSampler, the batch size will at least 8000+(m=1). But the largest batch size with my hardward configuration is 32 for my model.

[KevinMusgrave]

No it won't. Let's say you set m=2 and your dataloader batch size is 32. Then there will be 16 different classes in each batch.