[FEATURE] Add Semi-SL multilabel classification algorithm (#1805)

* Add multilabel semi sl cls

* Add semi sl multilabel recipies and modules

* Fix linters

* Fix multilabel classifier

* Fix linting issues

* Remove obsolete base model path in classificaiton templates

* Add cli test

* Update ssl configs

* Update multilabel semisl head to be consistent with smeisl hooks

* Add semi sl mlc

* Fix number of lalbeled images in semisl docs

* Fix multilabel semisl configs

* Add unit tests for mlc ssl heads

* Add tests for ssl mlc classifier

* Update licenses in tests

* Fix a typo in docstring

Co-authored-by: Jihwan Eom <[email protected]>

* Avoid extra multilabel model copying in config manager

* Move semisl helpers to utils

* Add a missing arg to docstring

* Update docs

* Fix linters

* Update MLC ssl docs

---------

Co-authored-by: Jihwan Eom <[email protected]>

Author: sovrasov

docs/source/guide/explanation/algorithms/classification/multi_label_classification.rst

@@ -52,11 +52,56 @@ In the table below the `mAP <https://en.wikipedia.org/w/index.php?title=Informat
 | EfficientNet-V2-S     | 91.91           | 77.28     | 71.52            | 80.24     |
 +-----------------------+-----------------+-----------+------------------+-----------+
 
-.. ************************
-.. Semi-supervised Learning
-.. ************************
+************************
+Semi-supervised Learning
+************************
 
-.. To be added soon
+Semi-SL (Semi-supervised Learning) is a type of machine learning algorithm that uses both labeled and unlabeled data to improve the performance of the model. This is particularly useful when labeled data is limited, expensive or time-consuming to obtain.
+
+To utilize unlabeled data during training, we use `BarlowTwins loss <https://arxiv.org/abs/2103.03230>`_ as an auxiliary loss for Semi-SL task solving. BarlowTwins enforces consistency across augmented versions of the same data (both labeled and unlabeled): each sample is augmented first with `Augmix <https://arxiv.org/abs/1912.02781>`_, then strongly augmented sample is generated by applying a pre-defined `RandAugment <https://arxiv.org/abs/1909.13719>`_ strategy on top of the basic augmentation.
+
+.. _mlc_cls_semi_supervised_pipeline:
+
+- ``BarlowTwins loss``: A specific implementation of Semi-SL that combines the use of a consistency loss with strong data augmentations, and a specific optimizer called Sharpness-Aware Minimization (`SAM <https://arxiv.org/abs/2010.01412>`_) to improve the performance of the model.
+
+- ``Adaptive loss auxiliary loss weighting``: A technique for assigning such a weight for an auxiliary loss that the resulting value is a predefined fraction of the EMA-smoothed main loss value. This method allows aligning contribution of the losses during different training phases.
+
+- ``Exponential Moving Average (EMA)``: A technique for maintaining a moving average of the model's parameters, which can improve the generalization performance of the model.
+
+- ``Additional techniques``: Other than that, we use several solutions that apply to supervised learning (No bias Decay, Augmentations, Early-Stopping, etc.)
+
+Please, refer to the :doc:`tutorial <../../../tutorials/advanced/semi_sl>` on how to train semi-supervised learning.
+Training time depends on the number of images and can be up to several times longer than conventional supervised learning.
+
+In the table below the mAP metric on some public datasets using our pipeline is presented.
+
++-----------------------+---------+----------------------+----------------+---------+----------------+---------+
+|        Dataset        | AerialMaritime 3 cls |         | VOC 2007 3 cls |         | COCO 14 5 cls  |         |
++=======================+======================+=========+================+=========+================+=========+
+|                       |   SL                 | Semi-SL |  SL            | Semi-SL |   SL           | Semi-SL |
++-----------------------+----------------------+---------+----------------+---------+----------------+---------+
+| MobileNet-V3-large-1x |  74.28               |  74.41  | 96.34          |  97.29  |  82.39         |  83.77  |
++-----------------------+----------------------+---------+----------------+---------+----------------+---------+
+|   EfficientNet-B0     |  79.59               |  80.91  | 97.75          |  98.59  | 83.24          |  84.19  |
++-----------------------+----------------------+---------+----------------+---------+----------------+---------+
+|  EfficientNet-V2-S    |  75.91               |  81.91  | 95.65          |  96.43  | 85.19          |  84.24  |
++-----------------------+----------------------+---------+----------------+---------+----------------+---------+
+
+AerialMaritime was sampled with 5 images per class. VOC was sampled with 10 images per class and COCO was sampled with 20 images per class.
+Additionel information abount the datasets can be found in the table below.
+
++-----------------------+----------------+----------------------+
+|        Dataset        | Labeled images | Unlabeled images     |
++=======================+================+======================+
+| AerialMaritime 3 cls  |  10            |  42                  |
++-----------------------+----------------+----------------------+
+| VOC 2007 3 cls        |  30            |  798                 |
++-----------------------+----------------+----------------------+
+| COCO 14 5 cls         |  95            |  10142               |
++-----------------------+----------------+----------------------+
+
+.. note::
+    This result can vary depending on the image selected for each class. Also, since there are few labeled settings for the Semi-SL algorithm. Some models may require larger datasets for better results.
 
 .. ************************
 .. Self-supervised Learning

otx/algorithms/classification/configs/base/data/semisl/data_pipeline.py

@@ -30,11 +30,12 @@
 
 __train_pipeline = [
     *__common_pipeline,
-    dict(type="PILImageToNDArray", keys=["img"]),
+    dict(type="PostAug", keys=dict(img_strong=__strong_pipeline)),
+    dict(type="PILImageToNDArray", keys=["img", "img_strong"]),
     dict(type="Normalize", **__img_norm_cfg),
-    dict(type="ImageToTensor", keys=["img"]),
+    dict(type="ImageToTensor", keys=["img", "img_strong"]),
     dict(type="ToTensor", keys=["gt_label"]),
-    dict(type="Collect", keys=["img", "gt_label"]),
+    dict(type="Collect", keys=["img", "img_strong", "gt_label"]),
 ]
 
 __unlabeled_pipeline = [

otx/algorithms/classification/configs/efficientnet_b0_cls_incr/semisl/model_multilabel.py

@@ -0,0 +1,30 @@
+"""EfficientNet-B0 config for semi-supervised multi-label classification."""
+
+# pylint: disable=invalid-name
+
+_base_ = ["../../../../../recipes/stages/classification/multilabel/semisl.yaml", "../../base/models/efficientnet.py"]
+
+model = dict(
+    task="classification",
+    type="SemiSLMultilabelClassifier",
+    backbone=dict(
+        version="b0",
+    ),
+    head=dict(
+        type="SemiLinearMultilabelClsHead",
+        use_dynamic_loss_weighting=True,
+        unlabeled_coef=0.1,
+        in_channels=-1,
+        aux_mlp=dict(hid_channels=0, out_channels=1024),
+        normalized=True,
+        scale=7.0,
+        loss=dict(type="AsymmetricAngularLossWithIgnore", gamma_pos=0.0, gamma_neg=1.0, reduction="sum"),
+        aux_loss=dict(
+            type="BarlowTwinsLoss",
+            off_diag_penality=1.0 / 128.0,
+            loss_weight=1.0,
+        ),
+    ),
+)
+
+fp16 = dict(loss_scale=512.0)

otx/algorithms/classification/configs/efficientnet_b0_cls_incr/template.yaml

@@ -15,7 +15,6 @@ entrypoints:
   base: otx.algorithms.classification.tasks.ClassificationTrainTask
   openvino: otx.algorithms.classification.tasks.ClassificationOpenVINOTask
   nncf: otx.algorithms.classification.tasks.nncf.ClassificationNNCFTask
-base_model_path: ../../adapters/deep_object_reid/configs/efficientnet_b0/template_experimental.yaml
 
 # Capabilities.
 capabilities:

otx/algorithms/classification/configs/efficientnet_v2_s_cls_incr/semisl/model_multilabel.py

@@ -0,0 +1,30 @@
+"""EfficientNet-V2 config for semi-supervised multi-label classification."""
+
+# pylint: disable=invalid-name
+
+_base_ = ["../../../../../recipes/stages/classification/multilabel/semisl.yaml", "../../base/models/efficientnet_v2.py"]
+
+model = dict(
+    task="classification",
+    type="SemiSLMultilabelClassifier",
+    backbone=dict(
+        version="s_21k",
+    ),
+    head=dict(
+        type="SemiLinearMultilabelClsHead",
+        use_dynamic_loss_weighting=True,
+        unlabeled_coef=0.1,
+        in_channels=-1,
+        aux_mlp=dict(hid_channels=0, out_channels=1024),
+        normalized=True,
+        scale=7.0,
+        loss=dict(type="AsymmetricAngularLossWithIgnore", gamma_pos=0.0, gamma_neg=1.0, reduction="sum"),
+        aux_loss=dict(
+            type="BarlowTwinsLoss",
+            off_diag_penality=1.0 / 128.0,
+            loss_weight=1.0,
+        ),
+    ),
+)
+
+fp16 = dict(loss_scale=512.0)

otx/algorithms/classification/configs/efficientnet_v2_s_cls_incr/template.yaml

@@ -15,7 +15,6 @@ entrypoints:
   base: otx.algorithms.classification.tasks.ClassificationTrainTask
   openvino: otx.algorithms.classification.tasks.ClassificationOpenVINOTask
   nncf: otx.algorithms.classification.tasks.nncf.ClassificationNNCFTask
-base_model_path: ../../adapters/deep_object_reid/configs/efficientnet_v2_s/template_experimental.yaml
 
 # Capabilities.
 capabilities:

otx/algorithms/classification/configs/mobilenet_v3_large_075_cls_incr/template_experiment.yaml

@@ -15,7 +15,6 @@ entrypoints:
   base: otx.algorithms.classification.tasks.ClassificationTrainTask
   openvino: otx.algorithms.classification.tasks.ClassificationOpenVINOTask
   nncf: otx.algorithms.classification.tasks.nncf.ClassificationNNCFTask
-base_model_path: ../../adapters/deep_object_reid/configs/mobilenet_v3_large_075/template_experimental.yaml
 
 # Capabilities.
 capabilities:

otx/algorithms/classification/configs/mobilenet_v3_large_1_cls_incr/semisl/model_multilabel.py

@@ -0,0 +1,29 @@
+"""MobileNet-V3-large-1 config for semi-supervised multi-label classification."""
+
+# pylint: disable=invalid-name
+
+_base_ = ["../../../../../recipes/stages/classification/multilabel/semisl.yaml", "../../base/models/mobilenet_v3.py"]
+
+model = dict(
+    task="classification",
+    type="SemiSLMultilabelClassifier",
+    backbone=dict(mode="large"),
+    head=dict(
+        type="SemiNonLinearMultilabelClsHead",
+        in_channels=960,
+        hid_channels=1280,
+        use_dynamic_loss_weighting=True,
+        unlabeled_coef=0.1,
+        aux_mlp=dict(hid_channels=0, out_channels=1024),
+        normalized=True,
+        scale=7.0,
+        loss=dict(type="AsymmetricAngularLossWithIgnore", gamma_pos=0.0, gamma_neg=1.0, reduction="sum"),
+        aux_loss=dict(
+            type="BarlowTwinsLoss",
+            off_diag_penality=1.0 / 128.0,
+            loss_weight=1.0,
+        ),
+    ),
+)
+
+fp16 = dict(loss_scale=512.0)

otx/algorithms/classification/configs/mobilenet_v3_large_1_cls_incr/template.yaml

@@ -15,7 +15,6 @@ entrypoints:
   base: otx.algorithms.classification.tasks.ClassificationTrainTask
   openvino: otx.algorithms.classification.tasks.ClassificationOpenVINOTask
   nncf: otx.algorithms.classification.tasks.nncf.ClassificationNNCFTask
-base_model_path: ../../adapters/deep_object_reid/configs/mobilenet_v3_large_1/template_experimental.yaml
 
 # Capabilities.
 capabilities:

otx/algorithms/classification/configs/mobilenet_v3_small_cls_incr/template_experiment.yaml

@@ -15,7 +15,6 @@ entrypoints:
   base: otx.algorithms.classification.tasks.ClassificationTrainTask
   openvino: otx.algorithms.classification.tasks.ClassificationOpenVINOTask
   nncf: otx.algorithms.classification.tasks.nncf.ClassificationNNCFTask
-base_model_path: ../../adapters/deep_object_reid/configs/mobilenet_v3_small/template_experimental.yaml
 
 # Capabilities.
 capabilities: