Faster training with augmentations in the loop

[ishank-juneja]

Earlier when I tried training my segmentation with MaskRCNN without any augmentations I had a training time of about 30 minutes.

Now when I have added augmentations to my training loop, I am seeing a reduced training time of about 20 minutes for the same config which is very confusing to me since the augmentations involve loading data from disk which I thought would have slowed down the training loop by some amount and certainly not speed it up.

To be clear the only thing I am changing between the 2 runs is one line in my build_train_loader which goes from,

@classmethod
    def build_train_loader(cls, cfg):
        # aug_imgnet_fg = ImagenetFgAugmentation()
        aug_imgnet_bg = ImagenetBgAugmentation()
        aug_outline_shape = OutlineShapeAugmentation()
        dataloader = build_detection_train_loader(cfg, mapper=DatasetMapperForNPY(cfg, is_train=True))
        return dataloader

to,

@classmethod
    def build_train_loader(cls, cfg):
        # aug_imgnet_fg = ImagenetFgAugmentation()
        aug_imgnet_bg = ImagenetBgAugmentation()
        aug_outline_shape = OutlineShapeAugmentation()
        dataloader = build_detection_train_loader(cfg, mapper=DatasetMapperForNPY(cfg, is_train=True, augmentations=[aug_outline_shape, aug_imgnet_bg]))
        return dataloader

I suspect this is because of a bug somewhere but wanted to check here