-
Notifications
You must be signed in to change notification settings - Fork 717
Description
tsai primarily implements BERT-like and InceptionTime architectures for self-supervised learning. BERT was designed for NLP tasks and InceptionTime is an ensemble of CNNs, which were designed for image tasks.
There are new state of the art models for self-supervised learning on images: SwAV and ReLIC. I believe SwAV is currently considered state of the art. ReLIC with stronger augmentations may be superior to SwAV. These architectures were trained and tested on image datasets, but, similar to the generalization of BERT and CNNs, they may generalize well to time series data.
This was intended more as an informational message to @oguiza than an issue to keep open until complete. Even if this is interesting to implement, it's a lot of work. I'll have to stick to using MVP and InceptionTimePlus for the time being, but I might get around to implementing one of these newer architectures after I've completed work using the existing architectures.