Transformer training tutorial

Author: pythonlessons

CHANGELOG.md

@@ -1,3 +1,15 @@
+## [1.1.0] - 2022-08-28
+### Changed
+- Changed `mltu.transformers.SpectrogramPadding` object, to pad spectrogram end with zeros instead of start
+
+### Added
+- Created `Tutorials/09_translation_transformer` tutorial, that shows how to train translation transformer model
+- Created `mltu.tensorflow.tokenizers` module, that contains `CustomTokenizer` for text data
+- Created `mltu.tensorflow.transformer.attention` module, that contains `BaseAttention`, `CrossAttention`, `GlobalSelfAttention` and `CausalSelfAttention` layers
+- Created `mltu.tensorflow.transformer.layers` module, that contains `positional_encoding` function, `PositionalEmbedding`, `FeedForward`, `EncoderLayer`, `DecoderLayer`, `Encoder`, `Decoder` layers and `Transformer` model 
+- Created `mltu.tensorflow.transformer.callbacks` module, that contains `EncDecSplitCallback` callback, to split Transformer model into separate encoder and decoder models
+- Created `mltu.tensorflow.transformer.utils` module, that contains `MaskedLoss` loss and `MaskedAccuracy` metric, used for training Transformer models
+
 ## [1.0.15] - 2022-07-15
 ### Changed
 - Fixed bug in `mltu.dataProvider.DataProvider` to work with `batch_postprocessors`.

README.md

@@ -23,4 +23,5 @@ Each tutorial has its own requirements.txt file for a specific mltu version. As
 5. [Introduction to speech recognition with TensorFlow](https://pylessons.com/speech-recognition), code in ```Tutorials\05_speech_recognition``` folder;
 6. [Introduction to PyTorch in a practical way](https://pylessons.com/pytorch-introduction), code in ```Tutorials\06_pytorch_introduction``` folder;
 7. [Using custom wrapper to simplify PyTorch models training pipeline](https://pylessons.com/pytorch-introduction), code in ```Tutorials\07_pytorch_wrapper``` folder;
-8. [Handwriting words recognition with PyTorch](https://pylessons.com/handwriting-recognition-pytorch), code in ```Tutorials\08_handwriting_recognition_torch``` folder;
+8. [Handwriting words recognition with PyTorch](https://pylessons.com/handwriting-recognition-pytorch), code in ```Tutorials\08_handwriting_recognition_torch``` folder;
+9. [Transformer training with TensorFlow for Translation task](https://pylessons.com/transformers-training), code in ```Tutorials\09_translation_transformer``` folder;

Tutorials/09_translation_transformer/README.md

@@ -0,0 +1,10 @@
+# Training TensorFlow Transformer model for Spanish to English translation task
+### In this tutorial, I'll walk through a practical example of Transformer Training for Language Translation tasks from Spanish to the English language
+
+<br><br>
+# **Detailed tutorial**:
+### [Transformer training with TensorFlow for Translation task](https://pylessons.com/transformers-training)
+
+<p align="center">
+    <img src="https://pylessons.com/media/Tutorials/transformers/transformers-training/transformers-training.png">
+</p>

Tutorials/09_translation_transformer/transformer.ipynb

@@ -1,4 +1,4 @@
-__version__ = "1.0.15"
+__version__ = "1.1.0"
 
 from .annotations.images import Image
 from .annotations.images import CVImage