Adapters are pretty cool. Let's see if we can implement them for our Transformer encoder models by integrating with Adapter Transformers.

A particular cool extension would be AdapterFusion, which would allow us to glue together Kiwi Adapter Transformer models for various language pairs into a single multilingual Kiwi model. This idea is a QE version of the the adapters for cross-lingual transfer. Lucky for us, AdapterFusion is also implemented in Adapter Transformers.