We are happy to announce Swarm 2.3.0 community release.
In this release, we have delivered key enhancements of supporting HuggingFace Trainer class (a framework for LLM fine-tuning) enabling federated LLM fine-tuning and training use-cases.
Features:
• Swarm support for HF Trainer class – a framework for LLM training and fine-tuning.
• Example of mini LLM fine-tuning – pre-trained BERT model text classification for sentiment analysis with a 300MB dataset using
LoRA optimization and without LoRA optimization.
• Swarm support for model parameter sizes > 2GB. Implemented a docker volume based file sharing protocol over protobuf.
Now the size is unlimited and is practically only limited by available disk space on the host docker volume.
• Manual installation support for this release. However SWARM Management UI (SLM-UI) can be used for monitoring the training.
• GIT Documentation Updates.