LLM Fine-Tuning with Axolotl - Pod Deployment

Pod-based LLM fine-tuning using Axolotl on RunPod.

Serverless Version: See llm-fine-tuning for API-based deployments.

🚀 Quick Start

Image: runpod/llm-finetuning:latest

Required Environment Variables

HF_TOKEN=your-huggingface-token
WANDB_API_KEY=your-wandb-key

# Training config (examples)
AXOLOTL_BASE_MODEL=TinyLlama/TinyLlama_v1.1
AXOLOTL_DATASETS=[{"path":"mhenrichsen/alpaca_2k_test","type":"alpaca"}]
AXOLOTL_ADAPTER=lora

⚠️ Critical: Volume Mounting

# ❌ NEVER mount to /workspace - overwrites everything!
# ✅ Mount to /workspace/data only

Training

# Training starts automatically, or manually:
axolotl train config.yaml

Inference (after training)

# Create vLLM config from example
cp vllm_config_example.yaml my_config.yaml
# Edit with your model path
./start_vllm.sh my_config.yaml

🏗️ Local Development

# Build and test
docker build -t llm-finetuning-pod .
docker-compose up

📚 Documentation

• Development Conventions - Development guide and best practices
• Axolotl Documentation - Complete configuration reference

🔧 Troubleshooting

Volume Mount Issues

# Symptoms: "No such file or directory" errors, infinite loops
# Cause: Mounting to /workspace overwrites container structure
# Solution: Mount to /workspace/data/ subdirectories only

Environment Variables Not Loading

# Variables must be set before container starts
env | grep AXOLOTL_

Authentication Issues

echo $HF_TOKEN
echo $WANDB_API_KEY

🏷️ Available Images

Tag	Description	Use Case
runpod/llm-finetuning:latest	Latest stable release	Production pods
runpod/llm-finetuning:dev	Development build	Testing new features

---

💡 Tip: For API-driven serverless deployments, check out the main llm-fine-tuning repository.