Phase 1.5: add soup chat, soup push, DPO trainer + smoke tests

- soup chat --model ./path: interactive terminal chat with LoRA adapters
  (auto-detects base model, supports /quit /clear /system commands)
- soup push --model ./path --repo user/model: upload to HuggingFace Hub
  (auto model card generation, token from env/cache/flag)
- DPO trainer: full DPOTrainerWrapper with LoRA + quantization support
  (configurable dpo_beta, preference data format {prompt, chosen, rejected})
- Smoke tests: real SFT + DPO training with tiny-gpt2 (pytest -m smoke)
- SFT trainer: fallback for models without chat_template
- Updated README, schema, formats, pyproject.toml, .gitignore

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Author: MakazhanAlpamys

.claude/settings.json

@@ -13,12 +13,15 @@
       "Bash(git remote*)",
       "Bash(ruff check*)",
       "Bash(ruff format*)",
+      "Bash(python -m ruff*)",
       "Bash(pytest*)",
       "Bash(python -m pytest*)",
       "Bash(pip install*)",
       "Bash(pip list*)",
       "Bash(soup *)",
-      "Bash(python -m soup_cli*)"
+      "Bash(python -m soup_cli*)",
+      "Bash(cd /c/Users/tokmo/peder/Soup && python -m pytest*)",
+      "Bash(cd /c/Users/tokmo/peder/Soup && ruff check*)"
     ]
   }
 }

.gitignore

@@ -34,3 +34,6 @@ wandb/
 # Secrets
 .env
 *.key
+
+# Internal plan (not for repo)
+plan.md

README.md

@@ -86,6 +86,50 @@ training:
 output: ./output
 ```
 
+## DPO Training
+
+Train with preference data using Direct Preference Optimization:
+
+```yaml
+base: meta-llama/Llama-3.1-8B-Instruct
+task: dpo
+
+data:
+  train: ./data/preferences.jsonl
+  format: dpo
+
+training:
+  epochs: 3
+  dpo_beta: 0.1
+  lora:
+    r: 64
+    alpha: 16
+  quantization: 4bit
+```
+
+## Chat with your model
+
+```bash
+# Chat with a LoRA adapter (auto-detects base model)
+soup chat --model ./output
+
+# Specify base model explicitly
+soup chat --model ./output --base meta-llama/Llama-3.1-8B-Instruct
+
+# Adjust generation
+soup chat --model ./output --temperature 0.3 --max-tokens 256
+```
+
+## Push to HuggingFace
+
+```bash
+# Upload model to HF Hub
+soup push --model ./output --repo your-username/my-model
+
+# Make it private
+soup push --model ./output --repo your-username/my-model --private
+```
+
 ## Data Formats
 
 Soup supports these formats (auto-detected):
@@ -105,6 +149,11 @@ Soup supports these formats (auto-detected):
 {"messages": [{"role": "user", "content": "Hi"}, {"role": "assistant", "content": "Hello!"}]}
 ```
 
+**DPO (preference pairs):**
+```json
+{"prompt": "Explain gravity", "chosen": "Gravity is a force...", "rejected": "I don't know"}
+```
+
 ## Data Tools
 
 ```bash
@@ -121,12 +170,14 @@ soup data validate ./data/train.jsonl --format alpaca
 |---|---|
 | LoRA / QLoRA fine-tuning | ✅ |
 | SFT (Supervised Fine-Tune) | ✅ |
-| DPO (Direct Preference Optimization) | 🔜 |
+| DPO (Direct Preference Optimization) | ✅ |
 | Auto batch size | ✅ |
 | Auto GPU detection (CUDA/MPS/CPU) | ✅ |
 | Live terminal dashboard | ✅ |
-| Alpaca / ShareGPT / ChatML formats | ✅ |
+| Alpaca / ShareGPT / ChatML / DPO formats | ✅ |
 | HuggingFace datasets support | ✅ |
+| Interactive model chat | ✅ |
+| Push to HuggingFace Hub | ✅ |
 | Experiment tracking | 🔜 |
 | Web dashboard | 🔜 |
 | Cloud mode (BYOG) | 🔜 |
@@ -143,7 +194,12 @@ soup data validate ./data/train.jsonl --format alpaca
 git clone https://github.com/MakazhanAlpamys/Soup.git
 cd Soup
 pip install -e ".[dev]"
+
+# Run unit tests (fast, no GPU needed)
 pytest tests/ -v
+
+# Run smoke tests (downloads tiny model, runs real training)
+pytest tests/ -m smoke -v
 ```
 
 ## License

pyproject.toml

@@ -33,6 +33,7 @@ dependencies = [
     "datasets>=2.14.0",
     "bitsandbytes>=0.41.0",
     "accelerate>=0.25.0",
+    "huggingface-hub>=0.16.0",
 ]
 
 [project.optional-dependencies]
@@ -57,3 +58,5 @@ select = ["E", "F", "I", "N", "W"]
 
 [tool.pytest.ini_options]
 testpaths = ["tests"]
+markers = ["smoke: slow smoke tests that download models and run training (run with: pytest -m smoke)"]
+addopts = "-m 'not smoke'"

soup_cli/cli.py

@@ -4,7 +4,7 @@
 from rich.console import Console
 
 from soup_cli import __version__
-from soup_cli.commands import data, init, train
+from soup_cli.commands import chat, data, init, push, train
 
 console = Console()
 
@@ -18,6 +18,8 @@
 # Register sub-commands
 app.command()(init.init)
 app.command()(train.train)
+app.command()(chat.chat)
+app.command()(push.push)
 app.add_typer(data.app, name="data", help="Dataset tools: inspect, convert, validate.")