Add SFT data generation pipeline for poetry training

Implements multi-agent orchestrator supporting both Claude Code agents and
OpenRouter models (Kimi K2, DeepSeek) for generating verified poetry with
dual reasoning traces (SYNTH stenographic + natural language).

Key components:
- src/abide/synth_trace.py: SYNTH trace synthesis using Baguettotron notation
- scripts/openrouter_generator.py: OpenRouter API integration with abide verification
- scripts/sft_orchestrator.py: Parallel agent coordination with configurable mix
- .claude/commands/generate-sft.md: Slash command interface
- config/sft_generation.yaml: Default configuration for top 10 learnable forms

Also adds GRPO training infrastructure for Baguettotron with variance-based
form selection and DAPO loss to prevent entropy collapse.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

Author: darrenangle

.beads/issues.jsonl

@@ -52,6 +52,7 @@
 {"id":"abide-4ei","title":"Add Tritina (3x3-line stanzas + 1-line envoi)","description":"","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-07T19:03:41.484179717-06:00","updated_at":"2025-12-08T11:38:10.128485781-06:00","closed_at":"2025-12-08T11:38:10.128485781-06:00"}
 {"id":"abide-4ie","title":"Blues Poem form implementation","description":"AAa tercets with L1 repeated/varied as L2, L3 rhymes","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-07T14:29:20.833270567-06:00","updated_at":"2025-12-07T14:44:48.770427318-06:00","closed_at":"2025-12-07T14:44:48.770427318-06:00","dependencies":[{"issue_id":"abide-4ie","depends_on_id":"abide-rgl","type":"blocks","created_at":"2025-12-07T14:29:36.510251614-06:00","created_by":"daemon"}]}
 {"id":"abide-4k6","title":"Ghazal form implementation","description":"5-15 couplets, AA BA CA pattern with radif (refrain) and qafiya (rhyme)","status":"closed","priority":1,"issue_type":"task","created_at":"2025-12-07T14:29:16.027733238-06:00","updated_at":"2025-12-07T14:44:48.767802115-06:00","closed_at":"2025-12-07T14:44:48.767802115-06:00","dependencies":[{"issue_id":"abide-4k6","depends_on_id":"abide-rgl","type":"blocks","created_at":"2025-12-07T14:29:36.474694476-06:00","created_by":"daemon"}]}
+{"id":"abide-4l9w","title":"Build SFT orchestrator script","description":"Create scripts/sft_orchestrator.py that coordinates form agents, manages model mix configuration, and handles progress tracking with append-only JSONL output","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-29T16:43:15.709428915-06:00","updated_at":"2025-12-29T16:51:14.531810751-06:00","closed_at":"2025-12-29T16:51:14.531810751-06:00","dependencies":[{"issue_id":"abide-4l9w","depends_on_id":"abide-ww9u","type":"blocks","created_at":"2025-12-29T16:43:45.519612388-06:00","created_by":"darren"}]}
 {"id":"abide-4pq7","title":"Audit StaircasePoem verification","description":"","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-11T10:55:22.217891418-06:00","updated_at":"2025-12-11T11:08:01.630787869-06:00","closed_at":"2025-12-11T11:08:01.630787869-06:00"}
 {"id":"abide-4tf","title":"Audit ChantRoyal verification","description":"","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-11T10:45:53.232943326-06:00","updated_at":"2025-12-11T11:08:01.639200354-06:00","closed_at":"2025-12-11T11:08:01.639200354-06:00"}
 {"id":"abide-4x3i","title":"Update training script and start background training","description":"Apply optimal params to train_grpo.py and launch training","status":"open","priority":2,"issue_type":"task","created_at":"2025-12-23T20:52:41.086438031-06:00","updated_at":"2025-12-23T20:52:41.086438031-06:00","dependencies":[{"issue_id":"abide-4x3i","depends_on_id":"abide-zuf1","type":"blocks","created_at":"2025-12-23T20:53:13.411609274-06:00","created_by":"darren"}]}
@@ -329,6 +330,7 @@
 {"id":"abide-nu6n","title":"Audit scoring: PrecisionVerse","description":"Check verify() scoring - fix if too lenient","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-23T07:12:14.948554602-06:00","updated_at":"2025-12-23T07:20:18.47301495-06:00","closed_at":"2025-12-23T07:20:18.47301495-06:00"}
 {"id":"abide-nuyk","title":"Audit scoring: BurnsStanza","description":"Check verify() scoring - fix if too lenient","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-23T07:06:19.301492189-06:00","updated_at":"2025-12-23T07:23:41.581061048-06:00","closed_at":"2025-12-23T07:23:41.581061048-06:00"}
 {"id":"abide-o54","title":"Add Burns Stanza/Standard Habbie (AAABAB with short B)","description":"","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-07T19:04:17.055144771-06:00","updated_at":"2025-12-08T11:38:10.11266736-06:00","closed_at":"2025-12-08T11:38:10.11266736-06:00"}
+{"id":"abide-o7s9","title":"Implement OpenRouter generator with abide verification loop","description":"Create scripts/openrouter_generator.py that uses OpenRouter API for Kimi K2 and other models, with retry loop based on abide scores","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-29T16:42:57.629915288-06:00","updated_at":"2025-12-29T16:48:50.981580033-06:00","closed_at":"2025-12-29T16:48:50.981580033-06:00","dependencies":[{"issue_id":"abide-o7s9","depends_on_id":"abide-ww9u","type":"blocks","created_at":"2025-12-29T16:43:30.28092095-06:00","created_by":"darren"}]}
 {"id":"abide-ohs","title":"Add Mesostic constraint (middle letters spell word)","description":"","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-07T19:04:26.581522622-06:00","updated_at":"2025-12-08T11:38:10.10476702-06:00","closed_at":"2025-12-08T11:38:10.10476702-06:00"}
 {"id":"abide-ojyo","title":"Audit Quatrain verification","description":"","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-11T10:52:45.616301354-06:00","updated_at":"2025-12-11T11:02:03.353080894-06:00","closed_at":"2025-12-11T11:02:03.353080894-06:00"}
 {"id":"abide-otk","title":"Improve rentrement verification in Roundel","description":"\n## Current State\nRoundel verifies: LineCount(11), StanzaCount(3), RhymeScheme\nHas Refrain with threshold=0.5 for partial match but not included in constraints!\n\n## Issue\nThe _refrain is created but NOT added to the constraint list!\nLines 248-254 show Refrain is initialized but constraints list only has line_count and rhyme\n\n## Fix\nAdd self._refrain to the constraints list\n\n## Files\n- src/abide/forms/rondel.py:200-279\n","status":"closed","priority":3,"issue_type":"task","created_at":"2025-12-11T00:48:16.279305177-06:00","updated_at":"2025-12-11T00:55:16.467899729-06:00","closed_at":"2025-12-11T00:55:16.467899729-06:00"}
@@ -340,6 +342,7 @@
 {"id":"abide-p5ka","title":"Audit scoring: BlankVerse","description":"Check verify() scoring - fix if too lenient","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-23T07:05:59.015257782-06:00","updated_at":"2025-12-23T07:22:26.118064351-06:00","closed_at":"2025-12-23T07:22:26.118064351-06:00"}
 {"id":"abide-pa98","title":"Audit scoring: GoldenRatio","description":"Check verify() scoring - fix if too lenient","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-23T07:09:16.991904061-06:00","updated_at":"2025-12-23T07:25:43.120652676-06:00","closed_at":"2025-12-23T07:25:43.120652676-06:00"}
 {"id":"abide-pcj9","title":"Audit Quatina verification","description":"","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-11T10:52:40.540319609-06:00","updated_at":"2025-12-11T11:08:01.632623229-06:00","closed_at":"2025-12-11T11:08:01.632623229-06:00"}
+{"id":"abide-pldd","title":"Create /generate-sft slash command","description":"Create .claude/commands/generate-sft.md that orchestrates parallel haiku agents and OpenRouter workers to generate verified poetry with dual reasoning traces","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-29T16:43:09.731029344-06:00","updated_at":"2025-12-29T16:51:14.524834028-06:00","closed_at":"2025-12-29T16:51:14.524834028-06:00","dependencies":[{"issue_id":"abide-pldd","depends_on_id":"abide-ww9u","type":"blocks","created_at":"2025-12-29T16:43:40.447443821-06:00","created_by":"darren"}]}
 {"id":"abide-pms","title":"Audit BlankVerse verification","description":"","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-11T10:45:17.766887446-06:00","updated_at":"2025-12-11T11:05:19.343041578-06:00","closed_at":"2025-12-11T11:05:19.343041578-06:00"}
 {"id":"abide-pne","title":"Verify Kyrielle refrain implementation","description":"\n## Question\nKyrielle should have a refrain (last line of each stanza repeats)\nNeed to check if this is currently verified in the implementation\n\n## Investigation\n1. Read kyrielle.py to check current constraints\n2. If refrain not verified, add Refrain constraint\n3. Traditional kyrielle: 8-syllable lines, AABB or ABAB rhyme, refrain line ends each quatrain\n\n## Files\n- src/abide/forms/kyrielle.py\n","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-11T00:47:36.57084964-06:00","updated_at":"2025-12-11T00:56:19.809815567-06:00","closed_at":"2025-12-11T00:56:19.809815567-06:00"}
 {"id":"abide-psr","title":"Audit Distich verification","description":"","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-11T10:47:12.623018471-06:00","updated_at":"2025-12-11T11:02:03.358048374-06:00","closed_at":"2025-12-11T11:02:03.358048374-06:00"}
@@ -393,6 +396,7 @@
 {"id":"abide-vghk","title":"Audit Mesostic verification","description":"","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-11T10:50:23.02576068-06:00","updated_at":"2025-12-11T11:00:38.436874926-06:00","closed_at":"2025-12-11T11:00:38.436874926-06:00"}
 {"id":"abide-vhcz","title":"Audit MonotoneMountain verification","description":"","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-11T10:50:43.292307731-06:00","updated_at":"2025-12-11T11:06:27.973305658-06:00","closed_at":"2025-12-11T11:06:27.973305658-06:00"}
 {"id":"abide-vksc","title":"Audit scoring: TriangularVerse","description":"Check verify() scoring - fix if too lenient","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-23T07:15:58.712355106-06:00","updated_at":"2025-12-23T07:25:30.570550107-06:00","closed_at":"2025-12-23T07:25:30.570550107-06:00"}
+{"id":"abide-vl8r","title":"Implement SYNTH trace synthesizer","description":"Create abide/synth_trace.py that converts rubric results to SYNTH stenographic format with confidence markers, section headers, and tree structures","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-29T16:43:03.74349666-06:00","updated_at":"2025-12-29T16:48:50.988037073-06:00","closed_at":"2025-12-29T16:48:50.988037073-06:00","dependencies":[{"issue_id":"abide-vl8r","depends_on_id":"abide-ww9u","type":"blocks","created_at":"2025-12-29T16:43:35.356668139-06:00","created_by":"darren"}]}
 {"id":"abide-vnpi","title":"Audit scoring: ThunderVerse","description":"Check verify() scoring - fix if too lenient","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-23T07:15:48.568643636-06:00","updated_at":"2025-12-23T07:25:30.574649396-06:00","closed_at":"2025-12-23T07:25:30.574649396-06:00"}
 {"id":"abide-voyp","title":"Audit scoring: Rondeau","description":"Check verify() scoring - fix if too lenient","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-23T07:13:16.252145369-06:00","updated_at":"2025-12-23T07:26:16.531436275-06:00","closed_at":"2025-12-23T07:26:16.531436275-06:00"}
 {"id":"abide-vvou","title":"Audit scoring: FibonacciPoem","description":"Check verify() scoring - fix if too lenient","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-23T07:08:56.512053375-06:00","updated_at":"2025-12-23T07:22:37.129386224-06:00","closed_at":"2025-12-23T07:22:37.129386224-06:00"}
@@ -410,6 +414,7 @@
 {"id":"abide-wqx","title":"Add Anaphora constraint (repeated opening words)","description":"","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-07T19:04:26.633413969-06:00","updated_at":"2025-12-08T11:38:10.103710923-06:00","closed_at":"2025-12-08T11:38:10.103710923-06:00"}
 {"id":"abide-wr3","title":"Audit FibonacciVerse verification","description":"","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-11T10:48:42.715489289-06:00","updated_at":"2025-12-11T11:06:27.976076299-06:00","closed_at":"2025-12-11T11:06:27.976076299-06:00"}
 {"id":"abide-wsnb","title":"Audit scoring: EchoEnd","description":"Check verify() scoring - fix if too lenient","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-23T07:08:15.940000365-06:00","updated_at":"2025-12-23T07:26:20.0345503-06:00","closed_at":"2025-12-23T07:26:20.0345503-06:00"}
+{"id":"abide-ww9u","title":"Epic: SFT Data Generation Pipeline for Reasoning Model Warmup","description":"","status":"in_progress","priority":2,"issue_type":"epic","created_at":"2025-12-29T16:31:23.689218876-06:00","updated_at":"2025-12-29T16:51:19.599143358-06:00"}
 {"id":"abide-wz1i","title":"Audit scoring: LiteraryBallad","description":"Check verify() scoring - fix if too lenient","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-23T07:10:33.514302683-06:00","updated_at":"2025-12-23T07:23:41.580708254-06:00","closed_at":"2025-12-23T07:23:41.580708254-06:00"}
 {"id":"abide-x2dt","title":"Audit scoring: Ghazal","description":"Check verify() scoring - fix if too lenient","status":"closed","priority":2,"issue_type":"task","created_at":"2025-12-23T07:09:11.876972622-06:00","updated_at":"2025-12-23T07:22:07.927394877-06:00","closed_at":"2025-12-23T07:22:07.927394877-06:00"}
 {"id":"abide-x4r","title":"Add EndWordPattern to Tritina","description":"\n## Current State\nTritina only verifies: LineCount(10), StanzaCount(4), StanzaSizes([3,3,3,1])\n\n## Missing Verification  \nEnd-word rotation pattern for 3 words: ABC -\u003e CAB -\u003e BCA\n- Stanza 1: A B C\n- Stanza 2: C A B  \n- Stanza 3: B C A\n- Envoi: all 3 words\n\n## Implementation\nUse EndWordPattern constraint with num_words=3, rotation=[2,0,1]\nWeight: 3.0 (defining characteristic)\n\n## Files\n- src/abide/forms/tina.py:29-112\n- src/abide/constraints/relational.py (EndWordPattern)\n","status":"closed","priority":1,"issue_type":"task","created_at":"2025-12-11T00:46:31.470485088-06:00","updated_at":"2025-12-11T00:51:20.134226961-06:00","closed_at":"2025-12-11T00:51:20.134226961-06:00"}

.claude/commands/generate-sft.md

@@ -0,0 +1,122 @@
+# Generate SFT Training Data
+
+Generate verified poetry SFT data with dual reasoning traces (SYNTH + natural).
+
+## Arguments
+$ARGUMENTS
+
+Parse the arguments to extract:
+- `--forms`: Comma-separated list of poetry forms (default: top 10 learnable)
+- `--num`: Number of examples per form (default: 100)
+- `--min-score`: Minimum abide score to accept (default: 0.8)
+- `--model-mix`: Ratio of Claude agents vs OpenRouter (0-1, default: 0.5)
+- `--openrouter-model`: Model for OpenRouter (default: moonshotai/kimi-k2)
+- `--output`: Output JSONL file (default: data/sft_dataset.jsonl)
+- `--backend`: Force backend: openrouter, claude, or mixed
+
+## Task
+
+You are orchestrating SFT data generation for poetry forms. For each form:
+
+1. **Spawn parallel haiku agents** (if using Claude backend):
+   - Each agent generates poems for its assigned form
+   - Agent verifies with abide until score >= min_score
+   - Agent produces both SYNTH and natural reasoning traces
+
+2. **Run OpenRouter workers** (if using OpenRouter backend):
+   - Use the openrouter_generator.py script
+   - Models: Kimi K2, DeepSeek, etc.
+   - Same verification loop with abide
+
+3. **Output format** (append-only JSONL):
+```json
+{
+  "form": "Sonnet",
+  "prompt": "Write a Sonnet about autumn in a melancholic tone",
+  "synth_trace": "<think>\nSonnet requirements:\n├─ lines: 14 ●\n...",
+  "natural_trace": "<think>\nI need to write a Sonnet about autumn...",
+  "poem": "When autumn's gentle breath...",
+  "score": 0.92,
+  "rubric": [...],
+  "model": "claude-haiku",
+  "timestamp": "2024-12-29T22:30:00Z"
+}
+```
+
+## Execution
+
+Run the orchestrator with parsed arguments:
+
+```bash
+python scripts/sft_orchestrator.py \
+  --forms "$FORMS" \
+  --num "$NUM" \
+  --min-score "$MIN_SCORE" \
+  --model-mix "$MODEL_MIX" \
+  --output "$OUTPUT"
+```
+
+Or for specific backends:
+- OpenRouter only: `--backend openrouter --model moonshotai/kimi-k2`
+- Claude only: `--backend claude`
+
+## SYNTH Trace Format
+
+The SYNTH traces use Baguettotron's stenographic notation:
+
+**Logical markers:**
+- `→` derivation/implication
+- `↺` iterative refinement
+- `?` uncertainty
+- `!/※` insight/breakthrough
+- `≈` approximation
+- `∴` conclusion
+
+**Confidence markers:**
+- `●` high confidence
+- `◐` medium confidence
+- `○` low confidence
+- `⚠` warning/risk
+
+**Verification:**
+- `☐` unverified
+- `☑` partial verification
+- `✓` confirmed
+
+**Structure:**
+```
+<think>
+FormName requirements:
+├─ constraint1: ●
+├─ constraint2: ◐
+└─ topic: theme
+
+### 1. Task Analysis
+→ planning steps...
+
+### 2. Composition Strategy
+⟨H≈0.5⟩ exploring options...
+
+### 3. Verification
+✓ line_count: 100%
+☑ syllables: 85%
+
+∴ FormName complete: ● high confidence
+</think>
+```
+
+## Progress Tracking
+
+Report progress periodically:
+- `[N/total] FormName: score=X.XX (model)`
+- Success/failure counts
+- Rate (poems/minute)
+- ETA for completion
+
+## Example Usage
+
+```
+/generate-sft --forms Sonnet,Haiku --num 10 --model-mix 0.7
+```
+
+This generates 20 poems total (10 Sonnets + 10 Haikus), with 70% using Claude agents and 30% using OpenRouter.

config/sft_generation.yaml

@@ -0,0 +1,61 @@
+# SFT Data Generation Configuration
+# Usage: python scripts/sft_orchestrator.py --config config/sft_generation.yaml
+
+pipeline:
+  # Poetry forms to generate (top 10 learnable from GRPO signal analysis)
+  forms:
+    - IrregularOde
+    - ConsonantCascade
+    - Sonnet
+    - CoprimeVerse
+    - Anaphora
+    - Mesostic
+    - Rubaiyat
+    - PetrarchanSonnet
+    - Etheree
+    - BroadBallad
+
+  # Number of verified examples per form
+  num_per_form: 100
+
+  # Minimum abide score to accept a poem
+  min_score: 0.8
+
+  # Maximum retry attempts per poem
+  max_retries: 5
+
+model_mix:
+  # Ratio of Claude Code agents vs OpenRouter workers
+  # 0.0 = all OpenRouter, 1.0 = all Claude, 0.5 = 50/50 mix
+  claude_ratio: 0.5
+
+  claude:
+    # Model for Claude Code agents: haiku or sonnet
+    model: haiku
+    # Number of parallel agents
+    parallel_agents: 4
+
+  openrouter:
+    # OpenRouter API key from environment
+    api_key_env: OPENROUTER_API_KEY
+
+    # Models to use (randomly selected per task)
+    models:
+      - moonshotai/kimi-k2
+      # - deepseek/deepseek-r1
+      # - anthropic/claude-sonnet-4
+
+    # Number of parallel workers
+    parallel_workers: 4
+
+output:
+  # Output JSONL file (append-only)
+  file: data/sft_dataset.jsonl
+
+  # Print progress checkpoint every N examples
+  checkpoint_every: 10
+
+# Topics and tones are loaded from:
+# - data/topics.txt (one per line)
+# - data/tones.txt (one per line)
+# If files don't exist, defaults are used.