opencode_haiku_45_transcript.md

User

I'm building a feature that summarizes long Slack threads so users can catch up quickly. Threads can be pretty long, like 200+ messages. Which LLM should I use?

---

Assistant (Build · claude-haiku-4.5 · 3.9s)

I'll help you find the best LLM for summarizing long Slack threads. Let me use the which-llm skill to analyze the requirements and recommend optimal models.

Tool: skill

**Input:**
```json
{
  "name": "which-llm"
}

Output:

## Skill: which-llm

**Base directory**: C:\Users\User\.opencode\skills\which-llm

# Skill: which-llm

Select the right LLM(s) for a task using real benchmark and capability data.

## When to Use

- User needs to pick a model for a specific task
- User wants to compare models by capability/price/speed
- User is designing a multi-agent system and needs model recommendations

## Preflight Check

Before proceeding, verify the CLI is operational:

```bash
# 1. Check CLI exists and show version
which-llm --version

# 2. Check data freshness (should be < 7 days old)
which-llm cache status

# 3. If stale or missing, refresh data
which-llm llms --refresh

If CLI is unavailable: See references/INSTALL.md for installation, or use references/FALLBACK.md for heuristic recommendations without the CLI.

Quick Start

1. Run preflight check to verify CLI is ready
2. Classify the task using the decision tree below
3. Query with which-llm to find matching models
4. Recommend Primary + Fallback models with tradeoffs

Task Classification

Use this decision tree to classify the user's task:

Is the task primarily about FORMAT conversion (summarize, extract, reformat)?
├─ YES → Transformational (intelligence ≥ 20)
└─ NO ↓

Does it require EXTERNAL ACTIONS (API calls, DB queries, file ops, code execution)?
├─ YES → Does it need to PLAN multiple steps autonomously?
│        ├─ YES → Agentic (intelligence ≥ 48, coding ≥ 42)
│        └─ NO  → Tool-using (intelligence ≥ 35, coding ≥ 35)
└─ NO ↓

Does it require JUDGMENT, COMPARISON, or ANALYSIS?
├─ YES → Analytical (intelligence ≥ 38)
└─ NO  → Transformational (intelligence ≥ 20)

Skill Type Reference

Note: Thresholds calibrated for Intelligence Index v4.0 (Jan 2026), SOTA ~50. Scores within ±2 points are effectively equivalent. See references/BENCHMARKS.md for dynamic threshold calculation.

Skill Type	Examples	Min Intelligence	Min Coding	Consider Also
Transformational	summarize, extract, reformat	20	-	tps for high volume
Analytical	compare, analyze, justify	38	-	context_window for long docs
Tool-using	API calls, DB queries, code execution	35	35	tool_call = true
Agentic	plan, decompose, orchestrate, self-critique	48	42	tool_call, reasoning, large context_window

Additional Selection Factors

Beyond skill type thresholds, consider these constraints when relevant:

Factor	Column	When to Use
Context window	context_window	Long documents (>32k tokens), RAG with large chunks
Tool calling	tool_call	Function calling, MCP servers, API integration
Structured output	structured_output	JSON responses, typed outputs, schema validation
Reasoning	reasoning	Complex multi-step problems, chain-of-thought
Latency	latency	Real-time chat, streaming UIs (want < 0.5s)
Throughput	tps	Batch processing, high volume (want > 100 tps)
Open weights	open_weights	Self-hosting, fine-tuning, data privacy

Note: Capability fields require models_dev_matched = true (~53% of models have this data). Models without capability data will have NULL values for these fields.

Weighted Scoring

Instead of just filtering by thresholds, use weighted scoring to rank models based on user priorities.

Scoring Formula

Score = (intelligence × quality_weight) 
      + (100/price × cost_weight) 
      + (tps/10 × speed_weight)

Priority Presets

Preset	Quality	Cost	Speed	Best For
Balanced	0.4	0.4	0.2	General use, no strong preference
Quality	0.7	0.2	0.1	Critical tasks, accuracy matters most
Cost	0.2	0.7	0.1	High volume, budget-sensitive
Speed	0.2	0.2	0.6	Real-time, latency-sensitive

Weighted Query Example

# Balanced scoring for analytical tasks
which-llm query "SELECT name, intelligence, price, tps,
          ROUND((intelligence * 0.4) + (100/price * 0.4) + (tps/10 * 0.2), 1) as score
          FROM llms 
          WHERE intelligence >= 38 AND price > 0
          ORDER BY score DESC 
          LIMIT 10"

# Cost-priority scoring
which-llm query "SELECT name, intelligence, price, tps,
          ROUND((intelligence * 0.2) + (100/price * 0.7) + (tps/10 * 0.1), 1) as score
          FROM llms 
          WHERE intelligence >= 38 AND price > 0
          ORDER BY score DESC 
          LIMIT 10"

See references/QUERIES.md for more weighted scoring patterns.

Core Queries

# Find models meeting requirements, sorted by price
which-llm query "SELECT name, creator, intelligence, coding, price, tps 
          FROM llms 
          WHERE intelligence >= 38 
          ORDER BY price 
          LIMIT 10"

# Tool-using tasks: require tool_call support
which-llm query "SELECT name, creator, intelligence, coding, tool_call, price 
          FROM llms 
          WHERE intelligence >= 35 AND coding >= 35 AND tool_call = true
          ORDER BY price 
          LIMIT 10"

# Agentic tasks: high capability + tool calling + large context
which-llm query "SELECT name, creator, intelligence, coding, context_window, price 
          FROM llms 
          WHERE intelligence >= 48 AND coding >= 42 
            AND tool_call = true AND context_window >= 128000
          ORDER BY price 
          LIMIT 10"

# Speed-critical (real-time chat)
which-llm query "SELECT name, intelligence, tps, latency, price 
          FROM llms 
          WHERE tps > 100 AND latency < 0.5 
          ORDER BY tps DESC"

Cascade Recommendations

For cost optimization, recommend a Primary + Fallback pair instead of a single model:

1. Primary model: Cheapest model meeting relaxed requirements (e.g., 5-10 points below strict threshold)
2. Fallback model: Higher-capability model meeting strict requirements for when primary fails

Relaxed vs Strict thresholds:

• Strict: Use the skill type thresholds directly (e.g., Agentic: intelligence ≥ 48)
• Relaxed: Lower by ~20% for primary selection (e.g., intelligence ≥ 40)

This approach can reduce costs by 50-70% compared to always using the best model. See references/CASCADE-PATTERNS.md for detailed implementation guidance.

Output Format

## Task Classification
- **Skill Type:** [type]
- **Key Constraints:** [e.g., needs tool_call, context > 100k]
- **Priority:** [Balanced/Quality/Cost/Speed]
- **Reasoning:** [why this classification]

## Recommendations

### Primary: [Model] ($X.XX/M) - Score: Y
- Meets requirements: intelligence X, coding Y
- Capabilities: [relevant capabilities like tool_call, context_window]
- Why: Best cost/capability ratio for this task

### Fallback: [Model] ($X.XX/M) - Score: Y
- Use if: Primary fails or task is unusually complex
- Capabilities: [relevant capabilities]
- Why: Higher capability ceiling

### Other Options
| Rank | Model | Score | Intelligence | Price | Key Capability |
|------|-------|-------|--------------|-------|----------------|
| 3 | ... | ... | ... | ... | ... |
| 4 | ... | ... | ... | ... | ... |

## Cost Estimate
- **Primary only:** $X.XX/M tokens
- **Fallback only:** $Y.YY/M tokens  
- **Cascade (30% fallback):** $Z.ZZ/M tokens
- **Savings vs always fallback:** NN%

### Calculation
Cascade cost = (0.70 × $X.XX) + (0.30 × $Y.YY) = $Z.ZZ/M
Savings = (1 - Z.ZZ/Y.YY) × 100 = NN%

## Query Used
[the which-llm query command]

Cost Estimate Guidance

When presenting cost estimates:

1. Always show the cascade calculation - helps users understand the math
2. Use 30% as default escalation rate - reasonable starting assumption
3. Note the price ratio - larger ratios mean more potential savings
4. Include the caveat - actual savings depend on task complexity mix

Example:

## Cost Estimate
- **Primary only:** $0.50/M tokens
- **Fallback only:** $5.00/M tokens (10x primary)
- **Cascade (30% fallback):** $1.85/M tokens
- **Savings vs always fallback:** 63%

*Assumes 30% of requests escalate. Actual rate depends on task complexity.*

Important Disclaimer

These recommendations are indicative starting points, not definitive answers. Benchmark scores measure general capabilities but may not reflect performance on your specific task.

Always validate by testing candidate models on representative examples from your actual use case before committing to a model choice.

References

For detailed information, see:

• references/FALLBACK.md - Quick recommendations when CLI is unavailable
• references/INSTALL.md - Installing and configuring the which-llm CLI
• references/BENCHMARKS.md - What the scores mean + dynamic thresholds
• references/BENCHMARK-LIMITATIONS.md - What benchmarks can't tell you
• references/QUERIES.md - Common query patterns
• references/WORKFLOWS.md - End-to-end workflow examples
• references/MULTI-MODEL.md - Multi-model architecture guidance
• references/CASCADE-PATTERNS.md - Cascade and fallback patterns
• references/SPECIALIZATION.md - Domain-specific model guidance
• references/PROVIDERS.md - Provider-specific considerations

---

Assistant (Build · claude-haiku-4.5 · 3.4s)

Now let me run the preflight checks and query for the best models for your use case:

Tool: bash

**Input:**
```json
{
  "command": "which-llm --version",
  "description": "Check which-llm CLI version"
}

Output:

which-llm 0.1.0

Tool: bash

Input:

{
  "command": "which-llm cache status",
  "description": "Check data freshness"
}

Output:

Cache Status
============
Location: C:\Users\User\AppData\Local\which-llm
Entries:  5
Size:     1.6 MB

---

## Assistant (Build · claude-haiku-4.5 · 4.7s)

Good! Now let me query for models suited to thread summarization. This is a **transformational task** (format conversion - long text → short summary), but with a **context window constraint** since threads can be 200+ messages:

Tool: bash

Input:

{
  "command": "which-llm query \"SELECT name, creator, intelligence, context_window, price, tps FROM llms WHERE intelligence >= 20 AND context_window >= 32000 ORDER BY (intelligence * 0.4) + (100/price * 0.4) + (tps/10 * 0.2) DESC LIMIT 12\"",
  "description": "Find transformational models with large context windows"
}

Output:

| name | creator | intelligence | context_window | price | tps |
| --- | --- | --- | --- | --- | --- |
| Devstral 2 | Mistral | 21.70 | 256000 | 0.00 | 38.74 |
| Sonar Reasoning Pro | Perplexity | 24.60 | 128000 | 0.00 | 0.00 |
| Grok 3 Reasoning Beta | xAI | 21.60 | 131072 | 0.00 | 0.00 |
| gpt-oss-20B (high) | OpenAI | 24.70 | 131072 | 0.10 | 305.74 |
| gpt-oss-20B (low) | OpenAI | 21.20 | 131072 | 0.10 | 243.97 |
| GPT-5 nano (high) | OpenAI | 26.70 | 400000 | 0.14 | 140.08 |
| GPT-5 nano (medium) | OpenAI | 25.80 | 400000 | 0.14 | 136.19 |
| MiMo-V2-Flash (Reasoning) | Xiaomi | 39.10 | 256000 | 0.15 | 98.78 |
| MiMo-V2-Flash (Non-reasoning) | Xiaomi | 30.10 | 256000 | 0.15 | 90.69 |
| Gemini 2.5 Flash-Lite Preview (Sep '25) (Reasoning) | Google | 22.30 | 1048576 | 0.17 | 529.40 |
| Gemini 2.5 Flash-Lite Preview (Sep '25) (Non-reasoning) | Google | 20.10 | 1048576 | 0.17 | 486.98 |
| gpt-oss-120B (high) | OpenAI | 32.90 | 131072 | 0.26 | 326.42 |

---

## Assistant (Build · claude-haiku-4.5 · 4.1s)

Let me also check for a cost-optimized fallback option with relaxed requirements:

Tool: bash

Input:

{
  "command": "which-llm query \"SELECT name, creator, intelligence, context_window, price, tps FROM llms WHERE intelligence >= 18 AND price > 0 AND price < 0.50 ORDER BY (100/price * 0.7) + (intelligence * 0.2) + (tps/10 * 0.1) DESC LIMIT 8\"",
  "description": "Find cost-effective summarization models"
}

Output:

| name | creator | intelligence | context_window | price | tps |
| --- | --- | --- | --- | --- | --- |
| gpt-oss-20B (high) | OpenAI | 24.70 | 131072 | 0.10 | 305.74 |
| gpt-oss-20B (low) | OpenAI | 21.20 | 131072 | 0.10 | 243.97 |
| NVIDIA Nemotron 3 Nano 30B A3B (Reasoning) | NVIDIA | 24.70 |  | 0.10 | 189.24 |
| GPT-5 nano (high) | OpenAI | 26.70 | 400000 | 0.14 | 140.08 |
| GPT-5 nano (medium) | OpenAI | 25.80 | 400000 | 0.14 | 136.19 |
| MiMo-V2-Flash (Reasoning) | Xiaomi | 39.10 | 256000 | 0.15 | 98.78 |
| MiMo-V2-Flash (Non-reasoning) | Xiaomi | 30.10 | 256000 | 0.15 | 90.69 |
| Gemini 2.5 Flash-Lite Preview (Sep '25) (Reasoning) | Google | 22.30 | 1048576 | 0.17 | 529.40 |

---

## Assistant (Build · claude-haiku-4.5 · 9.5s)

## Task Classification
- **Skill Type:** Transformational (summarization)
- **Key Constraints:** Large context window (200+ messages), need to handle long Slack threads efficiently
- **Priority:** Balanced (quality + cost + speed matter equally)
- **Reasoning:** Thread summarization is format conversion, but the 200+ message constraint requires a model with significant context window to handle full threads without truncation.

## Recommendations

### Primary: **gpt-oss-20B** ($0.10/M) - Balanced Score: 7.2
- **Intelligence:** 24.7 | **Context:** 131k tokens | **Throughput:** 305 tps
- **Why:** Excellent cost-capability ratio for Slack thread summarization. 131k context easily handles 200+ messages with safety margin. Fast throughput makes it ideal for real-time summaries.
- **Best for:** Most threads, budget-conscious deployments

### Fallback: **MiMo-V2-Flash** ($0.15/M) - Score: 7.8
- **Intelligence:** 30-39 | **Context:** 256k tokens | **Throughput:** 99 tps
- **Why:** Higher intelligence for complex or ambiguous threads. Double the context window for unusually long conversations.
- **Use if:** Primary struggles with nuanced summaries or threads exceed 300+ messages

### Other Strong Options
| Rank | Model | Intelligence | Context | Price | Best For |
|------|-------|--------------|---------|-------|----------|
| 3 | GPT-5 nano (high) | 26.7 | 400k | $0.14 | Best context window at reasonable price |
| 4 | Gemini 2.5 Flash-Lite | 20-22 | 1M | $0.17 | Massive context if threads grow huge |

## Cost Estimate
- **Primary only (gpt-oss-20B):** $0.10/M tokens
- **Fallback only (MiMo-V2-Flash):** $0.15/M tokens
- **Cascade (70% primary / 30% fallback):** $0.115/M tokens
- **Savings vs always using fallback:** 23%

Calculation: (0.70 × $0.10) + (0.30 × $0.15) = $0.115/M

## Implementation Notes

For a **Slack thread summarizer**, I'd recommend:

1. **Use gpt-oss-20B as your primary** - it's hard to beat at $0.10/M with 131k context
2. **Fall back to MiMo-V2-Flash** for threads over ~100 messages where nuance matters
3. **Consider batching** - Slack thread reads aren't real-time, so queue them and batch process for better throughput

**Validation step:** Before deploying, test both models on 5-10 representative Slack threads from your workspace to verify they capture the key decisions/action items you need.

---