Skip to content

Commit b610558

Browse files
danielbentesclaude
danielbentes
and
claude
committed
docs: add WHY_SIARE comparison guide
- Technology positioning matrix comparing SIARE vs alternatives - Detailed comparison with LangChain, DSPy, AutoRAG, Self-RAG, LangGraph - Six mutation types explained with scope and description - Quality-Diversity benefits and when to use SIARE - Updated docs README with new page in navigation Extracted from competitive analysis, focusing on technical capabilities rather than business strategy. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>
1 parent a8195dd commit b610558

File tree

2 files changed

+176
-0
lines changed

2 files changed

+176
-0
lines changed

‎docs/README.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ Welcome to the SIARE (Self-Improving Agentic RAG Engine) documentation.
1010

1111
| Goal | Guide |
1212
|------|-------|
13+
| Understand what makes SIARE unique | [Why SIARE?](WHY_SIARE.md) |
1314
| Run your first pipeline | [Quick Start](QUICKSTART.md) |
1415
| Configure SIARE | [Configuration](CONFIGURATION.md) |
1516
| Fix common issues | [Troubleshooting](TROUBLESHOOTING.md) |
@@ -43,6 +44,7 @@ Welcome to the SIARE (Self-Improving Agentic RAG Engine) documentation.
4344
```
4445
docs/
4546
├── Getting Started
47+
│ ├── WHY_SIARE.md # What makes SIARE unique
4648
│ ├── QUICKSTART.md # 10-minute first run
4749
│ ├── CONFIGURATION.md # All settings explained
4850
│ └── TROUBLESHOOTING.md # Common issues

‎docs/WHY_SIARE.md‎

Lines changed: 174 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,174 @@
1+
# Why SIARE?
2+
3+
**How SIARE compares to other RAG frameworks and what makes it unique.**
4+
5+
## The Problem with Manual RAG Tuning
6+
7+
Building RAG systems today means endless iteration:
8+
- Tweak prompts → benchmark → repeat
9+
- Try different chunking strategies → benchmark → repeat
10+
- Adjust retrieval parameters → benchmark → repeat
11+
- Add more agents → debug interactions → repeat
12+
13+
This process is **expensive**, **brittle**, and **never-ending** as your data and requirements change.
14+
15+
## How SIARE is Different
16+
17+
SIARE is the first RAG engine that treats pipeline configuration as **evolvable genetic material**. Instead of manual tuning, SIARE uses AI-driven evolution to automatically discover optimal strategies.
18+
19+
### What SIARE Does That Others Don't
20+
21+
| Capability | Description |
22+
|------------|-------------|
23+
| **SOP-as-Genome** | Treats entire multi-agent pipeline configurations as evolvable genetic material with `PromptGenome`, mutation operators, and ancestry tracking |
24+
| **Quality-Diversity Optimization** | Maintains Pareto frontier across multiple metrics and QD Grid for behavioral diversity—prevents convergence to single local optimum |
25+
| **Multi-Agent Topology Evolution** | Evolves agent roles, graph structure, tool assignments, and inter-agent communication patterns—not just prompts |
26+
| **Plugin-Based Prompt Evolution** | Adaptive strategy selection combining TextGrad, EvoPrompt, and MetaPrompt approaches |
27+
| **GenePool with Ancestry** | Full lineage tracking enabling "breeding" of successful configurations |
28+
| **Constraint-Aware Evolution** | Safety-first design with `validate_constraints()` before mutations |
29+
30+
---
31+
32+
## Comparison with Alternatives
33+
34+
### RAG Frameworks (LangChain, LlamaIndex, Haystack)
35+
36+
These frameworks provide excellent **building blocks** for RAG but require manual configuration:
37+
38+
- **What they do well**: Modular components, tool integrations, document processing
39+
- **What they don't do**: Autonomous pipeline evolution, systematic optimization, diversity maintenance
40+
41+
**SIARE's approach**: Use these frameworks as adapters within SIARE, then let evolution optimize the pipeline configuration.
42+
43+
### Prompt Optimization (DSPy)
44+
45+
DSPy pioneered treating prompts as tunable parameters with automated optimization:
46+
47+
- **What it does well**: Bayesian optimization of prompts, few-shot selection
48+
- **What it doesn't do**: Multi-agent topology evolution, Quality-Diversity optimization, pipeline structure mutation
49+
50+
**SIARE's approach**: DSPy optimizes prompts within a fixed pipeline; SIARE evolves the entire pipeline including topology.
51+
52+
### AutoML for RAG (AutoRAG, RAGSmith)
53+
54+
These tools automate hyperparameter search:
55+
56+
- **What they do well**: Grid search over configurations, benchmark evaluation
57+
- **What they don't do**: Quality-Diversity (they seek single optimum), multi-agent orchestration, ancestry tracking
58+
59+
**SIARE's approach**: Evolutionary algorithms with QD optimization maintain diverse solutions, not just the single best.
60+
61+
### Self-Improving RAG (Self-RAG, Adaptive-RAG)
62+
63+
Research on runtime adaptation:
64+
65+
- **What they do well**: Dynamic retrieval decisions, self-critique mechanisms
66+
- **What they don't do**: Design-time evolution, population-based search, pipeline mutation
67+
68+
**SIARE's approach**: Evolves pipelines at design-time; these approaches adapt at runtime. Complementary, not competing.
69+
70+
### Multi-Agent Frameworks (LangGraph, AutoGen, CrewAI)
71+
72+
Multi-agent orchestration platforms:
73+
74+
- **What they do well**: Agent collaboration, stateful workflows, role-based systems
75+
- **What they don't do**: Autonomous evolution of agent topologies, QD optimization, prompt genome evolution
76+
77+
**SIARE's approach**: Define agents as evolvable roles; let evolution discover optimal team structures.
78+
79+
---
80+
81+
## Technology Positioning Matrix
82+
83+
| Capability | LangChain | DSPy | AutoRAG | Self-RAG | LangGraph | **SIARE** |
84+
|-----------|-----------|------|---------|----------|-----------|-----------|
85+
| Multi-Agent Orchestration | ✅ | ❌ | ❌ | ❌ | ✅ | ✅ |
86+
| Prompt Evolution | ❌ | ✅ | ❌ | ❌ | ❌ | ✅ |
87+
| Pipeline Evolution | ❌ | ❌ | ✅ | ❌ | ❌ | ✅ |
88+
| Quality-Diversity | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ |
89+
| Evolutionary Algorithms | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ |
90+
| Topology Mutation | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ |
91+
| Ancestry Tracking | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ |
92+
| Constraint Validation | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ |
93+
| SOP Versioning | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ |
94+
95+
---
96+
97+
## SIARE's Six Mutation Types
98+
99+
SIARE evolves pipelines through six mutation operators:
100+
101+
| Mutation | Scope | Description |
102+
|----------|-------|-------------|
103+
| `PROMPT_CHANGE` | Prompts | Modify role prompts using TextGrad, EvoPrompt, or MetaPrompt |
104+
| `PARAM_TWEAK` | Parameters | Adjust temperature, max_tokens, top_k, etc. |
105+
| `ADD_ROLE` | Topology | Introduce new specialist agent |
106+
| `REMOVE_ROLE` | Topology | Remove underperforming agent |
107+
| `REWIRE_GRAPH` | Topology | Change edge connections between agents |
108+
| `CROSSOVER` | Structure | Combine successful elements from two SOPs |
109+
110+
---
111+
112+
## Quality-Diversity: Why Diversity Matters
113+
114+
Traditional optimization finds **one best solution**. But RAG has multiple valid objectives:
115+
- High accuracy vs. low latency
116+
- Comprehensive answers vs. concise answers
117+
- High recall vs. high precision
118+
119+
**Quality-Diversity optimization** maintains a diverse archive of high-performing solutions along the Pareto frontier. This means:
120+
121+
1. **No local optima trap**: Multiple search directions explored simultaneously
122+
2. **User choice**: Pick the solution that best fits your specific tradeoffs
123+
3. **Robustness**: If requirements change, alternative solutions already exist
124+
4. **Insight**: Understanding the capability frontier reveals fundamental tradeoffs
125+
126+
---
127+
128+
## When to Use SIARE
129+
130+
**SIARE is ideal when:**
131+
- You need to optimize across multiple metrics (accuracy, cost, latency)
132+
- Your domain is complex with many possible pipeline configurations
133+
- Manual tuning has plateaued or is too expensive
134+
- You want to maintain diverse solutions for different use cases
135+
- Your requirements evolve and pipelines need to adapt
136+
137+
**SIARE may be overkill when:**
138+
- You have a simple, well-understood retrieval task
139+
- A single metric dominates (pure accuracy, no cost/latency concerns)
140+
- Your pipeline is fixed by external constraints
141+
- You need a quick prototype without evolution
142+
143+
---
144+
145+
## Getting Started
146+
147+
Ready to stop tuning and start evolving?
148+
149+
```bash
150+
pip install siare
151+
152+
# Initialize a project
153+
siare init
154+
155+
# Run evolution
156+
siare evolve --generations 10 --metric quality
157+
158+
# Query your evolved pipeline
159+
siare run "How do I reset my password?"
160+
```
161+
162+
See the [Quick Start Guide](QUICKSTART.md) for detailed setup instructions.
163+
164+
---
165+
166+
## Learn More
167+
168+
- [System Architecture](architecture/SYSTEM_ARCHITECTURE.md) — How SIARE works internally
169+
- [Custom Extensions](guides/custom-extensions.md) — Add your own metrics and tools
170+
- [Use Cases](guides/USE_CASES.md) — Domain implementation patterns
171+
172+
---
173+
174+
*SIARE: Stop tuning. Start evolving.*

0 commit comments

Comments
 (0)