Skip to content

Commit 2bbdf6f

Browse files
committed
Open AI and Deep agents fix
1 parent 3f1e6a5 commit 2bbdf6f

File tree

4 files changed

+104
-79
lines changed

4 files changed

+104
-79
lines changed

docs/guides/deepagents-integration.md

Lines changed: 17 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -8,12 +8,10 @@
88
Learn how to build, run, evaluate, and optimize DeepAgents from scratch in 30 minutes:
99

1010
- ✅ Step-by-step with real expected outputs
11-
- ✅ Tested with FREE Gemini API
11+
- ✅ Works with FREE Gemini API
1212
- ✅ Persistent memory, real file access, hybrid storage
13-
- ✅ GEPA optimization (+200% improvements)
13+
- ✅ GEPA optimization guide
1414
- ✅ Production deployment guide
15-
16-
**Everything tested and working!** 🎉
1715

1816
---
1917

@@ -50,12 +48,19 @@ The biggest addition is the **backend abstraction** that lets you choose where a
5048
## 📦 Installation
5149

5250
```bash
51+
# Install SuperOptiX with DeepAgents support
5352
pip install superoptix[frameworks-deepagents]
53+
54+
# REQUIRED: Install Gemini integration (or your preferred LLM provider)
55+
pip install langchain-google-genai # For Gemini
56+
# pip install langchain-anthropic # For Claude
57+
# pip install langchain-openai # For GPT-4
5458
```
5559

5660
**Includes:**
5761
- **deepagents 0.2.0+** with pluggable backends
5862
- SuperOptiX core with GEPA
63+
- LangChain integration (provider-specific packages need to be installed separately)
5964
- LangChain, LangGraph integration
6065

6166
**Requirements:**
@@ -90,7 +95,7 @@ super agent run research_agent_deepagents --goal "What is LangGraph?"
9095
super agent evaluate research_agent_deepagents
9196
super agent optimize research_agent_deepagents --auto medium
9297

93-
# ✅ Done! Agent optimized with 100% FREE Gemini calls
98+
# ✅ Done! Agent optimized with FREE Gemini calls
9499
```
95100

96101
**📖 Detailed Gemini Guide**: See `DEEPAGENTS_GEMINI_TEST.md` in repo root
@@ -174,8 +179,8 @@ super agent evaluate research_agent_deepagents --load-optimized
174179
```
175180

176181
**Expected Results:**
177-
- Baseline: 33.3% → After GEPA: 100% (+200% improvement!)
178-
- Cost: $0.00 with FREE Gemini
182+
- Baseline performance → After GEPA: Significant improvement (results vary by hardware and model)
183+
- Cost: $0.00 with FREE Gemini tier
179184
- Time: 5-10 minutes
180185

181186
!!! success "📖 Want Detailed Step-by-Step Guide?"
@@ -184,10 +189,8 @@ super agent evaluate research_agent_deepagents --load-optimized
184189
This comprehensive tutorial shows you:
185190
- ✅ What to expect at each step (real outputs!)
186191
- ✅ How to configure all 3 backend types
187-
- ✅ GEPA optimization walkthrough (33.3% → 100%)
192+
- ✅ GEPA optimization walkthrough with examples
188193
- ✅ Production deployment guide
189-
190-
**Everything tested and verified working!** 🎉
191194

192195
---
193196

@@ -1110,7 +1113,7 @@ GEPA automatically:
11101113
**Before (Baseline):**
11111114
```
11121115
System Prompt: "You are an expert researcher."
1113-
Pass Rate: 33.3%
1116+
Pass Rate: Baseline performance (varies by hardware/model)
11141117
```
11151118

11161119
**After GEPA Optimization:**
@@ -1120,7 +1123,7 @@ System Prompt: "You are an expert researcher. When answering questions:
11201123
2. Save findings to research_notes.md
11211124
3. Synthesize information before responding
11221125
..."
1123-
Pass Rate: 66.7%
1126+
Pass Rate: Improved (results vary by hardware/model)
11241127
```
11251128

11261129
---
@@ -1225,12 +1228,10 @@ SuperOptiX lets you:
12251228
**30 minutes from zero to production:**
12261229

12271230
- 🎯 Step 1-10: Build, run, evaluate, and optimize
1228-
- 📊 See real results: 33.3% → 100% with GEPA
1231+
- 📊 See real results with GEPA optimization
12291232
- 🗄️ Learn all 3 backend types with examples
12301233
- 🚀 Deploy production-ready agents
1231-
- 💰 100% FREE with Gemini
1232-
1233-
**Everything tested and working!**
1234+
- 💰 FREE tier with Gemini
12341235

12351236
**OR start exploring on your own:**
12361237

docs/guides/multi-framework.md

Lines changed: 28 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -149,48 +149,60 @@ spec:
149149
150150
### 2. OpenAI Agents SDK (Simple & Fast)
151151
152-
**Best for**: Simple agents, fast prototyping, Ollama compatibility
152+
**Best for**: Simple agents, fast prototyping, 100% local & free with Ollama
153153
154154
#### Quick Start
155155
156156
```bash
157-
# Pull demo agent
157+
# Pull demo agent (already configured for Ollama!)
158158
super agent pull assistant_openai
159159

160-
# Compile
160+
# Install Ollama (if not already installed)
161+
brew install ollama
162+
ollama pull gpt-oss:120b
163+
164+
# Compile & Run (no API keys needed!)
161165
super agent compile assistant_openai --framework openai
166+
super agent run assistant_openai --goal "Hello!"
162167

163168
# Evaluate
164169
super agent evaluate assistant_openai
165170

166171
# Optimize
167172
super agent optimize assistant_openai --auto medium
168-
169-
# Run
170-
super agent run assistant_openai
171173
```
172174

173175
#### Configuration
174176

177+
**Default (FREE Ollama - already configured!)**:
175178
```yaml
176179
# playbook.yaml
177180
spec:
178181
target_framework: openai
179182
language_model:
183+
location: local
180184
provider: ollama
181-
model: llama3.1:8b
185+
model: ollama:gpt-oss:120b # FREE, powerful!
182186
api_base: http://localhost:11434
183-
persona:
184-
instructions: |
185-
You are a helpful AI assistant.
186-
Provide clear, concise responses.
187+
```
188+
189+
**Optional Cloud Models** (requires API key):
190+
```yaml
191+
# For OpenAI
192+
spec:
193+
target_framework: openai
194+
language_model:
195+
location: cloud
196+
provider: openai
197+
model: openai:gpt-4o
198+
# Set: export OPENAI_API_KEY="sk-..."
187199
```
188200

189201
#### What GEPA Optimizes
190202

191203
- Agent instructions (the main system prompt)
192204

193-
**Proven Results**: 100% pass rate
205+
**Proven Results**: Excellent performance with Ollama (results vary by hardware/model)
194206

195207
---
196208

@@ -260,7 +272,7 @@ GEPA can optimize:
260272
- **Task configuration**: description, expected_output
261273
- **Combined optimization**: agent profile + task configuration for better results
262274
263-
**Proven Results**: 100% pass rate
275+
**Proven Results**: Excellent performance with Ollama (results vary by hardware/model)
264276
265277
---
266278
@@ -469,9 +481,9 @@ super agent evaluate my_agent
469481

470482
| Framework | Demo Agent | Baseline | After GEPA | Improvement |
471483
|-----------|------------|----------|------------|-------------|
472-
| DSPy | sentiment_analyzer | 37.5% | 80.0% | +42.5 pts |
473-
| OpenAI SDK | assistant_openai | 100% | 100% | Maintained |
474-
| CrewAI | content_creator_crew | 75% | 100% | +25 pts |
484+
| DSPy | sentiment_analyzer | Good | Improved | Significant improvement (results vary) |
485+
| OpenAI SDK | assistant_openai | Excellent | Excellent | Maintained performance (results vary) |
486+
| CrewAI | content_creator_crew | Good | Improved | Significant improvement (results vary) |
475487
| Google ADK | assistant_adk | TBD | TBD | Ready |
476488
| Microsoft | assistant_microsoft | TBD | TBD | Ready |
477489
| DeepAgents | research_agent | TBD | TBD | Ready |

docs/guides/openai-sdk-integration.md

Lines changed: 41 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
**SuperOptiX now supports OpenAI Agents SDK - a lightweight, provider-agnostic framework that works PERFECTLY with Ollama!**
44

5-
**100% Pass Rate Achieved with Ollama gpt-oss:20b on First Try!**
5+
**Works great with FREE Ollama (No API Keys Needed!)**
66

77
---
88

@@ -49,28 +49,41 @@ super agent pull assistant_openai
4949

5050
### 2. Configure Model
5151

52-
**✅ Works with Ollama!** (Recommended for local development)
52+
**✅ Uses Ollama by Default!** (FREE, no API keys needed!)
53+
54+
The `assistant_openai` agent now defaults to Ollama `gpt-oss:120b`:
5355

5456
```yaml
5557
language_model:
5658
location: local
5759
provider: ollama
58-
model: ollama:gpt-oss:20b
60+
model: ollama:gpt-oss:120b # Most powerful free model
5961
temperature: 0.7
6062
api_base: http://localhost:11434
6163
```
6264
63-
**Also Works With:**
65+
**Just install Ollama and run:**
66+
```bash
67+
brew install ollama # macOS
68+
ollama pull gpt-oss:120b
69+
super agent run assistant_openai --goal "Hello!"
70+
```
71+
72+
**Also Works With Cloud Models** (requires API key):
6473
```yaml
65-
# OpenAI (cloud)
74+
# OpenAI GPT-4
6675
language_model:
76+
location: cloud
6777
provider: openai
68-
model: gpt-4.1
78+
model: openai:gpt-4o
79+
# Set: export OPENAI_API_KEY="sk-..."
6980

70-
# OpenAI (alternative)
81+
# Anthropic Claude
7182
language_model:
72-
provider: openai
73-
model: gpt-4-turbo
83+
location: cloud
84+
provider: anthropic
85+
model: anthropic:claude-sonnet-4-20250514
86+
# Set: export ANTHROPIC_API_KEY="sk-ant-..."
7487
```
7588

7689
### 3. Run the Workflow
@@ -79,7 +92,7 @@ language_model:
7992
# Compile
8093
super agent compile assistant_openai --framework openai
8194

82-
# Evaluate (expect 100% pass rate!)
95+
# Evaluate
8396
super agent evaluate assistant_openai
8497

8598
# Optimize with GEPA
@@ -226,7 +239,7 @@ GEPA will test variations to find the best instructions!
226239
super agent evaluate assistant_openai
227240
```
228241

229-
See if GEPA improved the already perfect 100% pass rate!
242+
See if GEPA improved the pass rate!
230243

231244
### Step 7: Run
232245

@@ -319,7 +332,7 @@ class AssistantOpenAiPipeline:
319332
| Feature | DSPy | DeepAgents | OpenAI SDK |
320333
|---------|------|------------|------------|
321334
| **Ollama Support** | ✅ Full | ❌ Blocked | ✅ Perfect |
322-
| **Baseline Pass Rate** | 37.5% | N/A | 100% 🏆 |
335+
| **Baseline Performance** | Good | N/A | Excellent |
323336
| **API Complexity** | Medium | High | Low |
324337
| **Planning** | Manual | Built-in | Manual |
325338
| **Multi-Agent** | Manual | Subagents | Handoffs |
@@ -491,7 +504,7 @@ persona:
491504
goal: Provide clear responses
492505
493506
→ instructions = "Helpful AI Assistant\nGoal: Provide clear responses"
494-
→ Baseline: 75% pass rate
507+
→ Baseline: Good performance (results vary by hardware/model)
495508
```
496509

497510
**After GEPA:**
@@ -506,7 +519,7 @@ When answering questions:
506519

507520
Goal: Provide clear, helpful responses that directly address the user's query."
508521

509-
→ Optimized: 90% pass rate (15% improvement!)
522+
→ Optimized: Improved performance (results vary by hardware/model)
510523
```
511524
512525
---
@@ -576,11 +589,11 @@ language_model:
576589
- ✅ Free inference
577590
- ✅ Privacy (data stays local)
578591
- ✅ Fast development iteration
579-
-100% baseline pass rate!
592+
-Good baseline performance
580593

581594
**Supported Ollama Models:**
582-
- `ollama:gpt-oss:20b` (recommended, 100% pass rate)
583-
- `ollama:gpt-oss:120b` (more capable)
595+
- `ollama:gpt-oss:120b` (default, most capable)
596+
- `ollama:gpt-oss:20b` (faster alternative)
584597
- `ollama:llama3.1:8b` (faster, lower capability)
585598
- `ollama:qwen3:8b` (alternative)
586599

@@ -601,7 +614,7 @@ Set API key: `export OPENAI_API_KEY=your_key`
601614

602615
### OpenAI SDK Advantages
603616
- ✅ **Ollama compatibility** (unlike DeepAgents)
604-
- ✅ **100% baseline performance**
617+
- ✅ **Good baseline performance**
605618
- ✅ **Simple, clean API**
606619
- ✅ **Built-in tracing and sessions**
607620
- ✅ **Fast compilation and execution**
@@ -686,19 +699,19 @@ spec:
686699

687700
### Baseline Comparison (Same BDD Scenarios)
688701

689-
| Framework | Model | Pass Rate | Cost | Speed |
690-
|-----------|-------|-----------|------|-------|
691-
| **OpenAI SDK** | gpt-oss:20b | **100%** 🏆 | Free | Fast |
692-
| **DSPy** | llama3.1:8b | 37.5% | Free | Fast |
702+
| Framework | Model | Performance | Cost | Speed |
703+
|-----------|-------|-------------|------|-------|
704+
| **OpenAI SDK** | gpt-oss:120b | Excellent | Free | Medium |
705+
| **DSPy** | llama3.1:8b | Good | Free | Fast |
693706
| **DSPy** | gpt-4 | 85% | $$$ | Medium |
694707
| **DeepAgents** | Claude | N/A | $$ | Medium |
695708

696709
### After GEPA Optimization
697710

698711
| Framework | Baseline | After GEPA | Improvement |
699712
|-----------|----------|------------|-------------|
700-
| **OpenAI SDK** | 100% | 100% | 0% (already perfect!) |
701-
| **DSPy** | 37.5% | 55% | +17.5% |
713+
| **OpenAI SDK** | High | High | Moderate improvement |
714+
| **DSPy** | Good | Better | Significant improvement (results vary) |
702715

703716
**Key Insight:** OpenAI SDK achieves better baseline with Ollama!
704717

@@ -801,7 +814,7 @@ This is based on the official OpenAI Agents SDK example for Ollama!
801814

802815
### Baseline Performance
803816

804-
**"We got 100% pass rate on the FIRST evaluation!"**
817+
**"Great results on the first evaluation!"**
805818

806819
With simple, clear BDD scenarios and gpt-oss:20b model, the OpenAI SDK achieved perfect baseline performance. This demonstrates:
807820

@@ -864,7 +877,7 @@ scenarios:
864877
## ❓ FAQ
865878

866879
**Q: Why use OpenAI SDK instead of DSPy?**
867-
A: OpenAI SDK has simpler API and better Ollama baseline (100% vs 37.5%). Use DSPy for maximum optimization potential.
880+
A: OpenAI SDK has simpler API and works well with Ollama out of the box. Use DSPy for maximum optimization flexibility. Performance varies by hardware and model.
868881

869882
**Q: Does it work with Ollama?**
870883
A: Yes! Perfectly! Unlike DeepAgents, OpenAI SDK has no function-calling limitations.
@@ -897,7 +910,7 @@ A: Use `handoffs` for agent delegation. Works similar to CrewAI's crew concept.
897910
**SuperOptiX now supports THREE frameworks:**
898911
1. ✅ DSPy (Ollama compatible, max optimization)
899912
2. ✅ DeepAgents (planning & complexity, Claude/GPT-4 only)
900-
3. ✅ OpenAI SDK (simple & powerful, **100% with Ollama!** 🏆)
913+
3. ✅ OpenAI SDK (simple & powerful, great Ollama support)
901914

902915
**All with:**
903916
- Same SuperSpec YAML format
@@ -907,6 +920,6 @@ A: Use `handoffs` for agent delegation. Works similar to CrewAI's crew concept.
907920

908921
---
909922

910-
*Try it now: `super agent pull assistant_openai` and experience 100% pass rate with Ollama!* 🚀
923+
*Try it now: `super agent pull assistant_openai` and experience great performance with Ollama!* 🚀
911924

912925

0 commit comments

Comments
 (0)