Ollama GEPA issue

Author: Shashikant86

CHANGELOG.md

@@ -8,14 +8,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ## [Unreleased]
 
 ### Added
-- Initial release of CodeOptiX
-- GEPA optimization engine
-- Bloom evaluation framework
-- Built-in behaviors: insecure-code, vacuous-tests, plan-drift
-- Support for multiple coding agents (Claude Code, Codex, Gemini CLI)
-- Multi-provider LLM support (OpenAI, Anthropic, Google, Ollama)
-- CI/CD integration
-- Comprehensive documentation and examples
 
 ### Changed
 
@@ -27,6 +19,27 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Security
 
+## [0.1.3] - 2025-12-27
+
+### Added
+- Ollama integration demo script (`examples/ollama_demo.py`) showcasing working local evaluations
+- Updated documentation highlighting Ollama integration fixes
+
+### Fixed
+- **Ollama Integration**: Fixed Ollama models to properly generate code and provide meaningful evaluation scores instead of always returning 100%. Now uses Ollama's chat API for better conversation handling and includes working demo script.
+
+## [0.1.2] - 2025-12-26
+
+### Added
+- Initial release of CodeOptiX
+- GEPA optimization engine
+- Bloom evaluation framework
+- Built-in behaviors: insecure-code, vacuous-tests, plan-drift
+- Support for multiple coding agents (Claude Code, Codex, Gemini CLI)
+- Multi-provider LLM support (OpenAI, Anthropic, Google, Ollama)
+- CI/CD integration
+- Comprehensive documentation and examples
+
 ## [0.1.0] - 2025-12-26
 
 ### Added

docs/guides/ollama-integration.md

@@ -1,6 +1,6 @@
 # Ollama Integration Guide
 
-CodeOptiX supports local Ollama models, allowing you to run evaluations without API keys!
+CodeOptiX supports local Ollama models, allowing you to run evaluations without API keys! ✅ **Now working correctly** - generates code and provides proper security evaluations.
 
 ---
 
@@ -14,6 +14,27 @@ CodeOptiX supports local Ollama models, allowing you to run evaluations without
 
 ---
 
+## ✅ Recent Updates
+
+**CodeOptiX now works correctly with Ollama!** Recent fixes ensure:
+
+- ✅ Proper code generation (not conversational responses)
+- ✅ Accurate security evaluations (detects real issues)
+- ✅ Meaningful scores (not always 100%)
+- ✅ Full evaluation pipeline support
+
+### 🚀 Try the Demo
+
+Test the Ollama integration with our interactive demo:
+
+```bash
+python examples/ollama_demo.py
+```
+
+This demo shows Ollama generating code, detecting security issues, and providing proper evaluation scores.
+
+---
+
 ## 📦 Installation
 
 ### Step 1: Install Ollama
@@ -140,11 +161,11 @@ export OLLAMA_BASE_URL=http://remote-server:11434
 
 | Model | Size | Speed | Quality | Use Case |
 |-------|------|-------|---------|----------|
-| `llama3.1:8b` | 4.9 GB | ⚡⚡⚡ | ⭐⭐⭐ | Fast, efficient |
-| `qwen3:8b` | 5.2 GB | ⚡⚡⚡ | ⭐⭐⭐ | Alternative 8B |
-| `gpt-oss:120b` | 65 GB | ⚡ | ⭐⭐⭐⭐⭐ | Best quality |
-| `gpt-oss:20b` | 13 GB | ⚡⚡ | ⭐⭐⭐⭐ | Good balance |
-| `llama3.2:3b` | 2.0 GB | ⚡⚡⚡⚡ | ⭐⭐ | Lightweight |
+| `llama3.2:3b` | 2.0 GB | ⚡⚡⚡⚡ | ⭐⭐⭐⭐ | **Best for CodeOptiX** - Fast, reliable code generation |
+| `llama3.1:8b` | 4.9 GB | ⚡⚡⚡ | ⭐⭐⭐ | Good balance, works well |
+| `qwen3:8b` | 5.2 GB | ⚡⚡⚡ | ⭐⭐⭐ | Alternative 8B model |
+| `gpt-oss:20b` | 13 GB | ⚡⚡ | ⭐⭐⭐⭐ | High quality, slower |
+| `gpt-oss:120b` | 65 GB | ⚡ | ⭐⭐⭐⭐⭐ | Best quality, requires powerful hardware |
 
 ### List Available Models
 
@@ -162,7 +183,17 @@ ollama pull <model-name>
 
 ## 💡 Usage Examples
 
-### Example 1: Basic Evaluation
+### Example 1: Try the Interactive Demo ⭐
+
+See Ollama working in action:
+
+```bash
+python examples/ollama_demo.py
+```
+
+This demo shows code generation, security evaluation, and proper scoring.
+
+### Example 2: Basic Evaluation
 
 ```bash
 codeoptix eval \
@@ -171,7 +202,7 @@ codeoptix eval \
   --llm-provider ollama
 ```
 
-### Example 2: With Custom Config
+### Example 3: With Custom Config
 
 ```bash
 codeoptix eval \
@@ -181,7 +212,7 @@ codeoptix eval \
   --llm-provider ollama
 ```
 
-### Example 3: Multiple Behaviors
+### Example 4: Multiple Behaviors
 
 ```bash
 codeoptix eval \
@@ -190,7 +221,7 @@ codeoptix eval \
   --llm-provider ollama
 ```
 
-### Example 4: Verbose Output
+### Example 5: Verbose Output
 
 ```bash
 codeoptix eval \
@@ -200,7 +231,7 @@ codeoptix eval \
   --verbose
 ```
 
-### Example 5: CI/CD Integration
+### Example 6: CI/CD Integration
 
 ```yaml
 # .github/workflows/codeoptix.yml
@@ -300,17 +331,15 @@ export OLLAMA_BASE_URL=http://localhost:11435
 - You need maximum speed
 - You're okay with API costs
 
-### ⚠️ Limitations
-
-While Ollama works great for evaluations, there are some limitations:
+### ⚠️ Known Limitations
 
 #### Evolution Support
-- **Limited support for `codeoptix evolve`**: The evolution feature uses GEPA optimization, which requires processing very long prompts. Ollama may fail with 404 errors or timeouts on complex evolution tasks.
-- **Recommendation**: Use cloud providers (OpenAI, Anthropic, Google) for full evolution capabilities. For basic evolution testing, try smaller models like `llama3.1:8b` with minimal iterations.
+- **Limited support for `codeoptix evolve`**: The evolution feature uses GEPA optimization, which requires processing very long prompts. Ollama may fail with timeouts on complex evolution tasks.
+- **Recommendation**: Use cloud providers (OpenAI, Anthropic, Google) for full evolution capabilities.
 
-#### Performance
-- Large models (e.g., `gpt-oss:120b`) require significant RAM and may be slow on consumer hardware.
-- Evolution tasks are computationally intensive and may not complete reliably with Ollama.
+#### Performance Considerations
+- Large models (e.g., `gpt-oss:120b`) require significant RAM and may be slow on consumer hardware
+- Evolution tasks are computationally intensive and may not complete reliably with Ollama
 
 For advanced features like evolution, consider cloud providers or contact us for tailored enterprise solutions.
 

docs/index.md

@@ -56,7 +56,7 @@ When AI coding agents dazzle with impressive code but leave you wondering about
 !!! tip "Ollama Support - No API Key Required!"
     **CodeOptiX supports Ollama** for evaluations - use local models without API keys:
 
-    - ✅ **Ollama integration** - Run evaluations with local models
+    - ✅ **Working Ollama integration** - Generates code and provides proper security evaluations
     - ✅ **No API key needed** - Perfect for open-source users
     - ✅ **Privacy-friendly** - All processing happens locally
     - ✅ **Free to use** - No cloud costs

examples/README.md

@@ -31,7 +31,23 @@ python examples/basic_adapter_usage.py
 - Executing tasks with different agents
 - Handling agent outputs
 
-### 3. Behavioral Spec Example (`behavioral_spec_example.py`) ⭐
+### 3. Ollama Local Demo (`ollama_demo.py`) ⭐
+
+**Local Ollama integration demo** showing that CodeOptiX now works correctly with Ollama.
+
+```bash
+python examples/ollama_demo.py
+```
+
+**What it shows:**
+- Ollama code generation working properly
+- Security evaluation detecting real issues
+- Proper scoring (not always 100%)
+- Local, privacy-friendly evaluations
+
+This is the **recommended starting point** for users who want to use CodeOptiX locally with Ollama.
+
+### 4. Behavioral Spec Example (`behavioral_spec_example.py`)
 
 **Complete end-to-end example** demonstrating a real-world behavioral spec scenario.
 
@@ -45,7 +61,7 @@ python examples/behavioral_spec_example.py
 - Real scenario: Database connection with secret management
 - Complete workflow from agent execution to prompt evolution
 
-This is the **recommended starting point** for understanding how CodeOptiX works in practice.
+This is the **recommended starting point** for understanding how CodeOptiX works in practice with cloud providers.
 
 ## Behavioral Spec Scenarios
 
@@ -89,23 +105,35 @@ pip install -e ".[dev,docs]"
 uv sync --dev --extra docs
 ```
 
-2. Set API keys (at least one):
-```bash
-export OPENAI_API_KEY="your-key"
-export ANTHROPIC_API_KEY="your-key"
-export GOOGLE_API_KEY="your-key"
-```
+2. Choose your LLM provider:
+
+   **For local Ollama usage:**
+   ```bash
+   # Install Ollama: https://ollama.com
+   ollama serve  # Start Ollama server
+   ollama pull llama3.2:3b  # Pull a model
+   ```
+
+   **For cloud providers (set at least one API key):**
+   ```bash
+   export OPENAI_API_KEY="your-key"
+   export ANTHROPIC_API_KEY="your-key"
+   export GOOGLE_API_KEY="your-key"
+   ```
 
 ### Run Examples
 
 ```bash
+# Ollama local demo (recommended for local usage)
+python examples/ollama_demo.py
+
 # Quick start with single behavior
 python examples/quickstart-single-behavior.py
 
 # Basic adapter usage
 python examples/basic_adapter_usage.py
 
-# Complete behavioral spec example (recommended)
+# Complete behavioral spec example (recommended for cloud providers)
 python examples/behavioral_spec_example.py
 ```