Skip to content

Commit 287952c

Browse files
committed
Add artifact side-channel with prompt integration
1 parent 91cc7e2 commit 287952c

File tree

16 files changed

+2238
-43
lines changed

16 files changed

+2238
-43
lines changed

README.md

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -157,6 +157,67 @@ Sample configuration files are available in the `configs/` directory:
157157

158158
See the [Configuration Guide](configs/default_config.yaml) for a full list of options.
159159

160+
## Artifacts Channel
161+
162+
OpenEvolve includes a **artifacts side-channel** that allows evaluators to capture build errors, profiling results, etc. to provide better feedback to the LLM in subsequent generations. This feature enhances the evolution process by giving the LLM context about what went wrong and how to fix it.
163+
164+
The artifacts channel operates alongside the traditional fitness metrics.
165+
166+
### Example: Compilation Failure Feedback
167+
168+
```python
169+
from openevolve.evaluation_result import EvaluationResult
170+
171+
return EvaluationResult(
172+
metrics={"compile_ok": 0.0, "score": 0.0},
173+
artifacts={
174+
"stderr": "SyntaxError: invalid syntax (line 15)",
175+
"traceback": "...",
176+
"failure_stage": "compilation"
177+
}
178+
)
179+
```
180+
181+
The next generation prompt will include:
182+
```
183+
## Last Execution Output
184+
### Stderr
185+
```
186+
SyntaxError: invalid syntax (line 15)
187+
```
188+
### Traceback
189+
```
190+
...
191+
```
192+
```
193+
194+
### Configuration
195+
196+
Artifacts can be controlled via configuration and environment variables:
197+
198+
```yaml
199+
# config.yaml
200+
evaluator:
201+
enable_artifacts: true
202+
203+
prompt:
204+
include_artifacts: true
205+
max_artifact_bytes: 4096 # 4KB limit in prompts
206+
artifact_security_filter: true
207+
```
208+
209+
```bash
210+
# Environment variable to disable artifacts
211+
export ENABLE_ARTIFACTS=false
212+
```
213+
214+
### Benefits
215+
216+
- **Faster convergence** - LLMs can see what went wrong and fix it directly
217+
- **Better error handling** - Compilation and runtime failures become learning opportunities
218+
- **Rich debugging context** - Full stack traces and error messages guide improvements
219+
- **Zero overhead** - When disabled, no performance impact on evaluation
220+
160221
## Examples
161222

162223
See the `examples/` directory for complete examples of using OpenEvolve on various problems:

0 commit comments

Comments
 (0)