Skip to content

Commit c779ac9

Browse files
authored
Merge pull request #52 from nerdsane/feat-artifact-side-channel
Feature: Artifact side channel
2 parents 970d079 + 72016e2 commit c779ac9

File tree

16 files changed

+2272
-43
lines changed

16 files changed

+2272
-43
lines changed

README.md

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -183,6 +183,67 @@ Sample configuration files are available in the `configs/` directory:
183183

184184
See the [Configuration Guide](configs/default_config.yaml) for a full list of options.
185185

186+
## Artifacts Channel
187+
188+
OpenEvolve includes a **artifacts side-channel** that allows evaluators to capture build errors, profiling results, etc. to provide better feedback to the LLM in subsequent generations. This feature enhances the evolution process by giving the LLM context about what went wrong and how to fix it.
189+
190+
The artifacts channel operates alongside the traditional fitness metrics.
191+
192+
### Example: Compilation Failure Feedback
193+
194+
```python
195+
from openevolve.evaluation_result import EvaluationResult
196+
197+
return EvaluationResult(
198+
metrics={"compile_ok": 0.0, "score": 0.0},
199+
artifacts={
200+
"stderr": "SyntaxError: invalid syntax (line 15)",
201+
"traceback": "...",
202+
"failure_stage": "compilation"
203+
}
204+
)
205+
```
206+
207+
The next generation prompt will include:
208+
```
209+
## Last Execution Output
210+
### Stderr
211+
```
212+
SyntaxError: invalid syntax (line 15)
213+
```
214+
### Traceback
215+
```
216+
...
217+
```
218+
```
219+
220+
### Configuration
221+
222+
Artifacts can be controlled via configuration and environment variables:
223+
224+
```yaml
225+
# config.yaml
226+
evaluator:
227+
enable_artifacts: true
228+
229+
prompt:
230+
include_artifacts: true
231+
max_artifact_bytes: 4096 # 4KB limit in prompts
232+
artifact_security_filter: true
233+
```
234+
235+
```bash
236+
# Environment variable to disable artifacts
237+
export ENABLE_ARTIFACTS=false
238+
```
239+
240+
### Benefits
241+
242+
- **Faster convergence** - LLMs can see what went wrong and fix it directly
243+
- **Better error handling** - Compilation and runtime failures become learning opportunities
244+
- **Rich debugging context** - Full stack traces and error messages guide improvements
245+
- **Zero overhead** - When disabled, no performance impact on evaluation
246+
186247
## Examples
187248

188249
See the `examples/` directory for complete examples of using OpenEvolve on various problems:

0 commit comments

Comments
 (0)