Skip to content

Commit fb5e654

Browse files
committed
update tests and controller
1 parent 286a70a commit fb5e654

File tree

5 files changed

+46
-17
lines changed

5 files changed

+46
-17
lines changed

README.md

Lines changed: 15 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ An open-source implementation of the AlphaEvolve system described in the Google
99
OpenEvolve is an evolutionary coding agent that uses Large Language Models to optimize code through an iterative process. It orchestrates a pipeline of LLM-based code generation, evaluation, and selection to continuously improve programs for a variety of tasks.
1010

1111
Key features:
12+
1213
- Evolution of entire code files, not just single functions
1314
- Support for multiple programming languages
1415
- Supports OpenAI-compatible APIs for any LLM
@@ -34,6 +35,7 @@ The controller orchestrates interactions between these components in an asynchro
3435
### Installation
3536

3637
To install natively, use:
38+
3739
```bash
3840
git clone https://github.com/codelion/openevolve.git
3941
cd openevolve
@@ -51,7 +53,7 @@ from openevolve import OpenEvolve
5153

5254
# Initialize the system
5355
evolve = OpenEvolve(
54-
initial_program_path="path/to/initial_program.py",
56+
initial_programs_paths=["path/to/initial_program.py"],
5557
evaluation_file="path/to/evaluator.py",
5658
config_path="path/to/config.yaml"
5759
)
@@ -83,6 +85,7 @@ python openevolve-run.py path/to/initial_program.py path/to/evaluator.py \
8385
```
8486

8587
When resuming from a checkpoint:
88+
8689
- The system loads all previously evolved programs and their metrics
8790
- Checkpoint numbering continues from where it left off (e.g., if loaded from checkpoint_50, the next checkpoint will be checkpoint_60)
8891
- All evolution state is preserved (best programs, feature maps, archives, etc.)
@@ -145,6 +148,7 @@ python scripts/visualizer.py --path examples/function_minimization/openevolve_ou
145148
```
146149

147150
In the visualization UI, you can
151+
148152
- see the branching of your program evolution in a network visualization, with node radius chosen by the program fitness (= the currently selected metric),
149153
- see the parent-child relationship of nodes and click through them in the sidebar (use the yellow locator icon in the sidebar to center the node in the graph),
150154
- select the metric of interest (with the available metric choices depending on your data set),
@@ -157,6 +161,7 @@ In the visualization UI, you can
157161
### Docker
158162

159163
You can also install and execute via Docker:
164+
160165
```bash
161166
docker build -t openevolve .
162167
docker run --rm -v $(pwd):/app --network="host" openevolve examples/function_minimization/initial_program.py examples/function_minimization/evaluator.py --config examples/function_minimization/config.yaml --iterations 1000
@@ -179,6 +184,7 @@ database:
179184
```
180185
181186
Sample configuration files are available in the `configs/` directory:
187+
182188
- `default_config.yaml`: Comprehensive configuration with all available options
183189

184190
See the [Configuration Guide](configs/default_config.yaml) for a full list of options.
@@ -205,18 +211,23 @@ return EvaluationResult(
205211
```
206212

207213
The next generation prompt will include:
214+
208215
```markdown
209216
## Last Execution Output
217+
210218
### Stderr
219+
211220
SyntaxError: invalid syntax (line 15)
212221
213222
### Traceback
223+
214224
...
215225
```
216226

217227
## Example: LLM Feedback
218228

219229
An example for an LLM artifact side channel is part of the default evaluation template, which ends with
230+
220231
```markdown
221232
Return your evaluation as a JSON object with the following format:
222233
{{
@@ -226,6 +237,7 @@ Return your evaluation as a JSON object with the following format:
226237
"reasoning": "[brief explanation of scores]"
227238
}}
228239
```
240+
229241
The non-float values, in this case the "reasoning" key of the json response that the evaluator LLM generates, will be available within the next generation prompt.
230242

231243
### Configuration
@@ -239,7 +251,7 @@ evaluator:
239251
240252
prompt:
241253
include_artifacts: true
242-
max_artifact_bytes: 4096 # 4KB limit in prompts
254+
max_artifact_bytes: 4096 # 4KB limit in prompts
243255
artifact_security_filter: true
244256
```
245257

@@ -266,6 +278,7 @@ A comprehensive example demonstrating OpenEvolve's application to symbolic regre
266278
[Explore the Symbolic Regression Example](examples/symbolic_regression/)
267279

268280
Key features:
281+
269282
- Automatic generation of initial programs from benchmark tasks
270283
- Evolution from simple linear models to complex mathematical expressions
271284
- Evaluation on physics, chemistry, biology, and material science datasets

configs/README.md

Lines changed: 12 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -5,23 +5,29 @@ This directory contains configuration files for OpenEvolve with examples for dif
55
## Configuration Files
66

77
### `default_config.yaml`
8+
89
The main configuration file containing all available options with sensible defaults. This file includes:
10+
911
- Complete documentation for all configuration parameters
1012
- Default values for all settings
1113
- **Island-based evolution parameters** for proper evolutionary diversity
1214

1315
Use this file as a template for your own configurations.
1416

1517
### `island_config_example.yaml`
18+
1619
A practical example configuration demonstrating proper island-based evolution setup. Shows:
20+
1721
- Recommended island settings for most use cases
1822
- Balanced migration parameters
1923
- Complete working configuration
2024

2125
### `island_examples.yaml`
26+
2227
Multiple example configurations for different scenarios:
28+
2329
- **Maximum Diversity**: Many islands, frequent migration
24-
- **Focused Exploration**: Few islands, rare migration
30+
- **Focused Exploration**: Few islands, rare migration
2531
- **Balanced Approach**: Default recommended settings
2632
- **Quick Exploration**: Small-scale rapid testing
2733
- **Large-Scale Evolution**: Complex optimization runs
@@ -34,9 +40,9 @@ The key new parameters for proper evolutionary diversity are:
3440

3541
```yaml
3642
database:
37-
num_islands: 5 # Number of separate populations
38-
migration_interval: 50 # Migrate every N generations
39-
migration_rate: 0.1 # Fraction of top programs to migrate
43+
num_islands: 5 # Number of separate populations
44+
migration_interval: 50 # Migrate every N generations
45+
migration_rate: 0.1 # Fraction of top programs to migrate
4046
```
4147
4248
### Parameter Guidelines
@@ -66,8 +72,8 @@ Then use with OpenEvolve:
6672
```python
6773
from openevolve import OpenEvolve
6874
evolve = OpenEvolve(
69-
initial_program_path="program.py",
70-
evaluation_file="evaluator.py",
75+
initial_program_paths=["program.py"],
76+
evaluation_file="evaluator.py",
7177
config_path="my_config.yaml"
7278
)
7379
```

openevolve/cli.py

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,11 +19,15 @@ def parse_args() -> argparse.Namespace:
1919
"""Parse command-line arguments"""
2020
parser = argparse.ArgumentParser(description="OpenEvolve - Evolutionary coding agent")
2121

22-
parser.add_argument("initial_program", help="Path to the initial program file", default=None)
23-
2422
parser.add_argument(
2523
"evaluation_file", help="Path to the evaluation file containing an 'evaluate' function"
2624
)
25+
parser.add_argument(
26+
"initial_program",
27+
nargs="?",
28+
help="Path to the initial program file",
29+
default=None,
30+
)
2731

2832
parser.add_argument("--config", "-c", help="Path to configuration file (YAML)", default=None)
2933

openevolve/controller.py

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -250,7 +250,13 @@ async def run(
250250

251251
if should_add_initial:
252252
logger.info("Adding initial programs to database")
253-
for code in self.initial_programs_code:
253+
254+
if len(self.initial_programs_code) > len(self.database.islands):
255+
raise ValueError(
256+
"Number of initial programs exceeds number of islands."
257+
)
258+
259+
for i, code in enumerate(self.initial_programs_code):
254260
initial_program_id = str(uuid.uuid4())
255261

256262
# Evaluate the initial program
@@ -267,7 +273,7 @@ async def run(
267273
)
268274

269275
# TODO. Should the island be incremented and reset here?
270-
self.database.add(initial_program)
276+
self.database.add(initial_program, 0, i)
271277
else:
272278
logger.info(
273279
f"Skipping initial program addition (resuming from iteration {start_iteration} "

tests/test_checkpoint_resume.py

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -86,7 +86,7 @@ async def run_test():
8686
mock_evaluator_class.return_value = mock_evaluator
8787

8888
controller = OpenEvolve(
89-
initial_program_path=self.test_program_path,
89+
initial_programs_paths=[self.test_program_path],
9090
evaluation_file=self.evaluator_path,
9191
config=self.config,
9292
output_dir=self.test_dir,
@@ -127,7 +127,7 @@ async def run_test():
127127
mock_evaluator_class.return_value = mock_evaluator
128128

129129
controller = OpenEvolve(
130-
initial_program_path=self.test_program_path,
130+
initial_programs_paths=[self.test_program_path],
131131
evaluation_file=self.evaluator_path,
132132
config=self.config,
133133
output_dir=self.test_dir,
@@ -169,7 +169,7 @@ async def run_test():
169169
mock_evaluator_class.return_value = mock_evaluator
170170

171171
controller = OpenEvolve(
172-
initial_program_path=self.test_program_path,
172+
initial_programs_paths=[self.test_program_path],
173173
evaluation_file=self.evaluator_path,
174174
config=self.config,
175175
output_dir=self.test_dir,
@@ -219,7 +219,7 @@ async def run_test():
219219
mock_evaluator_class.return_value = mock_evaluator
220220

221221
controller = OpenEvolve(
222-
initial_program_path=self.test_program_path,
222+
initial_programs_paths=[self.test_program_path],
223223
evaluation_file=self.evaluator_path,
224224
config=self.config,
225225
output_dir=self.test_dir,
@@ -269,7 +269,7 @@ async def run_test():
269269
mock_evaluator_class.return_value = mock_evaluator
270270

271271
controller = OpenEvolve(
272-
initial_program_path=self.test_program_path,
272+
initial_programs_paths=[self.test_program_path],
273273
evaluation_file=self.evaluator_path,
274274
config=self.config,
275275
output_dir=self.test_dir,

0 commit comments

Comments
 (0)