Request: stopping at a threshold score

[theahura]

Currently, open evolve implements an early stopping mechanism through the use of a patience value in a passed in config. Patience will kill an evolution if there is no score improvement above some threshold for N steps.

Another form of early stopping that is valuable is stopping once a certain score is reached. For example, you could imagine a distance score that you want to eventually converge to 0, and you want to keep running openevolve until the distance hits 0. In such a case, further iterations once the score is 0 is wasteful. This additional early stopping mechanism interacts with patience -- you could imagine wanting to stop early if the patience threshold is hit OR if the ideal score is hit, whichever happens first -- but you cannot directly use patience for this purpose without incurring some wasted computation cycles.

I think implementing early stopping in this way would require modifying parts of the config as well as the process_parallel.py file. I'm happy to take a stab at this, but would love some guidance about where to look / what needs changing

[codelion]

The target score stopping mechanism is implemented and should be working via the CLI:

# Stop when target score of 0.95 is reached
python openevolve-run.py program.py evaluator.py --config config.yaml --target-score 0.95

# Works with existing patience mechanism too
python openevolve-run.py program.py evaluator.py --config config.yaml --target-score 0.95 --iterations 1000

[theahura]

o sweet so to make the changes I want we just have to add this to the config then?

[theahura]

@codelion this does not work quite as expected.

Two issues.

• First, if no iterations value is passed in, it will use the default, which is set to 10k steps. This is obviously a lot, but its not the same as not having any iteration max limit at all
  https://github.com/codelion/openevolve/blob/main/openevolve/controller.py#L243

• Second, if a target score is passed in, it will look at the average of all numeric scores instead of the combined_score to evaluate the final result, resulting in it ending early (https://github.com/codelion/openevolve/blob/main/openevolve/process_parallel.py#L596)

[codelion]

thanks for checking this and yes it is a bug, happy to review a fix PR.

[theahura]

Here's a pass: #286