best_program_info iteration does not contain the iteration of the best program

[tom-dudley]

Hi, thanks for the great project. I was looking at setting up simple monitoring of progress by checking the best_program_info.json file for the latest checkpoint with:

watch -n 10 'jq ".metrics.combined_score, .iteration" "$(ls -d checkpoint_* | sort -V | tail -n1)/best_program_info.json"'

which would inform me how many iterations (and hence time) it took to get to a certain metric.

This file contains the current iteration though, not the iteration when the best program was achieved: https://github.com/codelion/openevolve/blob/main/openevolve/controller.py#L383

Would it make sense for this iteration value to be changed?

---

More generally, could you confirm a couple of things regarding iterations and generations:

1. Is a generation simply "how many ancestors do I have"? So a program might be found at iteration 100 with 4 ancestors and is hence generation 5, and a program might be found at generation 150 with 2 ancestors (as it was from a later new novel idea). Thus it's not like the algorithm is 'run for 10 generations, run 100 iterations per generation'. If this is indeed the case hopefully the ask above makes more sense given knowing the generation of the best program while interesting doesn't quite give details on how long it took to get there.
2. In the circle example README, when generations are mentioned (e.g. after 10 or 100 generations when showing the results) is this actually generations or is it iterations?

[codelion]

Would it make sense for this iteration value to be changed?

Yes, this is a bug, we need to change it to capture the iteration for the best program. Would appreciate a PR if you can fix it. Otherwise, I will try and address it over the weekend.

Generation is supposed to the number of ancestors, for the circle packing example we ran for 800 iterations to get those 10 generations.

[tom-dudley]

Would it make sense for this iteration value to be changed?

Yes, this is a bug, we need to change it to capture the iteration for the best program. Would appreciate a PR if you can fix it. Otherwise, I will try and address it over the weekend.

Generation is supposed to the number of ancestors, for the circle packing example we ran for 800 iterations to get those 10 generations.

Thanks, I'll try and take a look tonight.

[hafeezhmha]

Would it make sense for this iteration value to be changed?

Yes, this is a bug, we need to change it to capture the iteration for the best program. Would appreciate a PR if you can fix it. Otherwise, I will try and address it over the weekend.

Generation is supposed to the number of ancestors, for the circle packing example we ran for 800 iterations to get those 10 generations.

hi, I encountered the same bug, I have sent a pull request, can you take a look?
After the fix i can monitor the evolution process and can see both when the best program was discovered and the current iteration number

[codelion]

@tom-dudley can you verify if it is fixed now? It was addressed in #23

[tom-dudley]

Thanks both! That does fix this issue, although I'll admit the fact start_iteration appears to be used before it's set is a bit confusing to me - I'm having to init to to zero on my end to get things to work.

https://github.com/codelion/openevolve/blob/main/openevolve/controller.py#L147-L165

Small PR here if it is indeed required #24

[tom-dudley]

Given #25 was merged I think this is resolved.