Use multiple metrics in more of a Pareto optimization way?

[benjaminy]

My understanding is that currently the combined_score metric is the most important. In some contexts the average of other metric values is used. It seems like analyzing the metrics in more detail might be helpful. For example, when sampling versions it might work well to include versions with complementary metrics profiles. And when deciding which versions to evict it might be useful to look at some kind of Pareto hull. Is anyone thinking along these lines already? Would the maintainers be interested in PRs in this direction?

[jvm123]

I think it would be good. Also if it worked in combination with #109.

A flexible option might be an optional metrics_postprocessing() function in example/../evaluator.py, where the metrics generated there as well as the LLM feedback metrics can be cleaned up, merged or extended? Then the pareto hull suggestion, etc. would come in the form of utility functions, that one can use there if desired.

Notes on the current state:

• You can implement such logic in the evaluate() function for the individual use case
• With llm_feedback_weight, there is already a factor that the LLM-generated metrics are multiplied with. The LLM feedback metrics are not accessible from example/../evaluator.py at this point. There is no possibility to add extra logic for LLM feedback metrics yet.
• The availability of the optional functions in example/../evaluator.py is determined dynamically in openevolve/evaluator.py

Notes on the reasoning from AlphaEvolve:

• The AlphaEvolve paper flat out denies the necessity of weighing off different metrics, suggesting that good solutions typically perform well on a bunch of metrics at once
• With the MAP-Elites archive, high-performing solutions in different metrics are being reintroduced in the evolution

[codelion]

It is an interesting idea, but it is best implemented within the evaluator.py itself. This way you have the flexibility to define whatever metric you need.

[benjaminy]

Maybe I misunderstand the API to evaluation_file. What I'm suggesting is (at least conceptually) using the Pareto hull in operations like eviction to enforce population limit. For example if you had 4 programs with the following 2-dimensional metrics:
[10,3], [9,4], [8, 8], [6, 10] and want to reduce the population to 3, the [9,4] one is probably the best choice. Or during inspiration selection, it might be good to choose two programs with complementary metrics profiles, even if one of them is slightly worse in combined score than another option. As @jvm123 points out, this might not make a big enough difference to be worth doing. But is it even possible to do in evaluation_file?

[codelion]

Yeah, I think it will be good to see this with an example where optimizing multiple objectives this way would make the convergence of the best_program faster.

[benjaminy]

I agree. I will try to demonstrate an example. Intuitively I guess examples where first-order metrics are actually in tension with each other are where it might matter. For example, trying to optimize an engine for power and fuel efficiency. Obviously first-order metrics can be combined to a single number using some formula. But there could be interesting differences between solutions on different Pareto points with the same combined metric score.

[benjaminy]

Just leaving an inspiration here for myself or anyone else who finds time to work on this.

https://dl.acm.org/doi/abs/10.1145/3512290.3528823

[benjaminy]

I finally got around to looking at the AlphaEvolve white paper. I'm having trouble squaring the comment @jvm123 made about it. In the sub-section titled "Multiple scores" they say there is value in optimizing for multiple scores/metrics. They claim that even in applications where one metric is clearly the most important, there can still be diversity benefits to multi-objective optimization.

[jvm123]

@benjaminy I wrote

[the paper] denies the necessity of weighing off different metrics

This was my interpretation of the paragraph you mentioned, which reads:

we find that even if one metric is of particular interest, optimizing for multiple metrics often improves results for the single target metric. Perhaps this occurs because programs excelling under different evaluation criteria often possess distinct structures or logic

This seems to me as if even while one metric was of particular interest, they applied no effort to focus on that metric (i.e., give it a different weight), rather they maximized the sum with all metrics weighed equally.

Further:

by incorporating examples of these diverse, high-performing programs—each representing a different definition of “good”—into the prompts provided to the language model, we can stimulate the generation of more varied candidate solutions

This would happen by means of the MAP-elites archive.

Please correct me if wrong. Thanks!

[benjaminy]

I'm not certain if you're right or wrong, but when the white paper says:

AlphaEvolve allows for optimizing multiple user-provided scores

It would be odd to me if that just meant summing a bunch of metrics into a single combined score. The system wouldn't need to do anything special to "allow" that. What would it even mean to not allow that?