The issue of the temperature parameter

[Ethan-han921]

I’ve encountered something confusing while using the OpenEvolve framework, and I’d appreciate some clarification.
In the OpenAILLM class in your openai.py file, there is this code block when setting up generation parameters:

if self.api_base == "https://api.openai.com/v1" and str(self.model).lower().startswith("o"):
    # For o-series models
    params = {
        "model": self.model,
        "messages": formatted_messages,
        "max_completion_tokens": kwargs.get("max_tokens", self.max_tokens),
    }
else:
    params = {
        "model": self.model,
        "messages": formatted_messages,
        "temperature": kwargs.get("temperature", self.temperature),
        "top_p": kwargs.get("top_p", self.top_p),
        "max_tokens": kwargs.get("max_tokens", self.max_tokens),
    }

It seems that when the model name starts with "o" (e.g., "o4-mini"), the temperature parameter is not passed to the API.

I’ve also tested an example based on your function_minimization. I set temperature values like 0.3 vs 0.7 with o4-mini(just set one model) and observed significant behavioral differences in the output, which suggests that these parameters are indeed supported and have an effect.

This is my config.yaml

max_iterations: 10
checkpoint_interval: 5
log_level: "INFO"

  primary_model: "o4-mini"   
  primary_model_weight: 1.0
  temperature: 0.3

  api_base: "XXXX"

  api_key: "XXXX"
  max_tokens: 4096


prompt:

  include_artifacts: true
  system_message: "You are an expert programmer specializing in optimization algorithms. Your task is to improve a function minimization algorithm to find the global minimum of a complex function with many local minima. The function is f(x, y) = sin(x) * cos(y) + sin(x*y) + (x^2 + y^2)/20. Focus on improving the search_algorithm function to reliably find the global minimum, escaping local minima that might trap simple algorithms."
  evaluator_system_message: "You need to ensure each round score is improved by the last best one"
  num_top_programs: 1
  max_artifact_bytes: 4096
  use_template_stochasticity: true
  artifact_security_filter: true


database:
  population_size:  1
  archive_size: 1
  num_islands: 1
  elite_selection_ratio: 0.3
  exploitation_ratio: 0.7


evaluator:
  timeout: 60
  cascade_evaluation: true
  enable_artifacts: true


diff_based_evolution: false
allow_full_rewrites: true

Testing result

---

When Temperature = 0.7
0.767818. 0.954938 0.945888 0.952813 0.951586 0.779630 0.952998 0.797274 0.787243 0.986230 0.974237

Temperature = 0.3
0.767818 0.706419 0.784706 0.777148 0.988748 0.994253 0.702351 0.797274 0.999715 0.981818 0.999708

three questions:

Q1 Why are the two results different? If the temperature cannot be passed, the result should be the same.
Q2 Can OpenEvolve guarantee that each successive round’s score improves over the previous one and demonstrates a steadily converging trend?
Q3 Whether or not they can set elite_selection_ratio=1.0 and exploitation_ratio=1.0 at the same time?

Could you please provide some explanations? Thank you so much.

[codelion]

Yeah it was added for older models
• o1
• o1-mini
• o3-mini

They do not allow sampling via temperature. We can tighten the conditional to only skip it for these models and warn the user. Would appreciate a PR if you can do it I will take a look at it and merge.

[Ethan-han921]

Yeah it was added for older models • o1 • o1-mini • o3-mini

They do not allow sampling via temperature. We can tighten the conditional to only skip it for these models and warn the user. Would appreciate a PR if you can do it I will take a look at it and merge.

When I tried 'o3mini', give this config

llm:
  primary_model: "o3-mini"   
  primary_model_weight: 1.0
  temperature: 1.0
  top_p: 0.95

There is no warning or error, it shows:

2025-07-24 22:03:10,917 - INFO - Sampled model: o3-mini
[DEBUG] LLM params: dict_keys(['model', 'messages', 'max_completion_tokens'])
2025-07-24 22:03:10,925 - INFO - New MAP-Elites cell occupied: {'complexity': 6, 'diversity': 0}
2025-07-24 22:03:10,925 - INFO - Population size (3) exceeds limit (1), removing 2 programs
2025-07-24 22:03:10,925 - INFO - Population size after cleanup: 2
2025-07-24 22:03:10,925 - INFO - Iteration 1: Program 6f158a7a-30b7-4080-bf7c-f2ce5f09fa6f (parent: c6a4d687-8eb8-4c92-8479-c92d54f9badf) completed in 16.44s
2025-07-24 22:03:10,925 - INFO - Metrics: runs_successfully=1.0000, value_score=0.7352, distance_score=0.2402, overall_score=0.1813, standard_deviation_score=0.3251, speed_score=1.0000, reliability_score=1.0000, combined_score=0.5064, success_rate=1.0000
2025-07-24 22:03:27,838 - INFO - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
2025-07-24 22:03:28,608 - INFO - Evaluated program 4f83cf3e-cd76-4031-b26b-7add371dcf6e in 0.77s: runs_successfully=1.0000, value_score=0.9991, distance_score=0.9738, overall_score=0.9970, standard_deviation_score=0.9738, speed_score=1.0000, reliability_score=1.0000, combined_score=0.9852, success_rate=1.0000
2025-07-24 22:03:28,609 - INFO - Sampled model: o3-mini
[DEBUG] LLM params: dict_keys(['model', 'messages', 'max_completion_tokens'])
2025-07-24 22:03:28,616 - INFO - New MAP-Elites cell occupied: {'complexity': 9, 'diversity': 9}
2025-07-24 22:03:28,616 - INFO - Population size (3) exceeds limit (1), removing 2 programs
2025-07-24 22:03:28,616 - INFO - Population size after cleanup: 2
2025-07-24 22:03:28,616 - INFO - New best program 4f83cf3e-cd76-4031-b26b-7add371dcf6e replaces c6a4d687-8eb8-4c92-8479-c92d54f9badf (combined_score: 0.7678 → 0.9852, +0.2174)
2025-07-24 22:03:28,616 - INFO - Iteration 2: Program 4f83cf3e-cd76-4031-b26b-7add371dcf6e (parent: cb41e7cb-5427-4976-bb2b-fc18f6d95ee4) completed in 17.69s
2025-07-24 22:03:28,616 - INFO - Metrics: runs_successfully=1.0000, value_score=0.9991, distance_score=0.9738, overall_score=0.9970, standard_deviation_score=0.9738, speed_score=1.0000, reliability_score=1.0000, combined_score=0.9852, success_rate=1.0000
2025-07-24 22:03:28,617 - INFO - 🌟 New best solution found at iteration 2: 4f83cf3e-cd76-4031-b26b-7add371dcf6e
2025-07-24 22:03:46,329 - INFO - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
2025-07-24 22:03:46,522 - INFO - Evaluated program 2969095a-00c3-4e6b-97e7-1c510907d500 in 0.19s: runs_successfully=1.0000, value_score=0.9166, distance_score=0.3720, overall_score=0.9258, standard_deviation_score=0.3893, speed_score=1.0000, reliability_score=1.0000, combined_score=0.6289, success_rate=1.0000

[Ethan-han921]

it was added for older models • o1 • o1-mini • o3-mini

They do not allow sampling via temperature. We can tighten the conditional to only skip it for these models and warn the user. Would appreciate a PR if you can do it I will take a look at it and merge.

I will try.