fix: map MIPROv2 trial history to AcceptedIteration schema to prevent…

[Redtius]

Description

Hello again! While working with MIPROv2, I encountered a second issue that occurs at the very end of the optimization process. Even after the trials complete successfully, the library crashes when generating the final report.

The Issue:

MIPROv2 stores its internal trial history as raw dictionaries. However, the OptimizationReport Pydantic model requires the accepted_iterations field to be a list of AcceptedIteration objects. This leads to a ValidationError (especially on Pydantic v2), preventing the user from seeing the final results:
❌ BUG 2 REPRODUCED: [MIPROv2] • error ValidationError: 10 validation errors for OptimizationReport accepted_iterations.0.parent Field required [type=missing, input_value={'trial': 1, 'instr_idx':...o_idx': 0, 'score': 1.0}, input_type=dict] For further information visit https://errors.pydantic.dev/2.11/v/missing accepted_iterations.0.child Field required [type=missing, input_value={'trial': 1, 'instr_idx':...o_idx': 0, 'score': 1.0}, input_type=dict] For further information visit https://errors.pydantic.dev/2.11/v/missing accepted_iterations.0.module Field required [type=missing, input_value={'trial': 1, 'instr_idx':...o_idx': 0, 'score': 1.0}, input_type=dict] For further information visit https://errors.pydantic.dev/2.11/v/missing ##############...#################
accepted_iterations.1.before Field required [type=missing, input_value={'trial': 2, 'instr_idx':...o_idx': 0, 'score': 1.0}, input_type=dict] For further information visit https://errors.pydantic.dev/2.11/v/missing accepted_iterations.1.after Field required [type=missing, input_value={'trial': 2, 'instr_idx':...o_idx': 0, 'score': 1.0}, input_type=dict] For further information visit https://errors.pydantic.dev/2.11/v/missing • halted before first iteration

The Fix:

I have updated the _build_result method to map the trial history into the AcceptedIteration schema. This ensures the data is correctly validated while preserving the trial scores for the report.

Reproductible Case

I prepared a mock case that reproduces the issue without requiring any external API keys or complex setups:

from deepeval.models.base_model import DeepEvalBaseLLM
from deepeval.optimizer import PromptOptimizer
from deepeval.optimizer.algorithms import MIPROV2
from deepeval.dataset import Golden
from deepeval.prompt import Prompt
from deepeval.metrics import BaseMetric
from deepeval.test_case import LLMTestCase

class MockLLM(DeepEvalBaseLLM):
    def __init__(self): pass
    def load_model(self): return self
    def generate(self, prompt: str): return "response"
    async def a_generate(self, prompt: str): return "response"
    def get_model_name(self): return "LLM"

class MockMetric(BaseMetric):
    def __init__(self): self.threshold = 0.5
    def measure(self, test_case: LLMTestCase):
        self.score = 1.0
        return self.score
    async def a_measure(self, test_case: LLMTestCase): return self.measure(test_case)
    def is_successful(self): return True
    @property
    def __name__(self): return "Metric"

def model_callback(prompt: Prompt,thread_id:str) -> str: return "Mocked output"

mock_model = MockLLM()
goldens = [Golden(input="Hi", expected_output="Hello"),Golden(input="Hru?", expected_output="Fine, U?")]
prompt = Prompt(text_template="You are a mock assistant.")
optimizer = PromptOptimizer(
    algorithm=MIPROV2(num_candidates=1, num_trials=2),
    optimizer_model=mock_model,
    model_callback=model_callback,
    metrics=[MockMetric()]
)

# This is a bug too, i already sent another PR to fix it
optimizer.algorithm.optimizer_model = mock_model

try:
    # This should finish the trials and then crash when building the report
    result = optimizer.optimize(prompt=prompt, goldens=goldens)
    print(optimizer.optimization_report)
except Exception as e:
    print(f"\nBUG: {e}")

[vercel]

Someone is attempting to deploy a commit to the Confident AI Team on Vercel.

A member of the Team first needs to authorize it.

[greptile-apps]

PR author is not in the allowed authors list.

[Redtius]

@penguine-ip