Azure AI model as the judge evaluators break with gpt reasoning model

[luigiw]

• Package Name: azure-ai-evaluation
• Package Version: 1.12.0
• Operating System: MacOS
• Python Version: 3.12

Describe the bug
A clear and concise description of what the bug is.

Evaluator invocation fails when being configured with a reasoning model

To Reproduce
Steps to reproduce the behavior:

relevance_evaluator = RelevanceEvaluator(model_config=model_config, is_reasoning_model=True)

relevance_evaluator(query="What is Azure AI?", response="Azure AI is a set of services and tools for building AI applications.")

Above code breaks with a reasoning model and raises below error

2025-10-23 14:26:03 -0700 6307672064 azure.ai.evaluation._legacy.prompty._prompty ERROR    [0/10] AsyncAzureOpenAI request failed. BadRequestError: Error code: 400 - {'error': {'message': "Unsupported parameter: 'response_format'. In the Responses API, this parameter has moved to 'text.format'. Try again with the new parameter. See the API documentation for more information: https://platform.openai.com/docs/api-reference/responses/create.", 'type': 'invalid_request_error', 'param': None, 'code': 'unsupported_parameter'}}
Traceback (most recent call last):
  File "/Users/hanchiwang/miniconda3/envs/eval-agent/lib/python3.12/site-packages/azure/ai/evaluation/_legacy/prompty/_prompty.py", line 382, in _send_with_retries
    response = await client.chat.completions.create(**params)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/hanchiwang/miniconda3/envs/eval-agent/lib/python3.12/site-packages/openai/resources/chat/completions/completions.py", line 2603, in create
    return await self._post(
           ^^^^^^^^^^^^^^^^^
  File "/Users/hanchiwang/miniconda3/envs/eval-agent/lib/python3.12/site-packages/openai/_base_client.py", line 1794, in post
    return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/hanchiwang/miniconda3/envs/eval-agent/lib/python3.12/site-packages/openai/_base_client.py", line 1594, in request
    raise self._make_status_error_from_response(err.response) from None
openai.BadRequestError: Error code: 400 - {'error': {'message': "Unsupported parameter: 'response_format'. In the Responses API, this parameter has moved to 'text.format'. Try again with the new parameter. See the API documentation for more information: https://platform.openai.com/docs/api-reference/responses/create.", 'type': 'invalid_request_error', 'param': None, 'code': 'unsupported_parameter'}}
 

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
Add any other context about the problem here.

[github-actions]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @luigiw @needuv @singankit.