Skip to content

Add Azure OpenAI chat completion support #138726

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Add Azure OpenAI chat completion support #138726

wants to merge 4 commits into from

Conversation

@Jan-Kazlouski-elastic
Copy link
Contributor

@Jan-Kazlouski-elastic Jan-Kazlouski-elastic commented Nov 27, 2025 •
edited
Loading

Extending of an existing Azure OpenAI inference provider integration allowing chat_completion task to be executed as part of inference API with azureopenai provider.

Create chat completion endpoint 
RQ
{
    "service": "azureopenai",
    "service_settings": {
        "api_key": "{{azure-openai-key}}",
        "resource_name": "{{azure-resource-name}}",
        "deployment_id": "{{azure-deployment-id}}",
        "api_version": "{{azure-api-version}}"
    }
}
RS
{
    "inference_id": "azure-openai-chat-completion",
    "task_type": "chat_completion",
    "service": "azureopenai",
    "service_settings": {
        "resource_name": "{{azure-resource-name}}",
        "deployment_id": "{{azure-deployment-id}}",
        "api_version": "{{azure-api-version}}",
        "rate_limit": {
            "requests_per_minute": 120
        }
    }
}
Create chat completion endpoint (Unauthorized) 
RQ
{
    "service": "azureopenai",
    "service_settings": {
        "api_key": "{{invalid-azure-openai-key}}",
        "resource_name": "{{azure-resource-name}}",
        "deployment_id": "{{azure-deployment-id}}",
        "api_version": "{{azure-api-version}}"
    }
}
RS
{
    "error": {
        "root_cause": [
            {
                "type": "unified_chat_completion_exception",
                "reason": "Received an authentication error status code for request from inference entity id [azure-openai-chat-completion] status [401]. Error message: [{\"error\":{\"code\":\"401\",\"message\":\"Access denied due to invalid subscription key or wrong API endpoint. Make sure to provide a valid key for an active subscription and use a correct regional API endpoint for your resource.\"}}]"
            }
        ],
        "type": "status_exception",
        "reason": "Could not complete inference endpoint creation as validation call to service threw an exception.",
        "caused_by": {
            "type": "unified_chat_completion_exception",
            "reason": "Received an authentication error status code for request from inference entity id [azure-openai-chat-completion] status [401]. Error message: [{\"error\":{\"code\":\"401\",\"message\":\"Access denied due to invalid subscription key or wrong API endpoint. Make sure to provide a valid key for an active subscription and use a correct regional API endpoint for your resource.\"}}]"
        }
    },
    "status": 400
}
Create chat completion endpoint (Not Found) 
RQ
{
    "service": "azureopenai",
    "service_settings": {
        "api_key": "{{azure-openai-key}}",
        "resource_name": "{{invalid-azure-resource-name}}",
        "deployment_id": "{{invalid-azure-deployment-id}}",
        "api_version": "{{azure-api-version}}"
    }
}
RS
{
    "error": {
        "root_cause": [
            {
                "type": "status_exception",
                "reason": "Invalid host [https://jan-kazlouski-test1.openai.azure.com/openai/deployments/gpt-5-nano/chat/completions?api-version=2025-01-01-preview], please check that the URL is correct."
            }
        ],
        "type": "status_exception",
        "reason": "Could not complete inference endpoint creation as validation call to service threw an exception.",
        "caused_by": {
            "type": "status_exception",
            "reason": "Invalid host [https://jan-kazlouski-test1.openai.azure.com/openai/deployments/gpt-5-nano/chat/completions?api-version=2025-01-01-preview], please check that the URL is correct.",
            "caused_by": {
                "type": "unknown_host_exception",
                "reason": "No such host is known (jan-kazlouski-test1.openai.azure.com)"
            }
        }
    },
    "status": 400
}
Perform basic chat completion 
RQ
{
    "messages": [
        {
            "role": "user",
            "content": "What is deep learning?"
        }
    ],
    "max_completion_tokens": 100
}
RS
event: message
data: {"id":"","choices":[],"model":"","object":""}

event: message
data: {"id":"chatcmpl-CgXvBu72LBMQxmTLdl2TYJM0GHNSK","choices":[{"delta":{"content":"","role":"assistant"},"index":0}],"model":"gpt-5-nano-2025-08-07","object":"chat.completion.chunk"}

event: message
data: {"id":"chatcmpl-CgXvBu72LBMQxmTLdl2TYJM0GHNSK","choices":[{"delta":{},"finish_reason":"length","index":0}],"model":"gpt-5-nano-2025-08-07","object":"chat.completion.chunk"}

event: message
data: {"id":"chatcmpl-CgXvBu72LBMQxmTLdl2TYJM0GHNSK","choices":[],"model":"gpt-5-nano-2025-08-07","object":"chat.completion.chunk","usage":{"completion_tokens":100,"prompt_tokens":11,"total_tokens":111,"prompt_tokens_details":{"cached_tokens":0}}}

event: message
data: [DONE]
Perform chat completion with tool calling 
RQ
{
    "messages": [
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": "What's the price of a scarf?"
                }
            ]
        }
    ],
    "tools": [
        {
            "type": "function",
            "function": {
                "name": "get_current_price",
                "description": "Get the current price of a item",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "item": {
                            "id": "123"
                        }
                    }
                }
            }
        }
    ],
    "tool_choice": {
        "type": "function",
        "function": {
            "name": "get_current_price"
        }
    }
}
RS
event: message
data: {"id":"","choices":[],"model":"","object":""}

event: message
data: {"id":"chatcmpl-CgXxUUBq8TPfe3QwLrepIDkfmzFfs","choices":[{"delta":{"role":"assistant","tool_calls":[{"index":0,"id":"call_9XaPI6YYLMB1VDODuzxlOcEw","function":{"arguments":"","name":"get_current_price"},"type":"function"}]},"index":0}],"model":"gpt-5-nano-2025-08-07","object":"chat.completion.chunk"}

event: message
data: {"id":"chatcmpl-CgXxUUBq8TPfe3QwLrepIDkfmzFfs","choices":[{"delta":{"tool_calls":[{"index":0,"function":{"arguments":"{\""},"type":null}]},"index":0}],"model":"gpt-5-nano-2025-08-07","object":"chat.completion.chunk"}

event: message
data: {"id":"chatcmpl-CgXxUUBq8TPfe3QwLrepIDkfmzFfs","choices":[{"delta":{"tool_calls":[{"index":0,"function":{"arguments":"item"},"type":null}]},"index":0}],"model":"gpt-5-nano-2025-08-07","object":"chat.completion.chunk"}

event: message
data: {"id":"chatcmpl-CgXxUUBq8TPfe3QwLrepIDkfmzFfs","choices":[{"delta":{"tool_calls":[{"index":0,"function":{"arguments":"\":\""},"type":null}]},"index":0}],"model":"gpt-5-nano-2025-08-07","object":"chat.completion.chunk"}

event: message
data: {"id":"chatcmpl-CgXxUUBq8TPfe3QwLrepIDkfmzFfs","choices":[{"delta":{"tool_calls":[{"index":0,"function":{"arguments":"sc"},"type":null}]},"index":0}],"model":"gpt-5-nano-2025-08-07","object":"chat.completion.chunk"}

event: message
data: {"id":"chatcmpl-CgXxUUBq8TPfe3QwLrepIDkfmzFfs","choices":[{"delta":{"tool_calls":[{"index":0,"function":{"arguments":"arf"},"type":null}]},"index":0}],"model":"gpt-5-nano-2025-08-07","object":"chat.completion.chunk"}

event: message
data: {"id":"chatcmpl-CgXxUUBq8TPfe3QwLrepIDkfmzFfs","choices":[{"delta":{"tool_calls":[{"index":0,"function":{"arguments":"\"}"},"type":null}]},"index":0}],"model":"gpt-5-nano-2025-08-07","object":"chat.completion.chunk"}

event: message
data: {"id":"chatcmpl-CgXxUUBq8TPfe3QwLrepIDkfmzFfs","choices":[{"delta":{},"finish_reason":"tool_calls","index":0}],"model":"gpt-5-nano-2025-08-07","object":"chat.completion.chunk"}

event: message
data: {"id":"chatcmpl-CgXxUUBq8TPfe3QwLrepIDkfmzFfs","choices":[],"model":"gpt-5-nano-2025-08-07","object":"chat.completion.chunk","usage":{"completion_tokens":218,"prompt_tokens":134,"total_tokens":352,"prompt_tokens_details":{"cached_tokens":0}}}

event: message
data: [DONE]
  • - Have you signed the contributor license agreement?
  • - Have you followed the contributor guidelines?
  • - If submitting code, have you built your formula locally prior to submission with gradle check?
  • - If submitting code, is your pull request against main? Unless there is a good reason otherwise, we prefer pull requests against main and will backport as needed.
  • - If submitting code, have you checked that your submission is for an OS and architecture that we support?
  • - If you are submitting this code for a class then read our policy for that.

@elasticsearchmachine elasticsearchmachine added needs:triage Requires assignment of a team area label v9.3.0 external-contributor Pull request authored by a developer outside the Elasticsearch team labels Nov 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

external-contributor Pull request authored by a developer outside the Elasticsearch team needs:triage Requires assignment of a team area label v9.3.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@Jan-Kazlouski-elastic @elasticsearchmachine