[ML] Inference API removing _unified and using _stream instead

[jonathan-buttner]

This PR refactors the unified schema approach. Prior to this PR the to interact with the unified schema a request is made to _inference/completion/<inference endpoint id>/_unified

This PR refactors the code such that we no longer have a _unified endpoint. Instead we'll use the previous way with _stream:

PUT _inference/chat_completion/some_endpoint_id/_stream

To do this I created a new proxy action that is registered for the inference rest APIs. The reason we need a proxy action is because we need to know whether the request is the new unified schema or the input task_settings schema. The proxy action determines this by looking to see if the task type was included in the URL. If it wasn't, we retrieve the inference entity from storage and retrieve that task type that way. Once we have the task type we create a new request and route it to InferenceAction or UnifiedInferenceAction.

Other Notes

The unified action does not support non-streaming so if a request is made like:

POST http://localhost:9200/_inference/chat_completion/test2

An error will be returned

Error

{
    "error": {
        "root_cause": [
            {
                "type": "status_exception",
                "reason": "The [chat_completion] task type only supports streaming, please try again with the _stream API"
            }
        ],
        "type": "status_exception",
        "reason": "The [chat_completion] task type only supports streaming, please try again with the _stream API"
    },
    "status": 400
}

The stats/metrics aren't recorded until the internal action is handled.

Testing

Creating the endpoint

PUT http://localhost:9200/_inference/test2
{
    "service": "openai",
    "task_type": "chat_completion",
    "service_settings": {
        "api_key": <api key>,
        "model_id": "gpt-4o"
    }
}

Making a unified request

POST http://localhost:9200/_inference/chat_completion/test2/_stream
POST http://localhost:9200/_inference/test2/_stream
{
    "messages": [
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": "What's the price of a scarf?"
                }
            ]
        }
    ],
    "tools": [
        {
            "type": "function",
            "function": {
                "name": "get_current_price",
                "description": "Get the current price of a item",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "item": {
                            "id": "123"
                        }
                    }
                }
            }
        }
    ],
    "tool_choice": {
        "type": "function",
        "function": {
            "name": "get_current_price"
        }
    }
}

The response structure will be the same as it was before

[elasticsearchmachine]

Pinging @elastic/ml-core (Team:ML)

[davidkyle]

LGTM

Left a comment about the internal action name everything else is great

[davidkyle]

I'm not sure if it is safe to use the old InferenceAction name can be used here when it has different request and response classes. I'm thinking about a mixed cluster. To be safe please give it a new name.

Get, Put and Delete are cluster:monitor/xpack/inference/[get|put|delete] this could be cluster:monitor/xpack/inference/post or cluster:monitor/xpack/inference/inference`

[jonathan-buttner]

Ah ok, I'll switch to post 👍

[jonathan-buttner]

@davidkyle just wanted to confirm that this is what we want to do here right? Changing it to internal?

[davidkyle]

++ yes internal is good

[prwhelan]

nice

[elasticsearchmachine]

💔 Backport failed

Status	Branch	Result
❌	9.0	Commit could not be cherrypicked due to conflicts
❌	8.18	Commit could not be cherrypicked due to conflicts

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 121804

[jonathan-buttner]

💚 All backports created successfully

Status	Branch	Result
✅	8.x
✅	9.0
✅	8.18

Questions ?

Please refer to the Backport tool documentation