Skip to content

Conversation

@jonathan-buttner
Copy link
Contributor

WIP

This PR implements the changes to add chat completion support to EIS within the inference plugin.

The current state of this PR is hard coded to work against OpenAI.

Testing

Run elasticsearch with a few feature flags enabled:

run-es -Des.inference_unified_feature_flag_enabled=true -Des.elastic_inference_service_feature_flag_enabled=true

Create an endpoint

PUT http://localhost:9200/_inference/completion/test
{
    "service": "elastic",
    "service_settings": {
        "api_key": "key",
        "model_id": "gpt-4o"
    }
}

Send request

POST http://localhost:9200/_inference/completion/test/_unified
{
    "model": "gpt-4o",
    "messages": [
        {
            "role": "user",
            "content": "What is the weather like in Boston today?"
        }
    ],
    "stop": ["none"],
    "tools": [
        {
        "type": "function",
        "function": {
            "name": "get_current_weather",
            "description": "Get the current weather in a given location",
            "parameters": {
            "type": "object",
            "properties": {
                "location": {
                "type": "string",
                "description": "The city and state, e.g. San Francisco, CA"
                },
                "unit": {
                "type": "string",
                "enum": ["celsius", "fahrenheit"]
                }
            },
            "required": ["location"]
            }
        }
        }
    ],
    "tool_choice": "auto"
}

@jonathan-buttner jonathan-buttner added >enhancement :ml Machine learning Team:ML Meta label for the ML team v9.0.0 v8.18.0 labels Dec 9, 2024
@elasticsearchmachine
Copy link
Collaborator

Hi @jonathan-buttner, I've created a changelog YAML for you.

@jonathan-buttner
Copy link
Contributor Author

Closing in favor of: #118871

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

>enhancement :ml Machine learning Team:ML Meta label for the ML team v8.18.0 v9.0.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants