Provide InvokeModel API in AWS Platform

[hustxiayang]

Description:

Proposal: Integration of the InvokeModel API within the AWS Platform

We propose the addition of the InvokeModel API in our AWS platform, driven by the following key motivations: the need for better compatibility and existing issues with the current Converse API.

1. Enhanced Compatibility and Cross-Provider Services

Our primary goal is to provide seamless, cross-provider services to our users. For instance, both Google Cloud Platform (GCP) and Amazon Web Services (AWS) offer access to models like the Claude series. When a user makes a request to a model such as claude-sonnet-4, we want the underlying implementation to be identical across all providers. This approach allows us to fully utilize our resources, offer more resilient and efficient services, and ensure a consistent user experience with minimal configuration concerns.

Currently, the AWS Converse API has significant differences compared to the Anthropic Messages API (and the OpenAI API). While we have implemented numerous workarounds to mitigate these discrepancies, some incompatibilities are unavoidable and frequently cause integration issues.

2. Converse API Specific Issues

The Converse API itself presents several functional limitations and bugs that have led our users to avoid it. Below are two recent cases illustrating these problems:

• A. Inability to Force Model to Skip a Tool Call
  Users cannot explicitly instruct the model not to make a tool call after the model has previously initiated one.

  In the OpenAI API, users can set tool_choice to None or completely remove the tools definition to signal to the model that no further tool calls should be made. In the Converse API, neither of these options is available.

  • First, the Converse API does not support None as a valid tool_choice parameter, as documented in the AWS Bedrock API Reference.
  • Second, we cannot simply remove the tools definition during subsequent interactions. If a user asks a question, the model responds with a tool call, and the user then appends the tool results for the next request, attempting to remove the tools definition in that subsequent request results in a validation error:

    Error code: 400 - {'type': 'error', 'error': {'type': 'ValidationException:internal.amazon.com', 'code': '400', 'message': 'The toolConfig field must be defined when using toolUse and toolResult content blocks.'}}
    

    An online example of this issue can be found in the dify GitHub repository.

• B. Ambiguity of the tool_choice Parameter
  The specification for tool_choice in the Converse API is ambiguous.

  • In the OpenAI API, tool_choice is clearly defined as {"type": "function", "function": {"name": "function_name"}}.
  • In the Anthropic Messages API, it is {"type": "tool", "name": "tool_name"}.
  • However, in the Converse API, it uses the {function_name} directly as the tool_choice, which creates ambiguity. For example, it is possible to have tools named any or auto. The Converse API cannot distinguish whether the user intends to select the default behavior (any/auto) or specifically call a function named "any" or "auto".

---