Skip to content

[vertex-ai] fix: allow object as tool content #1279

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

[vertex-ai] fix: allow object as tool content #1279

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
623ef2e
fix: allow object as tool content
arturfromtabnine Aug 12, 2025
c90613f
address comments
arturfromtabnine Aug 26, 2025
708b86c
remove check for type of content key as arrays are supported in vertex
narengogi Aug 28, 2025
c7a652c
replicate changes in google as well
narengogi Aug 28, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion src/providers/google-vertex-ai/chatComplete.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,8 @@ export const VertexGoogleChatCompleteConfig: ProviderConfig = {
});
} else if (
message.role === 'tool' &&
typeof message.content === 'string'
(typeof message.content === 'string' ||
typeof message.content === 'object')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Code Refactor

Issue: The type check could be more robust for object validation
Fix: Consider validating that object content is not null and has expected structure
Impact: Prevents potential runtime errors with malformed object content

Suggested change
(typeof message.content === 'string' ||
typeof message.content === 'object')
(typeof message.content === 'string' ||
(typeof message.content === 'object' && message.content !== null))

Copy link
Collaborator

@narengogi narengogi Aug 12, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OpenAI spec only supports sending text content or array of text content parts as part of tool message content https://platform.openai.com/docs/api-reference/chat/create#chat_create-messages

I believe what you are trying to do is to send an object like this

{
  "role": "user",
  "parts": [
    {
      "functionResponse": {
        "name": "get_current_weather",
        "response": {
          "temperature": 20,
          "unit": "C"
        }
      }
    }
  ]
}

instead of

{
  "role": "user",
  "parts": [
    {
      "functionResponse": {
        "name": "get_current_weather",
        "response": { "content": "{'temperature': 20, 'unit': 'C'}"}
      }
    }
  ]
}

the current implementation is definitely incomplete, what we should be doing is

 else if (
            message.role === 'tool' &&
            typeof message.content === 'string'
          ) {
            let toolResponse;
            try {
              toolResponse = JSON.parse(message.content);
            } catch (e) {
              toolResponse = {
                content: message.content,
              };
            }
            parts.push({
              functionResponse: {
                name: message.name ?? 'gateway-tool-filler-name',
                response,
              },
            });
          }

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need to handle array of text contents as well, which we can change in a separate PR

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, we need to do a strict check for array, but your example is not what we are trying to send. Our client is open ai compatible, so its trying to send an array in message.content

{
role: 'tool',
tool_call_id: 'call_ty2u3gJiEIsf9TLOwUB67opO',
content: [
  {
   type: 'text',
   text: '{"filePath"}'
  }
 ]
}

with suggested change there no issue, vertex api for gemini models allows to have response.content as an array or string

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if we need to parse content like was suggested

            typeof message.content === 'string'
          ) {
            let toolResponse;
            try {
              toolResponse = JSON.parse(message.content);
            } catch (e) {
              toolResponse = {
                content: message.content,
              };
            }

) {
parts.push({
functionResponse: {
Expand Down
2 changes: 1 addition & 1 deletion src/providers/google/chatComplete.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ interface GoogleFunctionResponseMessagePart {
name: string;
response: {
name?: string;
content: string;
content: string | ContentType[];
};
};
}
Expand Down