Message content not appended after function or tool calling

[azel]

What happened?

If the model decides to call a function after generating content within the same API response, that generated content is not appended to the subsequent conversation. This can lead to unusual behavior when the model is prompted to perform chain-of-thought reasoning, as the initial thought is effectively 'lost.' The model will generate redundant content. In extreme cases, this can cause the model to get stuck in a loop.

Version Information

bb7303c7ec9e

Steps to Reproduce

1. Choose an agent that has access to tools or actions.
2. Provide the agent with a prompt, such as:

I need to know the typical lifespan of the largest land mammal alive today. Use Chain of Thought:

1. First, identify this specific animal using your biological knowledge.
2. Second, based on the animal identified, formulate a precise query for a tool to find its typical lifespan.
3. Finally, execute that formulated query using the appropriate tool.

3. The model then repeats the content it generated prior to the tool call.

What browsers are you seeing the problem on?

No response

Relevant log output

Screenshots

I set up Langfuse to monitor API calls. This behavior can be confirmed in the logs.

The output from the tool call response

In the subsequent API call, the 'content' field is an empty string.

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

When working with OpenAI spec, the actual "function call" is acceptable to have "empty content", the tool output is in the next content part:

{
  "model": "gpt-4o-mini",
  "stream": true,
  "tools": [
/* Tool definitions */
  ],
  "stream_options": {
    "include_usage": true
  },
  "include_reasoning": true,
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "I need to know the typical lifespan of the largest land mammal alive today. Use Chain of Thought:\n\n        First, identify this specific animal using your biological knowledge.\n        Second, based on the animal identified, formulate a precise query for a tool to find its typical lifespan.\n        Finally, execute that formulated query using the appropriate tool."
        }
      ]
    },
    {
      "role": "assistant",
/* Empty content */
      "content": "",
      "tool_calls": [
        {
          "id": "ccab3cd0-d9d7-49d9-b3fb-4d29ef8c822d",
          "type": "function",
          "function": {
            "name": "firecrawl_search_mcp_mcp-server-firecrawl",
            "arguments": "{\"query\":\"African elephant lifespan\"}"
          }
        }
      ]
    },
    {
      "role": "tool",
/* Tool output */
      "content": "URL: https://www.awf.org/news/elephants-live-longer-wild-study-shows\nTitle: Elephants Live Longer in the Wild, Study Shows\nDescription: The wild elephants lived three times as long on average, surviving to a median age of 56 years compared with 17 years for elephants living in captivity.\n\n\nURL: https://en.wikipedia.org/wiki/African_elephant\nTitle: African elephant - Wikipedia\nDescription: It is thought that this lineage has been isolated from the others for 2.4 million years.\n\n\nURL: https://www.wwf.org.uk/learn/fascinating-facts/elephants\nTitle: Our top 10 facts about elephants - WWF-UK\nDescription: Males only reach their full size at 35-40 years - that's well over half their lifespan as wild elephants can live for up to 60-70 years.\n\n\nURL: https://www.bbc.co.uk/bitesize/articles/zxm8h4j\nTitle: How long do elephants live for and which animal lives the longest?\nDescription: The average life expectancy of an African elephant is around 70 years, while their slightly smaller relative, the Asian elephant, has a life expectancy of ...\n\n\nURL: https://en.wikipedia.org/wiki/African_bush_elephant\nTitle: African bush elephant - Wikipedia\nDescription: The maximum lifespan of the African bush elephant is between 70 and 75 years. Its generation length is 25 years.\n",
      "name": "firecrawl_search_mcp_mcp-server-firecrawl",
      "tool_call_id": "ccab3cd0-d9d7-49d9-b3fb-4d29ef8c822d"
    },
    {
      "role": "assistant",
      "content": [
        {
          "type": "text",
          "text": "The typical lifespan of the largest land mammal, the African elephant, is around 60-70 years in the wild. Some sources suggest a maximum lifespan of 70-75 years.\n"
        }
      ]
    },
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "thanks"
        }
      ]
    }
  ]
}

I then get a reasonable response:

For Google, it gets formatted according to their API specs:

[
  {
    role: "user",
    parts: [
      {
        text: "I need to know the typical lifespan of the largest land mammal alive today. Use Chain of Thought:\n\n        First, identify this specific animal using your biological knowledge.\n        Second, based on the animal identified, formulate a precise query for a tool to find its typical lifespan.\n        Finally, execute that formulated query using the appropriate tool.",
      },
    ],
  },
  {
    role: "model",
    parts: [
      {
        text: "Okay, I will follow your instructions.\n\n1.  **Identify the animal:** The largest land mammal alive today is the African elephant.\n2.  **Formulate a query:** I will use the search tool to find the typical lifespan of an African elephant. The query will be: \"African elephant lifespan\"\n3.  **Execute the query:**\n",
      },
    ],
  },
  {
    role: "user",
    parts: [
      {
        functionResponse: {
          name: "firecrawl_search_mcp_mcp-server-firecrawl",
          response: {
            result: "URL: https://www.awf.org/news/elephants-live-longer-wild-study-shows\nTitle: Elephants Live Longer in the Wild, Study Shows\nDescription: The wild elephants lived three times as long on average, surviving to a median age of 56 years compared with 17 years for elephants living in captivity.\n\n\nURL: https://en.wikipedia.org/wiki/African_elephant\nTitle: African elephant - Wikipedia\nDescription: It is thought that this lineage has been isolated from the others for 2.4 million years.\n\n\nURL: https://www.wwf.org.uk/learn/fascinating-facts/elephants\nTitle: Our top 10 facts about elephants - WWF-UK\nDescription: Males only reach their full size at 35-40 years - that's well over half their lifespan as wild elephants can live for up to 60-70 years.\n\n\nURL: https://www.bbc.co.uk/bitesize/articles/zxm8h4j\nTitle: How long do elephants live for and which animal lives the longest?\nDescription: The average life expectancy of an African elephant is around 70 years, while their slightly smaller relative, the Asian elephant, has a life expectancy of ...\n\n\nURL: https://en.wikipedia.org/wiki/African_bush_elephant\nTitle: African bush elephant - Wikipedia\nDescription: The maximum lifespan of the African bush elephant is between 70 and 75 years. Its generation length is 25 years.\n",
          },
        },
      },
    ],
  },
  {
    role: "model",
    parts: [
      {
        text: "The typical lifespan of the largest land mammal, the African elephant, is around 60-70 years in the wild. Some sources suggest a maximum lifespan of 70-75 years.\n",
      },
    ],
  },
  {
    role: "user",
    parts: [
      {
        text: "thanks",
      },
    ],
  },
]

[danny-avila]

If you're using a google model via an OpenAI-compatible reverse proxy, it's up to them to interpret the OpenAI format correctily into Google specifications, as LC handles this natively when using Google directly as an endpoint.