fix: resolve #425 duplicate item error when using conversationId with tools #436

alhridoy · 2025-09-04T01:51:39Z

Issue Description

Resolves #425

When using the OpenAI Agents SDK with both conversationId and tools, users encounter a BadRequestError:

BadRequestError: 400 Duplicate item found with id rs_68b8366ad2848195beb3348e5860cc81029faaa0843e7ce0. Remove duplicate items from your input and try again.

This error occurs specifically when:

Using an Agent with tools
Providing a conversationId parameter
The agent makes tool calls that require multiple API interactions

Root Cause Analysis

The issue stems from a conflict between two behaviors:

OpenAI Responses API: When conversationId is provided, the API automatically retrieves and includes the conversation history
SDK Implementation: The SDK also sends the full conversation history via the input parameter
Result: Items with identical IDs appear twice in the request, triggering the duplicate ID validation error

Solution

This PR modifies the getInputItems function in packages/agents-openai/src/openaiResponsesModel.ts to intelligently filter input when a conversationId is present:

Key Changes

Enhanced Function Signature:

function getInputItems(
  input: ModelRequest['input'],
  conversationId?: string, // New optional parameter
): OpenAI.Responses.ResponseInputItem[]

Smart Input Filtering:

// When using conversationId, filter to avoid duplicates
let filteredInput = input;
if (conversationId) {
  // Find the last user message to identify the current turn
  let lastUserMessageIndex = -1;
  for (let i = input.length - 1; i >= 0; i--) {
    const item = input[i];
    if (isMessageItem(item) && item.role === 'user') {
      lastUserMessageIndex = i;
      break;
    }
  }
  
  // Only include items from the current turn
  if (lastUserMessageIndex >= 0) {
    filteredInput = input.slice(lastUserMessageIndex);
  } else {
    filteredInput = input; // Fallback to full input
  }
}

Updated Function Call:

const input = getInputItems(request.input, request.conversationId);

How It Works

The solution identifies the "current turn" by locating the last user message in the input array. When a conversationId is provided:

With conversationId: Only sends items from the current turn (last user message onwards)
Without conversationId: Sends the complete input as before (maintains backward compatibility)

This approach ensures that:

The OpenAI API receives the conversation history via conversationId
The SDK only sends new items from the current turn
No duplicate items are transmitted

Testing

This fix resolves the error in scenarios like:

import { Agent, run, tool } from '@openai/agents';
import { z } from 'zod';
import OpenAI from 'openai';

const weatherTool = tool({
  name: 'get_weather',
  description: 'Get weather for a city',
  parameters: z.object({ city: z.string() }),
  strict: true,
  async execute({ city }) {
    return `Weather in ${city}: sunny, 72°F`;
  },
});

async function main() {
  const client = new OpenAI();
  const conversation = await client.conversations.create({});
  
  const agent = new Agent({
    name: 'WeatherBot',
    instructions: 'Help users get weather information.',
    model: 'gpt-4o',
    tools: [weatherTool],
  });

  // This now works without throwing duplicate ID error
  const result = await run(
    agent, 
    "What's the weather in San Francisco?", 
    { conversationId: conversation.id }
  );
  
  console.log(result.finalOutput);
}

Benefits

✅ Fixes the duplicate ID error when using conversationId with tools
✅ Maintains full backward compatibility - no breaking changes
✅ Reduces API payload size by eliminating duplicate conversation data
✅ Preserves conversation context through the OpenAI API's built-in history retrieval
✅ Robust fallback handling for edge cases

Impact

This change enables developers to use the full power of the OpenAI Agents SDK with persistent conversations and tools without encountering the duplicate ID error, unlocking more sophisticated conversational AI applications.

Files Changed

packages/agents-openai/src/openaiResponsesModel.ts: Modified getInputItems function to handle conversationId filtering

Related Issues: This addresses similar issues reported by users when combining conversationId with tool-enabled agents.

When using the OpenAI Agents SDK with conversationId and tools, users encounter a 'Duplicate item found with id rs_xxx' error. This happens because the OpenAI Responses API automatically retrieves conversation history when conversationId is provided, while the SDK also sends the full conversation history, causing duplicate items. This fix modifies the getInputItems function to: - Accept an optional conversationId parameter - Filter input to only include the current turn when conversationId is present - Maintain full backward compatibility The solution identifies the current turn by finding the last user message and only includes items from that point onwards, preventing duplicates while preserving conversation context. Resolves the BadRequestError and enables proper use of conversationId with tool-enabled agents.

changeset-bot · 2025-09-04T01:51:43Z

🦋 Changeset detected

Latest commit: 4f00b05

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages

Name	Type
@openai/agents-openai	Patch
@openai/agents	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

seratch · 2025-09-04T11:26:22Z

Thanks for sending this. Indeed, this could be a solution for the issue. I will look into this shortly.

seratch · 2025-09-11T10:20:32Z

@codex review this PR; especially check if finding lastUserMessageIndex is sufficient for this bug fix

chatgpt-codex-connector

Codex Review: Here are some suggestions.

Reply with @codex fix comments to fix any unresolved comments.

About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you open a pull request for review, mark a draft as ready, or comment "@codex review". If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex fix this CI failure" or "@codex address that feedback".

chatgpt-codex-connector · 2025-09-11T10:22:44Z

packages/agents-openai/src/openaiResponsesModel.ts

+  // When using conversationId, the OpenAI Responses API automatically retrieves
+  // the conversation history. To avoid duplicate items with the same IDs,
+  // we need to filter out items that would already be present in the conversation.
+  // We keep only the items from the current turn (typically the last few items).
+  let filteredInput = input;
+  if (conversationId) {
+    // Find the last user message to identify the start of the current turn
+    let lastUserMessageIndex = -1;
+    for (let i = input.length - 1; i >= 0; i--) {
+      const item = input[i];
+      if (isMessageItem(item) && item.role === 'user') {
+        lastUserMessageIndex = i;
+        break;
+      }
+    }
+
+    // If we found a user message, only include items from that point onwards
+    // This represents the current turn's conversation
+    if (lastUserMessageIndex >= 0) {
+      filteredInput = input.slice(lastUserMessageIndex);


[P1] Filtering from last user message still resends duplicated tool items

The new filtering logic slices the input array starting at the last user message, but in a tool-driven turn the items that cause the duplicate-ID error are typically the assistant’s tool call and other model-generated items that follow that user message. During the follow‑up request that posts tool results, getTurnInput still includes the original user message and the model’s tool call (with the same IDs as already stored in the conversation). Because those items fall after the last user message, this function keeps them and they are resent alongside the tool output, so the API still receives duplicates and fails. To avoid the error you need to send only the items generated since the last request (e.g., by tracking the last response ID or filtering by new item IDs), not everything after the last user message.

Useful? React with 👍 / 👎.

I think this is a valid point.

@codex can you suggest alternative approaches here?

@codex can you suggest alternative approaches here? (path=packages/agents-openai/src/openaiResponsesModel.ts line=459 side=RIGHT)

For now, I can only help with PRs you've created.

Hi @seratch would you review the latest commit?
• Added heuristics in getInputItems to detect items already persisted in the conversation
when conversationId is set, slicing the input to send only newly generated turn data and
avoid duplicate IDs.
• Extended unit coverage to verify that stored items are skipped while fresh inputs continue
to flow when no persisted metadata exists.

arekbartnik · 2025-09-18T10:40:09Z

@seratch is there something we can do in userland before the fix in library?

IgorStetsiuk · 2025-09-29T15:28:20Z

Having the same issue (

Kenth06 · 2025-09-30T13:30:22Z

is there any update ?

…ith tools

seratch added bug Something isn't working package:agents-core labels Sep 4, 2025

seratch changed the title ~~fix: resolve duplicate item error when using conversationId with tools~~ fix: resolve #425 duplicate item error when using conversationId with tools Sep 4, 2025

seratch added package:agents-openai and removed package:agents-core labels Sep 4, 2025

Create gold-vans-complain.md

00a14f2

chatgpt-codex-connector bot reviewed Sep 11, 2025

View reviewed changes

seratch marked this pull request as draft September 25, 2025 06:29

fix(agents-openai): avoid duplicate items when using conversationId w…

4f00b05

…ith tools

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix: resolve #425 duplicate item error when using conversationId with tools #436

fix: resolve #425 duplicate item error when using conversationId with tools #436

alhridoy commented Sep 4, 2025 •

edited by seratch

Loading

Uh oh!

changeset-bot bot commented Sep 4, 2025 •

edited

Loading

Uh oh!

seratch commented Sep 4, 2025

Uh oh!

seratch commented Sep 11, 2025

Uh oh!

chatgpt-codex-connector bot left a comment

Uh oh!

chatgpt-codex-connector bot Sep 11, 2025

Uh oh!

seratch Sep 12, 2025

Uh oh!

seratch Sep 12, 2025

Uh oh!

chatgpt-codex-connector bot Sep 12, 2025

Uh oh!

alhridoy Oct 1, 2025

Uh oh!

arekbartnik commented Sep 18, 2025

Uh oh!

IgorStetsiuk commented Sep 29, 2025

Uh oh!

Kenth06 commented Sep 30, 2025

Uh oh!

Uh oh!

fix: resolve #425 duplicate item error when using conversationId with tools #436

Are you sure you want to change the base?

fix: resolve #425 duplicate item error when using conversationId with tools #436

Conversation

alhridoy commented Sep 4, 2025 • edited by seratch Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Issue Description

Root Cause Analysis

Solution

Key Changes

How It Works

Testing

Benefits

Impact

Files Changed

Uh oh!

changeset-bot bot commented Sep 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

seratch commented Sep 4, 2025

Uh oh!

seratch commented Sep 11, 2025

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector bot Sep 11, 2025

Choose a reason for hiding this comment

Uh oh!

seratch Sep 12, 2025

Choose a reason for hiding this comment

Uh oh!

seratch Sep 12, 2025

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector bot Sep 12, 2025

Choose a reason for hiding this comment

Uh oh!

alhridoy Oct 1, 2025

Choose a reason for hiding this comment

Uh oh!

arekbartnik commented Sep 18, 2025

Uh oh!

IgorStetsiuk commented Sep 29, 2025

Uh oh!

Kenth06 commented Sep 30, 2025

Uh oh!

Uh oh!

alhridoy commented Sep 4, 2025 •

edited by seratch

Loading

changeset-bot bot commented Sep 4, 2025 •

edited

Loading