Skip to content

Conversation

@tcdent
Copy link
Contributor

@tcdent tcdent commented Mar 25, 2025

  • Send all data from Responses to the API.
  • Include prompt(s).
  • Create reusable tooling for monkey patching.
  • Implement tagging.
  • Tests for openai instrumentation.

@codecov
Copy link

codecov bot commented Mar 25, 2025

@tcdent tcdent marked this pull request as ready for review March 26, 2025 21:16
@tcdent
Copy link
Contributor Author

tcdent commented Mar 26, 2025

Tests incoming. Pls begin review in the meantime.

@tcdent tcdent requested a review from dot-agi March 26, 2025 21:17
@tcdent
Copy link
Contributor Author

tcdent commented Mar 26, 2025

95% coverage let's gooo

@areibman areibman linked an issue Mar 31, 2025 that may be closed by this pull request
Copy link
Member

@dot-agi dot-agi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requested some changes - the PR needs a cleanup in terms of the comments (a lot of them) that decrease file visibility.

Spans are showing as requests instead of calls, tools etc which should also be looked at.

Also - instrumentation should belong to the instrumentations folder outside under third-party and not here.

@dot-agi
Copy link
Member

dot-agi commented Apr 1, 2025

Also unit tests failing - must be resolved.

@dot-agi
Copy link
Member

dot-agi commented Apr 1, 2025

An example of the Web Search tool call that we must instrument -

{
    "id": "msg_67ebd3c7da648191a0c479a3a90444b707f96ed5b5391c8a",
    "content": [
        {
            "annotations": [
                {
                    "end_index": 696,
                    "start_index": 603,
                    "title": "China's AI race creates tension at home",
                    "type": "url_citation",
                    "url": "https://www.ft.com/content/5192a8b1-a71e-409c-8edf-1fcccb9539ec?utm_source=openai"
                },
                {
                    "end_index": 1171,
                    "start_index": 1078,
                    "title": "BBC to use AI as it expands iPlayer offerings",
                    "type": "url_citation",
                    "url": "https://www.ft.com/content/89f476e2-a9b0-4f84-bb24-0e6a2fcd51b2?utm_source=openai"
                },
                {
                    "end_index": 1719,
                    "start_index": 1607,
                    "title": "Security teams embrace agentic AI",
                    "type": "url_citation",
                    "url": "https://www.axios.com/2025/03/27/agentic-ai-cybersecurity-microsoft-crowdstrike?utm_source=openai"
                },
                {
                    "end_index": 2266,
                    "start_index": 2090,
                    "title": "China's Zhipu AI launches free AI agent, intensifying domestic tech race",
                    "type": "url_citation",
                    "url": "https://www.reuters.com/technology/artificial-intelligence/chinas-zhipu-ai-launches-free-ai-agent-intensifying-domestic-tech-race-2025-03-31/?utm_source=openai"
                },
                {
                    "end_index": 3049,
                    "start_index": 2783,
                    "title": "Anime lessons in the limits of AI",
                    "type": "url_citation",
                    "url": "https://www.ft.com/content/291de779-2765-4a5c-997e-7139b766f105?utm_source=openai"
                },
                {
                    "end_index": 3049,
                    "start_index": 2783,
                    "title": "I've got bad news for you if you use ChatGPT, Perplexity, or Gemini as your main search tool - AI web search isn't worth your time, yet",
                    "type": "url_citation",
                    "url": "https://www.techradar.com/computing/artificial-intelligence/ive-got-bad-news-if-you-use-chatgpt-or-any-other-ai-as-your-main-search-tool?utm_source=openai"
                },
                {
                    "end_index": 3208,
                    "start_index": 3084,
                    "title": "China's AI race creates tension at home",
                    "type": "url_citation",
                    "url": "https://www.ft.com/content/5192a8b1-a71e-409c-8edf-1fcccb9539ec?utm_source=openai"
                },
                {
                    "end_index": 3341,
                    "start_index": 3211,
                    "title": "BBC to use AI as it expands iPlayer offerings",
                    "type": "url_citation",
                    "url": "https://www.ft.com/content/89f476e2-a9b0-4f84-bb24-0e6a2fcd51b2?utm_source=openai"
                },
                {
                    "end_index": 3478,
                    "start_index": 3344,
                    "title": "Security teams embrace agentic AI",
                    "type": "url_citation",
                    "url": "https://www.axios.com/2025/03/27/agentic-ai-cybersecurity-microsoft-crowdstrike?utm_source=openai"
                }
            ],
            "text": "Here are some recent developments in the field of artificial intelligence:\n\n**Advancements in Chinese AI Models**\n\nChinese tech companies are rapidly advancing their AI capabilities. DeepSeek's V3 model and Alibaba's Qwen series have shown significant improvements, operating efficiently even on mobile devices. This progress has intensified domestic competition, with leaders like Alibaba and Tencent gaining investor confidence, while others like Baidu face challenges. The success of these models is crucial for future growth, cloud platforms, enterprise services, and government contracts in China. ([ft.com](https://www.ft.com/content/5192a8b1-a71e-409c-8edf-1fcccb9539ec?utm_source=openai))\n\n**BBC Integrates AI into Services**\n\nThe BBC plans to incorporate AI into its iPlayer, Sounds, and sports coverage, aiming to offer personalized services to users. Internally, AI will assist in translating World Service content and creating live text pages for sports broadcasts. The BBC emphasizes maintaining its public service objectives without solely focusing on popularity. ([ft.com](https://www.ft.com/content/89f476e2-a9b0-4f84-bb24-0e6a2fcd51b2?utm_source=openai))\n\n**Security Teams Embrace Agentic AI**\n\nTo manage increasing threats and workloads, security teams are adopting agentic AI, which can take pre-approved actions based on analysis. Companies like Microsoft and CrowdStrike have integrated this technology into their platforms, enabling features such as automatic triage of notifications. While there's growing confidence in these systems, some skepticism about their reliability remains. ([axios.com](https://www.axios.com/2025/03/27/agentic-ai-cybersecurity-microsoft-crowdstrike?utm_source=openai))\n\n**Zhipu AI Launches Free AI Agent**\n\nChinese startup Zhipu AI has introduced AutoGLM Rumination, a free AI agent capable of tasks like web searches, travel planning, and research report writing. Powered by Zhipu's models, it rivals DeepSeek's R1 in performance but operates faster and requires fewer resources. This launch intensifies competition in China's AI market. ([reuters.com](https://www.reuters.com/technology/artificial-intelligence/chinas-zhipu-ai-launches-free-ai-agent-intensifying-domestic-tech-race-2025-03-31/?utm_source=openai))\n\n**Limitations of AI in Art and Search**\n\nRecent advancements in AI have led to an increase in AI-generated imitations of renowned artists' works. However, these creations often lack the intricate details and emotions of the originals, highlighting the limitations of AI in art. Additionally, AI tools like ChatGPT, Perplexity, and Google's Gemini are increasingly used for web searches but often present inaccurate information, indicating that AI is not yet ready to replace traditional search engines effectively. ([ft.com](https://www.ft.com/content/291de779-2765-4a5c-997e-7139b766f105?utm_source=openai), [techradar.com](https://www.techradar.com/computing/artificial-intelligence/ive-got-bad-news-if-you-use-chatgpt-or-any-other-ai-as-your-main-search-tool?utm_source=openai))\n\n\n## Recent Developments in AI:\n- [China's AI race creates tension at home](https://www.ft.com/content/5192a8b1-a71e-409c-8edf-1fcccb9539ec?utm_source=openai)\n- [BBC to use AI as it expands iPlayer offerings](https://www.ft.com/content/89f476e2-a9b0-4f84-bb24-0e6a2fcd51b2?utm_source=openai)\n- [Security teams embrace agentic AI](https://www.axios.com/2025/03/27/agentic-ai-cybersecurity-microsoft-crowdstrike?utm_source=openai) ",
            "type": "output_text"
        }
    ],
    "role": "assistant",
    "status": "completed",
    "type": "message"
}

@tcdent
Copy link
Contributor Author

tcdent commented Apr 1, 2025

Also unit tests failing - must be resolved.

Unit tests were passing and were broken when you pushed c38e03e

@dot-agi dot-agi force-pushed the openai-responses branch from 4c39766 to 2af8d96 Compare April 2, 2025 18:57
@dot-agi
Copy link
Member

dot-agi commented Apr 2, 2025

New notebooks added -

  • examples/openai_responses/web_search.ipynb
  • examples/openai_responses/multi_tool_orchestration.ipynb

@dot-agi dot-agi self-requested a review April 6, 2025 04:53
@bboynton97 bboynton97 self-requested a review April 7, 2025 15:41
@dot-agi dot-agi merged commit b6b984e into main Apr 7, 2025
8 of 10 checks passed
@dot-agi dot-agi deleted the openai-responses branch April 7, 2025 22:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

OpenAI Responses support

4 participants