Check potel sync #4639

sentrivana · 2025-07-29T14:10:23Z

No description provided.

Regular tox update

Update our OpenAI integration to support new APIs (`/responses`) and be OTel and Sentry AI Agents insights module compatible. Contains: - #4563 - #4564 - #4628 --------- Co-authored-by: Ivana Kellyer <[email protected]>

Fixes #4636

AI prompts/messages are potentially huge. * raise `DEFAULT_MAX_VALUE_LENGTH` (responsible for string trimming) from 1024 to 100 000 * adapt tests (and make them more generic, without hardcoded parts, where possible)

codecov · 2025-07-29T14:13:12Z

❌ 72 Tests Failed:

Tests completed	Failed	Passed	Skipped
20698	72	20626	1098

View the top 3 failed test(s) by shortest run time

tests.integrations.openai.test_openai::test_ai_client_span_responses_api

Stack Traces | 0.11s run time

.../integrations/openai/test_openai.py:1041: in test_ai_client_span_responses_api
    assert spans[0]["data"] == {
E   assert {'sentry.op': 'gen_ai.responses', 'sentry.source': <TransactionSource.CUSTOM: 'custom'>, 'sentry.origin': 'auto.ai.openai', 'sentry.name': 'responses gpt-4o', 'thread.id': '139959599934336', 'thread.name': 'MainThread', 'gen_ai.request.messages': 'How do I check if a Python object is an instance of a class?', 'gen_ai.system': 'openai', 'gen_ai.operation.name': 'responses', 'gen_ai.request.model': 'gpt-4o', 'gen_ai.response.model': 'response-model-id', 'gen_ai.response.text': '[{"id": "message-id", "content": [{"annotations": [], "text": "the model response", "type": "output_text"}], "role": "assistant", "status": "completed", "type": "message"}]', 'gen_ai.usage.input_tokens': 20, 'gen_ai.usage.input_tokens.cached': 5, 'gen_ai.usage.output_tokens': 10, 'gen_ai.usage.output_tokens.reasoning': 8, 'gen_ai.usage.total_tokens': 30} == {'gen_ai.operation.name': 'responses', 'gen_ai.request.messages': 'How do I check if a Python object is an instance of a class?', 'gen_ai.request.model': 'gpt-4o', 'gen_ai.system': 'openai', 'gen_ai.response.model': 'response-model-id', 'gen_ai.usage.input_tokens': 20, 'gen_ai.usage.input_tokens.cached': 5, 'gen_ai.usage.output_tokens': 10, 'gen_ai.usage.output_tokens.reasoning': 8, 'gen_ai.usage.total_tokens': 30, 'gen_ai.response.text': '[{"id": "message-id", "content": [{"annotations": [], "text": "the model response", "type": "output_text"}], "role": "assistant", "status": "completed", "type": "message"}]', 'thread.id': <ANY>, 'thread.name': <ANY>}
E     
E     Common items:
E     {'gen_ai.operation.name': 'responses',
E      'gen_ai.request.messages': 'How do I check if a Python object is an instance '
E                                 'of a class?',
E      'gen_ai.request.model': 'gpt-4o',
E      'gen_ai.response.model': 'response-model-id',
E      'gen_ai.response.text': '[{"id": "message-id", "content": [{"annotations": '
E                              '[], "text": "the model response", "type": '
E                              '"output_text"}], "role": "assistant", "status": '
E                              '"completed", "type": "message"}]',
E      'gen_ai.system': 'openai',
E      'gen_ai.usage.input_tokens': 20,
E      'gen_ai.usage.input_tokens.cached': 5,
E      'gen_ai.usage.output_tokens': 10,
E      'gen_ai.usage.output_tokens.reasoning': 8,
E      'gen_ai.usage.total_tokens': 30,
E      'thread.id': '139959599934336',
E      'thread.name': 'MainThread'}
E     Left contains 4 more items:
E     {'sentry.name': 'responses gpt-4o',
E      'sentry.op': 'gen_ai.responses',
E      'sentry.origin': 'auto.ai.openai',
E      'sentry.source': <TransactionSource.CUSTOM: 'custom'>}
E     
E     Full diff:
E       {
E           'gen_ai.operation.name': 'responses',
E           'gen_ai.request.messages': 'How do I check if a Python object is an instance of a class?',
E           'gen_ai.request.model': 'gpt-4o',
E           'gen_ai.response.model': 'response-model-id',
E           'gen_ai.response.text': '[{"id": "message-id", "content": [{"annotations": [], "text": "the model '
E           'response", "type": "output_text"}], "role": "assistant", "status": '
E           '"completed", "type": "message"}]',
E           'gen_ai.system': 'openai',
E           'gen_ai.usage.input_tokens': 20,
E           'gen_ai.usage.input_tokens.cached': 5,
E           'gen_ai.usage.output_tokens': 10,
E           'gen_ai.usage.output_tokens.reasoning': 8,
E           'gen_ai.usage.total_tokens': 30,
E     -     'thread.id': <ANY>,
E     -     'thread.name': <ANY>,
E     +     'sentry.name': 'responses gpt-4o',
E     +     'sentry.op': 'gen_ai.responses',
E     +     'sentry.origin': 'auto.ai.openai',
E     +     'sentry.source': <TransactionSource.CUSTOM: 'custom'>,
E     +     'thread.id': '139959599934336',
E     +     'thread.name': 'MainThread',
E       }

tests.integrations.openai.test_openai::test_ai_client_span_streaming_responses_async_api

Stack Traces | 0.112s run time

.../integrations/openai/test_openai.py:1167: in test_ai_client_span_streaming_responses_async_api
    assert spans[0]["data"] == {
E   assert {'sentry.op': 'gen_ai.responses', 'sentry.source': <TransactionSource.CUSTOM: 'custom'>, 'sentry.origin': 'auto.ai.openai', 'sentry.name': 'responses gpt-4o', 'thread.id': '139959599934336', 'thread.name': 'MainThread', 'gen_ai.request.messages': 'How do I check if a Python object is an instance of a class?', 'gen_ai.system': 'openai', 'gen_ai.operation.name': 'responses', 'gen_ai.request.model': 'gpt-4o', 'gen_ai.response.streaming': True, 'gen_ai.response.model': 'response-model-id', 'gen_ai.response.text': '[{"id": "message-id", "content": [{"annotations": [], "text": "the model response", "type": "output_text"}], "role": "assistant", "status": "completed", "type": "message"}]', 'gen_ai.usage.input_tokens': 20, 'gen_ai.usage.input_tokens.cached': 5, 'gen_ai.usage.output_tokens': 10, 'gen_ai.usage.output_tokens.reasoning': 8, 'gen_ai.usage.total_tokens': 30} == {'gen_ai.operation.name': 'responses', 'gen_ai.request.messages': 'How do I check if a Python object is an instance of a class?', 'gen_ai.request.model': 'gpt-4o', 'gen_ai.response.model': 'response-model-id', 'gen_ai.response.streaming': True, 'gen_ai.system': 'openai', 'gen_ai.usage.input_tokens': 20, 'gen_ai.usage.input_tokens.cached': 5, 'gen_ai.usage.output_tokens': 10, 'gen_ai.usage.output_tokens.reasoning': 8, 'gen_ai.usage.total_tokens': 30, 'gen_ai.response.text': '[{"id": "message-id", "content": [{"annotations": [], "text": "the model response", "type": "output_text"}], "role": "assistant", "status": "completed", "type": "message"}]', 'thread.id': <ANY>, 'thread.name': <ANY>}
E     
E     Common items:
E     {'gen_ai.operation.name': 'responses',
E      'gen_ai.request.messages': 'How do I check if a Python object is an instance '
E                                 'of a class?',
E      'gen_ai.request.model': 'gpt-4o',
E      'gen_ai.response.model': 'response-model-id',
E      'gen_ai.response.streaming': True,
E      'gen_ai.response.text': '[{"id": "message-id", "content": [{"annotations": '
E                              '[], "text": "the model response", "type": '
E                              '"output_text"}], "role": "assistant", "status": '
E                              '"completed", "type": "message"}]',
E      'gen_ai.system': 'openai',
E      'gen_ai.usage.input_tokens': 20,
E      'gen_ai.usage.input_tokens.cached': 5,
E      'gen_ai.usage.output_tokens': 10,
E      'gen_ai.usage.output_tokens.reasoning': 8,
E      'gen_ai.usage.total_tokens': 30,
E      'thread.id': '139959599934336',
E      'thread.name': 'MainThread'}
E     Left contains 4 more items:
E     {'sentry.name': 'responses gpt-4o',
E      'sentry.op': 'gen_ai.responses',
E      'sentry.origin': 'auto.ai.openai',
E      'sentry.source': <TransactionSource.CUSTOM: 'custom'>}
E     
E     Full diff:
E       {
E           'gen_ai.operation.name': 'responses',
E           'gen_ai.request.messages': 'How do I check if a Python object is an instance of a class?',
E           'gen_ai.request.model': 'gpt-4o',
E           'gen_ai.response.model': 'response-model-id',
E           'gen_ai.response.streaming': True,
E           'gen_ai.response.text': '[{"id": "message-id", "content": [{"annotations": [], "text": "the model '
E           'response", "type": "output_text"}], "role": "assistant", "status": '
E           '"completed", "type": "message"}]',
E           'gen_ai.system': 'openai',
E           'gen_ai.usage.input_tokens': 20,
E           'gen_ai.usage.input_tokens.cached': 5,
E           'gen_ai.usage.output_tokens': 10,
E           'gen_ai.usage.output_tokens.reasoning': 8,
E           'gen_ai.usage.total_tokens': 30,
E     -     'thread.id': <ANY>,
E     -     'thread.name': <ANY>,
E     +     'sentry.name': 'responses gpt-4o',
E     +     'sentry.op': 'gen_ai.responses',
E     +     'sentry.origin': 'auto.ai.openai',
E     +     'sentry.source': <TransactionSource.CUSTOM: 'custom'>,
E     +     'thread.id': '139959599934336',
E     +     'thread.name': 'MainThread',
E       }

tests.integrations.openai.test_openai::test_ai_client_span_responses_async_api

Stack Traces | 0.113s run time

.../integrations/openai/test_openai.py:1121: in test_ai_client_span_responses_async_api
    assert spans[0]["data"] == {
E   assert {'sentry.op': 'gen_ai.responses', 'sentry.source': <TransactionSource.CUSTOM: 'custom'>, 'sentry.origin': 'auto.ai.openai', 'sentry.name': 'responses gpt-4o', 'thread.id': '139959599934336', 'thread.name': 'MainThread', 'gen_ai.request.messages': 'How do I check if a Python object is an instance of a class?', 'gen_ai.system': 'openai', 'gen_ai.operation.name': 'responses', 'gen_ai.request.model': 'gpt-4o', 'gen_ai.response.model': 'response-model-id', 'gen_ai.response.text': '[{"id": "message-id", "content": [{"annotations": [], "text": "the model response", "type": "output_text"}], "role": "assistant", "status": "completed", "type": "message"}]', 'gen_ai.usage.input_tokens': 20, 'gen_ai.usage.input_tokens.cached': 5, 'gen_ai.usage.output_tokens': 10, 'gen_ai.usage.output_tokens.reasoning': 8, 'gen_ai.usage.total_tokens': 30} == {'gen_ai.operation.name': 'responses', 'gen_ai.request.messages': 'How do I check if a Python object is an instance of a class?', 'gen_ai.request.model': 'gpt-4o', 'gen_ai.response.model': 'response-model-id', 'gen_ai.system': 'openai', 'gen_ai.usage.input_tokens': 20, 'gen_ai.usage.input_tokens.cached': 5, 'gen_ai.usage.output_tokens': 10, 'gen_ai.usage.output_tokens.reasoning': 8, 'gen_ai.usage.total_tokens': 30, 'gen_ai.response.text': '[{"id": "message-id", "content": [{"annotations": [], "text": "the model response", "type": "output_text"}], "role": "assistant", "status": "completed", "type": "message"}]', 'thread.id': <ANY>, 'thread.name': <ANY>}
E     
E     Common items:
E     {'gen_ai.operation.name': 'responses',
E      'gen_ai.request.messages': 'How do I check if a Python object is an instance '
E                                 'of a class?',
E      'gen_ai.request.model': 'gpt-4o',
E      'gen_ai.response.model': 'response-model-id',
E      'gen_ai.response.text': '[{"id": "message-id", "content": [{"annotations": '
E                              '[], "text": "the model response", "type": '
E                              '"output_text"}], "role": "assistant", "status": '
E                              '"completed", "type": "message"}]',
E      'gen_ai.system': 'openai',
E      'gen_ai.usage.input_tokens': 20,
E      'gen_ai.usage.input_tokens.cached': 5,
E      'gen_ai.usage.output_tokens': 10,
E      'gen_ai.usage.output_tokens.reasoning': 8,
E      'gen_ai.usage.total_tokens': 30,
E      'thread.id': '139959599934336',
E      'thread.name': 'MainThread'}
E     Left contains 4 more items:
E     {'sentry.name': 'responses gpt-4o',
E      'sentry.op': 'gen_ai.responses',
E      'sentry.origin': 'auto.ai.openai',
E      'sentry.source': <TransactionSource.CUSTOM: 'custom'>}
E     
E     Full diff:
E       {
E           'gen_ai.operation.name': 'responses',
E           'gen_ai.request.messages': 'How do I check if a Python object is an instance of a class?',
E           'gen_ai.request.model': 'gpt-4o',
E           'gen_ai.response.model': 'response-model-id',
E           'gen_ai.response.text': '[{"id": "message-id", "content": [{"annotations": [], "text": "the model '
E           'response", "type": "output_text"}], "role": "assistant", "status": '
E           '"completed", "type": "message"}]',
E           'gen_ai.system': 'openai',
E           'gen_ai.usage.input_tokens': 20,
E           'gen_ai.usage.input_tokens.cached': 5,
E           'gen_ai.usage.output_tokens': 10,
E           'gen_ai.usage.output_tokens.reasoning': 8,
E           'gen_ai.usage.total_tokens': 30,
E     -     'thread.id': <ANY>,
E     -     'thread.name': <ANY>,
E     +     'sentry.name': 'responses gpt-4o',
E     +     'sentry.op': 'gen_ai.responses',
E     +     'sentry.origin': 'auto.ai.openai',
E     +     'sentry.source': <TransactionSource.CUSTOM: 'custom'>,
E     +     'thread.id': '139959599934336',
E     +     'thread.name': 'MainThread',
E       }

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}

sl0thentr0py and others added 9 commits July 28, 2025 15:19

Expose set_transaction_name (#4634)

070653a

tests: tox.ini update (#4635)

19ed1bb

Regular tox update

OpenAI integration update (#4612)

fcace85

Update our OpenAI integration to support new APIs (`/responses`) and be OTel and Sentry AI Agents insights module compatible. Contains: - #4563 - #4564 - #4628 --------- Co-authored-by: Ivana Kellyer <[email protected]>

fix(celery): Latency should be in milliseconds, not seconds (#4637)

fd7dca4

Fixes #4636

Considerably raise DEFAULT_MAX_VALUE_LENGTH (#4632)

4f9d326

AI prompts/messages are potentially huge. * raise `DEFAULT_MAX_VALUE_LENGTH` (responsible for string trimming) from 1024 to 100 000 * adapt tests (and make them more generic, without hardcoded parts, where possible)

release: 2.34.0

e84f6f3

Update CHANGELOG.md

e1848d4

Update changelog

72766a7

Merge branch 'release/2.34.0'

c273f67

sentrivana force-pushed the ivana/potel/check-sync branch from 3620b6d to c273f67 Compare July 29, 2025 14:59

sentrivana closed this Jul 30, 2025

sentrivana deleted the ivana/potel/check-sync branch October 20, 2025 07:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Check potel sync #4639

Check potel sync #4639

Uh oh!

sentrivana commented Jul 29, 2025

Uh oh!

codecov bot commented Jul 29, 2025 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Check potel sync #4639

Check potel sync #4639

Uh oh!

Conversation

sentrivana commented Jul 29, 2025

Uh oh!

codecov bot commented Jul 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

❌ 72 Tests Failed:

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

codecov bot commented Jul 29, 2025 •

edited

Loading