Skip to content

Add test recordings to all Agent telemetry tests that have OpenAI only network calls #44148

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
+79 −142
Open

Add test recordings to all Agent telemetry tests that have OpenAI only network calls #44148

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
12bd84d
First
dargilco Nov 22, 2025
6ffb72e
Second
dargilco Nov 22, 2025
9afe520
More from Copilot
dargilco Nov 22, 2025
b757160
Update test assets
dargilco Nov 22, 2025
2466189
run 'black'. Address Copilot comment
dargilco Nov 22, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion sdk/ai/azure-ai-projects/assets.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,5 @@
"AssetsRepo": "Azure/azure-sdk-assets",
"AssetsRepoPrefixPath": "python",
"TagPrefix": "python/ai/azure-ai-projects",
"Tag": "python/ai/azure-ai-projects_ca3d235eb7"
"Tag": "python/ai/azure-ai-projects_f19313ab67"
}
2 changes: 1 addition & 1 deletion sdk/ai/azure-ai-projects/samples/agents/sample_agent_basic_async.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
Before running the sample:
pip install "azure-ai-projects>=2.0.0b1" python-dotenv aiohttp
pip install "azure-ai-projects>=2.0.0b1" python-dotenv aiohttp
Set these environment variables with your own values:
1) AZURE_AI_PROJECT_ENDPOINT - The Azure AI Project endpoint, as found in the Overview
Expand Down
2 changes: 1 addition & 1 deletion sdk/ai/azure-ai-projects/samples/connections/sample_connections_async.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@

Before running the sample:

pip install "azure-ai-projects>=2.0.0b1" python-dotenv aiohttp
pip install "azure-ai-projects>=2.0.0b1" python-dotenv aiohttp

Set these environment variables with your own values:
1) AZURE_AI_PROJECT_ENDPOINT - Required. The Azure AI Project endpoint, as found in the overview page of your
Expand Down
2 changes: 1 addition & 1 deletion sdk/ai/azure-ai-projects/samples/datasets/sample_datasets_async.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@

Before running the sample:

pip install "azure-ai-projects>=2.0.0b1" python-dotenv aiohttp
pip install "azure-ai-projects>=2.0.0b1" python-dotenv aiohttp

Set these environment variables with your own values:
1) AZURE_AI_PROJECT_ENDPOINT - Required. The Azure AI Project endpoint, as found in the overview page of your
Expand Down
2 changes: 1 addition & 1 deletion sdk/ai/azure-ai-projects/samples/deployments/sample_deployments_async.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@

Before running the sample:

pip install "azure-ai-projects>=2.0.0b1" python-dotenv aiohttp
pip install "azure-ai-projects>=2.0.0b1" python-dotenv aiohttp

Set these environment variables with your own values:
1) AZURE_AI_PROJECT_ENDPOINT - Required. The Azure AI Project endpoint, as found in the overview page of your
Expand Down
2 changes: 1 addition & 1 deletion sdk/ai/azure-ai-projects/samples/indexes/sample_indexes_async.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@

Before running the sample:

pip install "azure-ai-projects>=2.0.0b1" python-dotenv aiohttp
pip install "azure-ai-projects>=2.0.0b1" python-dotenv aiohttp

Set these environment variables with your own values:
1) AZURE_AI_PROJECT_ENDPOINT - Required. The Azure AI Project endpoint, as found in the overview page of your
Expand Down
2 changes: 1 addition & 1 deletion sdk/ai/azure-ai-projects/samples/responses/sample_responses_basic_async.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@

Before running the sample:

pip install "azure-ai-projects>=2.0.0b1" python-dotenv aiohttp
pip install "azure-ai-projects>=2.0.0b1" python-dotenv aiohttp

Set these environment variables with your own values:
1) AZURE_AI_PROJECT_ENDPOINT - The Azure AI Project endpoint, as found in the Overview
Expand Down
2 changes: 1 addition & 1 deletion sdk/ai/azure-ai-projects/samples/telemetry/sample_telemetry_async.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@

Before running the sample:

pip install "azure-ai-projects>=2.0.0b1" python-dotenv aiohttp
pip install "azure-ai-projects>=2.0.0b1" python-dotenv aiohttp

Set these environment variables with your own values:
1) AZURE_AI_PROJECT_ENDPOINT - The Azure AI Project endpoint, as found in the overview page of your
Expand Down
95 changes: 32 additions & 63 deletions sdk/ai/azure-ai-projects/tests/agents/telemetry/test_responses_instrumentor.py
View file
Open in desktop

Large diffs are not rendered by default.

86 changes: 29 additions & 57 deletions sdk/ai/azure-ai-projects/tests/agents/telemetry/test_responses_instrumentor_async.py
View file
Open in desktop

Large diffs are not rendered by default.

20 changes: 6 additions & 14 deletions sdk/ai/azure-ai-projects/tests/agents/telemetry/test_responses_instrumentor_metrics.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,7 @@
from opentelemetry.sdk.metrics.export import InMemoryMetricReader
from opentelemetry import metrics
from openai import OpenAI

from devtools_testutils import recorded_by_proxy

from test_base import servicePreparer
from test_base import servicePreparer, recorded_by_proxy_httpx
from test_ai_instrumentor_base import TestAiAgentsInstrumentorBase, CONTENT_TRACING_ENV_VARIABLE

settings.tracing_implementation = "OpenTelemetry"
Expand Down Expand Up @@ -62,10 +59,9 @@ def _get_metrics_data(self):

return operation_duration_metrics, token_usage_metrics

@pytest.mark.skip(reason="recordings not working for responses API")
@pytest.mark.usefixtures("instrument_with_content")
@servicePreparer()
@recorded_by_proxy
@recorded_by_proxy_httpx
def test_metrics_collection_non_streaming_responses(self, **kwargs):
"""Test that metrics are collected for non-streaming responses API calls."""
self.cleanup()
Expand Down Expand Up @@ -104,10 +100,9 @@ def test_metrics_collection_non_streaming_responses(self, **kwargs):
# The test passes if we got here without errors and the API calls worked
assert True

@pytest.mark.skip(reason="recordings not working for responses API")
@pytest.mark.usefixtures("instrument_with_content")
@servicePreparer()
@recorded_by_proxy
@recorded_by_proxy_httpx
def test_metrics_collection_streaming_responses(self, **kwargs):
"""Test that metrics are collected for streaming responses API calls."""
self.cleanup()
Expand Down Expand Up @@ -155,10 +150,9 @@ def test_metrics_collection_streaming_responses(self, **kwargs):
# The test passes if we got here without errors and streaming worked
assert True

@pytest.mark.skip(reason="recordings not working for responses API")
@pytest.mark.usefixtures("instrument_with_content")
@servicePreparer()
@recorded_by_proxy
@recorded_by_proxy_httpx
def test_metrics_collection_conversation_create(self, **kwargs):
"""Test that metrics are collected for conversation creation calls."""
self.cleanup()
Expand Down Expand Up @@ -190,10 +184,9 @@ def test_metrics_collection_conversation_create(self, **kwargs):
# The test passes if we got here without errors and the conversation was created
assert True

@pytest.mark.skip(reason="recordings not working for responses API")
@pytest.mark.usefixtures("instrument_with_content")
@servicePreparer()
@recorded_by_proxy
@recorded_by_proxy_httpx
def test_metrics_collection_multiple_operations(self, **kwargs):
"""Test that metrics are collected correctly for multiple operations."""
self.cleanup()
Expand Down Expand Up @@ -237,10 +230,9 @@ def test_metrics_collection_multiple_operations(self, **kwargs):
# The test passes if we got here without errors and multiple calls worked
assert True

@pytest.mark.skip(reason="recordings not working for responses API")
@pytest.mark.usefixtures("instrument_without_content")
@servicePreparer()
@recorded_by_proxy
@recorded_by_proxy_httpx
def test_metrics_collection_without_content_recording(self, **kwargs):
"""Test that metrics are still collected when content recording is disabled."""
self.cleanup()
Expand Down
4 changes: 4 additions & 0 deletions sdk/ai/azure-ai-projects/tsp-location.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
directory: specification/ai/Azure.AI.Projects
commit: 78bfd335c31b8764578cfb9840f3b74349c10354
repo: Azure/azure-rest-api-specs-pr
additionalDirectories:
Loading