Skip to content

Commit 8ab753c

Browse files
ralf0131ralf0131
committed
feat: Add VCR-based integration tests for Google ADK Plugin
- Introduced new integration tests using pytest-vcr to record and replay real API interactions with the Google ADK. - Created fixtures in `conftest.py` for span, log, metric exporters, and tracer providers to facilitate testing. - Added multiple test cases to validate LLM chat functionality with and without content capture, ensuring compliance with OTel GenAI attributes. - Updated requirements files to include pytest-vcr and related dependencies for testing. - Documented the usage of VCR cassettes for recording and running tests. This enhancement improves the testing framework and ensures robust coverage for Google ADK instrumentation. Change-Id: [insert-change-id-here] Change-Id: I46e41e64288cc443f1757f3e3fa895561eb72532 Co-developed-by: Cursor <noreply@cursor.com>
1 parent cb8a4ed commit 8ab753c

9 files changed

+1178
-0
lines changed

‎instrumentation-loongsuite/loongsuite-instrumentation-google-adk/tests/cassettes/README.md‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
# VCR Cassettes
2+
3+
This directory contains recorded HTTP interactions for tests using pytest-vcr.
4+
5+
## What are cassettes?
6+
7+
Cassettes are YAML files that record HTTP requests and responses from real API calls. They allow tests to:
8+
- Run without needing real API keys (after initial recording)
9+
- Run faster (no network calls)
10+
- Be deterministic (same responses every time)
11+
12+
## How to use
13+
14+
### Recording new cassettes
15+
16+
1. Set your real API key:
17+
```bash
18+
export DASHSCOPE_API_KEY="your-real-api-key"
19+
```
20+
21+
2. Delete the old cassette file (if updating):
22+
```bash
23+
rm cassettes/test_name.yaml
24+
```
25+
26+
3. Run the test:
27+
```bash
28+
pytest tests/test_vcr_integration.py::test_name -v
29+
```
30+
31+
### Using existing cassettes
32+
33+
Just run the tests normally - no API key needed:
34+
```bash
35+
pytest tests/test_vcr_integration.py -v
36+
```
37+
38+
## File naming
39+
40+
Cassette files are automatically named after the test function:
41+
- Test: `test_agent_chat_completion()`
42+
- Cassette: `cassettes/test_agent_chat_completion.yaml`
43+
44+
## Security
45+
46+
Sensitive data (API keys, tokens) are automatically scrubbed by the VCR configuration in `conftest.py`.

‎instrumentation-loongsuite/loongsuite-instrumentation-google-adk/tests/cassettes/TestGoogleAdkPluginVcrIntegration.test_llm_chat_with_content_capture.yaml‎

Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
interactions:
2+
- request:
3+
body: |-
4+
{
5+
"messages": [
6+
{
7+
"role": "system",
8+
"content": "You are a helpful assistant. Answer questions briefly and accurately.\n\nYou are an agent. Your internal name is \"test_agent\". The description about you is \"A test agent for integration testing\"."
9+
},
10+
{
11+
"role": "user",
12+
"content": "What is 2+2? Answer with just the number."
13+
}
14+
],
15+
"model": "qwen-plus",
16+
"max_tokens": 100,
17+
"temperature": 0.7
18+
}
19+
headers:
20+
accept:
21+
- application/json
22+
accept-encoding:
23+
- gzip, deflate
24+
authorization:
25+
- Bearer test_dashscope_api_key
26+
connection:
27+
- keep-alive
28+
content-length:
29+
- '369'
30+
content-type:
31+
- application/json
32+
host:
33+
- dashscope.aliyuncs.com
34+
user-agent:
35+
- AsyncOpenAI/Python 2.6.0
36+
x-stainless-arch:
37+
- arm64
38+
x-stainless-async:
39+
- async:asyncio
40+
x-stainless-lang:
41+
- python
42+
x-stainless-os:
43+
- MacOS
44+
x-stainless-package-version:
45+
- 2.6.0
46+
x-stainless-raw-response:
47+
- 'true'
48+
x-stainless-read-timeout:
49+
- '600.0'
50+
x-stainless-retry-count:
51+
- '0'
52+
x-stainless-runtime:
53+
- CPython
54+
x-stainless-runtime-version:
55+
- 3.11.14
56+
method: POST
57+
uri: https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions
58+
response:
59+
body:
60+
string: |-
61+
{
62+
"choices": [
63+
{
64+
"message": {
65+
"role": "assistant",
66+
"content": "4"
67+
},
68+
"finish_reason": "stop",
69+
"index": 0,
70+
"logprobs": null
71+
}
72+
],
73+
"object": "chat.completion",
74+
"usage": {
75+
"prompt_tokens": 64,
76+
"completion_tokens": 1,
77+
"total_tokens": 65,
78+
"prompt_tokens_details": {
79+
"cached_tokens": 0
80+
}
81+
},
82+
"created": 1766719934,
83+
"system_fingerprint": null,
84+
"model": "qwen-plus",
85+
"id": "chatcmpl-11b4b20d-b213-9237-8e63-90af36963551"
86+
}
87+
headers:
88+
Content-Type:
89+
- application/json
90+
Date:
91+
- Fri, 26 Dec 2025 03:32:13 GMT
92+
Server:
93+
- istio-envoy
94+
Transfer-Encoding:
95+
- chunked
96+
Vary:
97+
- Origin,Access-Control-Request-Method,Access-Control-Request-Headers, Accept-Encoding
98+
content-length:
99+
- '369'
100+
req-arrive-time:
101+
- '1766719933346'
102+
req-cost-time:
103+
- '567'
104+
resp-start-time:
105+
- '1766719933914'
106+
x-dashscope-call-gateway:
107+
- 'true'
108+
x-envoy-upstream-service-time:
109+
- '567'
110+
x-request-id:
111+
- 11b4b20d-b213-9237-8e63-90af36963551
112+
status:
113+
code: 200
114+
message: OK
115+
version: 1

‎instrumentation-loongsuite/loongsuite-instrumentation-google-adk/tests/cassettes/TestGoogleAdkPluginVcrIntegration.test_llm_chat_without_content_capture.yaml‎

Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
interactions:
2+
- request:
3+
body: |-
4+
{
5+
"messages": [
6+
{
7+
"role": "system",
8+
"content": "You are a helpful assistant.\n\nYou are an agent. Your internal name is \"test_agent\". The description about you is \"Test agent\"."
9+
},
10+
{
11+
"role": "user",
12+
"content": "This is sensitive data that should not be captured"
13+
}
14+
],
15+
"model": "qwen-plus",
16+
"max_tokens": 50,
17+
"temperature": 0.7
18+
}
19+
headers:
20+
accept:
21+
- application/json
22+
accept-encoding:
23+
- gzip, deflate
24+
authorization:
25+
- Bearer test_dashscope_api_key
26+
connection:
27+
- keep-alive
28+
content-length:
29+
- '310'
30+
content-type:
31+
- application/json
32+
host:
33+
- dashscope.aliyuncs.com
34+
user-agent:
35+
- AsyncOpenAI/Python 2.6.0
36+
x-stainless-arch:
37+
- arm64
38+
x-stainless-async:
39+
- async:asyncio
40+
x-stainless-lang:
41+
- python
42+
x-stainless-os:
43+
- MacOS
44+
x-stainless-package-version:
45+
- 2.6.0
46+
x-stainless-raw-response:
47+
- 'true'
48+
x-stainless-read-timeout:
49+
- '600.0'
50+
x-stainless-retry-count:
51+
- '0'
52+
x-stainless-runtime:
53+
- CPython
54+
x-stainless-runtime-version:
55+
- 3.11.14
56+
method: POST
57+
uri: https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions
58+
response:
59+
body:
60+
string: |-
61+
{
62+
"choices": [
63+
{
64+
"message": {
65+
"role": "assistant",
66+
"content": "I understand that sensitive data should not be captured or shared. I'm here to help with any non-sensitive questions or tasks you might have. Let me know how I can assist!"
67+
},
68+
"finish_reason": "stop",
69+
"index": 0,
70+
"logprobs": null
71+
}
72+
],
73+
"object": "chat.completion",
74+
"usage": {
75+
"prompt_tokens": 50,
76+
"completion_tokens": 36,
77+
"total_tokens": 86,
78+
"prompt_tokens_details": {
79+
"cached_tokens": 0
80+
}
81+
},
82+
"created": 1766719936,
83+
"system_fingerprint": null,
84+
"model": "qwen-plus",
85+
"id": "chatcmpl-d5aee07a-c7b1-96eb-bfe0-3280718240c3"
86+
}
87+
headers:
88+
Content-Type:
89+
- application/json
90+
Date:
91+
- Fri, 26 Dec 2025 03:32:15 GMT
92+
Server:
93+
- istio-envoy
94+
Transfer-Encoding:
95+
- chunked
96+
Vary:
97+
- Origin,Access-Control-Request-Method,Access-Control-Request-Headers, Accept-Encoding
98+
content-length:
99+
- '540'
100+
req-arrive-time:
101+
- '1766719934498'
102+
req-cost-time:
103+
- '1405'
104+
resp-start-time:
105+
- '1766719935904'
106+
x-dashscope-call-gateway:
107+
- 'true'
108+
x-envoy-upstream-service-time:
109+
- '1405'
110+
x-request-id:
111+
- d5aee07a-c7b1-96eb-bfe0-3280718240c3
112+
status:
113+
code: 200
114+
message: OK
115+
version: 1

‎instrumentation-loongsuite/loongsuite-instrumentation-google-adk/tests/cassettes/TestGoogleAdkPluginVcrIntegration.test_metrics_recorded.yaml‎

Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
interactions:
2+
- request:
3+
body: |-
4+
{
5+
"messages": [
6+
{
7+
"role": "system",
8+
"content": "Answer briefly\n\nYou are an agent. Your internal name is \"metrics_test_agent\". The description about you is \"Metrics test\"."
9+
},
10+
{
11+
"role": "user",
12+
"content": "Hello"
13+
}
14+
],
15+
"model": "qwen-plus",
16+
"max_tokens": 50,
17+
"temperature": 0.7
18+
}
19+
headers:
20+
accept:
21+
- application/json
22+
accept-encoding:
23+
- gzip, deflate
24+
authorization:
25+
- Bearer test_dashscope_api_key
26+
connection:
27+
- keep-alive
28+
content-length:
29+
- '261'
30+
content-type:
31+
- application/json
32+
host:
33+
- dashscope.aliyuncs.com
34+
user-agent:
35+
- AsyncOpenAI/Python 2.6.0
36+
x-stainless-arch:
37+
- arm64
38+
x-stainless-async:
39+
- async:asyncio
40+
x-stainless-lang:
41+
- python
42+
x-stainless-os:
43+
- MacOS
44+
x-stainless-package-version:
45+
- 2.6.0
46+
x-stainless-raw-response:
47+
- 'true'
48+
x-stainless-read-timeout:
49+
- '600.0'
50+
x-stainless-retry-count:
51+
- '0'
52+
x-stainless-runtime:
53+
- CPython
54+
x-stainless-runtime-version:
55+
- 3.11.14
56+
method: POST
57+
uri: https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions
58+
response:
59+
body:
60+
string: |-
61+
{
62+
"choices": [
63+
{
64+
"message": {
65+
"role": "assistant",
66+
"content": "Hello! I'm metrics_test_agent. How can I assist you today?"
67+
},
68+
"finish_reason": "stop",
69+
"index": 0,
70+
"logprobs": null
71+
}
72+
],
73+
"object": "chat.completion",
74+
"usage": {
75+
"prompt_tokens": 40,
76+
"completion_tokens": 15,
77+
"total_tokens": 55,
78+
"prompt_tokens_details": {
79+
"cached_tokens": 0
80+
}
81+
},
82+
"created": 1766719939,
83+
"system_fingerprint": null,
84+
"model": "qwen-plus",
85+
"id": "chatcmpl-1a6591bb-4e29-9e9d-9421-f14b826b3c9d"
86+
}
87+
headers:
88+
Content-Type:
89+
- application/json
90+
Date:
91+
- Fri, 26 Dec 2025 03:32:18 GMT
92+
Server:
93+
- istio-envoy
94+
Transfer-Encoding:
95+
- chunked
96+
Vary:
97+
- Origin,Access-Control-Request-Method,Access-Control-Request-Headers, Accept-Encoding
98+
content-length:
99+
- '427'
100+
req-arrive-time:
101+
- '1766719937751'
102+
req-cost-time:
103+
- '787'
104+
resp-start-time:
105+
- '1766719938538'
106+
x-dashscope-call-gateway:
107+
- 'true'
108+
x-envoy-upstream-service-time:
109+
- '786'
110+
x-request-id:
111+
- 1a6591bb-4e29-9e9d-9421-f14b826b3c9d
112+
status:
113+
code: 200
114+
message: OK
115+
version: 1

0 commit comments

Comments
 (0)