Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions agentops/instrumentation/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,11 @@ def get_instance(self) -> BaseInstrumentor:
class_name="OpenAIAgentsInstrumentor",
provider_import_name="agents",
),
InstrumentorLoader(
module_name="agentops.instrumentation.google_generativeai",
class_name="GoogleGenerativeAIInstrumentor",
provider_import_name="google.genai",
),
]


Expand Down
33 changes: 33 additions & 0 deletions agentops/instrumentation/google_generativeai/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Google Generative AI (Gemini) Instrumentation

This module provides OpenTelemetry instrumentation for Google's Generative AI (Gemini) API. The instrumentation allows you to trace all API calls made using the `google-genai` Python SDK, capturing:

- Model parameters (temperature, max_tokens, etc.)
- Prompt content (with privacy controls)
- Response text and token usage
- Streaming metrics
- Token counting
- Performance and error data

## Supported Features

The instrumentation covers all major API methods including:

### Client-Based API
- `client.models.generate_content`
- `client.models.generate_content_stream`
- `client.models.count_tokens`
- `client.models.compute_tokens`
- And their corresponding async variants

## Metrics

The instrumentation captures the following metrics:

- Input tokens used
- Output tokens generated
- Total tokens consumed
- Operation duration
- Exception counts

These metrics are available as OpenTelemetry span attributes and can be viewed in your observability platform of choice when properly configured.
38 changes: 38 additions & 0 deletions agentops/instrumentation/google_generativeai/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
"""Google Generative AI (Gemini) API instrumentation.

This module provides instrumentation for the Google Generative AI (Gemini) API,
including content generation, streaming, and chat functionality.
"""

import logging
from typing import Collection

Check warning on line 8 in agentops/instrumentation/google_generativeai/__init__.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/__init__.py#L7-L8

Added lines #L7 - L8 were not covered by tests

def get_version() -> str:

Check warning on line 10 in agentops/instrumentation/google_generativeai/__init__.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/__init__.py#L10

Added line #L10 was not covered by tests
"""Get the version of the Google Generative AI SDK, or 'unknown' if not found

Attempts to retrieve the installed version of the Google Generative AI SDK using importlib.metadata.
Falls back to 'unknown' if the version cannot be determined.

Returns:
The version string of the Google Generative AI SDK or 'unknown'
"""
try:
from importlib.metadata import version
return version("google-genai")
except ImportError:
logger.debug("Could not find Google Generative AI SDK version")
return "unknown"

Check warning on line 24 in agentops/instrumentation/google_generativeai/__init__.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/__init__.py#L19-L24

Added lines #L19 - L24 were not covered by tests

LIBRARY_NAME = "google-genai"
LIBRARY_VERSION: str = get_version()

Check warning on line 27 in agentops/instrumentation/google_generativeai/__init__.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/__init__.py#L26-L27

Added lines #L26 - L27 were not covered by tests

logger = logging.getLogger(__name__)

Check warning on line 29 in agentops/instrumentation/google_generativeai/__init__.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/__init__.py#L29

Added line #L29 was not covered by tests

# Import after defining constants to avoid circular imports
from agentops.instrumentation.google_generativeai.instrumentor import GoogleGenerativeAIInstrumentor # noqa: E402

Check warning on line 32 in agentops/instrumentation/google_generativeai/__init__.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/__init__.py#L32

Added line #L32 was not covered by tests

__all__ = [

Check warning on line 34 in agentops/instrumentation/google_generativeai/__init__.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/__init__.py#L34

Added line #L34 was not covered by tests
"LIBRARY_NAME",
"LIBRARY_VERSION",
"GoogleGenerativeAIInstrumentor",
]
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
"""Attribute extractors for Google Generative AI instrumentation."""

from agentops.instrumentation.google_generativeai.attributes.common import (

Check warning on line 3 in agentops/instrumentation/google_generativeai/attributes/__init__.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/__init__.py#L3

Added line #L3 was not covered by tests
get_common_instrumentation_attributes,
extract_request_attributes,
)
from agentops.instrumentation.google_generativeai.attributes.model import (

Check warning on line 7 in agentops/instrumentation/google_generativeai/attributes/__init__.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/__init__.py#L7

Added line #L7 was not covered by tests
get_model_attributes,
get_generate_content_attributes,
get_stream_attributes,
get_token_counting_attributes,
)
from agentops.instrumentation.google_generativeai.attributes.chat import (

Check warning on line 13 in agentops/instrumentation/google_generativeai/attributes/__init__.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/__init__.py#L13

Added line #L13 was not covered by tests
get_chat_attributes,
)

__all__ = [

Check warning on line 17 in agentops/instrumentation/google_generativeai/attributes/__init__.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/__init__.py#L17

Added line #L17 was not covered by tests
"get_common_instrumentation_attributes",
"extract_request_attributes",
"get_model_attributes",
"get_generate_content_attributes",
"get_stream_attributes",
"get_chat_attributes",
"get_token_counting_attributes",
]
125 changes: 125 additions & 0 deletions agentops/instrumentation/google_generativeai/attributes/chat.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,125 @@
"""Chat attribute extraction for Google Generative AI instrumentation."""

from typing import Dict, Any, Optional, Tuple, List, Union

Check warning on line 3 in agentops/instrumentation/google_generativeai/attributes/chat.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/chat.py#L3

Added line #L3 was not covered by tests

from agentops.logging import logger
from agentops.semconv import SpanAttributes, LLMRequestTypeValues, MessageAttributes
from agentops.instrumentation.common.attributes import AttributeMap
from agentops.instrumentation.google_generativeai.attributes.common import (

Check warning on line 8 in agentops/instrumentation/google_generativeai/attributes/chat.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/chat.py#L5-L8

Added lines #L5 - L8 were not covered by tests
extract_request_attributes,
get_common_instrumentation_attributes,
)
from agentops.instrumentation.google_generativeai.attributes.model import (

Check warning on line 12 in agentops/instrumentation/google_generativeai/attributes/chat.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/chat.py#L12

Added line #L12 was not covered by tests
_extract_content_from_prompt,
_set_response_attributes,
)


def _extract_message_content(message: Any) -> str:

Check warning on line 18 in agentops/instrumentation/google_generativeai/attributes/chat.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/chat.py#L18

Added line #L18 was not covered by tests
"""Extract text content from a chat message.

Handles the various message formats in the Gemini chat API.

Args:
message: The message to extract content from

Returns:
Extracted text as a string
"""
if isinstance(message, str):
return message

Check warning on line 30 in agentops/instrumentation/google_generativeai/attributes/chat.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/chat.py#L29-L30

Added lines #L29 - L30 were not covered by tests

if isinstance(message, dict):
if "content" in message:
return _extract_content_from_prompt(message["content"])
if "text" in message:
return message["text"]

Check warning on line 36 in agentops/instrumentation/google_generativeai/attributes/chat.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/chat.py#L32-L36

Added lines #L32 - L36 were not covered by tests

if hasattr(message, "content"):
return _extract_content_from_prompt(message.content)

Check warning on line 39 in agentops/instrumentation/google_generativeai/attributes/chat.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/chat.py#L38-L39

Added lines #L38 - L39 were not covered by tests

if hasattr(message, "text"):
return message.text

Check warning on line 42 in agentops/instrumentation/google_generativeai/attributes/chat.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/chat.py#L41-L42

Added lines #L41 - L42 were not covered by tests

return ""

Check warning on line 44 in agentops/instrumentation/google_generativeai/attributes/chat.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/chat.py#L44

Added line #L44 was not covered by tests


def _set_chat_history_attributes(attributes: AttributeMap, args: Tuple, kwargs: Dict[str, Any]) -> None:

Check warning on line 47 in agentops/instrumentation/google_generativeai/attributes/chat.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/chat.py#L47

Added line #L47 was not covered by tests
"""Extract and set chat history attributes from the request.

Args:
attributes: The attribute dictionary to update
args: Positional arguments to the method
kwargs: Keyword arguments to the method
"""
messages = []
if 'message' in kwargs:
messages = [kwargs['message']]
elif args and len(args) > 0:
messages = [args[0]]
elif 'messages' in kwargs:
messages = kwargs['messages']

Check warning on line 61 in agentops/instrumentation/google_generativeai/attributes/chat.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/chat.py#L55-L61

Added lines #L55 - L61 were not covered by tests

if not messages:
return

Check warning on line 64 in agentops/instrumentation/google_generativeai/attributes/chat.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/chat.py#L63-L64

Added lines #L63 - L64 were not covered by tests

for i, message in enumerate(messages):
try:
content = _extract_message_content(message)
if content:
role = "user"

Check warning on line 70 in agentops/instrumentation/google_generativeai/attributes/chat.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/chat.py#L66-L70

Added lines #L66 - L70 were not covered by tests

if isinstance(message, dict) and "role" in message:
role = message["role"]
elif hasattr(message, "role"):
role = message.role

Check warning on line 75 in agentops/instrumentation/google_generativeai/attributes/chat.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/chat.py#L72-L75

Added lines #L72 - L75 were not covered by tests

attributes[MessageAttributes.PROMPT_CONTENT.format(i=i)] = content
attributes[MessageAttributes.PROMPT_ROLE.format(i=i)] = role
except Exception as e:
logger.debug(f"Error extracting chat message at index {i}: {e}")

Check warning on line 80 in agentops/instrumentation/google_generativeai/attributes/chat.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/chat.py#L77-L80

Added lines #L77 - L80 were not covered by tests


def get_chat_attributes(

Check warning on line 83 in agentops/instrumentation/google_generativeai/attributes/chat.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/chat.py#L83

Added line #L83 was not covered by tests
args: Optional[Tuple] = None,
kwargs: Optional[Dict[str, Any]] = None,
return_value: Optional[Any] = None,
) -> AttributeMap:
"""Extract attributes for chat session methods.

This function handles attribute extraction for chat session operations,
particularly the send_message method.

Args:
args: Positional arguments to the method
kwargs: Keyword arguments to the method
return_value: Return value from the method

Returns:
Dictionary of extracted attributes
"""
attributes = get_common_instrumentation_attributes()
attributes[SpanAttributes.LLM_SYSTEM] = "Gemini"
attributes[SpanAttributes.LLM_REQUEST_TYPE] = LLMRequestTypeValues.CHAT.value

Check warning on line 103 in agentops/instrumentation/google_generativeai/attributes/chat.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/chat.py#L101-L103

Added lines #L101 - L103 were not covered by tests

if kwargs:
kwargs_attributes = extract_request_attributes(kwargs)
attributes.update(kwargs_attributes)

Check warning on line 107 in agentops/instrumentation/google_generativeai/attributes/chat.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/chat.py#L105-L107

Added lines #L105 - L107 were not covered by tests

chat_session = None
if args and len(args) >= 1:
chat_session = args[0]

Check warning on line 111 in agentops/instrumentation/google_generativeai/attributes/chat.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/chat.py#L109-L111

Added lines #L109 - L111 were not covered by tests

if chat_session and hasattr(chat_session, "model"):
if isinstance(chat_session.model, str):
attributes[SpanAttributes.LLM_REQUEST_MODEL] = chat_session.model
elif hasattr(chat_session.model, "name"):
attributes[SpanAttributes.LLM_REQUEST_MODEL] = chat_session.model.name

Check warning on line 117 in agentops/instrumentation/google_generativeai/attributes/chat.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/chat.py#L113-L117

Added lines #L113 - L117 were not covered by tests

if args or kwargs:
_set_chat_history_attributes(attributes, args or (), kwargs or {})

Check warning on line 120 in agentops/instrumentation/google_generativeai/attributes/chat.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/chat.py#L119-L120

Added lines #L119 - L120 were not covered by tests

if return_value is not None:
_set_response_attributes(attributes, return_value)

Check warning on line 123 in agentops/instrumentation/google_generativeai/attributes/chat.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/chat.py#L122-L123

Added lines #L122 - L123 were not covered by tests

return attributes

Check warning on line 125 in agentops/instrumentation/google_generativeai/attributes/chat.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/chat.py#L125

Added line #L125 was not covered by tests
80 changes: 80 additions & 0 deletions agentops/instrumentation/google_generativeai/attributes/common.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
"""Common attribute extraction for Google Generative AI instrumentation."""

from typing import Dict, Any, Optional

Check warning on line 3 in agentops/instrumentation/google_generativeai/attributes/common.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/common.py#L3

Added line #L3 was not covered by tests

from agentops.logging import logger
from agentops.semconv import InstrumentationAttributes, SpanAttributes, LLMRequestTypeValues
from agentops.instrumentation.common.attributes import AttributeMap, get_common_attributes, _extract_attributes_from_mapping
from agentops.instrumentation.google_generativeai import LIBRARY_NAME, LIBRARY_VERSION

Check warning on line 8 in agentops/instrumentation/google_generativeai/attributes/common.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/common.py#L5-L8

Added lines #L5 - L8 were not covered by tests

# Common mapping for config parameters
REQUEST_CONFIG_ATTRIBUTES: AttributeMap = {

Check warning on line 11 in agentops/instrumentation/google_generativeai/attributes/common.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/common.py#L11

Added line #L11 was not covered by tests
SpanAttributes.LLM_REQUEST_TEMPERATURE: "temperature",
SpanAttributes.LLM_REQUEST_MAX_TOKENS: "max_output_tokens",
SpanAttributes.LLM_REQUEST_TOP_P: "top_p",
SpanAttributes.LLM_REQUEST_TOP_K: "top_k",
SpanAttributes.LLM_REQUEST_SEED: "seed",
SpanAttributes.LLM_REQUEST_SYSTEM_INSTRUCTION: "system_instruction",
SpanAttributes.LLM_REQUEST_PRESENCE_PENALTY: "presence_penalty",
SpanAttributes.LLM_REQUEST_FREQUENCY_PENALTY: "frequency_penalty",
SpanAttributes.LLM_REQUEST_STOP_SEQUENCES: "stop_sequences",
SpanAttributes.LLM_REQUEST_CANDIDATE_COUNT: "candidate_count",
}

def get_common_instrumentation_attributes() -> AttributeMap:

Check warning on line 24 in agentops/instrumentation/google_generativeai/attributes/common.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/common.py#L24

Added line #L24 was not covered by tests
"""Get common instrumentation attributes for the Google Generative AI instrumentation.

This combines the generic AgentOps attributes with Google Generative AI specific library attributes.

Returns:
Dictionary of common instrumentation attributes
"""
attributes = get_common_attributes()
attributes.update({

Check warning on line 33 in agentops/instrumentation/google_generativeai/attributes/common.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/common.py#L32-L33

Added lines #L32 - L33 were not covered by tests
InstrumentationAttributes.LIBRARY_NAME: LIBRARY_NAME,
InstrumentationAttributes.LIBRARY_VERSION: LIBRARY_VERSION,
})
return attributes

Check warning on line 37 in agentops/instrumentation/google_generativeai/attributes/common.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/common.py#L37

Added line #L37 was not covered by tests


def extract_request_attributes(kwargs: Dict[str, Any]) -> AttributeMap:

Check warning on line 40 in agentops/instrumentation/google_generativeai/attributes/common.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/common.py#L40

Added line #L40 was not covered by tests
"""Extract request attributes from the function arguments.

Extracts common request parameters that apply to both content generation
and chat completions, focusing on model parameters and generation settings.

Args:
kwargs: Request keyword arguments

Returns:
Dictionary of extracted request attributes
"""
attributes = {}

Check warning on line 52 in agentops/instrumentation/google_generativeai/attributes/common.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/common.py#L52

Added line #L52 was not covered by tests

if 'model' in kwargs:
model = kwargs["model"]

Check warning on line 55 in agentops/instrumentation/google_generativeai/attributes/common.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/common.py#L54-L55

Added lines #L54 - L55 were not covered by tests

# Handle string model names
if isinstance(model, str):
attributes[SpanAttributes.LLM_REQUEST_MODEL] = model

Check warning on line 59 in agentops/instrumentation/google_generativeai/attributes/common.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/common.py#L58-L59

Added lines #L58 - L59 were not covered by tests
# Handle model objects with _model_name or name attribute
elif hasattr(model, '_model_name'):
attributes[SpanAttributes.LLM_REQUEST_MODEL] = model._model_name
elif hasattr(model, 'name'):
attributes[SpanAttributes.LLM_REQUEST_MODEL] = model.name

Check warning on line 64 in agentops/instrumentation/google_generativeai/attributes/common.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/common.py#L61-L64

Added lines #L61 - L64 were not covered by tests

config = kwargs.get('config')

Check warning on line 66 in agentops/instrumentation/google_generativeai/attributes/common.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/common.py#L66

Added line #L66 was not covered by tests

if config:
try:
attributes.update(_extract_attributes_from_mapping(

Check warning on line 70 in agentops/instrumentation/google_generativeai/attributes/common.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/common.py#L68-L70

Added lines #L68 - L70 were not covered by tests
config.__dict__ if hasattr(config, '__dict__') else config,
REQUEST_CONFIG_ATTRIBUTES
))
except Exception as e:
logger.debug(f"Error extracting config parameters: {e}")

Check warning on line 75 in agentops/instrumentation/google_generativeai/attributes/common.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/common.py#L74-L75

Added lines #L74 - L75 were not covered by tests

if 'stream' in kwargs:
attributes[SpanAttributes.LLM_REQUEST_STREAMING] = kwargs['stream']

Check warning on line 78 in agentops/instrumentation/google_generativeai/attributes/common.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/common.py#L77-L78

Added lines #L77 - L78 were not covered by tests

return attributes

Check warning on line 80 in agentops/instrumentation/google_generativeai/attributes/common.py

View check run for this annotation

Codecov / codecov/patch

agentops/instrumentation/google_generativeai/attributes/common.py#L80

Added line #L80 was not covered by tests
Loading
Loading