Skip to content

Create a utility to automatically wire-up Open Telemetry to the Google Cloud Observability suite. #380

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 14 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions opentelemetry-configurator-gcp/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
.build
build
dist
.venv

22 changes: 22 additions & 0 deletions opentelemetry-configurator-gcp/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
.PHONY: all build test clean lint install

all: build test lint

build:
./tools/build.sh

test:
./tools/test.sh

lint:
./tools/lint.sh

install: build
pip install dist/*.whl

clean:
rm -rf .build
rm -rf dist
rm -rf .test
rm -rf .tox
rm -rf .venv
60 changes: 60 additions & 0 deletions opentelemetry-configurator-gcp/README.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
OpenTelemetry Google Cloud Configurator
========================================

Purpose
-------
Simplifies configuring the Open Telemetry library to write to Google Cloud Observability backends.


Usage
-----

There are two ways that this can be used:

1. With automatic instrumentation.
2. With manual instrumentation.


Automatic Instrumentation
^^^^^^^^^^^^^^^^^^^^^^^^^

To use this component with automatic instrumentation, simply invoke
``opentelemetry-instrument`` with the argument ``--configurator=gcp``.

::

opentelemetry-instrument \
--configurator=gcp \
python \
the/path/to/your/code.py

See `Python zero-code instrumentation for Python <https://opentelemetry.io/docs/zero-code/python/>`_


Manual Instrumentation
^^^^^^^^^^^^^^^^^^^^^^

You can also call the configurator code manually.

::

from opentelemetry.configurator.gcp import OpenTelemetryGcpConfigurator

OpenTelemetryGcpConfigurator().configure()


Installation
------------

You can use a standard Python package management tool like ``pip`` or ``uv`` to install.

The PyPi package is ``opentelemetry-configurator-gcp``.

For the automatic instrumentation, you must additionally install ``opentelemetry-distro``.


References
----------

* `Python zero-code instrumentation for Python <https://opentelemetry.io/docs/zero-code/python/>`_
* `Google Cloud Observability <https://cloud.google.com/stackdriver/docs>`_
27 changes: 27 additions & 0 deletions opentelemetry-configurator-gcp/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "opentelemetry-configurator-gcp"
dynamic = ["version"]
dependencies = [
"opentelemetry-api >= 1.30.0, <2",
"opentelemetry-sdk >= 1.30.0, <2",
"opentelemetry-exporter-gcp-logging >= 1.9.0a0, <2",
"opentelemetry-exporter-gcp-trace >= 1.9.0, <2",
"opentelemetry-exporter-gcp-monitoring >= 1.9.0a0, <2",
"opentelemetry-resourcedetector-gcp >= 1.9.0a0, <2",
]

[project.entry-points.opentelemetry_configurator]
gcp = "opentelemetry.configurator.gcp:OpenTelemetryGcpConfigurator"

[tool.hatch.build.targets.sdist]
include = ["*.py"]

[tool.hatch.build.targets.wheel]
packages = ["src/opentelemetry"]

[tool.hatch.version]
path = "src/opentelemetry/configurator/gcp/version.py"
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
"""Open Telemetry configurator for Google Cloud

This package provides the 'OpenTelemetryGcpConfigurator' which simplifies configuration of the
Open Telemetry library to route logs, traces, and metrics to Google Cloud Observability products
such as Cloud Logging, Cloud Trace, and Cloud Monitoring.

The OpenTelemetryGcpConfigurator can be invoked directly as in:

from opentelemetry.configurator.gcp import OpenTelemetryGcpConfigurator

OpenTelemetryGcpConfigurator().configure()

It can also be invoked automatically from the "opentelemetry-instrument" command,
which is part of the Open Telemetry zero-code instrumentation for Python. To
invoke it automatically, simply supply "--configurator=gcp" as a commandline
flag to the "opentelemetry-instrument" command. As an example:

opentelemetry-instrument \
--configurator=gcp \
python \
the/path/to/your/script.py

This automatic wiring is implemented using the registration mechanism in "pyproject.toml";
in particular, the "[project.entry-points.opentelemetry_configurator]" entry in that file
makes this component known to the auto-instrumentation system. And it being a class
that defines a "configure(self, **kwargs)" method makes it compatible with that API.
."""

from .configurator import OpenTelemetryGcpConfigurator
from .version import __version__

__all__ = [
"OpenTelemetryGcpConfigurator",
"__version__",
]
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
"""Defines the 'OpenTelemetryGcpConfigurator' class for simplifying Open Telemetry setup for GCP."""
from typing import Optional, Callable

from .flags import (
is_metrics_exporter_enabled,
is_logs_exporter_enabled,
is_traces_exporter_enabled,
is_resource_detector_enabled,
)
from .resource import get_resource
from .logs import configure_logs_exporter
from .metrics import configure_metrics_exporter
from .traces import configure_traces_exporter


def _bool_with_flag_default(value: Optional[bool], flag_lookup: Callable[None, bool]):
if value is not None:
return value
return flag_lookup()


class OpenTelemetryGcpConfigurator: # pylint: disable=too-few-public-methods
"""A class that can be used as a configurator in Open Telemetry zero-code instrumentation."""

def __init__(
self,
metrics_exporter_enabled:Optional[bool]=None,
logs_exporter_enabled:Optional[bool]=None,
traces_exporter_enabled:Optional[bool]=None,
resource_detector_enabled:Optional[bool]=None):
"""Initialize the configurator with optional parameters to direct the behavior.

No arguments are supplied when invoked from the zero-configuration system.

Args:
metrics_exporter_enabled: whether to enable metrics export. If unset,
falls back to an environment variable, allowing this argument to
be supplied even in zero-configuration scenarios. If that, too,
is unset, then the metrics export will be enabled.

logs_exporter_enabled: whether to enable logs export. If unset,
falls back to an environment variable, allowing this argument to
be supplied even in zero-configuration scenarios. If that, too,
is unset, then the logs export will be enabled.

traces_exporter_enabled: whether to enable trace export. If unset,
falls back to an environment variable, allowing this argument to
be supplied even in zero-configuration scenarios. If that, too,
is unset, then the trace export will be enabled.

resource_detector_enabled: whether to enable the GCP resource
detector (which is useful only when the code is running in
a GCP environment). If unset, falls back to an environment variable,
allowing this argument to be supplied even in zero-configuration
scenarios. If that, too, is unset, then the code will attempt
to determine if the code is likely deployed in GCP or not
based on the environment to enable the detector or not.

Environment Variables:

The following environment variables affect the defaults:

- OTEL_GCP_METRICS_EXPORTER_ENABLED
- OTEL_GCP_LOGS_EXPORTER_ENABLED
- OTEL_GCP_TRACES_EXPORTER_ENABLED
- OTEL_GCP_RESOURCE_DETECTOR_ENABLED
"""
self._metrics_exporter_enabled = _bool_with_flag_default(
metrics_exporter_enabled, is_metrics_exporter_enabled)
self._logs_exporter_enabled = _bool_with_flag_default(
logs_exporter_enabled, is_logs_exporter_enabled)
self._traces_exporter_enabled = _bool_with_flag_default(
traces_exporter_enabled, is_traces_exporter_enabled)
self._resource_detector_enabled = _bool_with_flag_default(
resource_detector_enabled, is_resource_detector_enabled)

def configure(self, **unused_kwargs):
"""Configure the Open Telemetry library to talk to GCP backends.

This function configures the Open Telemetry library to talk to GCP
backends, subject to the class initialization parameters.

Although this class does not inherit any explicit interface, this
function should be treated like an inherited method in that its
signature is dictated by the Open Telemetry auto-instrumentation.

Uses **unused_kwargs to allow future iterations of the Open Telemetry
library to introduce additional keyword arguments without breaking.
"""
resource = get_resource(include_gcp_detector=self._resource_detector_enabled)
if self._metrics_exporter_enabled:
configure_metrics_exporter(resource=resource)
if self._logs_exporter_enabled:
configure_logs_exporter(resource=resource)
if self._traces_exporter_enabled:
configure_traces_exporter(resource=resource)
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
"""Provides functions for querying the configuration of this library.

Other modules in this package use the 'flags' library to obtain default
behaviors where the behavior has not been explicitly specified.
"""
from typing import Optional

import os

from . import gcloud_env


def _str_to_optional_bool(s: str) -> Optional[bool]:
"""Converts a string to an optional boolean.

Args:
s: the string to convert to a boolean

Returns:
A boolean if the value is clearly false or clearly true.
None if the string does not match a known true/false pattern.
"""
lower_s = s.lower()
if lower_s in ['1', 'true', 't', 'y', 'yes', 'on']:
return True
if lower_s in ['0', 'false', 'f', 'n', 'no', 'off']:
return False
return None


def _get_bool_flag_from_env(env_var_name: str, default_value:Optional[bool]=None) -> Optional[bool]:
"""Retrieves a boolean value from an environment variable.

Args:
env_var_name: The name of the environment variable to retrieve.
default_value: The value to return if unset or has a non-bool value.

Returns:
The boolean value of the environment variable if set and valid.
Otherwise, falls back to the supplied default value.
"""
s = os.getenv(env_var_name)
if s is None:
return default_value
result = _str_to_optional_bool(s)
if result is not None:
return result
return default_value


def is_metrics_exporter_enabled():
"""Returns whether to enable metrics exporting by default."""
return _get_bool_flag_from_env(
'OTEL_GCP_METRICS_EXPORTER_ENABLED',
default_value=True
)


def is_logs_exporter_enabled():
"""Returns whether to enable logs exporting by default."""
return _get_bool_flag_from_env(
'OTEL_GCP_LOGS_EXPORTER_ENABLED',
default_value=True
)


def is_traces_exporter_enabled():
"""Returns whether to enable trace exporting by default."""
return _get_bool_flag_from_env(
'OTEL_GCP_TRACES_EXPORTER_ENABLED',
default_value=True
)


def is_resource_detector_enabled():
"""Returns whether to enable the GCP resource detector by default."""
result = _get_bool_flag_from_env(
'OTEL_GCP_RESOURCE_DETECTOR_ENABLED')
if result is not None:
return result
return gcloud_env.is_running_on_gcp()
Loading