Implemented Opentracing bridge (#388)

This implements a functional, albeit still limited OpenTracing bridge.
See Caveats section in the docs for a list of limitations.

closes #388
closes #249

commit a1e9582
Author: Benjamin Wohlwend <[email protected]>
Date:   Mon Jan 21 13:31:00 2019 +0100

    rename context to slightly less overloaded execution_context

commit b6e5a27
Author: Benjamin Wohlwend <[email protected]>
Date:   Mon Jan 21 12:09:52 2019 +0100

    make the internal context API a bit more explicit

Author: beniwohli

CHANGELOG.md

@@ -4,6 +4,7 @@
 [Check the diff](https://github.com/elastic/apm-agent-python/compare/v4.0.3...master)
 
  * Added support for collecting system and process metrics (#361)
+ * Added an OpenTracing bridge (#388)
  * Added `transaction.sampled` to errors (#371)
  * Added `transaction.type` to errors (#391)
  * Added parsing of `/proc/self/cgroup` to capture container meta data (#352)

docs/index.asciidoc

@@ -19,6 +19,8 @@ include::./flask.asciidoc[Flask support]
 
 include::./metrics.asciidoc[Metrics]
 
+include::./opentracing.asciidoc[OpenTracing]
+
 include::./advanced-topics.asciidoc[Advanced Topics]
 
 include::./api.asciidoc[API documentation]

docs/opentracing.asciidoc

@@ -0,0 +1,134 @@
+ifdef::env-github[]
+NOTE: For the best reading experience,
+please view this documentation at https://www.elastic.co/guide/en/apm/agent/java[elastic.co]
+endif::[]
+
+[[opentracing-bridge]]
+== Elastic APM OpenTracing bridge
+
+The Elastic APM OpenTracing bridge allows to create Elastic APM `Transactions` and `Spans`,
+using the OpenTracing API.
+In other words,
+it translates the calls to the OpenTracing API to Elastic APM and thus allows for reusing existing instrumentation.
+
+The first span of a service will be converted to an Elastic APM
+{apm-overview-ref-v}/transactions.html[`Transaction`],
+subsequent spans are mapped to Elastic APM
+{apm-overview-ref-v}/transaction-spans.html[`Span`].
+
+[float]
+[[opentracing-getting-started]]
+=== Getting started
+The first step in getting started with the OpenTracing API bridge is to install the `opentracing` library:
+
+[source,bash]
+----
+pip install elastic-apm[opentracing]
+----
+
+Or if you already have installed `elastic-apm`
+
+
+[source,bash]
+----
+pip install opentracing>=2.0.0
+----
+
+
+[float]
+[[opentracing-init-tracer]]
+=== Initialize tracer
+
+[source,python]
+----
+from elasticapm.contrib.opentracing import Tracer
+
+tracer = Tracer();
+----
+
+`Tracer` accepts the following optional arguments:
+
+  * `client_instance`: an already instantiated Elastic APM client
+  * `config`: a configuration dictionary, which will be used to instantiate a new Elastic APM client, 
+     e.g. `{"SERVER_URL": "https://example.org"}. See <<configuration, configuration>> for more information.
+  * `scope_manager`: a scope manager instance. Defaults to `ThreadLocalScopeManager` (see 
+
+
+[float]
+[[opentracing-elastic-apm-tags]]
+=== Elastic APM specific tags
+
+Elastic APM defines some tags which are not included in the OpenTracing API but are relevant in the context of Elastic APM.
+
+- `type` - sets the type of the transaction,
+  for example `request`, `ext` or `db`
+- `user.id` - sets the user id,
+  appears in the "User" tab in the transaction details in the Elastic APM UI
+- `user.email` - sets the user email,
+  appears in the "User" tab in the transaction details in the Elastic APM UI
+- `user.username` - sets the user name,
+  appears in the "User" tab in the transaction details in the Elastic APM UI
+- `result` - sets the result of the transaction. Overrides the default value of `success`.
+
+NOTE: these tags need to be set on the first span of an operation (e.g. an incoming request of your webapp).
+
+[float]
+[[opentracing-caveats]]
+=== Caveats
+Not all features of the OpenTracing API are supported.
+
+[float]
+[[opentracing-scope-managers]]
+==== Scope Managers
+Currently, only the `ThreadLocalScopeManager` is supported.
+Using other scope managers will lead to unpredictable and possibly app-breaking behaviour.
+
+[float]
+[[opentracing-instrumentation]]
+==== Instrumentation
+
+It is recommended to not use the built-in instrumentations of Elastic APM together with third-party OpenTracing instrumentations
+like https://pypi.org/project/opentracing_instrumentation/[opentracing_instrumentation] in the same service.
+If you would like to use such instrumentations, we recommend to disable the built-in instrumentation using the <<config-instrument,`instrument`>> config option.
+
+[float]
+[[opentracing-propagation]]
+==== Context propagation
+This bridge only supports the formats `Format.Builtin.TEXT_MAP` and `Format.Builtin.HTTP_HEADERS`.
+`Format.Builtin.BINARY` is currently not supported.
+
+[float]
+[[opentracing-references]]
+==== Span References
+Currently, this bridge only supports `child_of` references.
+Other references,
+like `follows_from` are not supported yet.
+
+[float]
+[[opentracing-baggage]]
+==== Baggage
+The `Span.set_baggage_item(key, value)` method is not supported.
+Baggage items are silently dropped.
+
+[float]
+[[opentracing-logs]]
+==== Logs
+Only exception logging is supported.
+Logging an Exception on the OpenTracing span will create an Elastic APM
+{apm-overview-ref-v}/errors.html[`Error`].
+Example:
+
+[source,python]
+----
+with tracer.start_active_span("transaction") as tx_scope:
+    try:
+        raise ValueError("oops")
+    except ValueError:
+        exc_type, exc_val, exc_tb = sys.exc_info()[:3]
+        tx_scope.span.log_kv({
+            "python.exception.type": exc_type,
+            "python.exception.val": exc_val,
+            "python.exception.tb": exc_tb
+        })
+----
+

elasticapm/base.py

@@ -25,7 +25,7 @@
 from elasticapm.conf import Config, constants
 from elasticapm.conf.constants import ERROR
 from elasticapm.metrics.base_metrics import MetricsRegistry
-from elasticapm.traces import Tracer, get_transaction
+from elasticapm.traces import Tracer, execution_context
 from elasticapm.utils import cgroup, compat, is_master_process, stacks, varmap
 from elasticapm.utils.encoding import keyword_field, shorten, transform
 from elasticapm.utils.module_import import import_string
@@ -292,7 +292,7 @@ def _build_msg_for_logging(
         """
         Captures, processes and serializes an event into a dict object
         """
-        transaction = get_transaction()
+        transaction = execution_context.get_transaction()
         if transaction:
             transaction_context = deepcopy(transaction.context)
         else:

elasticapm/context/base.py

@@ -0,0 +1,12 @@
+class BaseContext(object):
+    def set_transaction(self, transaction):
+        raise NotImplementedError
+
+    def get_transaction(self, clear=False):
+        raise NotImplementedError
+
+    def set_span(self, span):
+        raise NotImplementedError
+
+    def get_span(self):
+        raise NotImplementedError

elasticapm/context/contextvars.py

@@ -1,31 +1,33 @@
 from __future__ import absolute_import
 
 import contextvars
+from elasticapm.context.base import BaseContext
 
-elasticapm_transaction_var = contextvars.ContextVar("elasticapm_transaction_var")
-elasticapm_span_var = contextvars.ContextVar("elasticapm_span_var")
 
+class ContextVarsContext(BaseContext):
+    elasticapm_transaction_var = contextvars.ContextVar("elasticapm_transaction_var")
+    elasticapm_span_var = contextvars.ContextVar("elasticapm_span_var")
 
-def get_transaction(clear=False):
-    try:
-        transaction = elasticapm_transaction_var.get()
-        if clear:
-            set_transaction(None)
-        return transaction
-    except LookupError:
-        return None
+    def get_transaction(self, clear=False):
+        try:
+            transaction = self.elasticapm_transaction_var.get()
+            if clear:
+                self.set_transaction(None)
+            return transaction
+        except LookupError:
+            return None
 
+    def set_transaction(self, transaction):
+        self.elasticapm_transaction_var.set(transaction)
 
-def set_transaction(transaction):
-    elasticapm_transaction_var.set(transaction)
+    def get_span(self):
+        try:
+            return self.elasticapm_span_var.get()
+        except LookupError:
+            return None
 
+    def set_span(self, span):
+        self.elasticapm_span_var.set(span)
 
-def get_span():
-    try:
-        return elasticapm_span_var.get()
-    except LookupError:
-        return None
 
-
-def set_span(span):
-    elasticapm_span_var.set(span)
+execution_context = ContextVarsContext()

elasticapm/context/threadlocal.py

@@ -1,30 +1,33 @@
 import threading
 
-thread_local = threading.local()
-thread_local.transaction = None
-elasticapm_span_var = None
+from elasticapm.context.base import BaseContext
 
 
-def get_transaction(clear=False):
-    """
-    Get the transaction registered for the current thread.
+class ThreadLocalContext(BaseContext):
+    thread_local = threading.local()
+    thread_local.transaction = None
+    thread_local.span = None
 
-    :return:
-    :rtype: Transaction
-    """
-    transaction = getattr(thread_local, "transaction", None)
-    if clear:
-        thread_local.transaction = None
-    return transaction
+    def get_transaction(self, clear=False):
+        """
+        Get the transaction registered for the current thread.
 
+        :return:
+        :rtype: Transaction
+        """
+        transaction = getattr(self.thread_local, "transaction", None)
+        if clear:
+            self.thread_local.transaction = None
+        return transaction
 
-def set_transaction(transaction):
-    thread_local.transaction = transaction
+    def set_transaction(self, transaction):
+        self.thread_local.transaction = transaction
 
+    def get_span(self):
+        return getattr(self.thread_local, "span", None)
 
-def get_span():
-    return getattr(thread_local, "span", None)
+    def set_span(self, span):
+        self.thread_local.span = span
 
 
-def set_span(span):
-    thread_local.span = span
+execution_context = ThreadLocalContext()

elasticapm/contrib/django/context_processors.py

@@ -1,8 +1,8 @@
-from elasticapm.traces import get_transaction
+from elasticapm.traces import execution_context
 
 
 def rum_tracing(request):
-    transaction = get_transaction()
+    transaction = execution_context.get_transaction()
     if transaction and transaction.trace_parent:
         return {
             "apm": {

elasticapm/contrib/flask/__init__.py

@@ -22,7 +22,7 @@
 from elasticapm.conf import constants, setup_logging
 from elasticapm.contrib.flask.utils import get_data_from_request, get_data_from_response
 from elasticapm.handlers.logging import LoggingHandler
-from elasticapm.traces import get_transaction
+from elasticapm.traces import execution_context
 from elasticapm.utils import build_name_with_http_method_prefix
 from elasticapm.utils.disttracing import TraceParent
 
@@ -144,7 +144,7 @@ def rum_tracing():
             """
             Adds APM related IDs to the context used for correlating the backend transaction with the RUM transaction
             """
-            transaction = get_transaction()
+            transaction = execution_context.get_transaction()
             if transaction and transaction.trace_parent:
                 return {
                     "apm": {

elasticapm/contrib/opentracing/__init__.py

@@ -0,0 +1,2 @@
+from .span import OTSpan  # noqa: F401
+from .tracer import Tracer  # noqa: F401