Smarter, better, easier Django initialization (#77)

* Smarter, better, easier Django initialization

* Report Django middleware; Detect py console

* Priority to newer Django conventions; store list of middleware

* Add Python flavor and architecture into snapshot

* Default list type.

* Standardize on import style; clean up load order

* Move to use only one autowrapt entry point.

* Better version comparison.

* runtime is now deprecated; remove the file.

* Update documentation to follow changes.

Author: pglombardo

INSTALLATION.md

@@ -12,50 +12,70 @@ or to alternatively update an existing installation:
 
 # Automated
 
-The Instana package sensor can be enabled without any code modifications.  To do this, set the following environment variable for your Python application:
+The Instana package sensor can be enabled without any code modifications required.  To do this, install the package and set the following environment variable for your Python application:
 
-    AUTOWRAPT_BOOTSTRAP=runtime
+    AUTOWRAPT_BOOTSTRAP=instana
 
-This will cause the Instana Python package to automatically instrument your Python application.  Once it finds the host agent, it will begin to report Python metrics.
+This will cause the Instana Python package to automatically instrument your Python application.  Once it finds the Instana host agent, it will begin to report Python metrics.
 
 # Manual
 
-In any Python 2.7 or great application, to manually enable the Instana sensor, simply import the package:
+In any Python 2.7 or greater application, to manually enable the Instana sensor, simply import the package:
 
     import instana
 
-This will initialize the package and it will begin to report key Python metrics.
+# Flask
 
-# Django (Automated)
+To enable the Flask instrumentation, set the following environment variable in your _application boot environment_ and then restart your application:
 
-The Instana package offers a method to automatically instrument your Django application without any code changes required.  To enable the Django instrumetation, set the environment variable `AUTOWRAPT_BOOTSTRAP=django` (use `django19` for Django version 1.9.x) for your Python application.
+  `export AUTOWRAPT_BOOTSTRAP=flask`
+
+# WSGI Stacks
 
-# Django (Manual Installation)
+The Instana sensor includes WSGI middleware that can be added to any WSGI compliant stack.  This is automated for various stacks but can also be done manually for those we haven't added support for yet.
 
-To instead manually install the Instana Django middleware into your Django application, import the package and add `instana.django.InstanaMiddleware` to the top of your `MIDDLEWARE` (or `MIDDLEWARE_CLASSES` for earlier versions) in your `settings.py`:
+The general usage is:
 
-```Python
+```python
 import instana
+from instana.wsgi import iWSGIMiddleware
 
-MIDDLEWARE = [
-    'instana.django.InstanaMiddleware',
-    # ...
-    'django.contrib.messages.middleware.MessageMiddleware',
-    'django.middleware.clickjacking.XFrameOptionsMiddleware',
-]
+# Wrap the wsgi app in Instana middleware (iWSGIMiddleware)
+wsgiapp = iWSGIMiddleware(MyWSGIApplication())
 ```
 
-# Flask
+We are working to automate this for all major frameworks but in the meantime, here are some specific quick starts for those we don't have automatic support for yet.
 
-To enable the Flask instrumentation, set the following environment variable in your _application boot environment_ and then restart your application:
+## CherryPy WSGI
 
-  `export AUTOWRAPT_BOOTSTRAP=flask`
+```python
+import cherrypy
+import instana
+from instana.wsgi import iWSGIMiddleware
 
-# WSGI Stacks
+# My CherryPy application
+class Root(object):
+    @cherrypy.expose
+    def index(self):
+        return "hello world"
+
+cherrypy.config.update({'engine.autoreload.on': False})
+cherrypy.server.unsubscribe()
+cherrypy.engine.start()
+
+# Wrap the wsgi app in Instana middleware (iWSGIMiddleware)
+wsgiapp = iWSGIMiddleware(cherrypy.tree.mount(Root()))
+```
+
+In this example, we use uwsgi as the webserver and booted with:
+
+    uwsgi --socket 127.0.0.1:8080 --protocol=http --wsgi-file mycherry.py --callable wsgiapp -H ~/.local/share/virtualenvs/cherrypyapp-C1BUba0z
+
+Where `~/.local/share/virtualenvs/cherrypyapp-C1BUba0z` is the path to my local virtualenv from pipenv
 
-The Instana sensor bundles with it WSGI middleware.  The usage of this middleware is automated for various frameworks but for those that arent' supported yet, see the [WSGI documentation](WSGI.md) for details on how to manually add it to your stack.
+# uWSGI Webserver
 
-# uWSGI
+tldr; Make sure `enable-threads` and `lazy-apps` is enabled for uwsgi.
 
 ## Threads
 

README.md

@@ -19,7 +19,15 @@ The instana package will automatically collect key metrics from your Python proc
 
 `pip install instana` into the virtual-env or container ([hosted on pypi](https://pypi.python.org/pypi/instana))
 
-See our detailed [Installation document](INSTALLATION.md) for environment specific information covering Django, Flask, End-user Monitoring (EUM) and more.
+The Instana package can then be activated _without any code changes required_ by setting the following environment variable for your Python application:
+
+    export AUTOWRAPT_BOOTSTRAP=instana
+
+alternatively, if you prefer the manual method, simply import the `instana` package inside of your Python application:
+
+    import instana
+
+See our detailed [Installation document](INSTALLATION.md) for additional information covering Django, Flask, End-user Monitoring (EUM) and more.
 
 ## OpenTracing
 
@@ -47,8 +55,7 @@ Also note that under evented systems such as gevent, concurrence and/or greenlet
 
 ## Configuration
 
-See [Configuration.md](https://github.com/instana/python-sensor/blob/master/Configuration.md)
-
+For details on how to configure the Instana Python package, see [Configuration.md](https://github.com/instana/python-sensor/blob/master/Configuration.md)
 
 ## Documentation
 

WSGI.md

@@ -5,12 +5,6 @@
 from .tracer import InstanaTracer
 from .options import Options
 
-if "INSTANA_DISABLE_AUTO_INSTR" not in os.environ:
-    # Import & initialize instrumentation
-    # noqa: ignore=W0611
-    from .instrumentation import urllib3  # noqa
-    from .instrumentation import sudsjurko  # noqa
-
 """
 The Instana package has two core components: the sensor and the tracer.
 
@@ -29,6 +23,18 @@
 __maintainer__ = 'Peter Giacomo Lombardo'
 __email__ = '[email protected]'
 
+
+def load(module):
+    """
+    Method used to activate the Instana sensor via AUTOWRAPT_BOOTSTRAP
+    environment variable.
+    """
+    if "INSTANA_DEV" in os.environ:
+        print("==========================================================")
+        print("Instana: Loading...")
+        print("==========================================================")
+
+
 # For any given Python process, we only want one sensor as multiple would
 # collect/report metrics in duplicate, triplicate etc..
 #
@@ -65,3 +71,10 @@
 
 if "INSTANA_SERVICE_NAME" in os.environ:
     service_name = os.environ["INSTANA_SERVICE_NAME"]
+
+if "INSTANA_DISABLE_AUTO_INSTR" not in os.environ:
+    # Import & initialize instrumentation
+    # noqa: ignore=W0611
+    from .instrumentation import urllib3  # noqa
+    from .instrumentation import sudsjurko  # noqa
+    from .instrumentation.django import middleware  # noqa

instana/__init__.py

@@ -1,13 +1,14 @@
-from __future__ import print_function
+from __future__ import absolute_import
 import opentracing as ot
-from instana import internal_tracer
-from instana.log import logger
 import opentracing.ext.tags as ext
 import wrapt
-import os
+import sys
+from ...log import logger
+from ... import internal_tracer
 
 
-DJ_INSTANA_MIDDLEWARE = 'instana.django.InstanaMiddleware'
+
+DJ_INSTANA_MIDDLEWARE = 'instana.instrumentation.django.middleware.InstanaMiddleware'
 
 try:
     from django.utils.deprecation import MiddlewareMixin
@@ -63,65 +64,50 @@ def process_exception(self, request, exception):
             ec = self.span.tags.get('ec', 0)
             self.span.set_tag("ec", ec+1)
 
+
 def load_middleware_wrapper(wrapped, instance, args, kwargs):
     try:
         from django.conf import settings
 
         # Django >=1.10 to <2.0 support old-style MIDDLEWARE_CLASSES so we
         # do as well here
-        if hasattr(settings, 'MIDDLEWARE_CLASSES'):
-            if DJ_INSTANA_MIDDLEWARE in settings.MIDDLEWARE_CLASSES:
-                return wrapped(*args, **kwargs)
-
-            if type(settings.MIDDLEWARE_CLASSES) is tuple:
-                settings.MIDDLEWARE_CLASSES = (DJ_INSTANA_MIDDLEWARE,) + settings.MIDDLEWARE_CLASSES
-            elif type(settings.MIDDLEWARE_CLASSES) is list:
-                settings.MIDDLEWARE_CLASSES = [DJ_INSTANA_MIDDLEWARE] + settings.MIDDLEWARE_CLASSES
-            else:
-                logger.warn("Instana: Couldn't add InstanaMiddleware to Django")
-        elif hasattr(settings, 'MIDDLEWARE'):
+        if hasattr(settings, 'MIDDLEWARE'):
             if DJ_INSTANA_MIDDLEWARE in settings.MIDDLEWARE:
                 return wrapped(*args, **kwargs)
 
+            # Save the list of middleware for Snapshot reporting
+            internal_tracer.sensor.meter.djmw = settings.MIDDLEWARE
+
             if type(settings.MIDDLEWARE) is tuple:
                 settings.MIDDLEWARE = (DJ_INSTANA_MIDDLEWARE,) + settings.MIDDLEWARE
             elif type(settings.MIDDLEWARE) is list:
                 settings.MIDDLEWARE = [DJ_INSTANA_MIDDLEWARE] + settings.MIDDLEWARE
             else:
                 logger.warn("Instana: Couldn't add InstanaMiddleware to Django")
-        else:
-            logger.warn("Instana: Couldn't find middleware settings")
 
-        return wrapped(*args, **kwargs)
-    except Exception as e:
-            logger.warn("Instana: Couldn't add InstanaMiddleware to Django: ", e)
+        elif hasattr(settings, 'MIDDLEWARE_CLASSES'):
+            if DJ_INSTANA_MIDDLEWARE in settings.MIDDLEWARE_CLASSES:
+                return wrapped(*args, **kwargs)
 
-def hook(module):
-    """ Hook method to install the Instana middleware into Django >= 1.10 """
-    if "INSTANA_DEV" in os.environ:
-        print("==============================================================")
-        print("Instana: Running django hook")
-        print("==============================================================")
-    wrapt.wrap_function_wrapper(module, 'BaseHandler.load_middleware', load_middleware_wrapper)
+            # Save the list of middleware for Snapshot reporting
+            internal_tracer.sensor.meter.djmw = settings.MIDDLEWARE_CLASSES
 
+            if type(settings.MIDDLEWARE_CLASSES) is tuple:
+                settings.MIDDLEWARE_CLASSES = (DJ_INSTANA_MIDDLEWARE,) + settings.MIDDLEWARE_CLASSES
+            elif type(settings.MIDDLEWARE_CLASSES) is list:
+                settings.MIDDLEWARE_CLASSES = [DJ_INSTANA_MIDDLEWARE] + settings.MIDDLEWARE_CLASSES
+            else:
+                logger.warn("Instana: Couldn't add InstanaMiddleware to Django")
 
-def hook19(module):
-    try:
-        """ Hook method to install the Instana middleware into Django <= 1.9 """
-        if "INSTANA_DEV" in os.environ:
-            print("==============================================================")
-            print("Instana: Running django19 hook")
-            print("==============================================================")
-
-        if DJ_INSTANA_MIDDLEWARE in module.settings.MIDDLEWARE_CLASSES:
-            return
-
-        if type(module.settings.MIDDLEWARE_CLASSES) is tuple:
-            module.settings.MIDDLEWARE_CLASSES = (DJ_INSTANA_MIDDLEWARE,) + module.settings.MIDDLEWARE_CLASSES
-        elif type(module.settings.MIDDLEWARE_CLASSES) is list:
-            module.settings.MIDDLEWARE_CLASSES = [DJ_INSTANA_MIDDLEWARE] + module.settings.MIDDLEWARE_CLASSES
         else:
-            logger.warn("Instana: Couldn't add InstanaMiddleware to Django")
+            logger.warn("Instana: Couldn't find middleware settings")
 
+        return wrapped(*args, **kwargs)
     except Exception as e:
             logger.warn("Instana: Couldn't add InstanaMiddleware to Django: ", e)
+
+try:
+    if 'django' in sys.modules:
+        wrapt.wrap_function_wrapper('django.core.handlers.base', 'BaseHandler.load_middleware', load_middleware_wrapper)
+except:
+    pass

instana/instrumentation/django/__init__.py

@@ -1,34 +1,35 @@
 from __future__ import absolute_import
-import instana
-from instana.log import logger
+from distutils.version import LooseVersion
 import opentracing
 import opentracing.ext.tags as ext
 import wrapt
+from ..log import logger
+from .. import internal_tracer
 
 
 try:
     import suds # noqa
 
-    if (suds.version.__version__ <= '0.6'):
+    if (LooseVersion(suds.version.__version__) <= LooseVersion('0.6')):
         class_method = 'SoapClient.send'
     else:
         class_method = '_SoapClient.send'
 
     @wrapt.patch_function_wrapper('suds.client', class_method)
     def send_with_instana(wrapped, instance, args, kwargs):
-        context = instana.internal_tracer.current_context()
+        context = internal_tracer.current_context()
 
         # If we're not tracing, just return
         if context is None:
             return wrapped(*args, **kwargs)
 
         try:
-            span = instana.internal_tracer.start_span("soap", child_of=context)
+            span = internal_tracer.start_span("soap", child_of=context)
             span.set_tag('soap.action', instance.method.name)
             span.set_tag(ext.HTTP_URL, instance.method.location)
             span.set_tag(ext.HTTP_METHOD, 'POST')
 
-            instana.internal_tracer.inject(span.context, opentracing.Format.HTTP_HEADERS,
+            internal_tracer.inject(span.context, opentracing.Format.HTTP_HEADERS,
                                            instance.options.headers)
 
             rv = wrapped(*args, **kwargs)