[flask] rewrite Flask integration (#667)

* [flask] start work on new patch implementation

* [flask] trace more internal parts

* [flask] trace more things

* [flask] round out prototype of new tracing

* [flask] replace old patch.py with new_patch.py

* [flask] finish up prototype and deduplicate some code

* [flask] move wrapper helpers to wrappers.py

* [flask] add docstrings

* [flask] remove unused import

* [flask] Update documentation

* [flask] < 0.12.0 does not have Flask.finalize_request

* [flask] handle status code as a string

* [flask] use config API

* [flask] update version parsing

* [flask] patch signal receivers_for and add unpatch()

* [flask] add test for Flask signals

* [flask] fix patching/unpatching lists

* [flask] use template name as span resource name

* [flask] set test template directory

* [flask] add test cases for Flask helpers

* [flask] add test cases for Flask render functions

* [flask] add test helpers to check if something is wrapped

* [flask] simplify pin cloning logic

* [flask] add blueprint tests

* [flask] rename patch.py to monkey.py

* [flask] make sure do to do Pin(tracer=self.tracer) in tests

* [flask] fix spelling

* [flask] add patch/unpatch idempotency tests

* [flask] add initial support for v0.9

* [flask] update tests for v0.9

* [flask] use <= (0, 9) instead of == (0, 9)

* [flask] fix signal names

* [flask] add assertion message

* [flask] skip send_from_directory

* [flask] add find_span_by_name test helper

* [flask] add error handler test cases

* [flask] Use start_response instead of Flask.finalize_request for response code

* [flask] assert bytes equality

* [flask] add test caes for flask.views.View

* [flask] enable by default

* [flask] remove large TODO comment

* [flask] change 404 resource name to '<method> 404'

* [flask] fix py2 vs py3 base exception name

* [flask] add request lifecycle tests

* [flask] support unicode

* [flask] rewrite Flask autopatch tests

* [flask] run py27-flask09 tests in circleci

* [flask] add static file tests

* [flask] rename monkey.py back to patch.py

* [flask] update docstring for flask being enabled by default

* [flask] fix comments and docstrings

* [flask] use ddtrace.utils.importlib.func_name

* [core] modify Pin.get_from to accept multiple objects

* [flask] fix remaining get_arg_or_kwargs

* [flask] only use '<method> 404' if the endpoint is unknown

* [flask] use request.path for http.URL tag

* [flask] remove/fix TODOs

* [flask] Add Pin.find(*objs) helper

* [flask] only use 'def _wrap()' where necessary

* [flask] mark 5xx errors by default, allow config of additional error codes

* Update tests/contrib/flask/test_request.py

Co-Authored-By: brettlangdon <[email protected]>

* Update tests/contrib/flask/test_template.py

Co-Authored-By: brettlangdon <[email protected]>

* Update tests/contrib/flask/test_template.py

Co-Authored-By: brettlangdon <[email protected]>

* [flask] simplify fetching wrapped arg

* [flask] fix spelling mistake

* Update ddtrace/contrib/flask/patch.py

Co-Authored-By: brettlangdon <[email protected]>

* [flask] remove unnecessary 'func_name(func)' call

* Update tests/contrib/flask/test_blueprint.py

Co-Authored-By: brettlangdon <[email protected]>

* [flask] fix remaining comments from kyle

* [flask] fix flake8 issues

* [flask] test distributed tracing

* [flask] add find_span_parent helper

* [flask] add hook test cases

* [flask] fix spelling

* [flask] rename Pin.find to Pin._find and add tests

* [flask] one last Pin.find -> Pin._find

* [flask] try to parse endpoint/url rule in Flask.preprocess_request as well

* [flask] fix line too long issue

Author: brettlangdon

.circleci/config.yml

@@ -329,11 +329,14 @@ jobs:
     steps:
       - checkout
       - *restore_cache_step
-      - run: tox -e 'flask_contrib{,_autopatch}-{py27,py34,py35,py36}-flask{010,011,012,10}-blinker' --result-json /tmp/flask.1.results
-      - run: tox -e 'flask_cache_contrib-{py27,py34,py35,py36}-flask{010,011,012}-flaskcache{013}-memcached-redis{210}-blinker' --result-json /tmp/flask.2.results
-      - run: tox -e 'flask_cache_contrib_autopatch-{py27,py34,py35,py36}-flask{010,011,012}-flaskcache{013}-memcached-redis{210}-blinker' --result-json /tmp/flask.3.results
-      - run: tox -e 'flask_cache_contrib-{py27}-flask{010,011}-flaskcache{012}-memcached-redis{210}-blinker' --result-json /tmp/flask.4.results
-      - run: tox -e 'flask_cache_contrib_autopatch-{py27}-flask{010,011}-flaskcache{012}-memcached-redis{210}-blinker' --result-json /tmp/flask.5.results
+      - run: tox -e 'flask_contrib-{py27,py34,py35,py36}-flask{010,011,012,10}-blinker' --result-json /tmp/flask.1.results
+      - run: TOX_SKIP_DIST=False tox -e 'flask_contrib_autopatch-{py27,py34,py35,py36}-flask{010,011,012,10}-blinker' --result-json /tmp/flask.2.results
+      - run: tox -e 'flask_contrib-{py27}-flask{09}-blinker' --result-json /tmp/flask.3.results
+      - run: TOX_SKIP_DIST=False tox -e 'flask_contrib_autopatch-{py27}-flask{09}-blinker' --result-json /tmp/flask.4.results
+      - run: tox -e 'flask_cache_contrib-{py27,py34,py35,py36}-flask{010,011,012}-flaskcache{013}-memcached-redis{210}-blinker' --result-json /tmp/flask.5.results
+      - run: TOX_SKIP_DIST=False tox -e 'flask_cache_contrib_autopatch-{py27,py34,py35,py36}-flask{010,011,012}-flaskcache{013}-memcached-redis{210}-blinker' --result-json /tmp/flask.6.results
+      - run: tox -e 'flask_cache_contrib-{py27}-flask{010,011}-flaskcache{012}-memcached-redis{210}-blinker' --result-json /tmp/flask.7.results
+      - run: TOX_SKIP_DIST=False tox -e 'flask_cache_contrib_autopatch-{py27}-flask{010,011}-flaskcache{012}-memcached-redis{210}-blinker' --result-json /tmp/flask.8.results
       - persist_to_workspace:
           root: /tmp
           paths:
@@ -342,6 +345,9 @@ jobs:
             - flask.3.results
             - flask.4.results
             - flask.5.results
+            - flask.6.results
+            - flask.7.results
+            - flask.8.results
       - *save_cache_step
 
   gevent:

ddtrace/contrib/flask/__init__.py

@@ -1,35 +1,32 @@
 """
-The Flask trace middleware will track request timings and templates. It
-requires the `Blinker <https://pythonhosted.org/blinker/>`_ library, which
-Flask uses for signalling.
+The `Flask <http://flask.pocoo.org/>`_ integration will add tracing to all requests to your Flask application.
 
-To install the middleware, add::
+This integration will track the entire Flask lifecycle including user-defined endpoints, hooks,
+signals, and templating rendering.
 
-    from ddtrace import tracer
-    from ddtrace.contrib.flask import TraceMiddleware
+To configure tracing manually::
 
-and create a `TraceMiddleware` object::
+    from ddtrace import patch_all
+    patch_all()
 
-    traced_app = TraceMiddleware(app, tracer, service="my-flask-app", distributed_tracing=False)
+    from flask import Flask
 
-Here is the end result, in a sample app::
+    app = Flask(__name__)
 
-    from flask import Flask
-    import blinker as _
 
-    from ddtrace import tracer
-    from ddtrace.contrib.flask import TraceMiddleware
+    @app.route('/')
+    def index():
+        return 'hello world'
 
-    app = Flask(__name__)
 
-    traced_app = TraceMiddleware(app, tracer, service="my-flask-app", distributed_tracing=False)
+    if __name__ == '__main__':
+        app.run()
 
-    @app.route("/")
-    def home():
-        return "hello world"
 
-Set `distributed_tracing=True` if this is called remotely from an instrumented application.
-We suggest to enable it only for internal services where headers are under your control.
+You may also enable Flask tracing automatically via ddtrace-run::
+
+    ddtrace-run python app.py
+
 """
 
 from ...utils.importlib import require_modules
@@ -39,7 +36,11 @@ def home():
 
 with require_modules(required_modules) as missing_modules:
     if not missing_modules:
+        # DEV: We do this so we can `@mock.patch('ddtrace.contrib.flask._patch.<func>')` in tests
+        from . import patch as _patch
         from .middleware import TraceMiddleware
-        from .patch import patch
 
-        __all__ = ['TraceMiddleware', 'patch']
+        patch = _patch.patch
+        unpatch = _patch.unpatch
+
+        __all__ = ['TraceMiddleware', 'patch', 'unpatch']

ddtrace/contrib/flask/helpers.py

@@ -0,0 +1,44 @@
+from ddtrace import Pin
+import flask
+
+
+def get_current_app():
+    """Helper to get the flask.app.Flask from the current app context"""
+    appctx = flask._app_ctx_stack.top
+    if appctx:
+        return appctx.app
+    return None
+
+
+def with_instance_pin(func):
+    """Helper to wrap a function wrapper and ensure an enabled pin is available for the `instance`"""
+    def wrapper(wrapped, instance, args, kwargs):
+        pin = Pin._find(wrapped, instance, get_current_app())
+        if not pin or not pin.enabled():
+            return wrapped(*args, **kwargs)
+
+        return func(pin, wrapped, instance, args, kwargs)
+    return wrapper
+
+
+def simple_tracer(name, span_type=None):
+    """Generate a simple tracer that wraps the function call with `with tracer.trace()`"""
+    @with_instance_pin
+    def wrapper(pin, wrapped, instance, args, kwargs):
+        with pin.tracer.trace(name, service=pin.service, span_type=span_type):
+            return wrapped(*args, **kwargs)
+    return wrapper
+
+
+def get_current_span(pin, root=False):
+    """Helper to get the current span from the provided pins current call context"""
+    if not pin or not pin.enabled():
+        return None
+
+    ctx = pin.tracer.get_call_context()
+    if not ctx:
+        return None
+
+    if root:
+        return ctx.get_current_root_span()
+    return ctx.get_current_span()

ddtrace/contrib/flask/middleware.py

@@ -3,6 +3,7 @@
 from ... import compat
 from ...ext import http, errors, AppTypes
 from ...propagation.http import HTTPPropagator
+from ...utils.deprecation import deprecated
 
 import flask.templating
 from flask import g, request, signals
@@ -16,6 +17,7 @@
 
 class TraceMiddleware(object):
 
+    @deprecated(message='Use patching instead (see the docs).', version='1.0.0')
     def __init__(self, app, tracer, service="flask", use_signals=True, distributed_tracing=False):
         self.app = app
         log.debug('flask: initializing trace middleware')