Implement baked query (#659)

* implement baked query

Author: fantix

docs/how-to/bakery.rst

@@ -0,0 +1,191 @@
+Bake Queries
+============
+
+Baked queries are used to boost execution performance for constantly-used queries.
+Similar to the :doc:`orm/extensions/baked` in SQLAlchemy, GINO could also cache the
+object’s construction and string-compilation steps. Furthermore, GINO automatically
+manages a prepared statement for each baked query in every active connection in the
+pool. Executing baked queries is at least 40% faster than running normal queries, but
+you need to **bake them before creating the engine**.
+
+GINO provides two approaches for baked queries:
+
+1. Low-level :class:`~gino.bakery.Bakery` API
+2. High-level :meth:`Gino.bake() <gino.api.Gino.bake>` integration
+
+
+Use Bakery with Bare Engine
+---------------------------
+
+First, we need a bakery::
+
+    import gino
+
+    bakery = gino.Bakery()
+
+Then, let's bake some queries::
+
+    db_time = bakery.bake("SELECT now()")
+
+Or queries with parameters::
+
+    user_query = bakery.bake("SELECT * FROM users WHERE id = :uid")
+
+Let's assume we have this ``users`` table defined in SQLAlchemy Core::
+
+    import sqlalchemy as sa
+
+    metadata = sa.MetaData()
+    user_table = sa.Table(
+        "users", metadata,
+        sa.Column("id", sa.Integer, primary_key=True),
+        sa.Column("name", sa.String),
+    )
+
+Now we can bake a similar query with SQLAlchemy Core::
+
+    user_query = bakery.bake(
+        sa.select([user_table]).where(user.c.id == sa.bindparam("uid"))
+    )
+
+These baked queries are usually global, and supposed to be shared across the
+application. To run them, we need an engine with the bakery::
+
+    engine = await gino.create_engine("postgresql://localhost/", bakery=bakery)
+
+By doing so, GINO will bake the queries in the bakery. As new connections are added to
+the DB pool, the prepared statements are automatically created behind the scene.
+
+To execute the baked queries, you could treat the :class:`~gino.bakery.BakedQuery`
+instances as if they are the queries themselves, for example::
+
+    now = await engine.scalar(db_time)
+
+Pass in parameter values::
+
+    row = await engine.first(user_query, uid=123)
+
+
+Use the :class:`~gino.api.Gino` Integration
+--------------------------------------------
+
+In a more common scenario, there will be a :class:`~gino.api.Gino>` instance, which has
+usually a ``bind`` set - either explicitly or by the Web framework extensions::
+
+    from gino import Gino
+
+    db = Gino()
+
+    async def main():
+        async with db.with_bind("postgresql://localhost/"):
+            ...
+
+A :class:`~gino.bakery.Bakery` is automatically created in the ``db`` instance, and fed
+to the engine implicitly. You can immediately start to bake queries without further
+ado::
+
+    class User(db.Model):
+        __tablename__ = "users"
+
+        id = db.Column(db.Integer, primary_key=True)
+        name = db.Column(db.String)
+
+    db_time = db.bake("SELECT now()")
+    user_getter = db.bake(User.query.where(User.id == db.bindparam("uid")))
+
+And the execution is also simplified with the same ``bind`` magic::
+
+    async def main():
+        async with db.with_bind("postgresql://localhost/"):
+            print(await db_time.scalar())
+
+            user: User = await user_getter.first(uid=1)
+            print(user.name)
+
+To make things more easier, you could even define the baked queries directly on the
+model::
+
+    class User(db.Model):
+        __tablename__ = "users"
+
+        id = db.Column(db.Integer, primary_key=True)
+        name = db.Column(db.String)
+
+        @db.bake
+        def getter(cls):
+            return cls.query.where(cls.id == db.bindparam("uid"))
+
+        @classmethod
+        async def get(cls, uid):
+            return await cls.getter.one_or_none(uid=uid)
+
+Here GINO treats the ``getter()`` as a :meth:`~gino.declarative.declared_attr` with
+``with_table=True``, therefore it takes one positional argument ``cls`` for the ``User``
+class.
+
+
+How to customize loaders?
+-------------------------
+
+If possible, you could bake the additional execution options into the query::
+
+    user_getter = db.bake(
+        User.query.where(User.id == db.bindparam("uid")).execution_options(
+            loader=User.load(comment="Added by loader.")
+        )
+    )
+
+The :meth:`~gino.bakery.Bakery.bake` method accepts keyword arguments as execution
+options to e.g. simplify the example above into::
+
+    user_getter = db.bake(
+        User.query.where(User.id == db.bindparam("uid")),
+        loader=User.load(comment="Added by loader."),
+    )
+
+If the query construction is complex, :meth:`~gino.bakery.Bakery.bake` could also be
+used as a decorator::
+
+    @db.bake
+    def user_getter():
+        return User.query.where(User.id == db.bindparam("uid")).execution_options(
+            loader=User.load(comment="Added by loader.")
+        )
+
+Or with short execution options::
+
+    @db.bake(loader=User.load(comment="Added by loader."))
+    def user_getter():
+        return User.query.where(User.id == db.bindparam("uid"))
+
+Meanwhile, it is also possible to override the loader at runtime::
+
+    user: User = await user_getter.load(User).first(uid=1)
+    print(user.name)  # no more comment on user!
+
+.. hint::
+
+    This override won't affect the baked query - it's used only in this execution.
+
+
+What APIs are available on :class:`~gino.bakery.BakedQuery`?
+------------------------------------------------------------
+
+:class:`~gino.bakery.BakedQuery` is a :class:`~gino.api.GinoExecutor`, so it inherited
+all the APIs like :meth:`~gino.api.GinoExecutor.all`,
+:meth:`~gino.api.GinoExecutor.first`, :meth:`~gino.api.GinoExecutor.one`,
+:meth:`~gino.api.GinoExecutor.one_or_none`, :meth:`~gino.api.GinoExecutor.scalar`,
+:meth:`~gino.api.GinoExecutor.status`, :meth:`~gino.api.GinoExecutor.load`,
+:meth:`~gino.api.GinoExecutor.timeout`, etc.
+
+:class:`~gino.api.GinoExecutor` is actually the chained ``.gino`` helper API seen
+usually in queries like this::
+
+    user = await User.query.where(User.id == 123).gino.first()
+
+So a :class:`~gino.bakery.BakedQuery` can be seen as a normal query with the ``.gino``
+suffix, plus it is directly executable.
+
+.. seealso::
+
+    Please see API document of :mod:`gino.bakery` for more information.

src/gino/__init__.py

@@ -1,11 +1,17 @@
 from .api import Gino  # NOQA
+from .bakery import Bakery
 from .engine import GinoEngine, GinoConnection  # NOQA
 from .exceptions import *  # NOQA
 from .strategies import GinoStrategy  # NOQA
 
 
 def create_engine(*args, **kwargs):
-    """Shortcut for :func:`sqlalchemy.create_engine` with ``strategy="gino"``."""
+    """
+    Shortcut for :func:`sqlalchemy.create_engine` with ``strategy="gino"``.
+
+    .. versionchanged:: 1.1
+       Added the ``bakery`` keyword argument, please see :class:`~.bakery.Bakery`.
+    """
 
     from sqlalchemy import create_engine
 

src/gino/api.py

@@ -5,11 +5,11 @@
 from sqlalchemy.sql.base import Executable
 from sqlalchemy.sql.schema import SchemaItem
 
+from . import json_support
 from .crud import CRUDModel
 from .declarative import declarative_base, declared_attr
 from .exceptions import UninitializedError
 from .schema import GinoSchemaVisitor, patch_schema
-from . import json_support
 
 
 class GinoExecutor:
@@ -75,8 +75,7 @@ def model(self, model):
         """
         if model is not None:
             model = weakref.ref(model)
-        self._query = self._query.execution_options(model=model)
-        return self
+        return self.execution_options(model=model)
 
     def return_model(self, switch):
         """
@@ -86,8 +85,7 @@ def return_model(self, switch):
         information.
 
         """
-        self._query = self._query.execution_options(return_model=switch)
-        return self
+        return self.execution_options(return_model=switch)
 
     def timeout(self, timeout):
         """
@@ -97,8 +95,7 @@ def timeout(self, timeout):
         information.
 
         """
-        self._query = self._query.execution_options(timeout=timeout)
-        return self
+        return self.execution_options(timeout=timeout)
 
     def load(self, value):
         """
@@ -113,7 +110,20 @@ def load(self, value):
         information.
 
         """
-        self._query = self._query.execution_options(loader=value)
+        return self.execution_options(loader=value)
+
+    def execution_options(self, **options):
+        """
+        Set execution options to this query in a chaining call.
+
+        Read :meth:`~gino.engine.GinoConnection.execution_options` for more
+        information.
+
+        :param options: Multiple execution options.
+
+        .. versionadded:: 1.1
+        """
+        self._query = self._query.execution_options(**options)
         return self
 
     async def all(self, *multiparams, **params):
@@ -185,10 +195,7 @@ def iterate(self, *multiparams, **params):
         (:class:`Gino`) is bound, while metadata is found in this query.
 
         """
-        connection = self._query.bind.current_connection
-        if connection is None:
-            raise ValueError("No Connection in context, please provide one")
-        return connection.iterate(self._query, *multiparams, **params)
+        return self._query.bind.iterate(self._query, *multiparams, **params)
 
 
 class _BindContext:
@@ -354,6 +361,9 @@ def __init__(
         self._model = declarative_base(self, model_classes)
         self.declared_attr = declared_attr
         self.quoted_name = sa.sql.quoted_name
+        from .bakery import Bakery
+
+        self._bakery = Bakery()
         for mod in json_support, sa:
             for key in mod.__all__:
                 if not hasattr(self, key) and key not in self.no_delegate:
@@ -414,7 +424,7 @@ async def set_bind(self, bind, loop=None, **kwargs):
         if isinstance(bind, URL):
             from . import create_engine
 
-            bind = await create_engine(bind, loop=loop, **kwargs)
+            bind = await create_engine(bind, loop=loop, bakery=self._bakery, **kwargs)
         self.bind = bind
         return bind
 
@@ -429,6 +439,9 @@ def pop_bind(self):
         :return: :class:`~.engine.GinoEngine` or ``None`` if self is not bound.
 
         """
+        from .bakery import Bakery
+
+        self._bakery = Bakery()
         bind, self.bind = self.bind, None
         return bind
 
@@ -531,6 +544,23 @@ def transaction(self, *args, **kwargs):
         """
         return self.bind.transaction(*args, **kwargs)
 
+    def bake(self, func_or_elem=None, **execution_options):
+        """
+        A delegate of :meth:`Bakery.bake() <.bakery.Bakery.bake>`.
+
+        .. versionadded:: 1.1
+        """
+        return self._bakery.bake(func_or_elem, metadata=self, **execution_options)
+
+    @property
+    def bakery(self):
+        """
+        The bundled :class:`~.bakery.Bakery` instance.
+
+        .. versionadded:: 1.1
+        """
+        return self._bakery
+
 
 class _PlaceHolder:
     __slots__ = "_exception"
@@ -547,3 +577,6 @@ def __setattr__(self, key, value):
         if key == "_exception":
             return super().__setattr__(key, value)
         raise self._exception
+
+    def __bool__(self):
+        return False