ENH: Support ExtensionArray operators via a mixin

doc/source/extending.rst

@@ -122,66 +122,34 @@ By default, there are no operators defined for the class :class:`~pandas.api.ext
 There are two approaches for providing operator support for your ExtensionArray:
 
 1. Define each of the operators on your ExtensionArray subclass. 
-2. Use operators from pandas defined on the ExtensionArray subclass based on already defined
-   operators on the underlying elements.
+2. Use an operator implementation from pandas that depends on operators that are already defined
+   on the underlying elements (scalars) of the ExtensionArray.
 
-For the first approach, you will need to create a mixin class with a single class method,
-with the following signature:
+For the first approach, you define selected operators, e.g., ``_add__``, ``__le__``, etc. that
+you want your ExtensionArray subclass to support.
 
-.. code-block:: python
-
-    @classmethod
-    def _create_method(cls, op, coerce_to_dtype=True):
-
-The method ``create_method`` should return a method with the signature
-``binop(self, other)`` that returns the result of applying the operator ``op``
-to your ExtensionArray subclass.  Your mixin class will then become a base class
-for the provided :class:`ExtensionArithmeticOpsMixin` and
-:class:`ExtensionComparisonOpsMixin` classes. 
-
-For example, if your ExtensionArray subclass
-is called ``MyExtensionArray``, you could create a mixin class ``MyOpsMixin``
-that has the following skeleton:
-
-.. code-block:: python
-
-    class MyOpsMixin(object):
-    @classmethod
-    def _create_method(cls, op, coerce_to_dtype=True):
-        def _binop(self, other):
-            # Your implementation of the operator op
-        return _binop
-
-Then to use this class to define the operators for ``MyExtensionArray``, you can write:
-
-.. code-block:: python
-
-    class MyExtensionArray(ExtensionArray,
-                           ExtensionArithmeticOpsMixin(MyOpsMixin),
-                           ExtensionComparisonOpsMixin(MyOpsMixin))
-
-The mixin classes :class:`ExtensionArithmeticOpsMixin` and
-:class:`ExtensionComparisonOpsMixin` will then define the appropriate operators
-using your implementation of those operators in ``MyOpsMixin``.
-
-The second approach assumes that the underlying elements of the ExtensionArray 
+The second approach assumes that the underlying elements (i.e., scalar type) of the ExtensionArray 
 have the individual operators already defined.  In other words, if your ExtensionArray
 named ``MyExtensionArray`` is implemented so that each element is an instance 
 of the class ``MyExtensionElement``, then if the operators are defined 
 for ``MyExtensionElement``, the second approach will automatically
 define the operators for ``MyExtensionArray``.
 
-Two mixin classes, :class:`~pandas.api.extensions.ExtensionScalarArithmeticMixin` and
-:class:`~pandas.api.extensions.ExtensionScalarComparisonMixin`, support this second
+A mixin class, :class:`~pandas.api.extensions.ExtensionScalarOpscMixin` supports this second
 approach.  If developing an ``ExtensionArray`` subclass, for example ``MyExtensionArray``,
-simply include ``ExtensionScalarArithmeticMixin`` and/or 
-``ExtensionScalarComparisonMixin`` as parent classes of ``MyExtensionArray``
-as follows:
+simply include ``ExtensionScalarOpsMixin`` as a parent class of ``MyExtensionArray``
+and then call the methods :meth:`~MyExtensionArray.addArithmeticOps` and/or
+:meth:`~MyExtensionArray.addComparisonOps` to hook the operators into 
+your ``MyExtensionArray`` class, as follows:
 
 .. code-block:: python
 
-    class MyExtensionArray(ExtensionArray, ExtensionScalarArithmeticMixin,
+    class MyExtensionArray(ExtensionArray, ExtensionScalarOpsMixin,
                            ExtensionScalarComparisonMixin):
+        pass
+
+    MyExtensionArray.addArithmeticOps()
+    MyExtensionArray.addComparisonOps()
 
 Note that since ``pandas`` automatically calls the underlying operator on each
 element one-by-one, this might not be as performant as implementing your own

doc/source/whatsnew/v0.24.0.txt

@@ -19,26 +19,23 @@ A ``Series`` based on ``ExtensionArray`` now supports arithmetic and comparison
 operators.  There are two approaches for providing operator support for an ExtensionArray:
 
 1. Define each of the operators on your ExtensionArray subclass. 
-2. Use operators from pandas defined on the ExtensionArray subclass based on already defined
-   operators on the underlying elements.
+2. Use an operator implementation from pandas that depends on operators that are already defined
+   on the underlying elements (scalars) of the ExtensionArray.
 
 To use the first approach where you define your own implementation of the operators,
-use one or both of the mixin classes, :class:`ExtensionArithmeticOpsMixin` and
-:class:`ExtensionComparisonOpsMixin` that, by default, will create
-operators that are ``NotImplemented``.  To use those classes, you will need to create
-a class that has the implementation of the operator methods.  Details can be found in the
-:ref:`ExtensionArray Operator Support <extending.extension.operator>` documentation section.
+you define each operator such as `__add__`, __le__`, etc. on your ExtensionArray
+subclass.
 
 For the second approach, which is appropriate if your ExtensionArray contains 
 elements that already have the operators
-defined on a per-element basis, pandas provides two mixins,
-:class:`ExtensionScalarArithmeticMixin` and :class:`ExtensionScalarComparisonMixin`,
-that you can use that will automatically define the operators on your ExtensionArray
-subclass.
+defined on a per-element basis, pandas provides a mixin,
+:class:`ExtensionScalarOpsMixin` that you can use that can
+define the operators on your ExtensionArray subclass.
 If developing an ``ExtensionArray`` subclass, for example ``MyExtensionArray``,
-simply include ``ExtensionScalarArithmeticMixin`` and/or 
-``ExtensionScalarComparisonMixin`` as parent classes of ``MyExtensionArray``
-as follows:
+simply include ``ExtensionScalarOpsMixin`` as a parent class of ``MyExtensionArray``
+and then call the methods :meth:`~MyExtensionArray.addArithmeticOps` and/or
+:meth:`~MyExtensionArray.addComparisonOps` to hook the operators into 
+your ``MyExtensionArray`` class, as follows:
 
 .. code-block:: python
 

pandas/api/extensions/__init__.py

@@ -4,9 +4,5 @@
                                   register_series_accessor)
 from pandas.core.algorithms import take  # noqa
 from pandas.core.arrays.base import (ExtensionArray,    # noqa
-                                     ExtensionScalarArithmeticMixin,
-                                     ExtensionScalarComparisonMixin,
-                                     ExtensionScalarOpsMixin,
-                                     ExtensionArithmeticOpsMixin,
-                                     ExtensionComparisonOpsMixin)
+                                     ExtensionScalarOpsMixin)
 from pandas.core.dtypes.dtypes import ExtensionDtype  # noqa

pandas/core/arrays/__init__.py

@@ -1,7 +1,3 @@
 from .base import (ExtensionArray,    # noqa
-                   ExtensionScalarArithmeticMixin,
-                   ExtensionScalarComparisonMixin,
-                   ExtensionScalarOpsMixin,
-                   ExtensionArithmeticOpsMixin,
-                   ExtensionComparisonOpsMixin)
+                   ExtensionScalarOpsMixin)
 from .categorical import Categorical  # noqa

pandas/core/arrays/base.py

@@ -617,42 +617,60 @@ def _ndarray_values(self):
         return np.array(self)
 
 
-class ExtensionDefaultOpsMixin(object):
+
+class ExtensionOpsMixin(object):
     """
-    A base class for default ops, that returns NotImplemented
+    A base class for linking the operators to their dunder names
     """
     @classmethod
-    def _create_method(cls, op, coerce_to_dtype=True):
-        """
-        A class method that returns a method will correspond to an
-        operator for an ExtensionArray subclass, by always indicating
-        the operator is NotImplemented
-
-        Parameters
-        ----------
-        op: function
-            An operator that takes arguments op(a, b)
-        coerce_to_dtype:  bool
-            boolean indicating whether to attempt to convert
-            the result to the underlying ExtensionArray dtype
-            (default True)
-
-        Returns
-        -------
-        A method that can be bound to a method of a class
-        """
-
-        def returnNotImplemented(self, other):
-            return NotImplemented
+    def addArithmeticOps(cls):
+        cls.__add__ = cls._create_method(operator.add)
+        cls.__radd__ = cls._create_method(ops.radd)
+        cls.__sub__ = cls._create_method(operator.sub)
+        cls.__rsub__ = cls._create_method(ops.rsub)
+        cls.__mul__ = cls._create_method(operator.mul)
+        cls.__rmul__ = cls._create_method(ops.rmul)
+        cls.__pow__ = cls._create_method(operator.pow)
+        cls.__rpow__ = cls._create_method(ops.rpow)
+        cls.__mod__ = cls._create_method(operator.mod)
+        cls.__rmod__ = cls._create_method(ops.rmod)
+        cls.__floordiv__ = cls._create_method(operator.floordiv)
+        cls.__rfloordiv__ = cls._create_method(ops.rfloordiv)
+        cls.__truediv__ = cls._create_method(operator.truediv)
+        cls.__rtruediv__ = cls._create_method(ops.rtruediv)
+        if not PY3:
+            cls.__div__ = cls._create_method(operator.div)
+            cls.__rdiv__ = cls._create_method(ops.rdiv)
+
+        cls.__divmod__ = cls._create_method(divmod)
+        cls.__rdivmod__ = cls._create_method(ops.rdivmod)
+
+    @classmethod
+    def addComparisonOps(cls):
+        cls.__eq__ = cls._create_method(operator.eq, False)
+        cls.__ne__ = cls._create_method(operator.ne, False)
+        cls.__lt__ = cls._create_method(operator.lt, False)
+        cls.__gt__ = cls._create_method(operator.gt, False)
+        cls.__le__ = cls._create_method(operator.le, False)
+        cls.__ge__ = cls._create_method(operator.ge, False)
 
-        return returnNotImplemented
 
+class ExtensionScalarOpsMixin(ExtensionOpsMixin):
+    """A mixin for defining the arithmetic and logical operations on
+    an ExtensionArray class, where it is assumed that the underlying objects 
+    have the operators already defined.
 
-class ExtensionScalarOpsMixin(object):
-    """
-    A base class for the mixins for different operators.
-    Can also be used to define an individual method for a specific
-    operator using the class method create_method()
+    Usage
+    ------
+    If you have defined a subclass MyExtensionArray(ExtensionArray), then
+    use MyExtensionArray(ExtensionArray, ExtensionScalarOpsMixin) to
+    get the arithmetic operators.  After the definition of MyExtensionArray,
+    insert the lines
+
+    MyExtensionArray.addArithmeticOperators()
+    MyExtensionArray.addComparisonOperators()
+
+    to link the operators to your class.
     """
 
     @classmethod
@@ -678,11 +696,11 @@ def _create_method(cls, op, coerce_to_dtype=True):
 
         Example
         -------
-        Given an ExtensionArray subclass called MyClass, use
+        Given an ExtensionArray subclass called MyExtensionArray, use
 
-        >>> __add__ = ExtensionScalarOpsMixin.create_method(operator.add)
+        >>> __add__ = cls._create_method(operator.add)
 
-        in the class definition of MyClass to create the operator
+        in the class definition of MyExtensionArray to create the operator
         for addition, that will be based on the operator implementation
         of the underlying elements of the ExtensionArray
 
@@ -712,124 +730,3 @@ def convert_values(param):
 
         op_name = ops._get_op_name(op, True)
         return set_function_name(_binop, op_name, cls)
-
-
-def ExtensionArithmeticOpsMixin(base=ExtensionDefaultOpsMixin):
-    """A mixin that will define the arithmetic operators.
-
-    Parameters
-    ----------
-    base: class
-        A class with the class method _create_method() defined as follows:
-
-        @classmethod
-        def _create_method(cls, op, coerce_to_dtype=True):
-            Parameters
-            ----------
-            op: function
-                An operator that takes arguments op(a, b)
-            coerce_to_dtype:  bool
-                boolean indicating whether to attempt to convert
-                the result to the underlying ExtensionArray dtype
-                (default True)
-
-            Returns
-            -------
-            A method that can be bound to a method of a class.  That method
-            should return the result of op(self, other) where self is
-            an ExtensionArray subclass
-    """
-
-    class _ExtensionArithmeticOpsMixin(base):
-        __add__ = base._create_method(operator.add)
-        __radd__ = base._create_method(ops.radd)
-        __sub__ = base._create_method(operator.sub)
-        __rsub__ = base._create_method(ops.rsub)
-        __mul__ = base._create_method(operator.mul)
-        __rmul__ = base._create_method(ops.rmul)
-        __pow__ = base._create_method(operator.pow)
-        __rpow__ = base._create_method(ops.rpow)
-        __mod__ = base._create_method(operator.mod)
-        __rmod__ = base._create_method(ops.rmod)
-        __floordiv__ = base._create_method(operator.floordiv)
-        __rfloordiv__ = base._create_method(ops.rfloordiv)
-        __truediv__ = base._create_method(operator.truediv)
-        __rtruediv__ = base._create_method(ops.rtruediv)
-        if not PY3:
-            __div__ = base._create_method(operator.div)
-            __rdiv__ = base._create_method(ops.rdiv)
-
-        __divmod__ = base._create_method(divmod)
-        __rdivmod__ = base._create_method(ops.rdivmod)
-
-    _ExtensionArithmeticOpsMixin.__name__ = (
-        "ExtensionArithmeticOpsMixin_" + base.__name__)
-    return _ExtensionArithmeticOpsMixin
-
-
-def ExtensionComparisonOpsMixin(base=ExtensionDefaultOpsMixin):
-    """A mixin that will define the comparison operators.
-
-    Parameters
-    ----------
-    base: class
-        A class with the class method _create_method() defined as follows:
-
-        @classmethod
-        def _create_method(cls, op, coerce_to_dtype=True):
-            Parameters
-            ----------
-            op: function
-                An operator that takes arguments op(a, b)
-            coerce_to_dtype:  bool
-                boolean indicating whether to attempt to convert
-                the result to the underlying ExtensionArray dtype
-                (default True)
-
-            Returns
-            -------
-            A method that can be bound to a method of a class.  That method
-            should return the result of op(self, other) where self is
-            an ExtensionArray subclass
-    """
-    class _ExtensionComparisonOpsMixin(base):
-        __eq__ = base._create_method(operator.eq, False)
-        __ne__ = base._create_method(operator.ne, False)
-        __lt__ = base._create_method(operator.lt, False)
-        __gt__ = base._create_method(operator.gt, False)
-        __le__ = base._create_method(operator.le, False)
-        __ge__ = base._create_method(operator.ge, False)
-
-    _ExtensionComparisonOpsMixin.__name__ = (
-        "ExtensionComparisonOpsMixin_" + base.__name__)
-    return _ExtensionComparisonOpsMixin
-
-
-class ExtensionScalarArithmeticMixin(
-        ExtensionArithmeticOpsMixin(ExtensionScalarOpsMixin)):
-    """A mixin for defining the arithmetic operations on an ExtensionArray
-    class, where it is assumed that the underlying objects have the operators
-    already defined.
-
-    Usage
-    ------
-    If you have defined a subclass MyClass(ExtensionArray), then
-    use MyClass(ExtensionArray, ExtensionScalarArithmeticMixin) to
-    get the arithmetic operators
-    """
-    pass
-
-
-class ExtensionScalarComparisonMixin(
-        ExtensionComparisonOpsMixin(ExtensionScalarOpsMixin)):
-    """A mixin for defining the comparison operations on an ExtensionArray
-    class, where it is assumed that the underlying objects have the operators
-    already defined.
-
-    Usage
-    ------
-    If you have defined a subclass MyClass(ExtensionArray), then
-    use MyClass(ExtensionArray, ExtensionScalarComparisonMixin) to
-    get the comparison operators
-    """
-    pass