Un/structure fallback factories (#441)

* Un/structure fallback factories

* Finish up fallback hook factories

* Tweak docs

Author: Tinche

HISTORY.md

@@ -18,6 +18,8 @@
   ([#405](https://github.com/python-attrs/cattrs/pull/405))
 - The `omit` parameter of {py:func}`cattrs.override` is now of type `bool | None` (from `bool`).
   `None` is the new default and means to apply default _cattrs_ handling to the attribute, which is to omit the attribute if it's marked as `init=False`, and keep it otherwise.
+- Converters can now be initialized with custom fallback hook factories for un/structuring.
+  ([#331](https://github.com/python-attrs/cattrs/issues/311) [#441](https://github.com/python-attrs/cattrs/pull/441))
 - Fix {py:func}`format_exception() <cattrs.v.format_exception>` parameter working for recursive calls to {py:func}`transform_error <cattrs.transform_error>`.
   ([#389](https://github.com/python-attrs/cattrs/issues/389))
 - [_attrs_ aliases](https://www.attrs.org/en/stable/init.html#private-attributes-and-aliases) are now supported, although aliased fields still map to their attribute name instead of their alias by default when un/structuring.

docs/cattrs.rst

@@ -46,6 +46,14 @@ cattrs.errors module
    :undoc-members:
    :show-inheritance:
 
+cattrs.fns module
+-----------------
+
+.. automodule:: cattrs.fns
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
 cattrs.v module
 ---------------
 

docs/conf.py

@@ -284,6 +284,7 @@
 doctest_global_setup = (
     "import attr, cattr, cattrs;"
     "from attr import Factory, define, field;"
+    "from cattrs import Converter;"
     "from typing import *;"
     "from enum import Enum, unique"
 )
@@ -292,3 +293,4 @@
 copybutton_prompt_text = r">>> |\.\.\. "
 copybutton_prompt_is_regexp = True
 myst_heading_anchors = 3
+autoclass_content = "both"

docs/converters.md

@@ -1,10 +1,8 @@
 # Converters
 
 All _cattrs_ functionality is exposed through a {class}`cattrs.Converter` object.
-Global _cattrs_ functions, such as {meth}`cattrs.unstructure`, use a single
-global converter. Changes done to this global converter, such as registering new
-structure and unstructure hooks, affect all code using the global
-functions.
+Global _cattrs_ functions, such as {meth}`cattrs.unstructure`, use a single global converter.
+Changes done to this global converter, such as registering new structure and unstructure hooks, affect all code using the global functions.
 
 ## Global Converter
 
@@ -18,8 +16,7 @@ The following functions implicitly use this global converter:
 
 Changes made to the global converter will affect the behavior of these functions.
 
-Larger applications are strongly encouraged to create and customize a different,
-private instance of {class}`cattrs.Converter`.
+Larger applications are strongly encouraged to create and customize a different, private instance of {class}`cattrs.Converter`.
 
 ## Converter Objects
 
@@ -32,14 +29,52 @@ Currently, a converter contains the following state:
 - a reference to an unstructuring strategy (either AS_DICT or AS_TUPLE).
 - a `dict_factory` callable, used for creating `dicts` when dumping _attrs_ classes using `AS_DICT`.
 
-Converters may be cloned using the {meth}`cattrs.Converter.copy` method.
+Converters may be cloned using the {meth}`Converter.copy() <cattrs.BaseConverter.copy>` method.
 The new copy may be changed through the `copy` arguments, but will retain all manually registered hooks from the original.
 
+### Fallback Hook Factories
+
+By default, when a {class}`converter <cattrs.BaseConverter>` cannot handle a type it will:
+
+* when unstructuring, pass the value through unchanged
+* when structuring, raise a {class}`cattrs.errors.StructureHandlerNotFoundError` asking the user to add configuration
+
+These behaviors can be customized by providing custom [hook factories](usage.md#using-factory-hooks) when creating the converter.
+
+```python
+>>> from pickle import dumps
+
+>>> class Unsupported:
+...     """An artisinal (non-attrs) class, unsupported by default."""
+
+>>> converter = Converter(unstructure_fallback_factory=lambda _: dumps)
+>>> instance = Unsupported()
+>>> converter.unstructure(instance)
+b'\x80\x04\x95\x18\x00\x00\x00\x00\x00\x00\x00\x8c\x08__main__\x94\x8c\x04Test\x94\x93\x94)\x81\x94.'
+```
+
+This also enables converters to be chained.
+
+```python
+>>> parent = Converter()
+
+>>> child = Converter(
+...     unstructure_fallback_factory=parent._unstructure_func.dispatch, 
+...     structure_fallback_factory=parent._structure_func.dispatch,
+... )
+```
+
+```{note}
+`Converter._structure_func.dispatch` and `Converter._unstructure_func.dispatch` are slated to become public APIs in a future release.
+```
+
+```{versionadded} 23.2.0
+
+```
+
 ## `cattrs.Converter`
 
-The {class}`Converter <cattrs.Converter>` is a converter variant that automatically generates,
-compiles and caches specialized structuring and unstructuring hooks for _attrs_
-classes and dataclasses.
+The {class}`Converter <cattrs.Converter>` is a converter variant that automatically generates, compiles and caches specialized structuring and unstructuring hooks for _attrs_ classes, dataclasses and TypedDicts.
 
 `Converter` differs from the {class}`cattrs.BaseConverter` in the following ways:
 
@@ -53,7 +88,5 @@ The `Converter` used to be called `GenConverter`, and that alias is still presen
 
 ## `cattrs.BaseConverter`
 
-The {class}`BaseConverter <cattrs.BaseConverter>` is a simpler and slower Converter variant. It does no
-code generation, so it may be faster on first-use which can be useful
-in specific cases, like CLI applications where startup time is more
-important than throughput.
+The {class}`BaseConverter <cattrs.BaseConverter>` is a simpler and slower `Converter` variant.
+It does no code generation, so it may be faster on first-use which can be useful in specific cases, like CLI applications where startup time is more important than throughput.

docs/customizing.md

@@ -6,7 +6,7 @@ This section deals with customizing the unstructuring and structuring processes
 
 The default {class}`Converter <cattrs.Converter>`, upon first encountering an _attrs_ class, will use the generation functions mentioned here to generate the specialized hooks for it, register the hooks and use them.
 
-## Manual un/structuring hooks
+## Manual Un/structuring Hooks
 
 You can write your own structuring and unstructuring functions and register
 them for types using {meth}`Converter.register_structure_hook() <cattrs.BaseConverter.register_structure_hook>` and

docs/usage.md

@@ -95,10 +95,9 @@ MyRecord(a_string='test', a_datetime=DateTime(2018, 7, 28, 18, 24, 0, tzinfo=Tim
 MyRecord(a_string='test', a_datetime=DateTime(2018, 7, 28, 18, 24, 0, tzinfo=Timezone('+02:00')))
 ```
 
-## Using factory hooks
+## Using Factory Hooks
 
-For this example, let's assume you have some attrs classes with snake case attributes, and you want to
-un/structure them as camel case.
+For this example, let's assume you have some attrs classes with snake case attributes, and you want to un/structure them as camel case.
 
 ```{warning}
 A simpler and better approach to this problem is to simply make your class attributes camel case.
@@ -257,7 +256,7 @@ converter.register_structure_hook_factory(
 The `converter` instance will now un/structure every attrs class to camel case.
 Nothing has been omitted from this final example; it's complete.
 
-## Using fallback key names
+## Using Fallback Key Names
 
 Sometimes when structuring data, the input data may be in multiple formats that need to be converted into a common attribute.
 
@@ -305,7 +304,7 @@ class MyInternalAttr:
 
 _cattrs_ will now structure both key names into `new_field` on your class.
 
-```
+```python
 converter.structure({"new_field": "foo"}, MyInternalAttr)
 converter.structure({"old_field": "foo"}, MyInternalAttr)
 ```

src/cattrs/converters.py

@@ -11,7 +11,6 @@
     Dict,
     Iterable,
     List,
-    NoReturn,
     Optional,
     Tuple,
     Type,
@@ -56,12 +55,13 @@
     is_union_type,
 )
 from .disambiguators import create_default_dis_func, is_supported_union
-from .dispatch import MultiStrategyDispatch
+from .dispatch import HookFactory, MultiStrategyDispatch, StructureHook, UnstructureHook
 from .errors import (
     IterableValidationError,
     IterableValidationNote,
     StructureHandlerNotFoundError,
 )
+from .fns import identity, raise_error
 from .gen import (
     AttributeOverride,
     DictStructureFn,
@@ -79,6 +79,8 @@
 from .gen.typeddicts import make_dict_structure_fn as make_typeddict_dict_struct_fn
 from .gen.typeddicts import make_dict_unstructure_fn as make_typeddict_dict_unstruct_fn
 
+__all__ = ["UnstructureStrategy", "BaseConverter", "Converter", "GenConverter"]
+
 NoneType = type(None)
 T = TypeVar("T")
 V = TypeVar("V")
@@ -127,7 +129,20 @@ def __init__(
         unstruct_strat: UnstructureStrategy = UnstructureStrategy.AS_DICT,
         prefer_attrib_converters: bool = False,
         detailed_validation: bool = True,
+        unstructure_fallback_factory: HookFactory[UnstructureHook] = lambda _: identity,
+        structure_fallback_factory: HookFactory[StructureHook] = lambda _: raise_error,
     ) -> None:
+        """
+        :param detailed_validation: Whether to use a slightly slower mode for detailed
+            validation errors.
+        :param unstructure_fallback_factory: A hook factory to be called when no
+            registered unstructuring hooks match.
+        :param structure_fallback_factory: A hook factory to be called when no
+            registered structuring hooks match.
+
+        ..  versionadded:: 23.2.0 *unstructure_fallback_factory*
+        ..  versionadded:: 23.2.0 *structure_fallback_factory*
+        """
         unstruct_strat = UnstructureStrategy(unstruct_strat)
         self._prefer_attrib_converters = prefer_attrib_converters
 
@@ -143,13 +158,9 @@ def __init__(
 
         self._dis_func_cache = lru_cache()(self._get_dis_func)
 
-        self._unstructure_func = MultiStrategyDispatch(self._unstructure_identity)
+        self._unstructure_func = MultiStrategyDispatch(unstructure_fallback_factory)
         self._unstructure_func.register_cls_list(
-            [
-                (bytes, self._unstructure_identity),
-                (str, self._unstructure_identity),
-                (Path, str),
-            ]
+            [(bytes, identity), (str, identity), (Path, str)]
         )
         self._unstructure_func.register_func_list(
             [
@@ -175,7 +186,7 @@ def __init__(
         # Per-instance register of to-attrs converters.
         # Singledispatch dispatches based on the first argument, so we
         # store the function and switch the arguments in self.loads.
-        self._structure_func = MultiStrategyDispatch(BaseConverter._structure_error)
+        self._structure_func = MultiStrategyDispatch(structure_fallback_factory)
         self._structure_func.register_func_list(
             [
                 (lambda cl: cl is Any or cl is Optional or cl is None, lambda v, _: v),
@@ -237,7 +248,7 @@ def unstruct_strat(self) -> UnstructureStrategy:
             else UnstructureStrategy.AS_TUPLE
         )
 
-    def register_unstructure_hook(self, cls: Any, func: Callable[[Any], Any]) -> None:
+    def register_unstructure_hook(self, cls: Any, func: UnstructureHook) -> None:
         """Register a class-to-primitive converter function for a class.
 
         The converter function should take an instance of the class and return
@@ -254,17 +265,15 @@ def register_unstructure_hook(self, cls: Any, func: Callable[[Any], Any]) -> Non
             self._unstructure_func.register_cls_list([(cls, func)])
 
     def register_unstructure_hook_func(
-        self, check_func: Callable[[Any], bool], func: Callable[[Any], Any]
+        self, check_func: Callable[[Any], bool], func: UnstructureHook
     ) -> None:
         """Register a class-to-primitive converter function for a class, using
         a function to check if it's a match.
         """
         self._unstructure_func.register_func_list([(check_func, func)])
 
     def register_unstructure_hook_factory(
-        self,
-        predicate: Callable[[Any], bool],
-        factory: Callable[[Any], Callable[[Any], Any]],
+        self, predicate: Callable[[Any], bool], factory: HookFactory[UnstructureHook]
     ) -> None:
         """
         Register a hook factory for a given predicate.
@@ -276,9 +285,7 @@ def register_unstructure_hook_factory(
         """
         self._unstructure_func.register_func_list([(predicate, factory, True)])
 
-    def register_structure_hook(
-        self, cl: Any, func: Callable[[Any, Type[T]], T]
-    ) -> None:
+    def register_structure_hook(self, cl: Any, func: StructureHook) -> None:
         """Register a primitive-to-class converter function for a type.
 
         The converter function should take two arguments:
@@ -300,17 +307,15 @@ def register_structure_hook(
             self._structure_func.register_cls_list([(cl, func)])
 
     def register_structure_hook_func(
-        self, check_func: Callable[[Type[T]], bool], func: Callable[[Any, Type[T]], T]
+        self, check_func: Callable[[Type[T]], bool], func: StructureHook
     ) -> None:
         """Register a class-to-primitive converter function for a class, using
         a function to check if it's a match.
         """
         self._structure_func.register_func_list([(check_func, func)])
 
     def register_structure_hook_factory(
-        self,
-        predicate: Callable[[Any], bool],
-        factory: Callable[[Any], Callable[[Any, Any], Any]],
+        self, predicate: Callable[[Any], bool], factory: HookFactory[StructureHook]
     ) -> None:
         """
         Register a hook factory for a given predicate.
@@ -353,11 +358,6 @@ def _unstructure_enum(self, obj: Enum) -> Any:
         """Convert an enum to its value."""
         return obj.value
 
-    @staticmethod
-    def _unstructure_identity(obj: T) -> T:
-        """Just pass it through."""
-        return obj
-
     def _unstructure_seq(self, seq: Sequence[T]) -> Sequence[T]:
         """Convert a sequence to primitive equivalents."""
         # We can reuse the sequence class, so tuples stay tuples.
@@ -388,12 +388,6 @@ def _unstructure_union(self, obj: Any) -> Any:
 
     # Python primitives to classes.
 
-    @staticmethod
-    def _structure_error(_, cl: Type) -> NoReturn:
-        """At the bottom of the condition stack, we explode if we can't handle it."""
-        msg = f"Unsupported type: {cl!r}. Register a structure hook for it."
-        raise StructureHandlerNotFoundError(msg, type_=cl)
-
     def _gen_structure_generic(self, cl: Type[T]) -> DictStructureFn[T]:
         """Create and return a hook for structuring generics."""
         return make_dict_structure_fn(
@@ -742,7 +736,11 @@ def copy(
         prefer_attrib_converters: Optional[bool] = None,
         detailed_validation: Optional[bool] = None,
     ) -> "BaseConverter":
-        """Create a copy of the converter, keeping all existing custom hooks."""
+        """Create a copy of the converter, keeping all existing custom hooks.
+
+        :param detailed_validation: Whether to use a slightly slower mode for detailed
+            validation errors.
+        """
         res = self.__class__(
             dict_factory if dict_factory is not None else self._dict_factory,
             unstruct_strat
@@ -786,12 +784,27 @@ def __init__(
         unstruct_collection_overrides: Mapping[Type, Callable] = {},
         prefer_attrib_converters: bool = False,
         detailed_validation: bool = True,
+        unstructure_fallback_factory: HookFactory[UnstructureHook] = lambda _: identity,
+        structure_fallback_factory: HookFactory[StructureHook] = lambda _: raise_error,
     ):
+        """
+        :param detailed_validation: Whether to use a slightly slower mode for detailed
+            validation errors.
+        :param unstructure_fallback_factory: A hook factory to be called when no
+            registered unstructuring hooks match.
+        :param structure_fallback_factory: A hook factory to be called when no
+            registered structuring hooks match.
+
+        ..  versionadded:: 23.2.0 *unstructure_fallback_factory*
+        ..  versionadded:: 23.2.0 *structure_fallback_factory*
+        """
         super().__init__(
             dict_factory=dict_factory,
             unstruct_strat=unstruct_strat,
             prefer_attrib_converters=prefer_attrib_converters,
             detailed_validation=detailed_validation,
+            unstructure_fallback_factory=unstructure_fallback_factory,
+            structure_fallback_factory=structure_fallback_factory,
         )
         self.omit_if_default = omit_if_default
         self.forbid_extra_keys = forbid_extra_keys
@@ -1042,7 +1055,11 @@ def copy(
         prefer_attrib_converters: Optional[bool] = None,
         detailed_validation: Optional[bool] = None,
     ) -> "Converter":
-        """Create a copy of the converter, keeping all existing custom hooks."""
+        """Create a copy of the converter, keeping all existing custom hooks.
+
+        :param detailed_validation: Whether to use a slightly slower mode for detailed
+            validation errors.
+        """
         res = self.__class__(
             dict_factory if dict_factory is not None else self._dict_factory,
             unstruct_strat