You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
**cattrs** is an open source Python library for structuring and unstructuring data.
18
-
_cattrs_ works best with _attrs_ classes, dataclasses and the usual
19
-
Python collections, but other kinds of classes are supported by manually
20
-
registering converters.
16
+
**cattrs** is a Swiss Army knife for (un)structuring and validating data in Python.
17
+
In practice, that means it converts **unstructured dictionaries** into **proper classes** and back, while **validating** their contents.
21
18
22
-
Python has a rich set of powerful, easy to use, built-in data types like
23
-
dictionaries, lists and tuples. These data types are also the lingua franca
24
-
of most data serialization libraries, for formats like json, msgpack, cbor,
25
-
yaml or toml.
19
+
---
20
+
21
+
Python has a rich set of powerful, easy to use, built-in **unstructured** data types like dictionaries, lists and tuples.
22
+
These data types effortlessly convert into common serialization formats like JSON, MessagePack, CBOR, YAML or TOML.
23
+
24
+
But the data that is used by your **business logic** should be **structured** into well-defined classes, since not all combinations of field names or values are valid inputs to your programs.
25
+
The more trust you can have into the structure of your data, the simpler your code can be, and the fewer edge cases you have to worry about.
26
26
27
-
Data types like this, and mappings like `dict` s in particular, represent
28
-
unstructured data. Your data is, in all likelihood, structured: not all
29
-
combinations of field names or values are valid inputs to your programs. In
30
-
Python, structured data is better represented with classes and enumerations.
31
-
_attrs_ is an excellent library for declaratively describing the structure of
32
-
your data, and validating it.
27
+
When you're handed unstructured data (by your network, file system, database, ...), _cattrs_ helps to convert this data into trustworthy structured data.
28
+
When you have to convert your structured data into data types that other libraries can handle, _cattrs_ turns your classes and enumerations into dictionaries, integers and strings.
33
29
34
-
When you're handed unstructured data (by your network, file system, database...),
35
-
_cattrs_ helps to convert this data into structured data. When you have to
36
-
convert your structured data into data types other libraries can handle,
37
-
_cattrs_ turns your classes and enumerations into dictionaries, integers and
38
-
strings.
30
+
_attrs_ (and to a certain degree dataclasses) are excellent libraries for declaratively describing the structure of your data, but they're purposefully not serialization libraries.
31
+
*cattrs* is there for you the moment your `attrs.asdict(your_instance)` and `YourClass(**data)` start failing you because you need more control over the conversion process.
39
32
40
-
Here's a simple taste. The list containing a float, an int and a string
41
-
gets converted into a tuple of three ints.
33
+
34
+
## Examples
35
+
36
+
_cattrs_ works best with [_attrs_](https://www.attrs.org/) classes, and [dataclasses](https://docs.python.org/3/library/dataclasses.html) where simple (un-)structuring works out of the box, even for nested data:
42
37
43
38
```python
44
-
>>>import cattrs
39
+
>>>from attrs import define
40
+
>>>from cattrs import structure, unstructure
41
+
>>>@define
42
+
...classC:
43
+
... a: int
44
+
... b: list[str]
45
+
>>> instance = structure({'a': 1, 'b': ['x', 'y']}, C)
_cattrs_ works well with _attrs_ classes out of the box.
53
+
> [!IMPORTANT]
54
+
> Note how the structuring and unstructuring details do **not** pollute your class, meaning: your data model.
55
+
> Any needs to configure the conversion are done within *cattrs* itself, not within your data model.
56
+
>
57
+
> There are popular validation libraries for Python that couple your data model with its validation and serialization rules based on, for example, web APIs.
58
+
> We think that's the wrong approach.
59
+
> Validation and serializations are concerns of the edges of your program – not the core.
60
+
> They should neither apply design pressure on your business code, nor affect the performance of your code through unnecessary validation.
61
+
> In bigger real-world code bases it's also common for data coming from multiple sources that need different validation and serialization rules.
62
+
>
63
+
> 🎶 You gotta keep 'em separated. 🎶
64
+
65
+
*cattrs* also works with the usual Python collection types like dictionaries, lists, or tuples when you want to **normalize** unstructured data data into a certain (still unstructured) shape.
66
+
For example, to convert a list of a float, an int and a string into a tuple of ints:
51
67
52
68
```python
53
-
>>>from attrs import frozen
54
69
>>>import cattrs
55
70
56
-
>>>@frozen # It works with non-frozen classes too.
Here's a much more complex example, involving _attrs_ classes with type metadata.
76
+
Finally, here's a much more complex example, involving _attrs_ classes where _cattrs_ interprets the type annotations to structure and unstructure the data correctly, including Enums and nested data structures:
69
77
70
78
```python
71
79
>>>from enum import unique, Enum
@@ -92,73 +100,80 @@ Here's a much more complex example, involving _attrs_ classes with type metadata
92
100
>>>@define
93
101
...classDog:
94
102
... cuteness: int
95
-
... chip: Optional[DogMicrochip]=None
103
+
... chip: DogMicrochip|None=None
96
104
97
105
>>> p = unstructure([Dog(cuteness=1, chip=DogMicrochip(chip_id=1, time_chipped=10.0)),
Consider unstructured data a low-level representation that needs to be converted to structured data to be handled, and use `structure`.
107
-
When you're done, `unstructure` the data to its unstructured form and pass it along to another library or module.
108
-
Use [attrs type metadata](http://attrs.readthedocs.io/en/stable/examples.html#types) to add type metadata to attributes, so _cattrs_ will know how to structure and destructure them.
115
+
> [!TIP]
116
+
> Consider unstructured data a low-level representation that needs to be converted to structured data to be handled, and use `structure()`.
117
+
> When you're done, `unstructure()` the data to its unstructured form and pass it along to another library or module.
118
+
>
119
+
> Use [*attrs* type metadata](http://attrs.readthedocs.io/en/stable/examples.html#types) to add type metadata to attributes, so _cattrs_ will know how to structure and destructure them.
- Python versions supported: 3.8 and up. (Older Python versions are supported by older versions; see the changelog.)
113
121
114
122
## Features
115
123
116
-
- Converts structured data into unstructured data, recursively:
124
+
### Recursive Unstructuring
125
+
126
+
-_attrs_ classes and dataclasses are converted into dictionaries in a way similar to `attrs.asdict()`, or into tuples in a way similar to `attrs.astuple()`.
127
+
- Enumeration instances are converted to their values.
128
+
- Other types are let through without conversion. This includes types such as integers, dictionaries, lists and instances of non-_attrs_ classes.
129
+
- Custom converters for any type can be registered using `register_unstructure_hook`.
117
130
118
-
-_attrs_ classes and dataclasses are converted into dictionaries in a way similar to `attrs.asdict`, or into tuples in a way similar to `attrs.astuple`.
119
-
- Enumeration instances are converted to their values.
120
-
- Other types are let through without conversion. This includes types such as integers, dictionaries, lists and instances of non-_attrs_ classes.
121
-
- Custom converters for any type can be registered using `register_unstructure_hook`.
122
131
123
-
- Converts unstructured data into structured data, recursively, according to your specification given as a type.
124
-
The following types are supported:
132
+
### Recursive Structuring
125
133
126
-
-`typing.Optional[T]` and its 3.10+ form, `T | None`.
127
-
-`list[T]`, `typing.List[T]`, `typing.MutableSequence[T]`, `typing.Sequence[T]` (converts to a list).
128
-
-`tuple` and `typing.Tuple` (both variants, `tuple[T, ...]` and `tuple[X, Y, Z]`).
129
-
-`set[T]`, `typing.MutableSet[T]`, `typing.Set[T]` (converts to a set).
130
-
-`frozenset[T]`, `typing.FrozenSet[T]` (converts to a frozenset).
131
-
-`dict[K, V]`, `typing.Dict[K, V]`, `typing.MutableMapping[K, V]`, `typing.Mapping[K, V]` (converts to a dict).
132
-
-[`typing.TypedDict`](https://docs.python.org/3/library/typing.html#typing.TypedDict), ordinary and generic.
-[PEP 695 type aliases](https://docs.python.org/3/library/typing.html#type-aliases) on 3.12+
146
+
-_attrs_ classes with simple attributes and the usual `__init__`[^simple].
147
+
- All _attrs_ classes and dataclasses with the usual `__init__`, if their complex attributes have type metadata.
148
+
- Unions of supported _attrs_ classes, given that all of the classes have a unique field.
149
+
- Unions of anything, if you provide a disambiguation function for it.
150
+
- Custom converters for any type can be registered using `register_structure_hook`.
139
151
140
-
- All _attrs_ classes and dataclasses with the usual `__init__`, if their complex attributes have type metadata.
141
-
- Unions of supported _attrs_ classes, given that all of the classes have a unique field.
142
-
- Unions s of anything, given that you provide a disambiguation function for it.
143
-
- Custom converters for any type can be registered using `register_structure_hook`.
152
+
[^simple]: Simple attributes are attributes that can be assigned unstructured data, like numbers, strings, and collections of unstructured data.
153
+
154
+
155
+
### Batteries Included
156
+
157
+
_cattrs_ comes with pre-configured converters for a number of serialization libraries, including JSON (standard library, [_orjson_](https://pypi.org/project/orjson/), [UltraJSON](https://pypi.org/project/ujson/)), [_msgpack_](https://pypi.org/project/msgpack/), [_cbor2_](https://pypi.org/project/cbor2/), [_bson_](https://pypi.org/project/bson/), [PyYAML](https://pypi.org/project/PyYAML/), [_tomlkit_](https://pypi.org/project/tomlkit/) and [_msgspec_](https://pypi.org/project/msgspec/) (supports only JSON at this time).
144
158
145
-
_cattrs_ comes with preconfigured converters for a number of serialization libraries, including json, msgpack, cbor2, bson, yaml and toml.
146
159
For details, see the [cattrs.preconf package](https://catt.rs/en/stable/preconf.html).
147
160
161
+
148
162
## Design Decisions
149
163
150
-
_cattrs_ is based on a few fundamental design decisions.
164
+
_cattrs_ is based on a few fundamental design decisions:
151
165
152
166
- Un/structuring rules are separate from the models.
153
167
This allows models to have a one-to-many relationship with un/structuring rules, and to create un/structuring rules for models which you do not own and you cannot change.
154
168
(_cattrs_ can be configured to use un/structuring rules from models using the [`use_class_methods` strategy](https://catt.rs/en/latest/strategies.html#using-class-specific-structure-and-unstructure-methods).)
155
169
- Invent as little as possible; reuse existing ordinary Python instead.
156
170
For example, _cattrs_ did not have a custom exception type to group exceptions until the sanctioned Python [`exceptiongroups`](https://docs.python.org/3/library/exceptions.html#ExceptionGroup).
157
171
A side-effect of this design decision is that, in a lot of cases, when you're solving _cattrs_ problems you're actually learning Python instead of learning _cattrs_.
158
-
-Refuse the temptation to guess.
172
+
-Resist the temptation to guess.
159
173
If there are two ways of solving a problem, _cattrs_ should refuse to guess and let the user configure it themselves.
160
174
161
-
A foolish consistency is the hobgoblin of little minds so these decisions can and are sometimes broken, but they have proven to be a good foundation.
175
+
A foolish consistency is the hobgoblin of little minds, so these decisions can and are sometimes broken, but they have proven to be a good foundation.
176
+
162
177
163
178
## Additional documentation and talks
164
179
@@ -168,6 +183,7 @@ A foolish consistency is the hobgoblin of little minds so these decisions can an
168
183
-[Python has a macro language - it's Python (PyCon IT 2022)](https://www.youtube.com/watch?v=UYRSixikUTo)
169
184
-[Intro to cattrs 23.1](https://threeofwands.com/intro-to-cattrs-23-1-0/)
170
185
186
+
171
187
## Credits
172
188
173
189
Major credits to Hynek Schlawack for creating [attrs](https://attrs.org) and its predecessor, [characteristic](https://github.com/hynek/characteristic).
0 commit comments