dataclasses.asdict will mangle collection.Counter instances

[bscarlett]

BPO	39929
Nosy	@ericvsmith, @bscarlett

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/ericvsmith'
closed_at = None
created_at = <Date 2020-03-11.06:13:03.710>
labels = ['3.8', 'type-bug', 'library']
title = 'dataclasses.asdict will mangle collection.Counter instances'
updated_at = <Date 2020-03-11.09:24:12.536>
user = 'https://github.com/bscarlett'

bugs.python.org fields:

activity = <Date 2020-03-11.09:24:12.536>
actor = 'eric.smith'
assignee = 'eric.smith'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2020-03-11.06:13:03.710>
creator = 'brad.scarlett@gmail.com'
dependencies = []
files = []
hgrepos = []
issue_num = 39929
keywords = []
message_count = 2.0
messages = ['363884', '363893']
nosy_count = 2.0
nosy_names = ['eric.smith', 'brad.scarlett@gmail.com']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue39929'
versions = ['Python 3.8']

[bscarlett]

I noticed that dataclasses.asdict seems to incorrectly reconstruct collections.Counter objects with the counter values as tuple keys.

eg:

In [1]: from collections import Counter
In [2]: from dataclasses import dataclass, asdict
In [3]: c = Counter()
In [4]: c['stuff'] += 1
In [5]: @DataClass
...: class Bob:
...: c: Counter
...:
In [6]: b = Bob(c)
In [7]: c
Out[7]: Counter({'stuff': 1})
In [9]: b.c
Out[9]: Counter({'stuff': 1})
In [10]: asdict(b)
Out[10]: {'c': Counter({('stuff', 1): 1})}
In [11]: asdict(b)['c']
Out[11]: Counter({('stuff', 1): 1})

The Counter gets reconstructed with its item tuples as keys.

This problem seems to have similar aspects to https://bugs.python.org/issue35540

[ericvsmith]

The asdict API was a mistake, because it's not possible for it to know how to create all possible sub-objects. I can't decide what to do about it. I might just "abandon it in place", by documenting its problems and suggesting it not be used.

I should research what attrs does.

[rcgale]

I just encountered this as well, and it was pretty frustrating!

The offending part of _asdict_inner() in dataclasses looks like this:

    elif isinstance(obj, dict):
        return type(obj)((_asdict_inner(k, dict_factory),
                          _asdict_inner(v, dict_factory))
                         for k, v in obj.items())

The assumption here is that Counter can be re-initialized from a dict_items-like iterable, but that's not how it was implemented. For example:

>>> example_counter = Counter({"a": 1, "b": 2, "c": 3})
>>> Counter(example_counter)  # this works
Counter({'c': 3, 'b': 2, 'a': 1})

>>> Counter(example_counter.items())  # this mangles
Counter({('a', 1): 1, ('b', 2): 1, ('c', 3): 1})

This could be fixed by treating Counter differently in _asdict_inner(), something like:

    elif isinstance(obj, Counter):
        return Counter(dict((_asdict_inner(k, dict_factory),
                          _asdict_inner(v, dict_factory))
                         for k, v in obj.items()))
    elif isinstance(obj, dict):
        # ...

As to whether it should be fixed: Even if this function can't correctly handle every possible subclass of dict, shouldn't it at least support the ones that ship with Python?

[rcgale]

Another suggestion that could be cleaner: if a class implements __getstate__ and __setstate__, then _asdict_inner() ought to favor that approach. (Side note: why doesn't json use __getstate__?)