json.load() can raise UnicodeDecodeError, but this is not documented

[mattheww]

BPO	43340
Nosy	@rhettinger, @ericvsmith, @mattheww, @serhiy-storchaka, @kamilturek
PRs	bpo-43340: Document possible UnicodeDecodeErrors from json.load() #25173

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-02-27.18:24:21.488>
labels = ['type-bug', '3.8', '3.9', '3.10', 'docs']
title = 'json.load() can raise UnicodeDecodeError, but this is not documented'
updated_at = <Date 2021-04-08.06:12:00.019>
user = 'https://github.com/mattheww'

bugs.python.org fields:

activity = <Date 2021-04-08.06:12:00.019>
actor = 'rhettinger'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Documentation']
creation = <Date 2021-02-27.18:24:21.488>
creator = 'mattheww'
dependencies = []
files = []
hgrepos = []
issue_num = 43340
keywords = ['patch']
message_count = 5.0
messages = ['387780', '387794', '387795', '387851', '387879']
nosy_count = 6.0
nosy_names = ['rhettinger', 'eric.smith', 'mattheww', 'docs@python', 'serhiy.storchaka', 'kamilturek']
pr_nums = ['25173']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue43340'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']

Linked PRs

• gh-87506: Document that json.load*() can raise UnicodeDecodeError #127355
• [3.13] gh-87506: Document that json.load*() can raise UnicodeDecodeError (GH-127355) #128607
• [3.12] gh-87506: Document that json.load*() can raise UnicodeDecodeError (GH-127355) #128608
• gh-87506: Amend json.loads() post PR #127355 #128609
• [3.13] gh-87506: Amend json.loads() post PR GH-127355 (GH-128609) #128610
• [3.12] gh-87506: Amend json.loads() post PR GH-127355 (GH-128609) #128611

[mattheww]

The documentation for json.load() and json.loads() says:

« If the data being deserialized is not a valid JSON document, a JSONDecodeError will be raised. »

But this is not currently entirely true: if the data is provided in bytes form and is not properly encoded in one of the three accepted encodings, UnicodeDecodeError is raised instead.

(I have no opinion on whether the documentation or the behaviour should be changed.)

[ericvsmith]

As a rule we don't try and document every exception that can be raised. I could go either way on documenting encoding errors with the json module, although it seems pretty clear that an encoding error would be possible in this case.

[rhettinger]

Normally, we don't (or can't) enumerate all possible exceptions. But
in this case, it is worth expanding the documentation so that person can know which of two common input errors they need to catch:

"If the data being deserialized is not valid UTF-8 a UnicodeDecodeError will be raised, and if the decoded file is not
a valid JSON document, a JSONDecodeError will be raised".