email: encoded headers lose their quoting when refolded

[EmilStyrke]

BPO	43554
Nosy	@warsaw, @bitdancer, @maxking
Files	test_folding_bug.py: unittest displaying the issue

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-03-19.09:45:02.487>
labels = ['type-bug', 'library', '3.9']
title = 'email: encoded headers lose their quoting when refolded'
updated_at = <Date 2021-03-19.18:30:12.248>
user = 'https://bugs.python.org/EmilStyrke'

bugs.python.org fields:

activity = <Date 2021-03-19.18:30:12.248>
actor = 'shihai1991'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2021-03-19.09:45:02.487>
creator = 'Emil.Styrke'
dependencies = []
files = ['49893']
hgrepos = []
issue_num = 43554
keywords = []
message_count = 1.0
messages = ['389061']
nosy_count = 4.0
nosy_names = ['barry', 'r.david.murray', 'Emil.Styrke', 'maxking']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue43554'
versions = ['Python 3.9']

Linked PRs

• gh-87720 Fix additional header refolding-quoting edge case #134271

[EmilStyrke]

When a header with an encoded (QP or Base64) display_name is refolded, it may lose (some of) its encoding. If it then contains illegal "atext" tokens, an invalid header will result.

For example, From: =?utf-8?Q?a=2C=20123456789012345678901234567890123456?= <abc@example.com> will become From: a, 123456789012345678901234567890123456 <abc@example.com> This contains a comma character which needs to be quoted: correct rendering would be From: "a, 123456789012345678901234567890123456" <abc@example.com>. Note that this example isn't even folded to multiple lines, since the decoded text is short enough to fit in one line.

This can be triggered by BytesParser(policy=policy.default).parsebytes("From: =?utf-8?Q?a=2C=20123456789012345678901234567890123456?= <abc@example.com>").as_bytes(), but the offending code seems to be in or below email.policy.EmailPolicy.fold. See attached file for examples with and without folding.

[brychcy]

See #121284

[medmunds]

rfc2047 RFC-2047 encoded-word