What is wrong with this email related code

[rkd77]

Bug report

Bug description:

from email.header import Header
from email.message import EmailMessage
from email.policy import EmailPolicy

message = EmailMessage(EmailPolicy(utf8=True))
subject = 'TEST TEST SSL certificate for the domain a.b.c has been created'
message['Subject'] = Header(subject, 'utf-8').encode()

I guess it is a bug.

CPython versions tested on:

3.9

Operating systems tested on:

Linux

[ericvsmith]

You have not described what error you saw, if any. What result did you get, and what was the expected result?

[donBarbos]

Output for more context:

Traceback (most recent call last):
  File ".../test.py", line 7, in <module>
    message['Subject'] = Header(subject, 'utf-8').encode()
    ~~~~~~~^^^^^^^^^^^
  File ".../cpython/Lib/email/message.py", line 441, in __setitem__
    self._headers.append(self.policy.header_store_parse(name, val))
                         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File ".../cpython/Lib/email/policy.py", line 146, in header_store_parse
    raise ValueError("Header values may not contain linefeed "
                     "or carriage return characters")
ValueError: Header values may not contain linefeed or carriage return characters

I reproduced the error on versions from 3.9 to 3.14 (main branch) - the same result

The problem is that the Header.encode method uses separators such as linesep='\n' but EmailMessage does not accept headers containing control characters.
For your case it is suitable to use str(Header(subject, 'utf-8')) for which the Header class has a method __str__.

Thanks for openning issue but its title looks like a question that would be appropriate for the Python Discuss forum under the Help category.

if I misunderstood something, let me know

[ZeroIntensity]

I've marked this as pending because yeah, we don't want newlines in headers, and encode() happens to be returning a newline here, so this isn't a bug. But, @rkd77, if there's anything that can be improved in terms of documentation here, we'd be happy to fix that.

[Harry-Lees]

The RFC seems to indicate that \r\n is allowed in subject fields

Each header field is logically a single line of characters comprising
the field name, the colon, and the field body. For convenience
however, and to deal with the 998/78 character limitations per line,
the field body portion of a header field can be split into a multiple
line representation; this is called "folding". The general rule is

that wherever this standard allows for folding white space (not
simply WSP characters), a CRLF may be inserted before any WSP. For
example, the header field:

       Subject: This is a test

can be represented as:

       Subject: This
        is a test

https://datatracker.ietf.org/doc/html/rfc2822#section-2.2.3.

The docstring in Header.encode() also seems to indicate that \r\n should be a valid line separator

Optional linesep is a string to be used to separate the lines of
the value. The default value is the most useful for typical
Python applications, but it can be set to \r\n to produce RFC-compliant
line separators when needed.

even though the following code

from email.header import Header
from email.message import EmailMessage
from email.policy import EmailPolicy

message = EmailMessage(EmailPolicy(utf8=True))
subject = 'TEST TEST SSL certificate for the domain a.b.c has been created'

header =  Header(subject, 'utf-8').encode(linesep="\r\n")

message['Subject'] = header

still produces the same error

[rkd77]

For your case it is suitable to use str(Header(subject, 'utf-8')) for which the Header class has a method str.

No, it is not. If there is a real utf-8 character in subject, eg. ę . output of str(...) is utf-8 text instead of apropriately encoded subject.

[ZeroIntensity]

The RFC seems to indicate that \r\n is allowed in subject fields

It's only sort of valid with the abstraction here. It has to be used at the end of a header field, not anywhere else. Otherwise, you could inject other headers into the field e.g., Subject: Gotcha!\nReply-To: Evil <evil@evil.com>\n. (In fact, we recently fixed a security issue related to those kinds of shenanigans.)

@rkd77 This is because you're trying to mix the legacy Header class with the modern EmailMessage class. EmailMessage is designed to handle the encoding for you, not take results from encode. The issue is actually a mix of the maxlinelen and linesep parameters; the EmailMessage class doesn't like newlines in the headers, but encode will automatically insert newlines into messages that are long enough, which is why you're seeing the error.

You should either:

• Use Message instead of EmailMessage.
• Or, just don't use Header with EmailMessage; simply put the str you want in the subject and it will encode it for you.

[rkd77]

subject = 'TEST TEST SSL certificate for the domain ą.b.c has been created'
text = 'text'
message['Subject'] = subject
message.set_payload(text, 'utf-8')
return bytes(message.as_string(), 'utf-8')

It is not sufficient. subject ought to be encoded, it is utf-8 text here. So on non-utf8 MUA it won't be displayed correctly.

[rkd77]

Message with Header(...).encode() and return message.as_bytes() gives correct output.
Thanks.