Document that sys.stdout.write() and print() replaces a New line character with Carriage Return and New line characters.

[tballas]

This issue might serve as partial documentation.

This took a lot of digging to find. A carriage return sent over usb serial is replaced with a new line character when sent e.g. sys.stdout.write('String\n'), print('String\n', end='') or print('String') becomes 'String\r\n' on the pc side.

The source of the change seems to be the mp_hal_stdout_tx_strn_cooked() function in \lib\utils\stdout_helpers.c
It has the comment:

"Send "cooked" string of given length, where every occurrence of LF character is replaced with CR LF."

[amyreese]

Is there any way to work around this, short of modifying that function and building my own custom firmware to use?

[tannewt]

@jreese Modifying is easiest. What are you trying to do?

[amyreese]

Was attempting to use this to interface with a PC game over USB serial, where the game was designed for Arduinos and only expected LF, and would break on commands ending with CR/LF. However, after discussing it with the developer, they have since updated the game to be compatible with CR/LF.

It did eat up many hours of debugging efforts before I discovered that CircuitPython was silently doing this conversion. Certainly didn't help that count = sys.stdout.write("foo\n") returns 4, even though it's actually writing five characters to the serial console.

[tannewt]

Ok, thanks for the background. I agree we should document this somewhere. Maybe if/when we rework the serial in and out to the supervisor.

[dhalbert]

I thought this must have come up in MicroPython before, and sure enough, yes: micropython#1260

A feature was implemented in MicroPython to do raw output, sys.stdout.buffer.write(), but it's not turned on right now (MICROPY_PY_SYS_STDIO_BUFFER) in CircuitPython (nor for most board in MicroPython, it appears). I'm not sure whether just enabling this will work or whether we need to reimplement it a bit, but I think this is the way forward to solve this.

[pmp-p]

i think \n should not be replaced at all anywhere
Better change instead :
https://github.com/micropython/micropython/blob/master/py/modbuiltins.c#L402
to allow to chose beetween \r\n or \n for end of lines.
and also https://github.com/micropython/micropython/blob/master/py/modbuiltins.c#L452

ideally for fd < 3 => globally user defined and fd>2 could default to \n

[olivier-boesch]

Hello.
I'm also struggling with this replacement. Have you planned to work on this?
Can we help?
Thanks.

[tannewt]

We don't have plans to work on it. My priority would be to support a second CDC connection that is separate and doesn't do any munging.

[ladyada]

ditto on closing as wontfix, send people to dual CDC instead?

[dhalbert]

We now have a second CDC channel on many boards: https://circuitpython.readthedocs.io/en/latest/shared-bindings/usb_cdc/index.html. It sends an dreceives raw bytes, with no substitutions.