Arduino Nano ESP32 - Paste mode dropping characters after 200ish chars

[AwesomeCronk]

I recently received an Arduino Nano ESP32 in order to test using a ublox NORA-W106 with a different project. Any attempts I try to get code to the device via paste mode are failing miserably. I can get about 250 or so bytes across before it starts dropping characters like crazy. For example, here is the code that I tried to send vs what the board saw:

# This is the stub program which is downloaded to the uPython device. It receives commands from the host via stdin (REPL UART link).

import os, machine

buffer = b''
batch = 1024

runFlag = True
while runFlag:
    cmdText = input('\nready~')
    cmd, *params = cmdText.strip(== 'cd'missi    contin      on as print(ams) <       )); cst':
 ing p[0])
 pr(E)) djust B',  forpe = 'f               if lemSemSiz     + suf'

       ite: {}'me))       lenontinn asint('~ms) <     rint('~mplete    : fil((E)))rite'ssing p'); nu  excentinu  s   eonti     ifms'); Excepti       priif len(ut(d) lse:
d == ~~erro       whiHex[0:        buffch':


    '~~co:

I discovered this because my tool uPyFile wasn't able to get the stub program (the above code I tried to send) sent to the board. It works by starting paste mode (Ctrl+E), sending the raw Python code, and running it (Ctrl+D). With other ESP32s the uart buffer size has been capped at 64 bytes, so uPyFile has an option to limit the number of bytes it sends before flushing the serial port. Here's a sample of that code:

def writePort(data):
    pcLog.debug(data)
    # Write the data in batches to ensure the buffer never overflows
    remaining = data
    while len(remaining) > bufferSize:
        port.write(remaining[0:bufferSize])
        remaining = remaining[bufferSize:]
        port.flush()
    if len(remaining):
        port.write(remaining)
        port.flush()

The problem is, port.flush() doesn't appear to actually be flushing the port. It's supposed to wait until there are no characters left in the TX buffer so that the receiving device can consume them all before being bombarded with new data. However, it doesn't appear to be. The latest version of uPyFile (v3.2.1) was built with PySerial 3.5 on Ubuntu 20.04 LTS (the same machine I'm using right now).

I'm just trying to figure out if this is caused by something I messed up or by weird behavior with the Arduino board. If it's a problem on my end I want to fix it so my tool works. If it's not I still want to create a workaround so my tool works. 😝

If anyone would like to try replicating this, you'll need to download uPyFile from https://github.com/AwesomeCronk/uPyFile/releases and run this command: upyfile -p /dev/ttyACM0 -B 64 -v init. It will print out a bunch of verbose messaging detailing what exactly it is sending over the serial port and what it is receiving when it listens. If you're on Windows I don't have a binary built yet for 3.2.1, sorry about that!

[robert-hh]

For pasting code to the board you have to use paste mode or raw mode. The are made for that purpose. Normal REPL mode will not work for several reasons. One is the speed of processing oncoming characters, the other is the automatic indentation, and more.

[AwesomeCronk]

Maybe I wasn't clear enough in the original post, but this is using paste mode. Paste mode is dropping characters.

[Josverl]

Can you reproduce the issue in a smaller piece of code?

Does it only repro with compiled code, or also with plain Python?

Do you have a minimal repro?

It's hard to need to reverse engineer your code on both the host and MCU.

If you can isolate the issue in sending / receiving a block, that would make it simpler to help

What I've found useful in debugging other espxx serial issues is to connect a logic analiser to the RX/TX pins to watch what is actually going over the wire.

Also pretty sure your tool will not work well on windows as the tool appears to omit some dtr/rts handling needed for Esp reliable hard/ soft resets.

[AwesomeCronk]

Yes, the issue does reproduce for any chunk of code bigger than about 250 characters. I tried sending pieces of code, decreasing in size, and kept having the same issue. Here's a quick list of how many characters I pasted at once and how many it received before it got corrupted:

• 711 sent, 258 received ok
• 591 sent, 258 received ok
• 523 sent, 279 received ok
• 412 sent, 258 received ok
• 296 sent, 284 received ok
• 162 sent, 162 received ok
• 124 sent, 124 received ok

If I send a small batch of characters, say 100-150, then send another, both are received ok. I think this opens the door to a workaround to wait 100ms or so after flushing the serial port. It seems to me that the root issue is somewhere between PySerial flushing the serial port and the Arduino signalling when its buffer is flushed.

I should note, all of these were valid snippets of Micropython. They were raw code, not compiled. None of the code I have tried to send, either today or yesterday, was compiled.

As for minimal reproducible examples, any code at all will do it if it's bigger than about 250-300 characters. None of it ever gets to the point of running because it gets corrupted during the paste process. If you hit Ctrl+D after pasting, you will get syntax errors because of the corruption and it will not run.

I should be receiving an oscilloscope tonight, which should allow me to capture what's going on hardware-wise. Unfortunately I can't tell if the NORA-W106 (the ESP32) is connected straight to USB or has a USB/serial converter on it, so I don't know that I will have access to an actual UART. This is the board and Micropython image I used: https://micropython.org/download/ARDUINO_NANO_ESP32/ I used Micropython v1.21.0. Nothing has been modified on this image at all. No code has properly run, no configuration has been changed, no files edited.

The lack of DTR/RTS is intentional, my tool never reboots the device. In order to get the REPL it just rudely sends Ctrl+C to whatever script is running on the device and waits for it to give the REPL prompt (>>> ). It then enters paste mode (Ctrl+E), sends a stub program, and runs it (Ctrl+D). All file management afterward runs through the stub program, which has my own logic baked in to manage data flow.

[robert-hh]

Sorry for not reading carefully. Obviously your device is conuming the data slower that it is sent. Then, the data rate has to be lowered. Usual method are

• sending in chunks with delay in between (which you tested),
• flow control - which is not available
• reducing the baud rate. With generic USB this is not a problem can can be done purely at the PC side. If you device uses a USB/serial bridge, then the ESP32 would have to be adapted. As far as I understand, the NORA 106 has USB at GPIO19/20, which matches the ESP32-Sx mcu. That could have a generic USB interface.

[AwesomeCronk]

All good!

I tried a baud rate reduction from 115200 to 9600 with pretty much no change in results.

I added an option to my tool to use a delay instead of flow control. With a buffer size of 128 bytes and a delay of 200ms at 115200 baud, I was able to get 3566 characters of the stub program (total of 4943 chars sent) to the device before corruption appeared (line 106 of stub.py). At that point, 5 characters were dropped, then 247 more went through. After that, a newline and = (' were received, which does not appear in the program at all, indicating that most of the rest of the code sent was dropped. Nothing else went through after those five characters, not even the Ctrl+D to finish paste mode.

I tried various combinations of baud rate (9600 or 115200), flow delay (50ms or higher), and buffer size (less than 200) and all of them had the same result: about 3000 characters go through ok, then about 2-300 more with various drops in between, then none of the rest make it. I even tried running uPyFile to try sending the stub program, then when it failed, I killed it and switched to a serial terminal. After typing characters and hitting enter I saw the === paste mode prompt after each line.

The only thing I can think of is that the memory available to the USB interface on the NORA-W106 is filling up with the output buffer and, since I'm not actually reading data back until the whole stub is sent, it is failing to receive characters because it is out of memory. I've never used this board (or any ESP32-S3-based board) before though, so I can't be certain.

[robert-hh]

If not reading back the data is the culprit, you could use raw mode, which does not echo.
Send Ctrl-A, then paste the code, then send Ctrl-D and Ctrl-B to get back to REPL.
I tried that with an generic ESP32 and a 7k piece of code. Normal paste mode fails, but raw mode works.

[robert-hh]

Update: Seems to be a ESP32Sx issue, hardware or port. Trying the Raw mode paste works with a generic ESP32 and a ESP32C3, and fails with ESP32S2 and ESP32S3.

[AwesomeCronk]

Interesting. I might look into wiring a UART onto this board. My design down the line will have one instead of generic USB anyway.

I wish I had the tools and skillset to debug the ESP32-S3 itself while this was happening to try to identify the root cause of the problem.

[robert-hh]

Did you try to use mpremote? That one works with the ESP32Sx boards.
b.t.w. I tried the paste approach with other USB based boards like RP2, SAMD, Teensy 4.x, and it works for pasting in both raw mode and regular mode.