Writing data in parallel for multiple ws2812b led strings

[Lahaije]

Dear all,

I'm currently working on a project to automatically map a lot of ws2812b pixels in a tree. The mapping and calculations itself are done in python, and I run MicroPython on a ESP32 to drive the pixels.

On the ESP32, I use asyncio.start_server to receive data.
The build in neopixel library is used to send data to the led.
The relevant code for showing the led

string1 = neopixel.NeoPixel(4), 300)
string2 = neopixel.NeoPixel(5), 300)

for every frame:
  string1.write()
  string2.write()

The combination is working, but the performance is not what I expect. The servers appear to be not fully stable, and the frame rate is a bit low. (8 frames / sec).

I'm looking for a solution to increase the frame rate. At the moment data is first send to string1 and when complete, send to string2. I would like to send the data to both strings in parallel. I can't seem to find this option in the neopixel library. Are there any libraries supporting parallel data?

I plan to increase the number of to about 1000 pixels in the project. For the hardware, I would like to have up to 10 strings containing 100/150 pixels each. Writing 10 streams in parallel would considerately decrease the time to send a single frame. At the moment, I don't know if I can find 10 suitable pins on the esp32, but that's a different issue.

If relevant, the complete code is available at https://github.com/Lahaije/PixelTree/tree/main/src/MicroPython
esp32-20220618-v1.19.1.bin is installed at the ESP32

[jgriesfeller]

Since I was interested in doing a similar project in Micropython (this one to be precise). The conclusion of my research was that this not trivial in Arduino (due to the non flexible timing of the neopixels), and likely impossible in Micropython (besides one strip):

I have used with 8 strips in parallel the Adafruit_NeoPXL8 library (up to eight strips, needs samd21, samd51, rp2040 or ESP32S3 controllers)
There's also FastLED that should be able to do similar things. But I think Micropython is the wrong programming environment.

[Josverl]

For performance you need to have this delegated to a MCU peripheral. There may be multiple options for you, but before diving into that, how many strings of how many pixels are you looking for?

[Lahaije]

I'm planning on 10 strings of 100 pixels.
In the future, I might extend to 10 strings of 150 pixels.

[mattytrentini]

I see no reason why it wouldn't be possible in MicroPython...

It's likely you'd need to drive them from RMT to keep up the precise timing, but that would be valid for the other environments too.

It doesn't mean it would be easy in MicroPython - a lot of effort went into FastLED to optimise performance for example. But everything time-critical can be offloaded to hardware peripherals so it ought to be possible.

[mattytrentini]

No, none of the available Neopixel libraries (that I'm aware of!) perform parallel updates; you'll need to look at your own low-level implementation. Using multiple channels of RMT seems like it would be the most promising approach.

You may also want to consider investigating the Raspberry Pi Pico as there has been a lot of activity driving Neopixels by PIO.

[jgriesfeller]

Adafruit_NeoPXL8 does parallel updates (even DMA driven). But the number of peripherals is limited (e.g. 8 PIO for RP2040 and 8 RMT channels for the original ESP32) On the ESP there's also the I2S peripherals that can be used. Not sure if in parallel to RMT though.
I am also not saying that it's not possible in Micropython, but I think you need to do most so low level that you it's pretty similar to C/C++ that it might be easier to do it just with that.
I'll be very interested in the outcome of your work. I just don't have enough time to try myself.

[mattytrentini]

Would be interesting to try wrapping up Adafruit_NeoPXL8 (or FastLED). In general, I'd expect for these kind of performance-sensitive features at least some development would be in C with MicroPython APIs. And yes, definitely agree that we ought to be considering using hardware where possible.

[Lahaije]

It appears there is no readily available library to use. I'm going to try to make an implementation myself, but I'm unsure if I manage to pull it off.
Over the years I read quite some articles regarding the exact specifications for the NEO pixels. Most important for this attempt is the timing. Only sending a 0 bit has very strict timing. There is quite some slack in sending a 1, a reset or wait time before the next bit can be sent.
I'm going to base me on https://wp.josh.com/2014/05/13/ws2812-neopixels-are-not-so-finicky-once-you-get-to-know-them/
The important timing constrains from the article :

Symbol	Parameter	Min	Max	Units
T0H	0 code ,high voltage time	200	500	ns
T1H	1 code ,high voltage time	550	5,500	ns
TLD	data, low voltage time	450	5,000	ns
TLL	latch, low voltage time	6,000		ns

For timing, this means the following :
| 0 Bit | 200 ns high | 450 / 5000 ns low |
| 1 Bit | 550 ns high | 450 / 5000 ns low |
| reset | 0 ns high | 6000 ns low |

When sending a 0 bit, the pin should be set high between 200 and 500 ns. Sending a 1, the pin is allowed high between 550 and 5500 ns. After sending a bit (0 or 1) there is about 5000 ns before the next bit must be sent. This timing is not fixed as long as the next bit is sent before the reset. With these constrains, it is not necessary to send the bits in a fixed frequency. It's therefore also not needed to write to all strings in a fixed order. I'm currently considering the following logic :

num_pixels = int  # Max number of pixels in the longest string
num_strings = int  # Number of strings connected

write()
  for bit in range (8*num_pixels):
    runs = []  # Extract the next bit from each string. Something like [1 , 0 , 1, 0] in case of 4 strings

    zerobits = [i for i in range(num_strings) if not runs[i]]  # Index of string containing a 0 bit [1, 3]
    onebits =  [i for i in range(num_strings) if runs[i]]  # [0, 2]

    for index in onebits:
      Pin(index).value(1)

    ts = start_timer
    for index in zerobits:
      Pin(index).value(1)

    wait(time - ts > 200)

    for index in zerobits:
      Pin(index).value(0)

    wait(time - ts > 550)

    for index in onebits:
      Pin(index).value(0)

The above code is of course a bit simplified. I need to have a couple of open questions answered before I can start implementing anything.

• Is it possible to execute the complete write for all Neopixel data without getting an interrupt?
• Can " wait(time - ts > 200) " be executed without exceeding 500 ns (timing precise enough?)
• How long does is take to set a pin high or low?
• Is it even possible to accurately send data to neopixels calling Pin().value()

Since I'm driving > 1000 pixels, it's irrelevant how much power the esp32 will use. The power consumption of the ep32 will not significant contribute to the power consumption of the complete project.

[RomeoSimp]

So is there a library the controls WS2812B thru ESP32 using micropython? Is it possible?

[Lahaije]

I did some experimentation today to check what the capabilities within micropython are.
My esp is running with a clock frequency of 160 Mhz.
For sending a 0, I should make the Pin High somewhere between 32 and 80 clock cycles.

After some experimentation with the clock functions, I couldn't make 2 calls to time.ticks_cpu() faster than 661 clock cycles apart. Much too slow for transferring the data. To confirm, I wrote a very simple program to sent data to the neopixel

import machine
p0 = machine.Pin(4, machine.Pin.OUT)

for i in range(24):
    pp.on()
    pp.off()
time.sleep(1)

As expected, the first neopixel light up full brightness. This implementation is too slow to have any chance of sending a 0. Setting a pin high or low takes around 1000 / 4000 (I could not get a consistent reading) according to time.ticks_cpu().

I will explore some other options now.

Would it be feasible to take a similar approach in C ?
Maybe I can access my pins fast enough in C to get the speed required.

[mattytrentini]

You should probably watch Writing Fast and Efficient MicroPython (if you haven't already)!

[Lahaije]

Thank you for the link. I'm new to MicroPython and was totally unaware of the possibilities with assembly. From the talk, it seems possible to blink a led in 0.037 uSec (using @micropython.asm_thumb). 37 ns should be fast enough to send a logical 0. With 10 strings this increases to 370 ns for a logical 0 to all strings. This is below the maximum of 500 ns, making this a theoretical possibility.