ISRs, asyncio, threading question

[saraverbeecke]

Hi,

What is the best way to send a short message over I2C as fast as possible to a peripheral after an external interrupt has occurred?
I have a "clock pulse" signal coming in as an external input on a stm32f4. (max clock pulse train comes in at 100 Hz)
I already have an async application running with no time critical coroutines. Building a complete realtime system is not required, but the only event that needs accurate timing is sending a short (4 bytes) message over I2C to a peripheral as fast as possible after the interrupt occurs.

1. Using a separate thread waiting for a flag:
   Will cause CPU hogging so I tried using a time.sleep but this will cause irregular timings of the i2c.send call after an interrupt occurs

2. Using a ThreadSafeFlag in the ISR and a async coroutine waiting for the event?
   Will have to much latency as some other coroutines will need 20 ms

3. Using i2c.send directly in the ISR (is this even possible?)
   This cause a MemoryError so no go

4. Using micropython.schedule to schedule a function where the i2c.send call happens?
   This seems to be the solution but I'm not sure if this is capable of "pausing" an ongoing coroutine. Say a coroutine is running, will the function scheduled by micropython.schedule inside the ISR run after the coroutine is finished completely or will it run "in between" somewhere? The longest coroutine is about 20 ms so the chance the ISR is called during this task is high.

Any tips in handling this single "realtime event" in a non-realtime application?
I have read the ISR docs and the great info by @peterhinch on interrupts, threading and asyncio https://github.com/peterhinch/micropython-async/blob/master/v3/docs/INTERRUPTS.md, but still figuring out where the ISR will actually run when other tasks (coroutines) or functions or running.

Here some non-complete example code showing the 4 things I have tried already.

# Using a thread waiting for a flag
def pin_callback(pin):
    global flag, extint
    extint.disable()
    flag = True
    extint.enable()

def set_i2c_values():
    i2c.send(event_values, addr=ADDRESS, timeout=5)

running = True
def th_func(id):
    global running, flag
    try:
        print("Running thread...")
        while running:
            if flag:
                flag = False
                perform_step()
            else:
                time.sleep(0.001)
        print("Thread ended.")
    except Exception as e:
        print(e)
    finally:
        print("Thread exit.")

extint = ExtInt("PA5", ExtInt.IRQ_RISING, Pin.PULL_NONE, pin_callback)
t = _thread.start_new_thread(th_func, (1,))







# Using a coroutine as task waiting for a flag
thread_safe_flag = asyncio.ThreadSafeFlag()
def pin_callback(pin):
    global flag, extint
    extint.disable()
    thread_safe_flag.set()
    extint.enable()

def set_i2c_values():
    i2c.send(event_values, addr=ADDRESS, timeout=5)

async def clock_follower():
    while True:
        await thread_safe_flag.wait()
        set_i2c_values()

extint = ExtInt("PA5", ExtInt.IRQ_RISING, Pin.PULL_NONE, pin_callback)







# Directly inside ISR
def set_i2c_values(event_values):
    i2c.send(event_values, addr=ADDRESS, timeout=5)

event = bytearray([1,2,3,4])
def pin_callback(pin):
    extint.disable()
    set_i2c_values(event)
    extint.enable()

extint = ExtInt("PA5", ExtInt.IRQ_RISING, Pin.PULL_NONE, pin_callback)








# Using micropython schedule
def set_i2c_values(event_values):
    i2c.send(event_values, addr=ADDRESS, timeout=5)

event = bytearray([1,2,3,4])
def pin_callback(pin):
    extint.disable()
    micropython.schedule(set_i2c_values, event)
    extint.enable()

extint = ExtInt("PA5", ExtInt.IRQ_RISING, Pin.PULL_NONE, pin_callback)

[GitHubsSilverBullet]

The STM32F4 has multiple DMA channels available.
I'd prepare the entire DMA transfer to the I2C beforehand and use a hard IRQ Trigger.
Within that ISR I'd just trigger the DMA transfer.
It's hardly possible to make that any faster.

[saraverbeecke]

Ok thanks for the tip, I was not aware of it. Any example(s) how this can be done in Micropython for the stm32f4? I never used the DMA transfer in Micropython and the docs cover only rp2...

[GitHubsSilverBullet]

Well, you'll have to do it all manually.
Which means reading the STM32F407 datasheet (that's the one I used) in depth.
And manually access and modify the registers with Micropythons mem32[] command, the viper ptr32() or should you insist, the thumb assembler.
All those approaches worked perfectly with the three ADCs, don't see a reason why it shouldn't work with I2C.
Speed and reaction time will be in the sub µs range.
Have fun.

[bixb922]

Option 4: micropython.schedule will interrupt any running code except another scheduled routine. It will not wait for an asyncio coro to yield. So this option is really low latency, better than 1 to 3.

The DMA option described above is probably even faster.

[saraverbeecke]

Oh, so it will "pause" the coroutine even if it is not in an await state?

[GitHubsSilverBullet]

The DMA option described above is probably even faster.

At least a hundred times… ;-)
And that's a conservative estimate, because anything ›scheduled‹ can't even start before any currently running Micropython statement has completely finished.

But I agree, it all depends on the required reaction time.

[saraverbeecke]

Ok thanks. Does anyone know what the dma=True really does in the pyb.I2C init call? I tried this and put the send call in the ISR but nothing happened, maybe I did something else wrong. Did anyone used that option before?

[rkompass]

The thing is that currently we only have synchronous (i.e. blocking) SPI and I2C. Which are not suitable for an ISR.
Once there are asynchronous (non-blocking) SPi and I2C available certainly the transfers may be initiated from a ISR.
@GitHubsSilverBullet's DMA solution is like a hack that does the asynchronous transfer. Exactly what I wanted to suggest (while sitting in the sauna yet) :-)

The dma=True option certainly is referring to an implementation detail not to an asynchronous operation mode afaiu. So it would not change the general picture. However, given it is implemented it should not be too hard to change the API to include an asynchronous mode.

[peterhinch]

Using micropython.schedule has the merit of being easy, but expect latency in the region of 3ms. Latency can be kept low by periodically issuing gc.collect(). This is because doing this keeps GC time low, and if a hard ISR pre-empts GC, micropython.schedule will wait until the GC is complete.

If ~3ms is too long, then DMA is probably the only solution.

[saraverbeecke]

Thanks @peterhinch i'll use this for now. 3ms is acceptable. Any good rules of thumb for the gc.collect() rate?

[peterhinch]

Any good rules of thumb for the gc.collect() rate?

This is entirely application dependent. The concept is that, as an application runs, RAM blocks are allocated then fall out of scope. GC detects these and returns the unreferenced block to the pool. The length of time this takes increases with the number of blocks. I suggest running a task that does a GC (say) once per second, timing gc.collect(). Then increase the rate, and see what effect that has on the collect time.

One thing I omitted to mention: using a soft interrupt is the same as running a hard interrupt and using that to call micropython.schedule. Unless you have a specific need for a hard IRQ, just run a soft one and do the I2C in the service routine.