What resources are used by the WiFi stack?

[hanshuebner]

Hello,

on the WiFi Support page, it is stated that the WiFi stack requires both flash and RAM. Does it use resources beyond that? I'm particularly interested in DMA channels and PIOs because my application uses both and I observe differences in behavior of my code with and without WiFi. As MicroPython does not support DMA, I'm using direct register access which might collide with whatever allocation scheme is used by the network stack, but I'm speculating.

Any information on this would be appreciated!

Thanks,
Hans

[jimmo]

Yes, the rp2040 uses PIO to talk to the cyw43 wifi chip. (i.e. it's not using the regular SDIO mode that we use on the other ports like the Pyboard D, rather a special SPI mode that uses a PIO).

As MicroPython does not support DMA

There's no Python-level API for DMA but parts of MicroPython do use DMA (e.g. SPI). It's possible the cyw43 driver does also. A quick look at cyw43_bus_pio_spi.c (in the pico-sdk) suggests that it does.

[hanshuebner]

@jimmo Thank you for the pointer to the cyw43 driver - It does indeed look as if both a sizeable PIO program and two DMA channels are used, and that the driver uses the pio_claim_unused_sm and dma_claim_unused_channel to allocate its resources. Looking at rp2_pio.c in the MicroPython code, however, I could not figure out whether it would make sure that the Python code attempts to use a state machine that is already in use by the WiFi stack (the i2s driver seems to use pio_claim_unused_sm though).

I'm currently working on a project where I use both PIO blocks and interestingly, things mostly work - I had problems when trying to use state machine 0, and that is probably caused by its use in the cyw43 driver. When choosing a higher state machine number, though, things work fine - I find that surprising, given that both cyw43_bus_pio_spi.pio and my own PIO program occupy much of a PIO block's address space of 32 words. Is it possible to quickly switch between PIO programs, and that this is done by the WiFi driver?

Regarding DMA, I have been observing that things work most of the time, but sometimes I get DMA transfer failures that I have no explanation for. Given that I'm statically allocating my DMA channel numbers, but the WiFi driver uses dma_claim_unused_channel, this might be caused by the WiFi driver allocating a channel that I am using.

I think both problems should addressed in MicroPython by making the the PIO driver claim state machines when the user uses them and unclaiming them when usage ends. Also, it would make sense to expose the dma_claim_unused_channel and dma_channel_unclaim API calls so that DMA usage from MicroPython can be made to play nice with other drivers that use DMA. Would a new machine_dma module make sense for this?

[rkompass]

I think both problems should addressed in MicroPython by making the the PIO driver claim state machines when the user uses them and unclaiming them when usage ends. Also, it would make sense to expose the dma_claim_unused_channel and dma_channel_unclaim API calls so that DMA usage from MicroPython can be made to play nice with other drivers that use DMA. Would a new machine_dma module make sense for this?

For the DMA side I think a machine._dma module would be very helpful. For a start it would be sufficient to have
dma_claim_unused_channel and dma_channel_unclaim as well as two methods to register and unregister a callback function there.
I have routines for everything else, using register access in MP in my repo and would be happy to adapt the code to use these.

[hanshuebner]

@rkompass I'll try to come up with a PR to add a machine_dma module. I'll also try to figure out whether I can make the PIO MicroPython module be a nice citizen with respect to PIO allocation. I found that the PIO module already calls the required API functions to claim and unclaim state machines.

[hanshuebner]

I was going to head for a minimal solution, but thank you for pointing at the open PRs. I've found #10704 which seems to be fairly complete, providing a rp2.DMA class with documentation in #7670. What's not to like? There is also #7641 which seems to be having given up on.

For the DMA side I think a machine._dma module would be very helpful. For a start it would be sufficient to have
dma_claim_unused_channel and dma_channel_unclaim as well as two methods to register and unregister a callback function there.

Is there anything special about these callback functions that prevents normal IRQ handlers from not being suitable? Or what do you mean by "callback function" in this context?

[hanshuebner]

I've got this now:

>>> import machine
>>> dma = machine.DMA()
>>> dma.claim_unused_channel(True)
2
>>> dma.channel_unclaim(2)
>>> 

It is kind of tempting to just add the rest of the pico-sdk DMA API functions in the same fashion, but I don't want to put effort into something that would not have a chance to be merged. Thoughts?

[rkompass]

What's not to like?

Perhaps the api was too complicated.
In my class given you have state machine generating numbers (or reading something) and an array to take the values like

sm = StateMachine(0, sm_gen, freq=2000)
arr = array('l', (0,)*30)

it just requires:

dma = RP2_Dma(sm,  arr)       #  dma uses push_thresh=32 in @asm_pio to set dsize=2, but you may try with other dsize
dma.start()

to setup and run the DMA.
My philosophy was: If the values, like addresses of PIO channels or arrays, or word sizes, are determined, the internal logic of the class should be able to set them as default values, unless otherwise specified. And report contradictions if the defaults are not in agreement, unless they are specified, which would override the defaults, of course.
The class should help the user to solve things easily rather than require him to learn how to setup, unless it can't be avoided.

Would it be difficult to add a function for registering an interrupt service routine? Which would unregister the ISR if None is provided?

Using your functions I can replace

    chans = array('b', (0,0,0,0,0,0,0,0,0,0,0,0))    # stores 0 for free, or 1 for used DMA channel
    if chan is None:               # if not explicitly specified, get a free DMA channel
        try:
            chan = next((i for i, v in enumerate(RP2_Dma.chans) if v == 0))
        except StopIteration:
            raise ValueError('No more free dma channels')
    elif chan < 0 or chan > 11:
        raise ValueError('Dma channel outside range 0..11')

with chan = dma.claim_unused_channel(True)

If there were DMA.irq(handler=None, hard=False) like in in #7641 and #7650 I could use that to add the interrupt options.

Perhaps it would be appropriate to name the module _dma instead of dma. Perhaps is should reside in rp2 module not in machine, at least for now?
Later, given we have achieved that, and tested, we should ask for feedback.

[hanshuebner]

I've looked at the #7641 and #10704 again to see whether I can extract the IRQ handling parts, but that has shown to be more work than I can currently put into this.

#7641 has been sitting for too long to be easily resurrectable. #10704 seems to be relatively complete, but a deinit path is missing and the API could be a bit less involved.

As that there are now three different approaches to providing a complete DMA solution (@rkompass's library, #7641 and #10704), it would be helpful to know what the maintainer's opinion on the topic is - Is there a chance to get any of these solutions be merged into the official MicroPython port? (@jimmo @robert-hh @peterharperuk)

[andrewleech]

As much as I'd love a machine.DMA class to be feasible, I believe that across all the ports / mcu's involved on micropython there's really no chance of getting a consistent API that can realistically work for all chips.
DMA engines are just too different even across different ranges of chips one manufacturer, let alone across all manufacturer.

As such I think it's better to take the sort of work done here and put it into chip specific class eg rp2.DMA and leave machine out of it, which should also make it easier to progress one of these suggested PR's with a focus on only supporting RP2 with the cleanest API possible for its DMA engine.

[robert-hh]

The main-tainer of the repositoy is Damien with support by Jimmo. And yes, keeping a PR up-to-date is an ongoing work. I have no clear opinion whether a stand alone DMA class is useful or not. Usually DMA is used by the drivers for certain peripherals which embed the source & target of the transfer. Only the RP2 PIO seems one where these are exposed to Python scripts, and there it is useful.