Using second core as watchdog?

[ESPWarren]

I am using Pi Pico W which has two cores.

I am typically implementing simple units which have several sensors and send the data periodically to a MQTT broker.

It's annoying when the Pico stops / locks up / whatever. The watchdog function is not 100% reliable.

Given the two cores in the Pico, I want to know:

If all my sensor/MQTT code is running in one core, and locks up, does it affect any code running in the second core?

In not, then I could imagine a simple watchdog where core 2 has code monitoring core 1's loop. If the loop ceases, core 2 can trigger a relay and do a hardware restart...

Possible?

[karfas]

No experience with a Pico, but I think that any error stopping one core might as well lock the whole processor.
When you already plan to add external circuits ("relay"), why not add a dedicated hardware watchdog like the DS1286 ? This comes with a (maybe) battery-buffered RTC...

[robert-hh]

As an alternative to other watchdog chips which usually had only short alarm times I used an ATTiny as Watchdog, which itself sleeps most of the time, consuming less than 1 µA (it was hard to measure). See https://github.com/robert-hh/ATTiny-Watchdog-for-MicroPython
That worked well.

[ESPWarren]

Tell me Robert:

Using your ATTiny approach, have you ever had cases where despite having a watchdog that could trigger a machine reset, a power cycle was still necessary to bring the Pico back up?

[jimmo]

It's annoying when the Pico stops / locks up / whatever.

It depends what you mean by this... a regular Python exception on the "main" thread will not interrupt the second core, and so in this case a watchdog running on the second core will work.

Have you looked at machine.WDT ?

[robert-hh]

Can't tell. I used that in combination with an ESP32 acting as single channel LoRa Gateway, and for that it worked well. The events were rare. But the ESP32 reset input causes an internal power cycle, and so the reset behavior between ESP32 and Pico may be different. I should be expected that pulling Reset and doing a power cycle results in different internal states, especially for items where the reset state is not defined. And there may be a difference between a WDT reset and an external hardware reset.

[ESPWarren]

[I thought I had already sent a reply along these lines - but I can't see it!]

Thanks for the contrributions everyone. @jimmo - yes to machine.WDT. In my experience it's not great to be honest.

I have decided that the most likely 'bulletproof' approach is something that power cycles the Pico. I am refining that design and will get a custom PCB made to act as a baseboard into which the Pico plugs, but which also has some other useful things I need - the power-cycling watchdog, a few I2C breakouts and a modest non-volatile i2c ram t make power-cycling no problem with keeping state(s).

If anyone is interested in being kept in the loop in this, just PM me in Github.

Thanks again.

[peterhinch]

There is another way of looking at this problem: ask why is it locking up? When developing resilient MQTT we had tests running for months and nearly a million messages. Lockups never occurred.

While a WDT is great for truly unexpected events (lightning strikes nearby) I wouldn't advocate them as an alternative to robust code.

[ESPWarren]

Agreed - from the perspective of computer science, having the possibility of a watchdog should never be the reason to permit sloppy code.

I suspect that the desirability of a hardware power-cycling WDT is proportional to the pain involved in physically going to a remote sensor hub and power cycling it manually. One wonders: would they rely on robust code alone on the Mars rover? :)

But also, robust code is no protection against power brown-outs (which here in the UK in my urban environment seem to be more likely these days).

In addition, I can imagine situations where one does an OTA update and really does want the manchine to come up pristine. I know that in theory machine.reset() should do that, but it does seem that just now and then, variables / conection states are 'sticky' even given a machine reset(). That's not a coding issue but a flash issue - and discovering that on a remote hub could be very problematic.

The extensive MQTT testing you did, was that on the Pico?

And if not, do you feel that extensive tests done on ESP-32 platforms 'carry over' to different microcontrollers running micropython?

Thanks

[peterhinch]

I agree that a WDT has its place. You seemed to be describing a situation where crashes were rather frequent: in my view they should be rare, with an external cause. I would develop without a WDT, applying it only once the code had been proved to be robust.

We haven't done long term testing on the Pico W. We did on ESP8266, ESP32, and Pyboard D. The architecture of the Pico W is sufficiently close to that of the Pyboard D that I'd be surprised if it didn't perform well. AIUI it uses the same separate WiFi chip and driver and the CPU architecture is ARM (though not the same version).