web server and _thread not working

[awsiuk]

Hi Group,

I use Micropython v1.21.0 on RP2040 with simple code:

import network
import socket
import machine
import json
import urandom
import time
import _thread

data={
    "temp": 0.0,
    "hum": 0.0,
    "AQI": 0,
    "TVOC": 0,
    "eCO2": 0,
    "lux": 0,
    "pressure": 0.0}

#read configuration for the application
def read_the_config():
    global config
    with open("pogoda.json") as json_data_file:
        config = json.load(json_data_file)

def t_web_server(ip):
    address = (ip, 80)
    addr = socket.getaddrinfo('0.0.0.0', 80)[0][-1]
    
    connection = socket.socket()
    connection.bind(addr)
    connection.listen(1)
    print(connection)
    
    while True:
        print("Web server is waiting for connection")
        client, client_addr = connection.accept()
        print("Incoming connection from:", str(client_addr))
        try:
            print("handling the client request")
            request = client.recv(2048)
            f=open("index.html","r")
            html_main=str(f.read())
            client.send('HTTP/1.0 200 OK\r\nContent-type: text/html\r\n\r\n')
            client.send(html_main)
            time.sleep(0.1)
            
        except:
            print("ERROR: unable to handle the client")
        print("Closing connection to:", str(client_addr))
        client.close()
    print("exiting")
    return


###############
read_the_config()
global config
print(config)

#setting up WiFi
global wlan
wlan = network.WLAN(network.STA_IF)
wlan.active(True)
wlan.connect(config['SSID'], config['PASSWORD'])
while wlan.isconnected() == False:
    print('Waiting for WiFi connection...')
    time.sleep(1)
ip = wlan.ifconfig()[0]
print(f'Wifi connected on {ip}')

#initiate collecting data thread
_thread.start_new_thread(t_web_server, (ip,) )

while True:
    time.sleep(1)

it is almost ok, expect that after 2nd connection to RP2040 it no longer accept connections and stops responding to ping. There is no crash on RP2040 and it seems program is still running awaiting on connections, but nothing happens. try section does not trigger as there is no connection error, it simply does not even start.
If i run t_web_server function outside _thread it works properly. So either i do something wrong with _thread init, or there is a BUG for network support within the _thread. Can you help me with this?

[jimmo]

Your code looks like it should work, so this makes me think there must be a bug with the network stack running on the second core.

Does it work if you run the web server on the main thread, and do your other work on the second thread?

[awsiuk]

I was browsing documentation and to my understanding ARM Cortex M0/M0+ has just 1 core as per ARM documentation. It has 2 stage pipelines to increase performance but it does not mean that it has 2 cores or 2 CPUs. To me that is single core and there is no way to chose different core as there is just 1 core. It supports interrupts so web server + TCP client should be possible. I'm worried that either:

1. CPU is being occupied waiting for connection and then other actions are not possible
2. There is a BUG in firmware of micropython that is not handling interrupts properly.

[rkompass]

Hello awsiuk,

the rp2040 has two processor cores, as you can see from the schematics (taken from its datasheet).

Threading in MP with the rp2040 is different from other boards in that it tries to allow for truely employing both cores in parallel.
This is somewhat experimental and there are certain issues known as e.g. that interrupt service routines on the second core make problems. These will be solved in the long run, but not immediately.

[awsiuk]

Indeed if we use this schematic, but based on this schematic it also tells that the 2nd core is connected to the hardware.
Also I'm a bit confused with the documentation as raspberry say that it is the ARM Cortex M0+ CPU which is single core (to my understanding). This is very confusing. Unless there are 2x ARM cortex M0+ CPUs shown as Prox0 and Proc1.
So if you say that, it also means that my suspicion about handling the IRQ is correct and that is indeed a BUG that in future probably will be fixed (not near future). I guess then only solution untill future will come then is to use ASYNCIO. I'm guessing if I would use such microcontroller but programing in C I would not have this issue?

[rkompass]

Yes, both processors (i.e. both ARM cortex M0+ cores) have full access to the hardware. They are symmetric. Still in Micropython the network and interrupt stuff is by standard only fully established (and tested) on Proc0 (i.e core 0).
By programming in C you would have to control all inter-processor task assignment and scheduling by yourself.
I'm not an expert. So I don't know what kind of libraries are there that might help you.
On the other hand, my impression (as non-expert) is that the solution of the MP threading problems is just a time consuming task of tracing many subtle details that are involved in the current partly unsatisfactory state: threading works, but interrupts function well only on one core. This obviously is on the developers todo list with other things having higher priority.
Asyncio also has limitation across cores (you cannot assign asyncio tasks across both cores and expect it to work).
What you can do is to work with asyncio as if there were only one core. And then, once this is established you may experiment to delegate a certain computation-intensive task onto the second core. You would benefit from reading into thread-safe flag and thread-safe queue for this.

[awsiuk]

That is probably something that I will do. I do not need more than 1 core. Based on code pasted in this thread i will not do much more than this so 1 core should be fully sufficient.

Thanks!

[awsiuk]

Hi,

Thanks for looking into this.
Yes, it works if you put function into the "main" meaning:

#initiate collecting data thread
#_thread.start_new_thread(t_web_server, (ip,) )

while True:
    t_web_server(ip)

then it works correctly. If i create just a test function that holds loop with some simple math and print then thread works.

def t_test(ip)
   t=0
  while True:
      print("IP:" + str(ip))
      time.sleep(1)
      t+=1
 
#initiate collecting data thread
_thread.start_new_thread(t_test, (ip,) )

while True:
    time.sleep(1)

There is an issue with network support. Network works just in 1 loop run. Ping of the RP2040 before connecting to port 80 works, after connection to port 80 no longer works. It looks like network stack stop working correctly and i have no clue how to debug this.

If i put web server within thread then it stops working after 1 loop run, so i stared with something simple like above to see where is the issue. I need 1 thread that will either run web server or will send data. Since sending data function is simpler i prefer to put that one in the thread. But if i do this:

import network
import socket
import machine
import json
import urandom
import time
import _thread
import struct

DATA=""

#for random function needed to generate session identifier
keys = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz1234567890'


data={
    "temp": 0.0,
    "hum": 0.0,
    "AQI": 0,
    "TVOC": 0,
    "eCO2": 0,
    "lux": 0,
    "pressure": 0.0}

#read configuration for the application
def read_the_config():
    global config
    with open("pogoda.json") as json_data_file:
        config = json.load(json_data_file)


zabbix_data = {
    "request": "agent data",
    "data": [
        {"id":1, "host": "node_test.la-tech.pl", "key": "custom.pogodynka.temp", "value": 24.5, "clock": 0,"ns": 0},
        {"id":2, "host": "node_test.la-tech.pl", "key": "custom.pogodynka.hum", "value": 35.5, "clock": 0,"ns": 0},
        {"id":3, "host": "node_test.la-tech.pl", "key": "custom.pogodynka.AQI", "value": 3, "clock": 0,"ns": 0},
        {"id":4, "host": "node_test.la-tech.pl", "key": "custom.pogodynka.TVOC", "value": 211, "clock": 0,"ns": 0},
        {"id":5, "host": "node_test.la-tech.pl", "key": "custom.pogodynka.eCO2", "value": 965, "clock": 0,"ns": 0},
        {"id":6, "host": "node_test.la-tech.pl", "key": "custom.pogodynka.lux", "value": 2222, "clock": 0,"ns": 0},
        {"id":7, "host": "node_test.la-tech.pl", "key": "custom.pogodynka.pressure", "value": 888, "clock": 0,"ns": 0}
        ],
    "session": ""
}

def randstr(length=32, aSeq=keys):
    return ''.join((urandom.choice(aSeq) for _ in range(length)))


#this function updates package structure based on collected data
def build_packet():
    #get time counters need by zabbix server
    #this is the time when data was collected
    epoch=time.localtime()
    nstime=time.time_ns()   
    
    #setting session time
    zabbix_data["session"]=randstr()
    #putting data into the zabbix packet
    for item in zabbix_data["data"]:
        item["clock"]=epoch
        item["ns"]=nstime
        key=item["key"].split(".")[2]
        item["value"]=data[key]

    #return content of packet that should be sent to zabbix server
    return b"ZBXD\1" + struct.pack("<II", len(str(json.dumps(zabbix_data))), 0) + str(json.dumps(zabbix_data))


def t_zabbix_active(ip):
    
    while True:
        s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        print("connecting to zabbix")
        s.connect(("10.254.7.6", 10051))
        print(len(str(zabbix_data)))

        s.send(build_packet())
        out=s.recv(4096)
        print(out)
        s.close()
        print("zabbix connection closed")
        time.sleep(10)



###############
read_the_config()
global config
print(config)

#setting up WiFi
global wlan
wlan = network.WLAN(network.STA_IF)
wlan.active(True)
wlan.connect(config['SSID'], config['PASSWORD'])
while wlan.isconnected() == False:
    print('Waiting for WiFi connection...')
    time.sleep(1)
ip = wlan.ifconfig()[0]
print(f'Wifi connected on {ip}')

#initiate collecting data thread
_thread.start_new_thread(t_zabbix_active, (ip,))

while True:

    time.sleep(1)

then i get:

Unhandled exception in thread started by <function t_zabbix_active at 0x200163a0>
Traceback (most recent call last):
  File "<stdin>", line 101, in t_zabbix_active
OSError: [Errno 103] ECONNABORTED

and Error reffers to line with: s.connect(("10.254.7.6", 10051))

if i do:

#initiate collecting data thread
#_thread.start_new_thread(t_zabbix_active, (ip,))

while True:
    t_zabbix_active(ip)

then all is perfect:

Wifi connected on 10.254.0.13
connecting to zabbix
822
b'ZBXD\x01Z\x00\x00\x00\x00\x00\x00\x00{"response":"success","info":"processed: 7; failed: 0; total: 7; seconds spent: 0.000262"}'
zabbix connection closed
connecting to zabbix
1188
b'ZBXD\x01Z\x00\x00\x00\x00\x00\x00\x00{"response":"success","info":"processed: 7; failed: 0; total: 7; seconds spent: 0.000232"}'
zabbix connection closed
connecting to zabbix
1181
b'ZBXD\x01Z\x00\x00\x00\x00\x00\x00\x00{"response":"success","info":"processed: 7; failed: 0; total: 7; seconds spent: 0.000224"}'
zabbix connection closed
connecting to zabbix
1188
b'ZBXD\x01Z\x00\x00\x00\x00\x00\x00\x00{"response":"success","info":"processed: 7; failed: 0; total: 7; seconds spent: 0.000206"}'
zabbix connection closed

[scruss]

Network functions only work on one core of the Raspberry Pi Pico W. The second core has no connection to the network hardware.

[awsiuk]

Well... I'm not sure if that statement is correct. If we read the Raspberry documentation, indeed they mention dual-core. They also mention that the RP2040 is based on Cortex-M0+ CPU. That means that the architecture of RP2040 CPU is:

Cortex M4 introduce the FPU as separate unit that could be treated as the 2nd core, but still it is not CPU, it is FPU. RP2040 is not M4 based it's M0+ based thus even the FPU is not there.
What Cortex M0+ introduce is way of queuing the instructions for most effective CPU use and keep it busy. This is achieved by implementing the 2 stage pipeline. To my understanding the information on Raspberry page is incorrect and refer to this 2 stage pipeline.
If we would assume the 2 core architecture then by consequence the 2nd core is connected to the hardware as on diagram:

That would mean that threads should be properly handled.
ARM Cortex A family introduce the core concept into the CPU and core count is between 1-4 cores. ARM Cortex M family has just 1 core. At least based on reading datasheets i can conclude this. Since I'm not the hardware developer i can only suspect 2 scenarios:

1. IRQs are not properly handled by firmware/software
2. There is no way to handle properly interrupts due to architecture and it needs to be done as suggested on other post by ASYNCIO.

I have insufficient knowledge to conclude this, however in Pentium 1 times, there was just 1 CPU and we could multitask on windows 95 by using interrupts. So my best guess would be that IRQs are not properly handled, but still due to that Pentium 1 was more of CISC and here we have RISC architecture 2nd also might be valid. From the other hand our ARM based phones are working just fine with single or multi core which suggest point 1.

[scruss]

Well... I'm not sure if that statement is correct. …

It is: RP2040 Datasheet, pages 9–10, “Why is the chip called RP2040?”: 1. Number of processor cores (2)

That means that the architecture of RP2040 CPU is: … That would mean that threads should be properly handled.

There are two M0+ cores sharing memory, but there's no explicit thread management in the hardware. MicroPython expects the user to manage memory between threads on RP2040. There is no OS to manage it. Don't bring IRQs into it, please — they mean something else in MicroPython. The Threads · micropython/micropython Wiki maybe be of help.

My previous The second core has no connection to the network hardware. statement may not be correct. On systems that use FreeRTOS on RP2040 (like Earle Philhower's Arduino port), they've chosen to make wifi only available on Core 0. There doesn't seem to be any definition documentation saying that you can't use both cores with wifi, but there have been many Raspberry Pi forum threads about problems using wifi and _thread, most of which are resolved by stopping using _thread.

[rkompass]

My previous The second core has no connection to the network hardware. statement may not be correct.
You are right: It was not correct (at least that is my current understanding: fully symmetry on the hardware side, so all pins/connections to the Wifi-chip are similarly accessible from both cores).

What you tell about the Arduino port is interesting. The RP2040 seems like an experiment what is possible without using FreeRTOS. Which is not finished yet.

[jimmo]

@scruss @awsiuk

The second core has no connection to the network hardware.

I assume this is relating to the Pico W. In general the rp2040 has no network peripheral and it would be up to the driver to manage whatever connection (e.g. SPI) to external network hardware (from either core). For example, the Pico W uses the PIO to do 1-wire SPI to the CYW43 WiFi chip.

This is very different to e.g. the ESP32 where one of the CPU cores is very involved in the core radio functionality.

The goal for MicroPython is that you should be able to use the high-level network and socket modules from either core, however clearly there are issues with accessing it from the second core right now. There's a bit of an open question about how this should work without a GIL (i.e. what if both threads attempt to read from the same socket), but definitely exclusively using it from one or the other core should work.

My understanding is that anything preventing this right now is a bug with locking in the socket binding layer, LWIP, or the PIO) not a hardware limitation.