Microdot uasyncio webserver and running other tasks

[MrTinkerman]

Hi all,

I'm trying to set up a webserver that does datalogging.
For testing purposes I've connected a DS18b20 temperateure sensor.

Ok, so what should this server do and how far did I get:

I would like:
-- A webserver that does datalogging, say logging a temperature to a text file every 5 minutes
-- I would like to access the logfiles or maybe have them sent somewhere once or twice a day

What I have so far:
-- A Teensy 4.1 running MicroPython 1.19.1
-- I got ethernet, the temperature sensor, writing to SD card and a Microdot webserver working

See my code below.

And at this point I'm confused.

It turns out I have to use uasyncio to schedule runing the logging function?
I've gone over (u)asyncio documentation all day but gotten nowhere.

Let's say I want the webserver to be available and run the logging function every 5 minutes.
How can I achieve that?

This thread seems to be about my issue: miguelgrinberg/microdot#17
The key thing I do not understand how to do is:

[QUOTE]
start async functions before calling app.run() and release control to the web server.
[/QUOTE]

Thanks for making it to here and I hope you have a suggestion.

My code:

# LOG TEMPERATURE ==========================================

async def log_temperature():
    
    print('logging temperature')
    
    with open("/sd/test01.txt", "a") as file:             # 'a' for append to file
        
        ds_sensor.convert_temp()                          # Read temperature
        time.sleep_ms(750)
        temp = ds_sensor.read_temp(ds_rom)                # Temperature
        ct = rtc.now()                                    # Current time [YYYY, M, D, h, m, s, us, ?]
        log_entry = f'{ct[0]}.{ct[1]:02d}.{ct[2]:02d} {ct[3]:02d}:{ct[4]:02d}:{ct[5]:02d}  {temp}'
        file.write(f'{log_entry}\r\n')


# SETUP =================================================

from machine import Pin, RTC, SDCard
import onewire, ds18x20, network, time, uasyncio, uos
from microdot_asyncio import Microdot

ds_pin = Pin(23)                                           # DS18b20 temperature sensor on pin ##
ds_sensor = ds18x20.DS18X20(onewire.OneWire(ds_pin))
roms = ds_sensor.scan()
ds_rom = roms[0]                                           # ds_rom is rom to read temperature from

rtc = RTC()                                                # Create real time clock intance

sd = SDCard()                                              # Initialize SD card, filesystem and mount
vfs = uos.VfsFat(sd)
uos.mount(vfs, '/sd')

server = Microdot()                                        # Create Microdot web server

@server.route('/')                                         # Main route 
async def index_html(request):
    
    log_temperature()
    print('index.html requested')
    return f'Server online, time= {rtc.now()[3]:02d}:{rtc.now()[4]:02d}:{rtc.now()[5]:02d}'                      

# MAIN ================================================

if __name__ == '__main__':
    
    eth = network.LAN(0)                                 # Initialize ethernet
                                                         # https://docs.micropython.org/en/latest/library/network.LAN.html?highlight=lan%20module
                                                         # TODO: catch network cable not connected / error connecting, keep trying
    
    print(f'Webserver available on: {eth.ifconfig()[0]}')
    
    server.run(eth.ifconfig()[0], port=80)               # Start webserver on IP address, port 80
                                                         # def run(self, host='0.0.0.0', port=5000, debug=False, ssl=None):

[Carglglz]

@MrTinkerman

This will log the temperature every 5 minutes

import uasyncio as asyncio
# LOG TEMPERATURE ==========================================


async def log_temperature(loop=True):
    while True:
        print("logging temperature")

        with open("/sd/test01.txt", "a") as file:  # 'a' for append to file
            ds_sensor.convert_temp()  # Read temperature
            await asyncio.sleep_ms(750)
            temp = ds_sensor.read_temp(ds_rom)  # Temperature
            ct = rtc.now()  # Current time [YYYY, M, D, h, m, s, us, ?]
            log_entry = f"{ct[0]}.{ct[1]:02d}.{ct[2]:02d} {ct[3]:02d}:{ct[4]:02d}:{ct[5]:02d}  {temp}"
            file.write(f"{log_entry}\r\n")
        if not loop:
            return
        await asyncio.sleep(300)

This will also log the temperature when a request is received

@server.route('/')                                         # Main route 
async def index_html(request):
    
    await log_temperature(loop=False)
    print('index.html requested')
    return f'Server online, time= {rtc.now()[3]:02d}:{rtc.now()[4]:02d}:{rtc.now()[5]:02d}' 

This runs the logger and the server concurrently

# MAIN ================================================
async def main(_eth):
    logger_task = asyncio.create_task(log_temperature())
    server_task = asyncio.create_task(server.start_server(_eth.ifconfig()[0],
                                                          port=80))
    await asyncio.gather(logger_task, server_task)

if __name__ == '__main__':

    eth = network.LAN(0)                                 # Initialize ethernet
                                                         # https://docs.micropython.org/en/latest/library/network.LAN.html?highlight=lan%20module
                                                         # TODO: catch network cable not connected / error connecting, keep trying

    print(f'Webserver available on: {eth.ifconfig()[0]}')

    asyncio.run(main(eth))

I haven't tested this but I think it should work 👍🏼

To explain a little bit, if you want to run async "functions" (coroutines), you need to:

• Run the event loop, calling asyncio.run([async_function]), e.g. asyncio.run(main(eth))
• await a coroutine , e.g. await log_temperature(loop=False)
• or create a task from a coroutine, e.g. logger_task = asyncio.create_task(log_temperature())
• await asyncio.gather(*tasks) that runs tasks concurrently , e.g. await asyncio.gather(logger_task, server_task)

This tutorial from @peterhinch is the best option learn about MicroPython uasyncio

Also check aiorepl which makes debugging asyncio apps way easier

[MrTinkerman]

Ok,

One step closer:
https://microdot.readthedocs.io/en/latest/intro.html#including-dynamic-components-in-the-url-path

#A special type path can be used to capture the remainder of the path as a single argument:

@app.get('/tests/<path:path>')
def get_test(request, path):
    return 'Test: ' + path

So I wrote this:

@server.route('<path:path>')                                 # Catch all
async def handle_request(request, path):                     # Request properies: .method .path .args .headers .cookies .app
    print('catch all triggered')                             # .content_type .content_length .client_addr
    print(request.path)
    print('/www/'+path)
    return send_file('/www/'+path)

Which results in this:

Traceback (most recent call last):
  File "microdot_asyncio.py", line 369, in dispatch_request
  File "microdot_asyncio.py", line 439, in _invoke_handler
  File "<stdin>", line 63, in handle_request
  File "microdot.py", line 674, in send_file
OSError: [Errno 19] ENODEV

Not quite there yet :(

[MrTinkerman]

Ah ok:

ENODEV, meaning that filesystem was not mounted

So I changed my code to:

@server.route('<path:path>')                                 # Catch all
async def handle_request(request, path):                     # Request properies: .method .path .args .headers .cookies .app
    print('catch all triggered')                             # .content_type .content_length .client_addr
    print(request.path)
    print('/www/'+path)
    return send_file('/sd/www/'+path)

Where /sd is mounted by:

sd = SDCard()                                                # SD card
vfs = uos.VfsFat(sd)
uos.mount(vfs, '/sd')

[bixb922]

Microdot integrates a template engine, that is one way to go, I don't know many details.

Another strategy is to write @route services that return lists or dicts, which are converted by Microdot to json and shipped to the browser. You call these services from Javascript using XMLHttpRequest(), and then use Javascript DOM and the json data received from the server to format the page. Only static html pages are served. On the upside, the Micropython code is simple and clean and the presentation layer is done in the browser (which has plenty of cpu, ram and battery to do so). On the downside, you'll have to code also in Javascript...

[MrTinkerman]

That could actually be a more suitable strategy for future projects. I've done this before in Aduino/C++, serve one HTML/Javascript file that only fetches data from the server once it's loaded in the broswer, thanks.

[bixb922]

Yep, code is much clearer, lighter and json makes shipping data easy.