Is using mbedtls in esp-idf ROM a better option in my case?

[burumdev]

Hello,

I developed a device authorization app for my esp32-c3 with esp-mbedtls: https://github.com/burumdev/esp32-padlock

The problem is currently I can only run one embassy task in task pool due to memory overflows. And available memory for other tasks is rather limited at this point.

If I use mbedtls from IDF, will I be able to better fit this authorization section to an app that can actually do other things? I'm aware of #75 and it could help a bit.

Thanks

[ivmarkov]

In short, no.

a) "mbedtls from IDF" is not "in ROM"
b) you have an (S)RAM problem, not a flash/ROM problem
c) "mbedtls from IDF" and esp-mbedtls have similar RAM memory requirements. Or will have, once #75 is resolved

[yanshay]

Here is my experience:

Every TLS session takes about 40kb-45kb, esp-wifi takes around 50kb-60kb.
Add to that the various buffers used in every web task.
Embassy tasks could also take a significant memory for the tasks (which can be to some extent optimized but not a lot).
Also need enough space for the stack.
etc.

This piles up pretty quickly and memory is exhausted very fast.

If you can switch to esp32-s3, you can use PSRAM for most things. ESP-WIFI will always need the internal RAM, and so does the stack. TLS for now require internal RAM but could move to PSRAM in the future and all the rest I push to the heap in PSRAM.

So eventually the major internal memory bottleneck is esp-mbedtls with the large memory need for TLS sessions in the internal RAM, but I hope it will be resolved in the future and would allow using PSRAM for that as well.

[burumdev]

Thanks!

[ivmarkov]

Here is my experience:

Every TLS session takes about 40kb-45kb, esp-wifi takes around 50kb-60kb. Add to that the various buffers used in every web

We should work on reducing this number. I think the "IDF mbedtls" uses something like 32KB max which is possible to reduce even down to 18KB-16KB.

#75 (or a successor of it) should strive to do that.
And then as @yanshay says, ideally a successor of #75 should allow putting everything mbedtls-related (or most of it) in PSRAM.

[yanshay]

BTW (not related to the question) - for user facing server side using TLS with embedded device is less useful IMO because its always self signed certificates and user is warned in the browser about insecure app, and switch to advanced, accept risks, etc. a reall bad user experience, and such a pain that I'm not sure it is that useful anyway.

[burumdev]

Yes, you're right for end-user professional settings. But an organization can use it's internal certificate authority and install it to company computers and browsers by using tools like mkcert or step-ca for intranet settings.

[AnthonyGrondin]

Hello,

I developed a device authorization app for my esp32-c3 with esp-mbedtls: https://github.com/burumdev/esp32-padlock

The problem is currently I can only run one embassy task in task pool due to memory overflows. And available memory for other tasks is rather limited at this point.

If I use mbedtls from IDF, will I be able to better fit this authorization section to an app that can actually do other things? I'm aware of #75 and it could help a bit.

Thanks

Your server task could be optimized. I suggest using something like edge-http for multiple handlers, which would reduce the memory ballooning of spawning multiple times the same server task, and only "duplicate" the required resources to serve multiple clients. Your server task loads a copy of your certificates per instance, while using edge-http, you'd only need to load it once. Etc.

I know that the C3 is more memory contrained, but you should be able to support 2 handlers at least.

As for flash usage, your HTML could be optimized, if you use something like https://github.com/wilsonzlin/minify-html/ in your build.rs to minify it, and 90% of it is shared between lock and unlock, you could use the same template and just do template binding.

This is a bit hacky, but you could do something like:

// Inject extra data as json in the HTML page
connection.write_all(b"<script type=\"application/json\" id=\"unlock-state\">").await?;
connection.write_all(&buffer[..len]).await?;
connection.write_all(b"</script>\n").await?;
connection
    .write_all(include_bytes!('your-webpage.html'))

And then in your HTML do:

<script>
      const data = JSON.parse(document.getElementById("unlock-state").textContent);
</script>

To fetch computed injected data

[burumdev]

I tried with edge-http and I still couldn't make 2 tasks work.
In addition I had to reduce the first heap allocation size from 64K to 48K to avoid stack overflows. But edge-http version works much better with post method supported and responses look like they're faster now.

https://github.com/burumdev/esp32-padlock

I'll look into minifying htmls. Other than that I think c6 is a better device for these things.

I removed cloned version of esp-mbedtls and pull it from git revision now. I'll update the rev once #75 gets merged.

Thanks a lot for your help.

[yanshay]

With heap at 48kb you can’t expect two TLs sessions simultaneously.
You’ll have to focus on optimizing other memory usage if at all possible to increase the heap size.
I spent quite some time on that to get things working properly. And it’s not fun to be on the edge of memory, the bugs of stack overrunning heap are sometime difficult to identify related to that. It’s not always a panic.