-
-
Notifications
You must be signed in to change notification settings - Fork 95
OTA implementation & fixes for RTL8720CF #302
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
30af0a2 to
7082519
Compare
|
Hi @fcol95, I've described steps how to install esphome with working OTA in #44 (comment) . You can skip step 0, because ltchiptool >= 4.11.5 already contains the mentioned fix. |
|
@prokoma thanks for your quick reply, just had deleted my comment when stumbling on your open PR in ltchiptool repo and understanding a bit better the issue! thanks for the link to your custom solution! |
|
Just tested the OTA through esphome and your "source: https://github.com/prokoma/libretiny#55aacc8" framework, works quite well, thanks for the effort, would be glad to see that merged in the production release soon! |
|
Just want to confirm this works perfectly for me in esphome once the latest libretiny commits are pulled in. Would love to see this in master! |
|
I have used this with a tuya smart weather station that uses WBR3 module and can confirm it works well. Thanks!! |
|
Hi @prokoma , could you please update your fork to include this commit from the original repo? It cannot be compiled with the newest esphome without this change. |
|
Rebased on latest LibreTiny dev version. |
|
confirmed working with ESPHome Had to manually upload Had to manually generate uf2 but than rtl87xx:
board: bw15
framework:
version: 1.9.2
source: https://github.com/prokoma/libretiny/archive/refs/heads/ambz2-ota.zipedit: with patch it just works diff --git a/builder/family/realtek-ambz2.py b/builder/family/realtek-ambz2.py
index f67d97b..144fa94 100644
--- a/builder/family/realtek-ambz2.py
+++ b/builder/family/realtek-ambz2.py
@@ -437,7 +437,6 @@ queue.BuildLibraries()
image_part_table = "${BUILD_DIR}/image_part_table.${FLASH_PART_TABLE_OFFSET}.bin"
image_bootloader = "${BUILD_DIR}/image_bootloader.${FLASH_BOOT_OFFSET}.bin"
image_firmware_is = "${BUILD_DIR}/image_firmware_is.${FLASH_OTA1_OFFSET}.bin"
-image_firmware_is_ota = "${BUILD_DIR}/image_firmware_is_ota.${FLASH_OTA1_OFFSET}.bin"
env.Replace(
# linker command (dual .bin outputs)
LINK='${LTCHIPTOOL} link2bin ${BOARD_JSON} "" ""',
@@ -445,8 +444,8 @@ env.Replace(
UF2OTA=[
# use unmodified image for flasher
f"{image_firmware_is},{image_firmware_is}=flasher:ota1,ota2",
- # use patched OTA image for device
- f"{image_firmware_is_ota},{image_firmware_is_ota}=device:ota1,ota2",
+ # use same image for device OTA
+ f"{image_firmware_is},{image_firmware_is}=device:ota1,ota2",
# having flashed an application image, update the bootloader and partition table (incl. keys)
f"{image_bootloader},{image_bootloader}=flasher:boot,boot",
f"{image_part_table},{image_part_table}=flasher:part_table,part_table",This works with ESPHome dev rtl87xx:
board: bw15
framework:
version: 1.9.2
source: https://github.com/bdraco/libretiny/archive/refs/heads/ambz2-ota.zip |
|
I wanted to follow up on this PR. I understand you have broader plans for LibreTiny architecture, but I'd like to explain why merging this now would help the project. I have a collection of BW15 boards (RTL8720CF) that I use as test targets. The problem: without OTA support, I can't include these boards in our release testing workflow. Flashing them requires manual intervention with ltchiptool, or using a fork which makes things brittle and more likely we end up skipping testing due to time constraints. I've tried to get other RTL boards but obtaining them has proven difficult, and buying devices to guess which chip they have in them isn't a good plan either because they are even harder to debug and get serial logs on. We had tons of WiFi and thread safety issues on LibreTiny in ESPHome that only just got fixed because I was able to obtain test boards and had a clear path to easily flash and iterate on that doesn't involve clamps and tiny wires and soldering things smaller than my eyes can see. The practical result is that LibreTiny RTL code doesn't get tested during ESPHome releases. Bugs slip through or the platform falls behind because it's too painful to iterate on. If this PR gets merged, I can add RTL8720CF to my manual test matrix. That means LibreTiny RTL code actually gets exercised during releases and we catch problems early instead of finding out later from users. I get that it might not align perfectly with your future architecture plans, but working OTA with real testing seems better than untested code waiting for a rewrite. Is there anything specific about the PR that concerns you, or something I can help with to get it over the line? |
|
Hi The main issues were discussed over in ltchiptool's repository: libretiny-eu/ltchiptool#46 |
|
Thanks for being open to this. I spent some time today digging into how the RTL8720CF bootloader actually decides which image to boot, and I think I can address some of the concerns from the ltchiptool discussion. From what I can tell, the bootloader doesn't actually use serial numbers for boot selection. It just checks if the first byte of each image is valid (not inverted). The serial numbers seem to be informational only. The SDK's own So the mechanism in this PR appears to be correct for the hardware. It's not a workaround, it's how Realtek designed it. The main limitation is there's no automatic rollback if the new image is broken since the bootloader has no fallback mechanism. ESPHome's safe_mode catches most failures, but very early crashes would need UART recovery. That's a hardware limitation though, not something we can fix in software. |
|
Interesting. The Realtek documentation seems to disagree (after all, I must have gotten that serial number counter idea from somewhere) -- but that wouldn't be surprising at all. It's worth noting that there are at least two different variants of the bootloader: stock Realtek (the one in SDK, the one LT has access to) and modified Tuya (with unknown modifications). I'm not sure which one you're using on that BW15 (they can be reflashed), but it's entirely possible that they implement a different method for choosing the active application. Serial numbers aside, there are some indications that one of the bootloaders could be using a bit flag value in the system partition (just like on AmebaZ). So it appears we have three different possible checks:
We need to choose a solution that would work in both cases. There are commercial devices other than Tuya (for example TP-Link Tapo plugs), which, I assume, use the stock bootloader (or even something entirely different). As for automatic rollback - yes, I know it's not possible, and it doesn't have to be. The feature is entirely useless without any bootloader-based bootloop checks, and it was added only to conform with Arduino's Update library. It should be dropped since there's no real use case for that. |
|
For reference/testing, https://github.com/divadiow/FlashDumps/tree/main/IoT/RTL8720CF has multiple full dumps, including the original BW15 firmware (expected Realtek bootloader), several Tuya firmwares (expected Tuya bootloader), and a handful of other firmwares (unknown bootloaders) for bootloader compatibility testing. Edit: this may not cover serparate ROMs, not sure how important that is. |
|
I see I think requiring confirmation across all bootloader variants is effectively the same as "never merge." This PR has been open for over a year, and it took that long for someone like me (not even the original author) to stumble accross it and start pushing for it. The reality is:
I'd suggest we merge with documented limitations. Something like: "OTA tested on stock Realtek bootloader (BW15). Other bootloader variants (Tuya, etc.) are untested and may require UART recovery if OTA fails." Users already need UART access for initial flashing anyway, so UART recovery for edge cases isn't a new burden. And if someone with a different bootloader variant reports issues, we can investigate then with actual hardware and logs rather than speculating about theoretical problems. Perfect is the enemy of good here. Working OTA on known hardware is better than no OTA on any hardware. |
|
Just to be clear, I fully agree the bootloader variance is a real risk. I’m not claiming this is universally safe, only that we merge a tested path with explicit limits, and iterate based on real reports rather than blocking indefinitely. |
|
Makes sense - I agree that this should be released. Currently I'm not able to test whether this PR works - so I'll have to take your word for it. I know that many people have reported this to work just fine, so I'll trust that too. I'm not sure about the Regarding the I think it should be fine to leave this as-is - meaning that the OTA image will not have a bit-flipped byte at all, and will be "enabled" directly after writing. The C code will still have that Let me know if the proposed solution would work for you: merge #302 directly and leave libretiny-eu/ltchiptool#46 unmerged for now. |
|
Opened #344 with a working version based on this PR. Re: your questions - The original And yeah I'm not using the patched image, same firmware image for both flasher and device OTA. The enable/disable logic is still there but its basically a no-op since images are already enabled when written. Retested with mutliple OTAs on ESPHome, working fine. |

I've cleaned the history and removed the code to pull ltchiptool from git. You need to make a release of ltchiptool first before merging this PR and also update the minimum required version in builder/main.py.