Support for latest Espressif ESP8266 frameworks: NONOS-SDK 3.0.4 and RTOS-SDK 3.4

[freedib]

This Pull Request implements support for the latest Espressif ESP8266 frameworks

Changes are explained in each commit of the Pull Request

Support for the Espressif NONOS-SDK requires two additional packages

• The latest Espressif NONOS-SDK @ 3.0.5 framework with additional platformio's package.json
  GIT: https://github.com/freedib/framework-esp8266-nonos-sdk.git
• a new tool called genbin.py in replacement of esptool which can create the two legacy OTA bin files in one call
  GIT: https://github.com/freedib/tool-genbin.git
  The builder appends -DSPI_FLASH_SIZE_MAP=n to CCFLAGS for the compiler. main.c should handle this value accordingly to Espressif documentation.
  OTA is activated the same way than ESP-IDF thru a build.partition in board definition. If used, two bin files are generated, and the first one uploaded. Then, the two bin files can be uploaded to a web server for FOTA update.

Support for the Espressif RTOS-SDK @ 3.4 is very similar to the latest ESP-IDF platform with ESP8266 in mind. Support for old RTOS-SDK @ 1.5 have been removed.
It requires one additional package:

• The latest Espressif RTOS-SDK @ 3.4 framework with additional platformio's package.json
  GIT: https://github.com/freedib/framework-esp8266-rtos-sdk.git
  All other tools (Cmake and Ninja) are already packaged for platformio.
  No change are required to boards definition.

Embed_files support have been included in esp8266-nonos-sdk and Arduino builders. It should replace the one proposed in Pull Request #267 to support the new esp8266-rtos-sdk builder

[freedib]

RTOS-SDK @ 3.4 requires a new toolchain-xtensa @ 8.4.0.
I packaged it locally for my Linux system, but I could prepare a version suitable for the Registry

[freedib]

tool-genbin have been renamed to tool-genbin-esp8266.
GIT: https://github.com/freedib/tool-genbin-esp8266.git

[qnxsgwy]

Thank you for your contribution, hope to support RTOS-SDK 3.4 as soon as possible, the current version is too old

[omar-mohamed-khallaf]

When will it be merged?

[YashxD]

I am sure a lot of folks are eagerly awaiting this PR to be merged. Is there any particular reason it's being held back? Can I contribute in any way?

[h1aji]

@freedib I am getting this error when i try to install

remote: Counting objects: 100% (105/105), done.
remote: Compressing objects: 100% (79/79), done.
remote: Total 105 (delta 54), reused 46 (delta 13), pack-reused 0
Receiving objects: 100% (105/105), 46.42 KiB | 864.00 KiB/s, done.
Resolving deltas: 100% (54/54), done.
Platform Manager: espressif8266@4.2.0+sha.032d938 has been installed!
Tool Manager: Installing platformio/toolchain-xtensa @ ~2.100300.0
Unpacking  [####################################]  100%          
Tool Manager: toolchain-xtensa@2.100300.220621 has been installed!
Tool Manager: Installing platformio/tool-genbin-esp8266 @ ~1.0.0
Error: Could not find the package with 'platformio/tool-genbin-esp8266 @ ~1.0.0' requirements for your system 'linux_x86_64'

Any chance to fix it?

[freedib]

Because this pull request have not been merged, tool-genbin-esp8266 have not been added to platformio registry.
From your log, it seems also that "pio run" downloads the current espressif8266 framework (version 4.2.0) and not the one of the pull request (version 4.0.1)
You have to modify your platformio.ini file to specify where to get components.

If developing for NONOS SDK:
[env]
platform = https://github.com/freedib/platform-espressif8266.git
framework = esp8266-nonos-sdk
platform_packages =
framework-esp8266-nonos-sdk @ https://github.com/freedib/framework-esp8266-nonos-sdk@3.0.5
tool-genbin-esp8266 @ https://github.com/freedib/tool-genbin-esp8266@1.0.0

If developing for RTOS SDK on Linux 64:
[env]
platform = https://github.com/freedib/platform-espressif8266.git
framework = esp8266-rtos-sdk
platform_packages =
framework-esp8266-rtos-sdk @ https://github.com/freedib/framework-esp8266-rtos-sdk.git@3.4
toolchain-xtensa @ https://github.com/freedib/toolchain-xtensa-8.4.0-linux-amd64.git@8.4.0
tool-genbin-esp8266 @ https://github.com/freedib/tool-genbin-esp8266@1.0.0

[h1aji]

@freedib Thanks it all works except mdns from components.

Building in release mode
Linking .pio/build/esp12e/firmware.elf
/home/hmurad/.platformio/packages/toolchain-xtensa@src-5e1dba65eabc364c0001e57aa616c039/bin/../lib/gcc/xtensa-lx106-elf/8.4.0/../../../../xtensa-lx106-elf/bin/ld: .pio/build/esp12e/src/webserver.o:(.bss.clientIvol+0x0): multiple definition of `clientIvol'; .pio/build/esp12e/src/main.o:(.bss.clientIvol+0x0): first defined here
/home/hmurad/.platformio/packages/toolchain-xtensa@src-5e1dba65eabc364c0001e57aa616c039/bin/../lib/gcc/xtensa-lx106-elf/8.4.0/../../../../xtensa-lx106-elf/bin/ld: .pio/build/esp12e/src/main.o:(.literal.app_main+0xf4): undefined reference to `mdns_init'
/home/hmurad/.platformio/packages/toolchain-xtensa@src-5e1dba65eabc364c0001e57aa616c039/bin/../lib/gcc/xtensa-lx106-elf/8.4.0/../../../../xtensa-lx106-elf/bin/ld: .pio/build/esp12e/src/main.o: in function `app_main':
/home/hmurad/Documents/PlatformIO/Projects/Le-Radio/src/main.c:745: undefined reference to `mdns_init'
/home/hmurad/.platformio/packages/toolchain-xtensa@src-5e1dba65eabc364c0001e57aa616c039/bin/../lib/gcc/xtensa-lx106-elf/8.4.0/../../../../xtensa-lx106-elf/bin/ld: /home/hmurad/Documents/PlatformIO/Projects/Le-Radio/src/main.c:750: undefined reference to `mdns_hostname_set'
/home/hmurad/.platformio/packages/toolchain-xtensa@src-5e1dba65eabc364c0001e57aa616c039/bin/../lib/gcc/xtensa-lx106-elf/8.4.0/../../../../xtensa-lx106-elf/bin/ld: /home/hmurad/Documents/PlatformIO/Projects/Le-Radio/src/main.c:755: undefined reference to `mdns_instance_name_set'
/home/hmurad/.platformio/packages/toolchain-xtensa@src-5e1dba65eabc364c0001e57aa616c039/bin/../lib/gcc/xtensa-lx106-elf/8.4.0/../../../../xtensa-lx106-elf/bin/ld: /home/hmurad/Documents/PlatformIO/Projects/Le-Radio/src/main.c:760: undefined reference to `mdns_service_add'
/home/hmurad/.platformio/packages/toolchain-xtensa@src-5e1dba65eabc364c0001e57aa616c039/bin/../lib/gcc/xtensa-lx106-elf/8.4.0/../../../../xtensa-lx106-elf/bin/ld: /home/hmurad/Documents/PlatformIO/Projects/Le-Radio/src/main.c:763: undefined reference to `mdns_service_add'
/home/hmurad/.platformio/packages/toolchain-xtensa@src-5e1dba65eabc364c0001e57aa616c039/bin/../lib/gcc/xtensa-lx106-elf/8.4.0/../../../../xtensa-lx106-elf/bin/ld: .pio/build/esp12e/src/interface.o:(.literal.setHostname+0x2c): undefined reference to `mdns_service_remove'
/home/hmurad/.platformio/packages/toolchain-xtensa@src-5e1dba65eabc364c0001e57aa616c039/bin/../lib/gcc/xtensa-lx106-elf/8.4.0/../../../../xtensa-lx106-elf/bin/ld: .pio/build/esp12e/src/interface.o:(.literal.setHostname+0x30): undefined reference to `mdns_hostname_set'
/home/hmurad/.platformio/packages/toolchain-xtensa@src-5e1dba65eabc364c0001e57aa616c039/bin/../lib/gcc/xtensa-lx106-elf/8.4.0/../../../../xtensa-lx106-elf/bin/ld: .pio/build/esp12e/src/interface.o:(.literal.setHostname+0x34): undefined reference to `mdns_instance_name_set'
/home/hmurad/.platformio/packages/toolchain-xtensa@src-5e1dba65eabc364c0001e57aa616c039/bin/../lib/gcc/xtensa-lx106-elf/8.4.0/../../../../xtensa-lx106-elf/bin/ld: .pio/build/esp12e/src/interface.o:(.literal.setHostname+0x38): undefined reference to `mdns_service_add'
/home/hmurad/.platformio/packages/toolchain-xtensa@src-5e1dba65eabc364c0001e57aa616c039/bin/../lib/gcc/xtensa-lx106-elf/8.4.0/../../../../xtensa-lx106-elf/bin/ld: .pio/build/esp12e/src/interface.o: in function `heapSize':
/home/hmurad/Documents/PlatformIO/Projects/Le-Radio/src/interface.c:1155: undefined reference to `mdns_service_remove'
/home/hmurad/.platformio/packages/toolchain-xtensa@src-5e1dba65eabc364c0001e57aa616c039/bin/../lib/gcc/xtensa-lx106-elf/8.4.0/../../../../xtensa-lx106-elf/bin/ld: .pio/build/esp12e/src/interface.o: in function `setHostname':
/home/hmurad/Documents/PlatformIO/Projects/Le-Radio/src/interface.c:1161: undefined reference to `mdns_service_remove'
/home/hmurad/.platformio/packages/toolchain-xtensa@src-5e1dba65eabc364c0001e57aa616c039/bin/../lib/gcc/xtensa-lx106-elf/8.4.0/../../../../xtensa-lx106-elf/bin/ld: /home/hmurad/Documents/PlatformIO/Projects/Le-Radio/src/interface.c:1162: undefined reference to `mdns_hostname_set'
/home/hmurad/.platformio/packages/toolchain-xtensa@src-5e1dba65eabc364c0001e57aa616c039/bin/../lib/gcc/xtensa-lx106-elf/8.4.0/../../../../xtensa-lx106-elf/bin/ld: /home/hmurad/Documents/PlatformIO/Projects/Le-Radio/src/interface.c:1163: undefined reference to `mdns_instance_name_set'
/home/hmurad/.platformio/packages/toolchain-xtensa@src-5e1dba65eabc364c0001e57aa616c039/bin/../lib/gcc/xtensa-lx106-elf/8.4.0/../../../../xtensa-lx106-elf/bin/ld: /home/hmurad/Documents/PlatformIO/Projects/Le-Radio/src/interface.c:1165: undefined reference to `mdns_service_add'
/home/hmurad/.platformio/packages/toolchain-xtensa@src-5e1dba65eabc364c0001e57aa616c039/bin/../lib/gcc/xtensa-lx106-elf/8.4.0/../../../../xtensa-lx106-elf/bin/ld: /home/hmurad/Documents/PlatformIO/Projects/Le-Radio/src/interface.c:1166: undefined reference to `mdns_service_add'
collect2: error: ld returned 1 exit status
*** [.pio/build/esp12e/firmware.elf] Error 1

VScode finds mdns.h library fine, but when i run compilation i see this error.

[freedib]

@h1aji Right. I didn't test mDNS :-(
mDNS is not enabled by default. You have to execute "pio run -t menuconfig" to enable it in sdkconfig.
In menuconfig, it is in "Component Config" / "mDNS" / "Enable mDNS".
When mDNS is activated, the side effect is that IPv6 is automatically enabled ("Component Config" / "LWIP" / "Enable IPv6")

[Bastian0302]

Hallo,

i was searching for a long time and now found this.
But i get an error while installing the packages.

TypeError: stat: path should be string, bytes, os.PathLike or integer, not NoneType: File "C:\Users\Bastian\.platformio\penv\lib\site-packages\platformio\builder\main.py", line 174: env.SConscript("$BUILD_SCRIPT") File "C:\Users\Bastian\.platformio\packages\tool-scons\scons-local-4.5.2\SCons\Script\SConscript.py", line 598: return _SConscript(self.fs, *files, **subst_kw) File "C:\Users\Bastian\.platformio\packages\tool-scons\scons-local-4.5.2\SCons\Script\SConscript.py", line 285: exec(compile(scriptdata, scriptname, 'exec'), call_stack[-1].globals) File "C:\Users\Bastian\.platformio\platforms\espressif8266\builder\main.py", line 250: target_elf = env.BuildProgram() File "C:\Users\Bastian\.platformio\packages\tool-scons\scons-local-4.5.2\SCons\Util\envs.py", line 242: return self.method(*nargs, **kwargs) File "C:\Users\Bastian\.platformio\penv\lib\site-packages\platformio\builder\tools\piobuild.py", line 60: env.ProcessProgramDeps() File "C:\Users\Bastian\.platformio\packages\tool-scons\scons-local-4.5.2\SCons\Util\envs.py", line 242: return self.method(*nargs, **kwargs) File "C:\Users\Bastian\.platformio\penv\lib\site-packages\platformio\builder\tools\piobuild.py", line 120: env.BuildFrameworks(env.get("PIOFRAMEWORK")) File "C:\Users\Bastian\.platformio\packages\tool-scons\scons-local-4.5.2\SCons\Util\envs.py", line 242: return self.method(*nargs, **kwargs) File "C:\Users\Bastian\.platformio\penv\lib\site-packages\platformio\builder\tools\piobuild.py", line 331: SConscript(env.GetFrameworkScript(name), exports="env") File "C:\Users\Bastian\.platformio\packages\tool-scons\scons-local-4.5.2\SCons\Script\SConscript.py", line 662: return method(*args, **kw) File "C:\Users\Bastian\.platformio\packages\tool-scons\scons-local-4.5.2\SCons\Script\SConscript.py", line 598: return _SConscript(self.fs, *files, **subst_kw) File "C:\Users\Bastian\.platformio\packages\tool-scons\scons-local-4.5.2\SCons\Script\SConscript.py", line 285: exec(compile(scriptdata, scriptname, 'exec'), call_stack[-1].globals) File "C:\Users\Bastian\.platformio\platforms\espressif8266\builder\frameworks\esp8266-rtos-sdk.py", line 60: assert os.path.isdir(TOOLCHAIN_DIR) File "C:\Users\Bastian\.platformio\python3\lib\genericpath.py", line 42: st = os.stat(s))

Im working on windows.

Thanks.

[h1aji]

@Bastian0302 what python version are you using?

[Bastian0302]

I installed 3.10.11 with the check box „add to path“ but still not working.

Update: I uninstalled Paython, deleted .platformio,and isntalled all new.
But still same error.

Additional Information:
`Tool Manager: Installing git+https://github.com/freedib/toolchain-xtensa-8.4.0-linux-amd64.git @ 8.4.0
Tool Manager: toolchain-xtensa@8.4.0+sha.61d3129 has been installed!
Verbose mode can be enabled via -v, --verbose option
CONFIGURATION: https://docs.platformio.org/page/boards/espressif8266/d1_mini_pro.html
PLATFORM: Espressif 8266 (4.0.1+sha.4979251) > WeMos D1 mini Pro
HARDWARE: ESP8266 80MHz, 80KB RAM, 16MB Flash
PACKAGES:

• framework-esp8266-rtos-sdk @ 3.4.0+sha.519046b
• tool-cmake @ 3.16.4
• tool-esptool @ 1.413.0 (4.13)
• tool-esptoolpy @ 1.30000.201119 (3.0.0)
• tool-genbin-esp8266 @ 1.0.0+sha.baddadf
• tool-ninja @ 1.9.0`

I have no clue where i can start finding the error.

[remuslazar]

@ivankravets it will be really awesome if this PR could get merged! ESP8266 is still an awesome and cheap device to use. 🙏

[jeecrypt]

I would like to return to PlatformIO, but the old version of RTOS SDK does not allow me to do this. I'm really looking forward to the update. I want stability.

[milo1000]

Apparently this project is dead, year passed and nobody cares to merge most important PR.

[cziter15]

@ivankravets is there any chance to get this merged? It might be not so trivial as it links two external dependencies (RTOS sdk repo and tool-genbin), but please at least give it a try.

I'm porting https://github.com/cziter15/ksIotFrameworkLib to espidf, but it would be nice to support esp8266 too.

[tbb98]

@ivankravets @valeros
Is it possible to merge this?
Even into an another branch, if You can't really test it?

[KrazyOTTO]

@Bastian0302 did you fix the issue? if yes than please let me know im having the same issue on my end aswell.

[HomeAutomationComponents]

I can’t fix the problem. And new models of the ESP32 make it unnecessary to investigate further at this topic.

[freedib]

Hello. I can help fixing the issue. Are we talking of comment on May 30, 2023?
The platform was tested on Linux, but if there is still interest in this platform, I can make full tests for Windows and correct it.
It is true that for new projects, esp32 variants are the right way to go, but I guess the existing esp8266 will be there for many years.

[freedib]

I fixed the problem. The labeling of the toolchain was incorrect. I made also a few fixes on the platform to support windows, so please make a git pull on it.
For esp8266-rtos platformio.ini should contain:

[env]
platform = https://github.com/freedib/platform-espressif8266.git
framework = esp8266-rtos-sdk
platform_packages =
framework-esp8266-rtos-sdk @ https://github.com/freedib/framework-esp8266-rtos-sdk.git@3.4
toolchain-xtensa @ https://github.com/freedib/toolchain-xtensa-windows_amd64-8.4.0.git@8.4.0

The missing feature of the platform is automatic generation of sdkconfig. Actually I generate this file using the framework with the command: idf.py menuconfig.
The sdk is different on esp8266 from esp32 where this file is generated automatically when calling cmake. I'm thinking about how integrate this feature.

[omar-mohamed-khallaf]

In case it might help you, here's how I set up a project and use cmake without Makefile (i.e. sdkconfig is autogenerated).
Project directory (from hello_world):

.
├── build.sh
├── CMakeLists.txt
├── CMakePresets.json
├── components
│   ├── common
│   │   ├── CMakeLists.txt
│   │   ├── include
│   │   │   ├── globals.hpp
│   │   │   ├── mqtt.hpp
│   │   │   └── nvs.hpp
│   │   ├── Kconfig.projbuild
│   │   └── src
│   │       ├── mqtt.cpp
│   │       └── nvs.cpp
│   ├── smartconfig
│   │   ├── CMakeLists.txt
│   │   ├── include
│   │   │   ├── smartconfig.hpp
│   │   │   └── wifi.hpp
│   │   ├── Kconfig.projbuild
│   │   └── src
│   │       ├── smartconfig.cpp
│   │       └── wifi.cpp
│   └── utils
│       ├── CMakeLists.txt
│       ├── include
│       │   └── utils.hpp
│       └── src
│           └── utils.cpp
├── main
│   ├── CMakeLists.txt
│   ├── include
│   └── src
│       └── main.cpp
├── partitions.csv
├── res
│   └── rootca.pem
├── sdkconfig
└── sdkconfig.old

1. Export variables

export PATH="$PATH":/opt/toolchains/xtensa-lx106-elf/bin/
export IDF_PATH=/opt/sdks/esp8266-rtos-sdk
export ESPPORT=/dev/ttyUSB0
export ESPBAUD=921600

2. In project source directory, create a venv environment to install python dependencies.

python -m venv .venv
. .venv/bin/activate
pip install -r $IDF_PATH/requirements.txt

3. For components, I replace component.mk with CMakeLists.txt for example in ./components/smartconfig/CMakeLists.txt:

idf_component_register(
    SRCS
    "src/smartconfig.cpp"
    "src/wifi.cpp"
    INCLUDE_DIRS
    "include"
    PRIV_REQUIRES
    utils
    common
)

and in ./main/CMakeLists.txt:

idf_component_register(SRCS "src/main.cpp" INCLUDE_DIRS "include")

4. Use cmake to build:

cmake -B cmake-build-release -S . -DCMAKE_BUILD_TYPE=Release
cmake --build cmake-build-release --target flash

I use CMakePresets but that's the basic idea.

[freedib]

Thanks. You right. sdkconfig is auto-generated by cmake when we pass only -S and -B parameters.
But if you want use a different sdkconfig name like in platformio where the name is sdkconfig.{$board} and call cmake with parameter -DSDKCONFIG=sdkconfig.esp12e, we get an error about the file doesn't exists. We have even the problem with -DSDKCONFIG=sdkconfig.
With espidf we don't have the issue.
I was able to modify project.cmake in RTOS-SDK 3.4 to avoid checking if the file exists and now cmake works as expected with parameter -DSDKCONFIG and file file is generated as expected.
With this patch, I was able to name sdkconfig files the same way as in platform-espressif32.

platform-espressif8266 and framework-esp8266-rtos-sdk have been updated to support it

[tbb98]

i wonder when they will press the button

[HomeAutomationComponents]

There now really good boards with esp32 and esp8266 layout, Pins are compatible and I replaced a lot of them. Only products like Plug Switches are no longer usable because you can’t replace the chip.

Sad but that’s how it goes.

[panda-io]

Hello @valeros Please consider to merge the PR, we are waiting to revive our boards. Arduino core is not the only choice, let's code also with c/c++. Thank you!