diff --git a/src/content/docs/guides/cli.mdx b/src/content/docs/guides/cli.mdx index eec53c5e79..5de0104e78 100644 --- a/src/content/docs/guides/cli.mdx +++ b/src/content/docs/guides/cli.mdx @@ -130,6 +130,9 @@ The `esphome compile ` validates the configuration and compiles the firm **`--only-generate`** : If set, only generates the C++ source code and does not compile the firmware. +**`--native-idf`** +: If set, uses the ESP-IDF framework instead of PlatformIO to compile the project. +Please see [Native ESP-IDF](/guides/native-idf) for details. ### `upload` Command diff --git a/src/content/docs/guides/index.mdx b/src/content/docs/guides/index.mdx index 9998b1f1cd..d35ac968f6 100644 --- a/src/content/docs/guides/index.mdx +++ b/src/content/docs/guides/index.mdx @@ -16,6 +16,7 @@ title: "Guides" - [YAML Configuration in ESPHome](/guides/yaml/) - [Configuration Types](/guides/configuration-types/) - [Command Line Interface](/guides/cli/) +- [Native ESP-IDF](/guides/native_idf/) - [Frequently Asked Questions](/guides/faq/) - [Troubleshooting](/guides/troubleshooting/) - [Security Best Practices](/guides/security_best_practices/) diff --git a/src/content/docs/guides/native_idf.mdx b/src/content/docs/guides/native_idf.mdx new file mode 100644 index 0000000000..bfd40f931f --- /dev/null +++ b/src/content/docs/guides/native_idf.mdx @@ -0,0 +1,153 @@ +--- +description: "Native ESP-IDF" +title: "Native ESP-IDF" +--- +# Automatic ESP-IDF Framework Installation + +ESPHome can automatically download, install, and manage the ESP-IDF framework and its Python environment, so you no longer need to manually clone ESP-IDF, run `install.sh`, or set `IDF_PATH` yourself. + +This page explains how the automatic ESP-IDF management works, where it stores data on disk, and how you can customize it with environment variables. + +## Overview + +With the automatic framework management: + +- ESPHome downloads a specific ESP-IDF release archive (v5.5.2 by default) into a per-version folder under the ESPHome data directory. +- ESPHome runs the ESP-IDF tool installer (`idf_tools.py`) to install the required tools such as `cmake` and `ninja` if they are not already available. +- ESPHome creates and manages a dedicated Python virtual environment per ESP-IDF version and installs the ESP-IDF Python dependencies into it. +- Builds and ESP-IDF tools (including `idf.py` and `esptool`) run using this managed environment, so you do not need to source `export.sh` or set `IDF_PATH`. + +> [!NOTE] +> If you already provide an IDF_PATH environment variable, ESPHome will honor it and use your existing ESP-IDF installation instead of managing its own copy. + +## Storage locations + +By default, ESPHome stores ESP-IDF frameworks and Python environments under its data directory: + +- Base directory: `/idf` +- Frameworks: `/idf/frameworks//` +- Python environments: `/idf/penvs//` + +On a typical installation, `` is the ESPHome data directory (for example `./.esphome`). + +You can override the base directory with the `ESPHOME_ESP_IDF_PREFIX` environment variable: + +```bash +export ESPHOME_ESP_IDF_PREFIX="/opt/esphome-esp-idf" +``` + +## Default ESP-IDF version and features + +ESPHome selects a default ESP-IDF version and feature set when it needs to install the framework automatically: + +- Default ESP-IDF version: `ESPHOME_IDF_DEFAULT_VERSION` (defaults to `5.5.2` if not set). +- Default targets: `ESPHOME_IDF_DEFAULT_TARGETS` (defaults to `all`). +- Default tools: `ESPHOME_IDF_DEFAULT_TOOLS` (defaults to `cmake;ninja`). +- Tools that must be installed even if present on the system: `ESPHOME_IDF_DEFAULT_TOOLS_FORCE` (defaults to `required`). +- Default Python requirement feature sets: `ESPHOME_IDF_DEFAULT_FEATURES` (defaults to `core`). + +All of these values can be customized via environment variables, using semicolon-separated lists where applicable: + +```bash +# Use a different ESP-IDF version +export ESPHOME_IDF_DEFAULT_VERSION="5.5.3" + +# Only install tools for specific targets +export ESPHOME_IDF_DEFAULT_TARGETS="esp32s2;esp32c3" + +# Force installation of extra tools +export ESPHOME_IDF_DEFAULT_TOOLS="cmake;ninja;openocd" + +# Control which requirement sets are installed +export ESPHOME_IDF_DEFAULT_FEATURES="core;gdbgui" +``` + +When no explicit tool list is provided, ESPHome determines which tools to install by merging `ESPHOME_IDF_DEFAULT_TOOLS` and `ESPHOME_IDF_DEFAULT_TOOLS_FORCE`, and then skipping tools that are already available on the system unless they are listed as “force”. + +## Download mirrors + +ESPHome downloads the ESP-IDF framework and its Python constraints file from configurable mirror URLs. + +### Framework archives + +Framework archives are downloaded from the list of URLs in `ESPHOME_IDF_FRAMEWORK_MIRRORS`. + +The default value is: + +```text +https://github.com/espressif/esp-idf/releases/download/v{VERSION}/esp-idf-v{VERSION}.zip +``` + +The `{VERSION}` placeholder is replaced by the ESP-IDF version (for example `5.5.2`). + +You can provide multiple mirrors separated by semicolons; ESPHome will try each mirror in order until the download succeeds: + +```bash +export ESPHOME_IDF_FRAMEWORK_MIRRORS="\ +https://my.local.mirror/esp-idf-v{VERSION}.zip;\ +https://github.com/espressif/esp-idf/releases/download/v{VERSION}/esp-idf-v{VERSION}.zip" +``` + + +### Python constraints file + +ESPHome also downloads an ESP-IDF Python constraints file to ensure that the Python environment uses versions compatible with the selected ESP-IDF release. + +The list of URLs is configured by `ESP_IDF_CONSTRAINTS_MIRRORS`, which defaults to: + +```text +https://dl.espressif.com/dl/esp-idf/espidf.constraints.v{VERSION}.txt +``` + +The `{VERSION}` placeholder is replaced by the ESP-IDF framework version detected in the installed framework. + +You can provide multiple mirrors separated by semicolons; ESPHome will try each mirror in order until the download succeeds: + +```bash +export ESP_IDF_CONSTRAINTS_MIRRORS="\ +https://my.local.mirror/espidf.constraints.v{VERSION}.txt;\ +https://dl.espressif.com/dl/esp-idf/espidf.constraints.v{VERSION}.txt" +``` + +## Python virtual environment + +For each ESP-IDF version, ESPHome creates a dedicated Python virtual environment under: + +```text +/penvs// +``` + +where `` is either the ESPHome data directory or `ESPHOME_ESP_IDF_PREFIX`. + +The process: + +1. Creates a virtual environment using the host Python interpreter. +2. Downloads the ESP-IDF Python constraints file for the exact ESP-IDF version. +3. Upgrades `pip` and `setuptools` inside the environment using those constraints. +4. Installs the ESP-IDF requirements from `tools/requirements/requirements..txt` for each configured feature (`core`, `gdbgui`, `ci`, etc.). + +To avoid unnecessary re-installs, ESPHome keeps a stamp file per virtual environment that tracks: + +- ESPHome's internal ESP-IDF management version. +- The ESP-IDF version of the framework. +- The Python interpreter version used to create the environment. + +If the stamp file matches the expected values, ESPHome reuses the existing virtual environment; otherwise it deletes and recreates it. + +## Environment used for builds and tools + +When ESPHome needs to run ESP-IDF tools (for example `idf.py` or `esptool`), it: + +1. Ensures that the ESP-IDF framework and Python environment for the requested version are installed. +2. Builds an environment dictionary with: + - `IDF_TOOLS_PATH` pointing to the ESPHome ESP-IDF tools directory. + - `IDF_PATH` pointing to the selected ESP-IDF framework directory. + - `ESP_IDF_VERSION` set to the framework version. + - `IDF_PYTHON_ENV_PATH` pointing to the virtual environment directory. +3. Modifies `PATH` so that python resolves to the managed environment's interpreter. +4. Adds required tool paths and variables returned by the ESP-IDF `export` logic. + +This environment is then used for: + +- `idf.py` commands (for example when building ESP32 firmware). +- `esptool` invocations, such as when creating `factory.bin` images. \ No newline at end of file