esptool: bump to v4.8.1

Update esptool to latest release.

Signed-off-by: Sylvio Alves <[email protected]>

Author: sylvioalves

tools/esptool_py/CHANGELOG.md

@@ -1,3 +1,102 @@
+## v4.8.1 (2024-09-25)
+
+### Bug Fixes
+
+- **esp32c2**: Add esp32c2 eco4 rom magic value
+- **packaging**: Correctly exclude the unwanted sub/modules
+
+## v4.8.0 (2024-09-18)
+
+### New Features
+
+- **espefuse**: Supports wafer efuse versions for esp32c61
+- **esptool**: add new command SFDP read
+- **esptool**: Add option to retry connection in a loop
+- **efuse**: Updates efuse table for esp32c5
+- **efuse**: Updates efuse table for esp32p4
+- **esp32c61**: Added stub flasher support
+- **cli**: add autocompletions
+- **esptool**: allow picking UART by VID/PID/Name
+- **esp32c5**: Add USB-serial/JTAG stub support
+- **esp32c5**: Add UART stub support
+- **esptool**: Print key_purpose name for get_security_info cmd
+- **espefuse**: Adds support extend efuse table by user CSV file
+- **espefuse**: Adds efuse dump formats: separated(default) and united(new)
+- **espefuse**: Adds incompatible eFuse settings check for S3
+- **reset**: Apply reconnections to the whole reset sequence, not line transitions
+- **reset**: Automatically reconnect if port disconnects during reset
+- **esp32-p4**: Add ECO1 magic number
+- **espsecure**: Add support for secure boot v2 using ECDSA-P384 signatures
+- **write_flash**: retry flashing if chip disconnects
+- **espefuse**: Allow filtering efuses based on command line arguments
+- **esploader**: Enable context manager for esp instances
+- **espefuse**: Added check for correctness of written data
+- **espefuse**: Improves help for burn_efuse cmd
+- **esp32s3**: clear boot control register on hard reset
+- **esp32-p4**: add spi-connection restriction to ROM class
+- add UF2 IDs for ESP32-C5 and ESP32-C61
+- **espefuse**: Updates efuses for C5 and C61
+- **esp32c61**: add c61 basic flash support (no_stub)
+- **esp32c5**: skipped the stub check for esp32c5 mp
+- **esp32c5**: base support of esp32c5 mp (no stub)
+- Added warning when secure boot enabled
+- **cmds/write_flash**: Recalculated SHA digest for image binary
+- print flash voltage in flash_id command
+- **esptool**: Adds wafer and pkg versions
+- **espefuse**: Update adc_info commands for all chips
+- **espefuse**: Adds new efuses for esp32p4
+- **espefuse**: Allow the espefuse.py to work when coding scheme == 3
+- **err_defs**: Add ROM bootloader flash error definitions
+- Use ruff instead of flake8 and black both in pre-commit and CI
+- **esp32p4**: Enable USB-serial/JTAG in flasher stub
+- **espefuse**: Postpone some efuses to burn them at the very end
+- add advisory port locking
+- **espefuse**: check_error --recover chip even if there are num_errors
+- **espefuse**: Adds new efuses for esp32c6 and esp32h2
+- **esp32c5**: add target esp32c5 beta3
+
+### Bug Fixes
+
+- **esptool**: Fix esp32c61 flash frequency config
+- **esptool**: Fix incorrect chip version for esp32c5
+- **write_flash**: Verify if files will fit against the real flash size when possible
+- **remote_ports**: Disable reset sequence when a socket is used
+- **bitstring**: Restricted bitstring dependency to fix 32-bit compatibility
+- **esp32_d0wdr2_v3**: Print correct chip name
+- pass error message to exception in OTG mode
+- **bin_image**: add check for ELF file segment when saving RAM segments
+- **docs**: Add a note about entering manual bootloader mode
+- **esp32c5**: Fix MAC reading for esptool
+- Erase non-aligned bytes with --no-stub
+- **esp32-c5**: Use a longer reset delay with usb-serial/jtag to stabilize boot-up
+- **espefuse**: Use stub class if stub flasher is running
+- Do not append SHA256 when `--ram-only-header`
+- **elf2image**: add ELF flags to merge condition
+- ram_only_header: pad flash segment to next boundary
+- sort segments if ram_only_header is used
+- **espefuse**: Fix efuse base addr for esp32c5 MP
+- fix type annotation to comply with mypy
+- **espefuse**: Fix burn_key for ECDSA_KEY, it can read pem file
+- **secure_download_mode**: Disable secure boot detection and print more info
+- **esptool**: clear boot control register on ESP32-S3
+- **intelhex**: catch unicode decode errors when converting hex to binary
+- ROM doesn't attach in-package flash chips
+- close file gracefully in espsecure
+- Fixed glitches on RTS line when no_reset option on Windows
+- **merge_bin**: treat files starting with colon as raw files
+- Index image segments from 0 instead of 1
+- **read_flash**: add flash size arg to enable reading past 2MB without stub
+- **read_flash**: flush transmit buffer less often to inrease throughput
+- **esptool**: Proper alignment for SoCs with offset load
+- ignore resetting on unsupported ports
+- **esptool**: Remove the shebang from uf2_writer.py
+
+### Code Refactoring
+
+- Migrated esp_rfc2217_server into standalone subpackage
+- **test/esptool**: Updated tests according to SHA recomputation for binary
+- **style**: Comply with black>=24.0.0
+
 ## v4.7.0 (2023-12-13)
 
 ### New Features

tools/esptool_py/MANIFEST.in

@@ -1,11 +1,11 @@
 include README.md
 include LICENSE
-include esptool/targets/stub_flasher/*.json
+include esptool/targets/stub_flasher/1/*
+include esptool/targets/stub_flasher/2/*
 include espefuse/efuse_defs/*.yaml
 # sdist includes test/test*.py by default, but esptool.py tests
 # are so far only intended to run from the git repo itself
 prune test
-prune flasher_stub
 prune .github
 prune docs
 exclude .git*

tools/esptool_py/ci/Sign-File.ps1

@@ -0,0 +1,78 @@
+[CmdletBinding()]
+param (
+    [Parameter()]
+    [String]
+    $Path
+)
+
+
+function FindSignTool {
+    $SignTool = "signtool.exe"
+    if (Get-Command $SignTool -ErrorAction SilentlyContinue) {
+        return $SignTool
+    }
+    $SignTool = "${env:ProgramFiles(x86)}\Windows Kits\10\bin\x64\signtool.exe"
+    if (Test-Path -Path $SignTool -PathType Leaf) {
+        return $SignTool
+    }
+    $SignTool = "${env:ProgramFiles(x86)}\Windows Kits\10\bin\x86\signtool.exe"
+    if (Test-Path -Path $SignTool -PathType Leaf) {
+        return $SignTool
+    }
+    $sdkVers = "10.0.22000.0", "10.0.20348.0", "10.0.19041.0", "10.0.17763.0"
+    Foreach ($ver in $sdkVers)
+    {
+        $SignTool = "${env:ProgramFiles(x86)}\Windows Kits\10\bin\${ver}\x64\signtool.exe"
+        if (Test-Path -Path $SignTool -PathType Leaf) {
+            return $SignTool
+        }
+    }
+    "signtool.exe not found"
+    Exit 1
+}
+
+function SignEsptool {
+    param(
+        [Parameter()]
+        [String]
+        $Path
+    )
+
+    $SignTool = FindSignTool
+    "Using: $SignTool"
+    $CertificateFile = [system.io.path]::GetTempPath() + "certificate.pfx"
+
+    if ($null -eq $env:CERTIFICATE) {
+        "CERTIFICATE variable not set, unable to sign the file"
+        Exit 1
+    }
+
+    if ("" -eq $env:CERTIFICATE) {
+        "CERTIFICATE variable is empty, unable to sign the file"
+        Exit 1
+    }
+
+    $SignParameters = @("sign", "/tr", 'http://timestamp.digicert.com', "/td", "SHA256", "/f", $CertificateFile, "/fd", "SHA256")
+    if ($env:CERTIFICATE_PASSWORD) {
+        "CERTIFICATE_PASSWORD detected, using the password"
+        $SignParameters += "/p"
+        $SignParameters += $env:CERTIFICATE_PASSWORD
+    }
+    $SignParameters += $Path
+
+    [byte[]]$CertificateBytes = [convert]::FromBase64String($env:CERTIFICATE)
+    [IO.File]::WriteAllBytes($CertificateFile, $CertificateBytes)
+
+    &$SignTool $SignParameters
+
+    if (0 -eq $LASTEXITCODE) {
+        Remove-Item $CertificateFile
+    } else {
+        Remove-Item $CertificateFile
+        "Signing failed"
+        Exit 1
+    }
+
+}
+
+SignEsptool ${Path}

tools/esptool_py/ci/download_flasher_stubs.py

@@ -0,0 +1,91 @@
+#!/usr/bin/env python
+#
+# SPDX-FileCopyrightText: 2024 Espressif Systems (Shanghai) CO LTD
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+import glob
+import os
+import urllib.request
+
+STUBS = (
+    {
+        "STUB_SET_VERSION": "1",
+        "DOWNLOAD_URL": "https://github.com/espressif/esptool-legacy-flasher-stub/releases/download",
+        "TAG_URL": "https://github.com/espressif/esptool-legacy-flasher-stub/releases/tag",
+        "VERSION": "v1.3.0",
+        "FILE_LIST": (
+            "esp32",
+            "esp32c2",
+            "esp32c3",
+            "esp32c5",
+            "esp32c5beta3",
+            "esp32c6",
+            "esp32c61",
+            "esp32c6beta",
+            "esp32h2",
+            "esp32h2beta1",
+            "esp32h2beta2",
+            "esp32p4",
+            "esp32s2",
+            "esp32s3",
+            "esp32s3beta2",
+            "esp8266",
+        ),
+        "LICENSE": "released as Free Software under GNU General Public License Version 2 or later",
+    },
+    {
+        "STUB_SET_VERSION": "2",
+        "DOWNLOAD_URL": "https://github.com/esp-rs/esp-flasher-stub/releases/download",
+        "TAG_URL": "https://github.com/esp-rs/esp-flasher-stub/releases/tag",
+        "VERSION": "v0.3.0",
+        "FILE_LIST": (
+            "esp32",
+            "esp32c2",
+            "esp32c3",
+            "esp32c6",
+            "esp32h2",
+            "esp32s2",
+            "esp32s3",
+        ),
+        "LICENSE": "dual licensed under the Apache License Version 2.0 or the MIT license",
+    },
+)
+
+DESTINATION_DIR = os.path.join("esptool", "targets", "stub_flasher")
+
+README_TEMPLATE = """# Licensing
+
+The binaries in JSON format distributed in this directory are {LICENSE}. They were released at {URL} from where the sources can be obtained.
+"""
+
+
+def main():
+    for stub_set in STUBS:
+        dest_sub_dir = os.path.join(DESTINATION_DIR, stub_set["STUB_SET_VERSION"])
+
+        """ The directory is cleaned up so we would detect if a stub was just committed into the repository but the
+        name was not added into the FILE_LIST of STUBS. This would be an unwanted state because the checker would not
+        detect any changes in that stub."""
+        for old_file in glob.glob(os.path.join(dest_sub_dir, "*.json")):
+            print(f"Removing old file {old_file}")
+            os.remove(old_file)
+
+        for file_name in stub_set["FILE_LIST"]:
+            file = ".".join((file_name, "json"))
+            url = "/".join((stub_set["DOWNLOAD_URL"], stub_set["VERSION"], file))
+            dest = os.path.join(dest_sub_dir, file)
+            print(f"Downloading {url} to {dest}")
+            urllib.request.urlretrieve(url, dest)
+
+        with open(os.path.join(dest_sub_dir, "README.md"), "w") as f:
+            print(f"Writing README to {f.name}")
+            f.write(
+                README_TEMPLATE.format(
+                    LICENSE=stub_set["LICENSE"],
+                    URL="/".join((stub_set["TAG_URL"], stub_set["VERSION"])),
+                )
+            )
+
+
+if __name__ == "__main__":
+    main()

tools/esptool_py/ci/espressif.ico

@@ -0,0 +1,28 @@
+{# This changelog template is used for automatically generated GH release notes. #}
+{# It is passed to commitizen's --template option in a GH Actions workflow run. #}
+
+{% for entry in tree %}
+
+{% for change_key, changes in entry.changes.items() %}
+
+{% if change_key %}
+### {{ change_key }}
+{% endif %}
+
+{% for change in changes %}
+{% if change.scope %}
+- **{{ change.scope }}**: {{ change.message }}
+{% elif change.message %}
+- {{ change.message }}
+{% endif %}
+{% endfor %}
+{% endfor %}
+{% endfor %}
+
+Thanks to <FILL OUT CONTRIBUTORS>, and others for contributing to this release!
+
+# Results of checking the release against common anti-virus SW
+
+<Upload the release binaries to VirusTotal and ADD a link to the report here>
+
+The failures are probably false positives. You can mark esptool as safe in your anti-virus SW, or [install esptool from source](https://docs.espressif.com/projects/esptool/en/latest/installation.html).

tools/esptool_py/ci/gh_changelog_template.md.j2

@@ -0,0 +1,40 @@
+# SPDX-FileCopyrightText: 2022-2023 Espressif Systems (Shanghai) CO LTD
+#
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+import argparse
+import re
+
+LINE_RE = re.compile(r"^__version__ = ['\"]([^'\"]*)['\"]")
+NEW_LINE = '__version__ = "{}"\n'
+
+
+def patch_file(path, new_version):
+    assert ".dev" in new_version
+    new_version = new_version.lstrip("v")
+
+    with open(path, "r") as fin:
+        lines = fin.readlines()
+
+    for i, line in enumerate(lines, start=0):
+        m = LINE_RE.search(line)
+        if m:
+            lines[i] = NEW_LINE.format(new_version)
+            break
+
+    with open(path, "w") as fout:
+        fout.writelines(lines)
+
+
+def main():
+    parser = argparse.ArgumentParser()
+    parser.add_argument("file", help="Path to script with __version__")
+    parser.add_argument(
+        "--version", help="Development version specifier to patch the version to"
+    )
+    args = parser.parse_args()
+    patch_file(args.file, args.version)
+
+
+if __name__ == "__main__":
+    main()

tools/esptool_py/ci/patch_dev_release.py

@@ -0,0 +1,7 @@
+#!/bin/bash
+
+# Init tokens for tests
+softhsm2-util --init-token --label softhsm-test-token --pin 1234 --so-pin 123456 --slot 0
+softhsm2-util --init-token --label softhsm-test-token-1 --pin 1234 --so-pin 123456 --slot 1
+softhsm2-util --init-token --label softhsm-test-token-2 --pin 1234 --so-pin 123456 --slot 2
+softhsm2-util --init-token --label softhsm-test-token-3 --pin 1234 --so-pin 123456 --slot 3

tools/esptool_py/ci/setup_softhsm2.sh

@@ -166,6 +166,10 @@ Depending on the kind of hardware you have, it may also be possible to manually
 - For development boards produced by Espressif, this information can be found in the respective getting started guides or user guides. For example, to manually reset a development board, hold down the **Boot** button (``{IDF_TARGET_STRAP_BOOT_GPIO}``) and press the **EN** button (``EN`` (``CHIP_PU``)).
 - For other types of hardware, try pulling ``{IDF_TARGET_STRAP_BOOT_GPIO}`` down.
 
+.. note::
+
+   If esptool is able to reset the chip but for some reason the chip is not entering into bootloader mode then hold down the Boot button (or pull down ``{IDF_TARGET_STRAP_BOOT_GPIO}``) while you start esptool and keep it down during reset.
+
 .. only:: esp8266
 
    .. _boot-log-esp8266:

tools/esptool_py/docs/en/advanced-topics/boot-mode-selection.rst

@@ -10,6 +10,10 @@ Positional arguments:
 - ``eFuse name``
 - ``value``
 
+Optional arguments:
+
+* ``--force``. Suppress an error to burn eFuses. The tool checks for incompatible eFuse states to prevent them from burning and potentially **bricking the chip**. Use this flag only if you are sure. This will suppress the eFuse incompatibility error.
+
 It can be list of eFuse names and values (like EFUSE_NAME1 1 EFUSE_NAME2 7 EFUSE_NAME3 10 etc.).
 
 New values can be a numeric value in decimal or hex (with "0x" prefix). eFuse bits can only be burned from 0 to 1, attempting to set any back to 0 will have no effect. Most eFuses have a limited bit width (many are only 1-bit flags). Longer eFuses (MAC addresses, keys) can be set with this command, but it's better to use a specific command (``burn_custom_mac``, ``burn_key``) for a specific field.