Cleanups to release notes (#5480)

Co-authored-by: Keith Burzinski <[email protected]>

Author: bdraco

content/changelog/2025.10.0.md

@@ -25,15 +25,16 @@ while introducing groundbreaking new features.
 - **Z-Wave Proxy** - Bridge Z-Wave devices to Home Assistant through ESPHome
 - **Arduino as IDF Component** - Unified ESP32 framework architecture (breaking change)
 - **Enhanced ePaper Display Support** - New unified SPI ePaper component
-- **Security Enhancements** - SHA256 OTA authentication and password deprecation
+- **Security Enhancements** - SHA256 OTA authentication, password deprecation, and API hardening with OOM protection
 - **API Improvements** - Home Assistant action responses and configurable connection limits
 - **Extensive Memory Optimizations** - Flash and RAM savings across all platforms
 - **New Sensor Components** - WTS01 and LM75B temperature sensors
 
 ## Z-Wave Proxy
 
 The new {{< docref "/components/zwave_proxy" >}} component enables network-based connectivity for Z-Wave hardware
-by proxying serial communication between a Z-Wave modem SoC and Z-Wave JS over WiFi or Ethernet.
+by proxying serial communication between a Z-Wave modem SoC and Z-Wave JS over WiFi or Ethernet. This component was
+specifically designed for the [Home Assistant ZWA-2 Z-Wave Adapter](https://www.home-assistant.io/connect/zwa-2/), which combines an ESP32 and a Z-Wave modem in a single device, but could support other Z-Wave hardware in the future.
 
 **Key Features:**
 
@@ -47,24 +48,51 @@ by proxying serial communication between a Z-Wave modem SoC and Z-Wave JS over W
 This component is particularly useful for optimizing Z-Wave mesh network topology by placing the Z-Wave controller in
 a central location, even when your server is in a basement or closet.
 
-## Arduino as IDF Component (Breaking Change)
+## Arduino as IDF Component (Major Architectural Change)
 
-A significant architectural change in this release integrates Arduino as an ESP-IDF component rather than a
-separate framework. This unification brings:
+This release includes a fundamental change in how ESP32 Arduino builds work - **Arduino is now integrated as an ESP-IDF component** rather than a separate framework. This is the biggest architectural change for ESP32 in ESPHome's history.
 
-**Benefits:**
+Previously, Arduino came as a precompiled framework with everything baked in - you got whatever upstream decided, whether you needed it or not. This led to bloated builds that often **couldn't even fit on devices** with limited flash space. You were stuck with the upstream choices and locked out of ESP-IDF features.
 
-- **Better compatibility** between Arduino and ESP-IDF features
-- **Simplified build process** with unified toolchain
-- **Access to latest ESP-IDF features** while maintaining Arduino compatibility
-- **Improved memory efficiency** through shared libraries
-- **Future-proof architecture** aligning with Espressif's direction
+Now Arduino is built as an ESP-IDF component alongside your configuration. ESPHome compiles only what you actually use, eliminating the bloat. You get access to ESP-IDF features while still using Arduino libraries when you need them. This aligns with Espressif's recommended approach and is how the Arduino ecosystem is evolving.
 
-**Migration Notes:**
+**Memory Savings:**
 
-- Most configurations will work without changes
-- Custom components may need minor adjustments
-- Framework version handling has been improved (see breaking changes section)
+- **20-30KB RAM savings** on Arduino builds
+- **Additional ~8KB RAM savings** if using the web server
+- **Smaller binary sizes** overall
+- Better memory efficiency through shared libraries
+
+**Development Benefits:**
+
+- **Faster fixes for underlying components** - Many components now share the same implementations between Arduino and ESP-IDF, eliminating duplicate maintenance
+- **Access to latest ESP-IDF 5.4** features while using Arduino libraries
+- **Reduced maintenance burden** - One framework to maintain instead of two separate paths
+- **Future-proof** - Follows Espressif's direction for the ESP32 ecosystem
+
+**Build Time Trade-off:**
+
+- Arduino builds now take longer (Arduino is compiled on top of ESP-IDF)
+- ESP-IDF builds are now **2-3x faster** than Arduino builds
+- **Consider migrating to ESP-IDF** if build time matters to you
+
+### Should You Migrate to ESP-IDF?
+
+**We recommend ESP-IDF for most users**, especially for:
+
+- Bluetooth Proxy devices (lower memory usage)
+- New projects
+- Configurations where you want faster builds
+
+Arduino is still fully supported and works great if you need it for specific Arduino libraries. See our {{< docref "/components/esp32#migrating-from-arduino" "Arduino to ESP-IDF migration guide" >}} for help switching - it's easier than you might think!
+
+### What You Need to Know
+
+**Most users:** Your configuration will work without changes. ESPHome handles the framework integration automatically.
+
+**External component developers:** If you maintain external components using Arduino-specific APIs, review [Espressif's Arduino as ESP-IDF component documentation](https://docs.espressif.com/projects/arduino-esp32/en/latest/esp-idf_component.html) and test your components with this release.
+
+**Breaking change details:** Framework version handling has been improved with stricter validation. See the breaking changes section for specifics.
 
 ## ePaper SPI Display Component
 
@@ -86,25 +114,74 @@ new architecture provides a more maintainable codebase for future ePaper display
 
 ## Security Enhancements
 
-ESPHome 2025.10.0 significantly improves security with several key changes:
+ESPHome 2025.10.0 significantly improves security with several key changes. We conducted an internal security audit that didn't find any privilege escalation issues but identified many hardening opportunities. We've addressed these in this release and will continue to focus on internal security testing in the coming months.
 
 ### SHA256 OTA Authentication
 
+OTA authentication has been upgraded from MD5 to SHA256 for better security:
+
 - **Modern cryptographic hashing** replaces legacy MD5 authentication
 - **Hardware acceleration** on supported ESP32 variants (S3, etc.)
-- **Backward compatible** transition period
 - **Non-blocking authentication** for better performance
+- **Graceful transition period** - Both SHA256 and MD5 are accepted through 2025.12.x
+- **MD5 will be rejected starting with 2026.1.0** on platforms that support SHA256
+
+During the transition, you'll see a warning when MD5 is used. After 2026.1.0, downgrading to older ESPHome versions will require serial flashing instead of OTA.
+
+**Bonus improvement:** Extended handshake timeout (20s) makes OTA more reliable on poor WiFi connections without freezing the device.
 
-### API Password Deprecation
+### API Password Deprecation - Heads Up
 
-- **API passwords are now deprecated** in favor of encryption keys
-- **Mutual exclusivity** enforced between password and encryption
-- **Improved security model** with modern cryptographic practices
-- **Migration warnings** guide users to encryption-based authentication
+If you're still using password authentication for the API, now's the time to switch to encryption. Password support will be removed in **version 2026.1.0**.
+
+This release adds better warnings so you'll know if you need to update your configuration:
+
+- **You'll see a clear warning** if you're using passwords - no surprises when support is removed
+- **Can't use both** - if you try to configure both password and encryption, ESPHome will stop you (this combination has never worked properly and causes connection failures)
+- **Easy migration** - just switch from `password:` to `encryption:` with a key
+- **You have time** - password support works until 2026.1.0, giving you plenty of time to update
 
 > [!warning]
-> API password authentication is deprecated and will be removed in a future release. Please migrate to
-> encryption-based authentication using the `encryption:` key.
+> If you're using `password:` in your API configuration, please migrate to `encryption:` before version 2026.1.0.
+> Password authentication will be removed at that time. You can't use both password and encryption together -
+> pick one (and encryption is the better choice for security).
+
+## Dashboard Improvements
+
+The ESPHome Dashboard now uses WebSockets instead of HTTP polling for real-time updates:
+
+- **Instant updates** - Device status changes appear immediately (no more 1-2 second polling delay)
+- **Snappier interface** - Everything feels more responsive
+- **Lower server load** - More efficient than constant HTTP polling
+- **Logs start instantly** - The logs command now skips external component updates, eliminating 5-30 second delays when internet is unavailable
+
+If you manage multiple devices, the dashboard will feel noticeably faster and more responsive. Viewing logs is now instant even in offline or isolated network environments.
+
+## Device Onboarding Improvements
+
+Getting your ESPHome devices online has never been easier. This release brings major improvements to the initial setup experience, particularly for ESP32-IDF devices and the [Made for ESPHome](https://esphome.io/guides/made_for_esphome/) program.
+
+### Captive Portal for ESP-IDF
+
+**This is huge.** ESP-IDF devices now have full captive portal support, which means you don't have to figure out which IP address to type into your browser anymore. When you connect to the device's access point, your phone or computer automatically detects the captive portal and opens the configuration page - just like connecting to hotel or airport WiFi.
+
+This brings ESP-IDF up to feature parity with Arduino for onboarding, and since most new ESPHome devices are running ESP-IDF, this dramatically improves the out-of-box experience. The implementation includes a lightweight DNS server that runs in ESPHome's event loop, responding to all DNS queries with the device's IP address to trigger captive portal detection on iOS, Android, and other platforms.
+
+### Improved Improv BLE Performance
+
+[Improv Wi-Fi over BLE](https://www.improv-wifi.com/ble/) is the standard way to provision ESPHome devices over Bluetooth, and we've made it significantly faster and more reliable:
+
+**Performance improvements:**
+
+- **26x faster service lookups** for typical 3-service configurations by replacing HashMap with vectors
+- **1KB flash savings** plus reduced RAM usage
+- **Better discoverability** - Device names are now visible to passive BLE scanners, making ESPHome devices easier to find on a wider range of BLE scanning technologies
+
+**Why passive scanning matters:** Many home automation systems and BLE scanners only support passive scanning (not active scanning). Making the device name visible to passive scanners means your devices show up in more scanning apps and are easier to identify during setup.
+
+These improvements directly support the [Made for ESPHome program](https://esphome.io/guides/made_for_esphome/), which requires Improv provisioning via BLE for Wi-Fi projects to ensure a consistent, high-quality setup experience across all ESPHome-compatible products.
+
+The Made for ESPHome program requires Improv BLE provisioning for Wi-Fi projects. See the [certification requirements](https://esphome.io/guides/made_for_esphome/) for details.
 
 ## Home Assistant Action Responses
 
@@ -141,29 +218,34 @@ inclusion and saving memory.
 
 ## Memory & Performance Optimizations
 
-ESPHome 2025.10.0 continues aggressive optimization efforts:
+ESPHome 2025.10.0 brings major memory efficiency improvements, with **nearly 3KB of RAM savings** on typical configurations. As ESPHome has grown more capable over the years, memory usage has naturally increased with new features. ESP8266 devices with their ~40KB of free RAM were starting to feel the squeeze, and ESP32-C3 Bluetooth proxies were experiencing stability issues under load.
 
-**Flash Savings:**
+**We've made these use cases much better.** Through systematic optimization efforts, we've reclaimed precious RAM and improved stability across the board. ESP8266 users get more headroom for complex configurations. ESP32-C3 Bluetooth proxy users get better stability with multiple active connections. All platforms benefit from faster performance and lower memory overhead.
 
-- **Lock component**: 388 bytes flash + 23 bytes RAM per lock via bitmask optimization
-- **ESP32 BLE Server**: 1KB flash + 26x faster service lookups using vectors instead of HashMap
-- **Event Emitter**: 2.6KB flash + 2.3x faster via vector replacement
-- **Logger**: 35-72% faster log formatting through optimized string handling
-- **Web Server**: Significant reduction through lookup tables and IDF server unification
+The improvements come from replacing heavy C++ STL containers (vectors, maps, sets) with lighter alternatives and moving data from RAM to flash storage where possible. Each optimization might save a few hundred bytes, but they add up quickly when applied across the codebase.
+
+**This is just the beginning** - we're continuing this optimization work in 2025.11.0 and beyond. Expect more memory savings and performance improvements in upcoming releases.
+
+**RAM Savings:**
 
-**RAM Optimizations:**
+- **~3KB total RAM savings on ESP8266, ~1-2KB on ESP32** from all optimizations combined
+- **Lock component**: 388 bytes flash + 23 bytes RAM per lock via bitmask optimization
+- **ESP32 BLE Server**: 1KB flash savings by replacing HashMap with vectors (26x faster lookups)
+- **Event Emitter**: 2.6KB flash savings via vector replacement (2.3x faster)
+- **mDNS/OpenThread**: StaticVector eliminates dynamic allocation overhead
+- **Script names**: Moved to flash storage instead of RAM
+- **Component metadata**: Named structs replace std::pair overhead
+- **API**: Zero-copy techniques eliminate unnecessary data copying in Bluetooth proxy, Z-Wave proxy, and base API calls (authentication, time zones), reducing allocations and improving performance
 
-- **StaticVector** for mDNS/OpenThread services with compile-time capacity
-- **Script names** stored in flash instead of RAM
-- **Component metadata** using named structs instead of std::pair
-- **Reduced allocations** through zero-copy API techniques
+**Performance Improvements:**
 
-**Architecture Improvements:**
+- **Logger**: 35-72% faster log formatting through optimized string handling
+- **Web Server**: Significant reduction through lookup tables and IDF server unification
+- **Scheduler**: Reduced function call overhead
+- **OTA**: Non-blocking authentication improves responsiveness
+- **USB Host**: Dedicated FreeRTOS task for continuous event processing prevents data corruption during high-frequency transfers
 
-- **WebSocket-based dashboard** replaces polling for real-time updates
-- **Optimized logger** with improved timestamp formatting
-- **Scheduler enhancements** with reduced function call overhead
-- **Non-blocking OTA authentication** improves responsiveness
+These optimizations benefit all platforms, but ESP8266 users will notice the most dramatic stability improvements.
 
 ## New Components & Features
 
@@ -186,6 +268,20 @@ ESPHome 2025.10.0 continues aggressive optimization efforts:
 - **QMC5883L DRDY pin** for maximum sampling rate
 - **USB host/UART** support on ESP32-P4
 
+## For Contributors
+
+If you contribute to ESPHome or maintain external components, you'll notice some big improvements to the development experience:
+
+**CI Testing Improvements:**
+
+- **Much faster CI for large PRs** - Large PR test time reduced from ~1 hour to ~20 minutes through intelligent component grouping
+- **Release builds 6-7x faster** - Release test time reduced from 12+ hours to 1h40m
+- **Smarter component testing** - Changed components get isolated test runners with full validation, while dependency-only components are grouped for speed
+- **Better feedback loops** - Get results faster, making it easier to iterate on contributions
+- **More reliable testing** - APT package caching (2m30s → 5s), retry logic for downloads, and utilization of a larger available disk partition (66GB instead of 23GB)
+
+These improvements make it much easier to contribute to ESPHome by reducing wait times and making the CI system more efficient and reliable.
+
 ## Breaking Changes
 
 ESPHome 2025.10.0 includes several breaking changes requiring attention: