Merge branch 'develop' into copilot/handle-wifi-issues-after-ota-update

Author: doudar

.github/copilot-instructions.md

@@ -0,0 +1,130 @@
+# SmartSpin2k - ESP32 Smart Trainer Firmware
+
+SmartSpin2k is an ESP32-based DIY smart trainer project that converts any spin bike into a connected fitness device compatible with Zwift, TrainerRoad, and other training apps. The firmware controls stepper motor resistance, handles BLE communication, serves a web interface, and manages sensor data.
+
+Always reference these instructions first and fallback to search or bash commands only when you encounter unexpected information that does not match the info here.
+
+## Working Effectively
+
+### Bootstrap Environment
+Install required tools and dependencies:
+- `sudo apt-get update && sudo apt-get install -y build-essential git python3 python3-pip`
+- `pip install platformio pre-commit`
+- `pre-commit install --hook-type pre-push`
+
+### Build the Firmware
+- **CRITICAL**: Build takes 15-45 minutes depending on network connectivity. NEVER CANCEL. Set timeout to 60+ minutes.
+- `pio run --environment release` -- builds ESP32 firmware. NEVER CANCEL: Build takes 15-45 minutes on first run due to platform/toolchain downloads.
+- `pio run --target buildfs` -- builds filesystem. Takes 2-5 minutes.
+- **Network Issues**: If platform downloads fail with HTTPClientError, this is due to firewall/network restrictions. The build cannot proceed without internet access to download ESP32 toolchain.
+
+### Testing
+- **CRITICAL**: Native tests take 5-15 minutes. NEVER CANCEL. Set timeout to 30+ minutes.
+- `pio test --environment native` -- runs unit tests using Unity framework. NEVER CANCEL: Takes 5-15 minutes on first run.
+- Tests validate sensor data parsing, power calculations, BLE communication, and stepper motor control.
+- **Network Issues**: Native platform download may fail with HTTPClientError due to firewall restrictions.
+
+### Code Quality and Validation
+- `pre-commit run --all-files` -- runs license header insertion. Takes 1-2 minutes.
+- `pio check -e debug` -- runs cppcheck static analysis on debug environment. Takes 2-5 minutes. **Network Issues**: May fail with HTTPClientError due to platform download restrictions.
+- Python build scripts (always work):
+  - `python git_tag_macro.py` -- generates firmware version from git tags
+  - `python build_date_macro.py` -- generates build timestamp  
+  - `python cert_updater.py` -- updates SSL certificates (may fail with network issues)
+
+### Run the Application
+- **Build First**: Always complete the bootstrap and build steps before attempting to run.
+- The application runs on ESP32 hardware - cannot be executed in the sandbox environment.
+- Web interface available at device IP on port 80 when running on hardware.
+- BLE services broadcast as "SmartSpin2k" when running on hardware.
+
+## Validation
+
+### Manual Testing Scenarios
+After making code changes, always validate:
+1. **Build Validation**: Ensure `pio run --environment release` completes successfully.
+2. **Test Validation**: Ensure `pio test --environment native` passes all Unity tests.
+3. **Code Quality**: Run `pre-commit run --all-files` and fix any license header issues.
+4. **BLE Service Changes**: When modifying BLE services, verify characteristic UUIDs match the CustomCharacteristic.md specification.
+5. **Power Calculations**: When changing power table or ERG mode code, run tests in test_pt_lookup_*.cpp files.
+6. **Sensor Data**: When modifying sensor parsing, validate with tests in test_*Data.cpp files.
+
+### Critical Areas to Test
+- **Power Table**: Always validate power lookup and resistance calculations after changes to Power_Table.cpp or PowerTable_Helpers.cpp
+- **BLE Services**: Test characteristic read/write operations when modifying BLE_*_Service.cpp files
+- **ERG Mode**: Validate resistance control when changing ERG_Mode.cpp
+- **Stepper Control**: Test motor control when modifying stepper-related code in Main.cpp
+
+## Common Tasks
+
+### Repository Structure
+Key directories and their purpose:
+```
+/src                 -- Main ESP32 firmware source code
+/lib/SS2K/src        -- Core library with sensor parsing and data structures
+/include             -- Header files and configuration
+/test                -- Unity unit tests for native environment
+/data                -- Web interface HTML/CSS files
+/Hardware            -- 3D printing files and PCB designs
+/.github/workflows   -- CI/CD pipeline definitions
+```
+
+### Important Files
+- `platformio.ini` -- Build configuration for ESP32 and native environments
+- `include/settings.h` -- Hardware pin definitions and configuration constants
+- `CustomCharacteristic.md` -- BLE characteristic specification and usage
+- `src/Main.cpp` -- Main firmware entry point and setup
+- `lib/SS2K/src/sensors/` -- Sensor data parsing classes
+
+### Build Dependencies
+External libraries loaded automatically by PlatformIO:
+- NimBLE-ESP32 for Bluetooth Low Energy
+- TMCStepper for stepper motor control
+- FastAccelStepper for smooth motor movement
+- ArduinoJson for configuration and web API
+- ArduinoWebsockets for real-time web communication
+
+### Configuration
+- Default device name: "SmartSpin2k"
+- Default WiFi password: "password"
+- Web interface served on port 80
+- BLE service UUID: "77776277-7877-7774-4466-896665500000"
+- Over-the-air update URL: configured in settings.h
+
+### Hardware Compatibility
+- ESP32 DevKit v1 board (primary target)
+- TMC2209 stepper motor driver
+- Custom PCB designs in Hardware/ directory
+- Support for multiple bike mount configurations
+
+### Known Issues and Limitations
+- **Network Connectivity**: Platform and toolchain downloads may fail due to firewall restrictions. All build commands (`pio run`, `pio test`, `pio check`) require internet access on first run.
+- **SSL Certificates**: cert_updater.py may fail to fetch current certificates due to network restrictions
+- **Hardware Testing**: Cannot test actual motor control or BLE communication without physical hardware
+- **Build Times**: Initial builds require internet access and take 15-45 minutes due to large platform downloads
+
+### Troubleshooting Common Issues
+- **HTTPClientError during build**: This indicates network/firewall restrictions preventing platform downloads. No workaround available in restricted environments.
+- **Platform not found**: Run `pio platform install espressif32` to manually install the ESP32 platform (requires internet).
+- **Test failures**: Ensure you're running tests in native environment: `pio test -e native`
+- **SSL certificate warnings**: Update certificates with `python cert_updater.py` or manually update `include/cert.h`
+- **Build flag errors**: The Python scripts in build_flags must execute successfully. Test them individually if build fails.
+
+### Environment Verification
+Before working on the project, verify your environment:
+```bash
+# Check tools are installed
+which python3 pio pre-commit
+# Verify project configuration
+pio project config
+# Test build scripts
+python git_tag_macro.py && python build_date_macro.py
+```
+
+### Debugging Tips
+- Use `pio device monitor` to view serial output when connected to ESP32 hardware
+- Check `include/cert.h` if experiencing SSL errors during firmware updates
+- Monitor memory usage with DEBUG_STACK enabled in settings.h
+- BLE debugging available through web interface at `/develop.html`
+
+Always run `pre-commit run --all-files` before completing changes to ensure code meets project standards.

.github/workflows/copilot-setup-steps.yml

@@ -0,0 +1,92 @@
+name: "Copilot Setup Steps"
+
+on:
+  workflow_dispatch:
+  push:
+    paths:
+      - .github/workflows/copilot-setup-steps.yml
+  pull_request:
+    paths:
+      - .github/workflows/copilot-setup-steps.yml
+
+jobs:
+  # IMPORTANT: Job name must be exactly this.
+  copilot-setup-steps:
+    # You can upgrade to a larger Ubuntu runner label later if needed (e.g. ubuntu-4-core)
+    runs-on: ubuntu-latest
+    timeout-minutes: 40
+
+    # Minimize permissions; Copilot will get its own token later.
+    permissions:
+      contents: read
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v5
+        with:
+          # fetch-depth overridden internally for Copilot anyway; keep explicit for normal runs
+          fetch-depth: 0
+
+      - name: Set up Python
+        uses: actions/setup-python@v6
+        with:
+          python-version: '3.x'
+
+      - name: Cache PlatformIO core and packages
+        uses: actions/cache@v4
+        with:
+          path: |
+            ~/.platformio
+          key: pio-${{ runner.os }}-${{ hashFiles('platformio.ini', 'dependencies.lock') }}
+          restore-keys: |
+            pio-${{ runner.os }}-
+
+      - name: Install PlatformIO Core
+        run: |
+          python -m pip install --upgrade pip
+            pip install --upgrade platformio
+
+      - name: Show PlatformIO info
+        run: |
+          platformio --version
+          platformio system info
+
+      - name: Pre-download native test dependencies
+        run: |
+          platformio pkg install -e native || echo "Native env install attempted"
+
+      - name: Pre-download ESP32 toolchains and libraries (release env)
+        run: |
+          # Only install packages (faster + ensures offline availability later)
+          platformio pkg install -e release
+
+      - name: Warm build cache (lightweight compile) for release env
+        run: |
+          # A full build ensures toolchains, frameworks, and libs are all present
+          # If this becomes too slow, you can remove this step and rely on pkg install only.
+          platformio run -e release || exit 0
+
+      - name: Warm build cache for native env (tests)
+        run: |
+          platformio run -e native || exit 0
+
+      - name: Validate native tests (non-blocking)
+        run: |
+          set +e
+          platformio test -e native
+          # Do not fail setup if tests fail; Copilot will handle fixes.
+          exit 0
+
+      - name: Summarize cached packages
+        run: |
+          du -sh ~/.platformio/packages/* || true
+          du -sh ~/.platformio/platforms/* || true
+
+      - name: Guidance for future adjustments
+        shell: bash
+        run: |
+          echo "Setup steps completed. You can:"
+          echo " - Upgrade runner: change runs-on to ubuntu-4-core if builds are slow." \
+               "\n - Enable LFS: add 'with: lfs: true' to checkout step if using Git LFS." \
+               "\n - Trim steps: remove warm build steps if time exceeds limits." \
+               "\n - Add more pkg installs for other environments if created later."

.gitignore

@@ -22,5 +22,4 @@ C/Users/
 /managed_components
 dependencies.lock
 .vscode/settings.json
-dependencies.lock
-dependencies.lock
+*.map

CHANGELOG.md

@@ -32,6 +32,29 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Hardware
 
+
+## [25.9.17]
+
+### Added
+
+### Changed
+- Fixed incline mode handling negative numbers. 
+
+### Hardware
+
+
+## [25.9.8]
+
+### Added
+
+### Changed
+- Only check battery level on initial connection. This is to fix Tempo power meter drops. 
+- Moved battery information to the SpinBLEAdvertisedDevice class.
+- When adding to SpinBLEAdvertisedDevice, check adevname as well as address to prevent duplicates.
+- Stopped reusing BLE clients for better connection reliability.  
+
+### Hardware
+
 ## [25.8.26]
 
 ### Added

dependencies.lock

@@ -75,7 +75,7 @@ dependencies:
   idf:
     source:
       type: idf
-    version: 5.4.1
+    version: 5.4.2
   joltwallet/littlefs:
     component_hash: fe3d04a59a4c370408b0e0b69d9096c06371b9ee12ad8e06b9d52ac63ab1570c
     dependencies:
@@ -96,6 +96,6 @@ direct_dependencies:
 - espressif/network_provisioning
 - idf
 - joltwallet/littlefs
-manifest_hash: e871ea1e1aff9421b8e57e1c5a4db8daeef71b071a2a17f239e5feebc4acdfc3
+manifest_hash: e7ce7a886dfdb3cc5cd59acbdb4ff333c0a91c6e4de55b9fd6323d866c7cc691
 target: esp32
 version: 2.0.0

include/BLE_Common.h

@@ -29,7 +29,7 @@
 // maxInterval – [in] The maximum connection interval in 1.25ms units.
 // latency – [in] The number of packets allowed to skip (extends max interval).
 // timeout – [in] The timeout time in 10ms units before disconnecting.
-const uint16_t connectionParams[] = {24, 168, 1, 200};
+const uint16_t connectionParams[] = {24, 48, 0, 200};
 
 // Vector of supported BLE services and their corresponding characteristic UUIDs
 struct BLEServiceInfo {
@@ -42,8 +42,8 @@ namespace BLEServices {
 const std::vector<BLEServiceInfo> SUPPORTED_SERVICES = {{CYCLINGPOWERSERVICE_UUID, CYCLINGPOWERMEASUREMENT_UUID, "Cycling Power Service"},
                                                         {CSCSERVICE_UUID, CSCMEASUREMENT_UUID, "Cycling Speed And Cadence Service"},
                                                         {HEARTSERVICE_UUID, HEARTCHARACTERISTIC_UUID, "Heart Rate Service"},
-                                                        {ECHELON_DEVICE_UUID, ECHELON_SERVICE_UUID, "Echelon Device"}, // Two lines for Echelon
-                                                        {ECHELON_SERVICE_UUID, ECHELON_DATA_UUID, "Echelon Service"}, // Because one is for search, the other for data
+                                                        {ECHELON_DEVICE_UUID, ECHELON_SERVICE_UUID, "Echelon Device"},  // Two lines for Echelon
+                                                        {ECHELON_SERVICE_UUID, ECHELON_DATA_UUID, "Echelon Service"},   // Because one is for search, the other for data
                                                         {FITNESSMACHINESERVICE_UUID, FITNESSMACHINEINDOORBIKEDATA_UUID, "Fitness Machine Service"},
                                                         {HID_SERVICE_UUID, HID_REPORT_DATA_UUID, "HID Service"},
                                                         {FLYWHEEL_UART_SERVICE_UUID, FLYWHEEL_UART_TX_UUID, "Flywheel UART Service"}};
@@ -145,33 +145,29 @@ class SpinBLEAdvertisedDevice {
  private:
   QueueHandle_t dataBufferQueue = nullptr;
 
-  bool isPostConnected = false;
+  void clearState(bool resetAdvertisedDevice);  // NEW
 
- public:  // eventually these should be made private
-  // // TODO: Do we dispose of this object?  Is so, we need to de-allocate the queue.
-  // //       This destructor was called too early and the queue was deleted out from
-  // //       under us.
-  // ~SpinBLEAdvertisedDevice() {
-  //   if (dataBuffer != nullptr) {
-  //     Serial.println("Deleting queue");
-  //     vQueueDelete(dataBuffer);
-  //   }
-  // }
+ public:
+  SpinBLEAdvertisedDevice() { clearState(true); }  // NEW
 
   const NimBLEAdvertisedDevice* advertisedDevice = nullptr;
   NimBLEAddress peerAddress;
 
+  std::string uniqueName = "";  // Stable identifier using adevName2UniqueName()
+
   int connectedClientID = BLE_HS_CONN_HANDLE_NONE;
   BLEUUID serviceUUID   = (uint16_t)0x0000;
   BLEUUID charUUID      = (uint16_t)0x0000;
-  bool isHRM            = false;
-  bool isPM             = false;
-  bool isCSC            = false;
-  bool isCT             = false;
-  bool isRemote         = false;
-  bool doConnect        = false;
-  void setPostConnected(bool pc) { isPostConnected = pc; }
-  bool getPostConnected() { return isPostConnected; }
+  Measurement batt;
+  bool isHRM           = false;
+  bool isPM            = false;
+  bool isCSC           = false;
+  bool isCT            = false;
+  bool isRemote        = false;
+  bool doConnect       = false;
+  bool isPostConnected = false;
+  unsigned long lastDataUpdateTime = 0;  // Reset disconnect detection timestamp
+
   void set(const NimBLEAdvertisedDevice* device, int id = BLE_HS_CONN_HANDLE_NONE, BLEUUID inServiceUUID = (uint16_t)0x0000, BLEUUID inCharUUID = (uint16_t)0x0000);
   void reset(bool resetAdvertisedDevice = true);
   bool enqueueData(uint8_t* data, size_t length, NimBLEUUID serviceUUID, NimBLEUUID charUUID);
@@ -188,21 +184,18 @@ class SpinBLEClient {
   boolean connectedCT            = false;
   boolean connectedSpeed         = false;
   boolean connectedRemote        = false;
-  boolean doScan                 = false;
-  int intentionalDisconnect      = 0;
+  boolean doScan                 = true; //Set to true so there's an initial scan on startup
   long int cscCumulativeCrankRev = 0;
   double cscLastCrankEvtTime     = 0.0;
   long int cscCumulativeWheelRev = 0;
   double cscLastWheelEvtTime     = 0.0;
-  int reconnectTries             = MAX_RECONNECT_TRIES;
 
   BLERemoteCharacteristic* pRemoteCharacteristic = nullptr;
 
   // BLEDevices myBLEDevices;
   SpinBLEAdvertisedDevice myBLEDevices[NUM_BLE_DEVICES];
 
   void start();
-  // void serverScan(bool connectRequest);
   bool connectToServer();
   // Check for duplicate services of BLEClient and remove the previously
   // connected one.

include/BLE_Fitness_Machine_Service.h

@@ -17,8 +17,9 @@ class BLE_Fitness_Machine_Service {
   void update();
   bool spinDown(uint8_t response);
   void processFTMSWrite();
-
+  
  private:
+  int calculateResistanceFromPosition();
   BLEService *pFitnessMachineService;
   BLECharacteristic *fitnessMachineFeature;
   BLECharacteristic *fitnessMachineIndoorBikeData;

include/ERG_Mode.h

@@ -19,7 +19,6 @@ class ErgMode {
    // What used to be in the ERGTaskLoop(). This is the main control function for ERG Mode and the powertable operations.
   void runERG();
   void computeErg();
-  void computeResistance();
   void _writeLog(float currentIncline, float newIncline, int currentSetPoint, int newSetPoint, int currentWatts, int newWatts, int currentCadence, int newCadence);
 
  private:

include/Main.h

@@ -74,7 +74,7 @@ class SS2K {
   void FTMSModeShiftModifier();
   static void rxSerial(void);
   void txSerial();
-  void pelotonConnected();
+  bool pelotonConnected();
   void goHome(bool bothDirections = false);
 
   SS2K() {