Merge branch 'add-esp32-s3-support' into add-freenove-board-support

Author: argetlam-coder

Changelog.md

@@ -1,3 +1,22 @@
+## [16.0.0-RC6] - 2024-xx-xx
+
+For a full list of changes see [Full list of changes](https://github.com/jomjol/AI-on-the-edge-device/compare/v15.7.0...v16.0.0)
+
+#### Known issues
+Please check the [issues](https://github.com/jomjol/AI-on-the-edge-device/issues) and
+[discussions](https://github.com/jomjol/AI-on-the-edge-device/discussions) before reporting a new issue.
+
+#### Core Changes
+Only changes since RC5 are listed:
+- [#3436](https://github.com/jomjol/AI-on-the-edge-device/pull/3436) Added basic authentification of the Web Interface and the REST API, see https://jomjol.github.io/AI-on-the-edge-device-docs/Password-Protection
+- xxx
+
+  **:warning: Please check your Homeassistant instance to make sure it is handled correctly!**
+
+#### Bug Fixes
+Only changes since RC5 are listed:
+ - xxx
+
 ## [16.0.0-RC5] - 2024-12-05
 
 For a full list of changes see [Full list of changes](https://github.com/jomjol/AI-on-the-edge-device/compare/v15.7.0...v16.0.0)
@@ -8,25 +27,25 @@ Please check the [issues](https://github.com/jomjol/AI-on-the-edge-device/issues
 
 #### Core Changes
 Only changes since RC4 are listed:
-- Removed `Autostart` parameter and make the flow to be always enabled (#3423)
-- Enable `Flow start` menu entry in UI (#3423)
-- Updated the Homeassistant Discovery topics (#3332):
+- [#3423](https://github.com/jomjol/AI-on-the-edge-device/pull/3423) Removed `Autostart` parameter and make the flow to be always enabled 
+- [#3423](https://github.com/jomjol/AI-on-the-edge-device/pull/3423) Enable `Flow start` menu entry in UI 
+- [#3332](https://github.com/jomjol/AI-on-the-edge-device/pull/3332) Updated the Homeassistant Discovery topics :
     - `raw` has now set the `State Class` to `measurement`. Before it was always set to `""`. 
     - `value` has now only set the `State Class` to `total_increasing` if the parameter `Allow Negative Rates` is **not** set. Else it uses `measurement` since the rate could also be negative. Before it was always set to `total_increasing`.
     - The `rate_per_time_unit` topic of an **Energy** meter needs a `Device Class`=`power`. For `gas` and `water` it should be `volume_flow_rate`. Before it was always set to `""`.
-    - Added button for `flow start` (#3415)
-    - Added support for Domoticz MQTT integration (#3359)
+    - [#3415](https://github.com/jomjol/AI-on-the-edge-device/pull/3415) Added button for `flow start` 
+    - [#3359](https://github.com/jomjol/AI-on-the-edge-device/pull/3359) Added support for Domoticz MQTT integration 
     - Added Date and time to overview page
     - Updated submodules and models
 
   **:warning: Please check your Homeassistant instance to make sure it is handled correctly!**
 
 #### Bug Fixes
-Only changes since RC3 are listed:
- - Added fix for ledintensity (#3418)
- - Added fix for OV2640 brightness contrast saturation (#3417)
- - Added fix for 'AnalogToDigitTransitionStart' always using 9.2 regardless of the configured value (#3393)
- - Addef fix for HA menu entry (#3342)
+Only changes since RC4 are listed:
+ - [#3418](https://github.com/jomjol/AI-on-the-edge-device/pull/3418) Added fix for ledintensity 
+ - [#3417](https://github.com/jomjol/AI-on-the-edge-device/pull/3417) Added fix for OV2640 brightness contrast saturation 
+ - [#3393](https://github.com/jomjol/AI-on-the-edge-device/pull/3393) Added fix for 'AnalogToDigitTransitionStart' always using 9.2 regardless of the configured value 
+ - [#3342](https://github.com/jomjol/AI-on-the-edge-device/pull/3342) Added fix for HA menu entry 
 
 
 ## [16.0.0-RC4] - 2024-10-06
@@ -39,13 +58,13 @@ Please check the [issues](https://github.com/jomjol/AI-on-the-edge-device/issues
 
 #### Core Changes
 Only changes since RC3 are listed:
-- Update esp32-camera submodule to `v2.0.13` (#3316)
-- Added contributor list (#3317)
-- Added files for demo mode (#3315)
+- [#3316](https://github.com/jomjol/AI-on-the-edge-device/pull/3316) Update esp32-camera submodule to `v2.0.13` 
+- [#3317](https://github.com/jomjol/AI-on-the-edge-device/pull/3317) Added contributor list 
+- [#3315](https://github.com/jomjol/AI-on-the-edge-device/pull/3315) Added files for demo mode 
 
 #### Bug Fixes
 Only changes since RC2 are listed:
-- Added delay in InitCam (#3313) to fix `Camera not detected` issues
+- [#3313](https://github.com/jomjol/AI-on-the-edge-device/pull/3313) Added delay in InitCam  to fix `Camera not detected` issues
 
 
 ## [16.0.0-RC3] - 2024-10-05
@@ -62,7 +81,7 @@ Only changes since RC2 are listed:
 
 #### Bug Fixes
 Only changes since RC2 are listed:
-- Re-did revertion of TFlite submodule update as certain modules crash with it (#3269) (change was lost)
+- [#3269](https://github.com/jomjol/AI-on-the-edge-device/pull/3269) Re-did revertion of TFlite submodule update as certain modules crash with it  (change was lost)
 
 
 ## [16.0.0-RC2] - 2024-10-04
@@ -76,13 +95,13 @@ Please check the [issues](https://github.com/jomjol/AI-on-the-edge-device/issues
 #### Core Changes
 Only changes since RC1 are listed:
 - Updated parameter documentation pages
-- Rename/remove unused parameters (#3291)
-- Migrate-cam-parameters (#3288)
+- [#3291](https://github.com/jomjol/AI-on-the-edge-device/pull/3291) Rename/remove unused parameters 
+- [#3288](https://github.com/jomjol/AI-on-the-edge-device/pull/3288) Migrate-cam-parameters 
 
 #### Bug Fixes
 Only changes since RC1 are listed:
-- Reverted TFlite submodule update as certain modules crash with it (#3269)
-- Changed the webhook UploadImg to false (#3279)
+- [#3269](https://github.com/jomjol/AI-on-the-edge-device/pull/3269) Reverted TFlite submodule update as certain modules crash with it 
+- [#3279](https://github.com/jomjol/AI-on-the-edge-device/pull/3279) Changed the webhook UploadImg to false 
 - Changed default value from boolean to numeric value in parameter camDenoise documentation
 - Updated config page
 
@@ -96,24 +115,24 @@ Please check the [issues](https://github.com/jomjol/AI-on-the-edge-device/issues
 
 #### Core Changes
 Those are just the major changes:
-- Add support for OV5640 camera (#3063)
+- [#3063](https://github.com/jomjol/AI-on-the-edge-device/pull/3063) Add support for OV5640 camera 
 - New tflite-Models
-- Homeassistant service discovery: derive node_id when using nested topics (#3088)
-- Added Prometheus/OpenMetrics exporter (#3081)
-- Added Webhook (#3148, #3163, #3174)
-- Add rate threshold parameter (#3195)
-- Added a Delay between the WiFi reconnections (#3068)
+- [#3088](https://github.com/jomjol/AI-on-the-edge-device/pull/3088) Homeassistant service discovery: derive node_id when using nested topics 
+- [#3081](https://github.com/jomjol/AI-on-the-edge-device/pull/3081) Added Prometheus/OpenMetrics exporter 
+- [#3148](https://github.com/jomjol/AI-on-the-edge-device/pull/3148), [#3163](https://github.com/jomjol/AI-on-the-edge-device/pull/3163), [#3174](https://github.com/jomjol/AI-on-the-edge-device/pull/3148), [#3163](https://github.com/jomjol/AI-on-the-edge-device/pull/3163), [#3174](https://github.com/jomjol/AI-on-the-edge-device/pull/3174) Added Webhook 
+- [#3195](https://github.com/jomjol/AI-on-the-edge-device/pull/3195) Add rate threshold parameter 
+- [#3068](https://github.com/jomjol/AI-on-the-edge-device/pull/3068) Added a Delay between the WiFi reconnections 
 - Web UI improvements
 - Various minor changes
 - Update platformIO to 6.9.0 (Contains ESP IDF 5.3.1)
 
 #### Bug Fixes
 Those are just the major changes:
-- Handle crash on corrupted model (#3220)
-- Bugfix for boot loop (#3175)
-- Bugfix for time stamp (#3180)
-- Handle empty prevalue.ini gracefully (#3162)
-- Added note about only TLS 1.2 is supported (#3213)
+- [#3220](https://github.com/jomjol/AI-on-the-edge-device/pull/3220) Handle crash on corrupted model 
+- [#3175](https://github.com/jomjol/AI-on-the-edge-device/pull/3175) Bugfix for boot loop 
+- [#3180](https://github.com/jomjol/AI-on-the-edge-device/pull/3180) Bugfix for time stamp 
+- [#3162](https://github.com/jomjol/AI-on-the-edge-device/pull/3162) Handle empty prevalue.ini gracefully 
+- [#3213](https://github.com/jomjol/AI-on-the-edge-device/pull/3213) Added note about only TLS 1.2 is supported 
 
 ## [15.7.0] - 2024-02-17
 

code/components/jomjol_controlGPIO/server_GPIO.cpp

@@ -25,6 +25,7 @@
 #include "server_mqtt.h"
 #endif //ENABLE_MQTT
 
+#include "basic_auth.h"
 
 static const char *TAG = "GPIO";
 QueueHandle_t gpio_queue_handle = NULL;
@@ -458,7 +459,7 @@ void GpioHandler::registerGpioUri()
     httpd_uri_t camuri = { };
     camuri.method    = HTTP_GET;
     camuri.uri       = "/GPIO";
-    camuri.handler   = callHandleHttpRequest;
+    camuri.handler   = APPLY_BASIC_AUTH_FILTER(callHandleHttpRequest);
     camuri.user_ctx  = (void*)this;    
     httpd_register_uri_handler(_httpServer, &camuri);
 }

code/components/jomjol_controlcamera/server_camera.cpp

@@ -10,6 +10,8 @@
 #include "ClassLogFile.h"
 #include "esp_log.h"
 
+#include "basic_auth.h"
+
 #include "../../include/defines.h"
 
 static const char *TAG = "server_cam";
@@ -280,27 +282,27 @@ void register_server_camera_uri(httpd_handle_t server)
     camuri.method = HTTP_GET;
 
     camuri.uri = "/lighton";
-    camuri.handler = handler_lightOn;
+    camuri.handler = APPLY_BASIC_AUTH_FILTER(handler_lightOn);
     camuri.user_ctx = (void *)"Light On";
     httpd_register_uri_handler(server, &camuri);
 
     camuri.uri = "/lightoff";
-    camuri.handler = handler_lightOff;
+    camuri.handler = APPLY_BASIC_AUTH_FILTER(handler_lightOff);
     camuri.user_ctx = (void *)"Light Off";
     httpd_register_uri_handler(server, &camuri);
 
     camuri.uri = "/capture";
-    camuri.handler = handler_capture;
+    camuri.handler = APPLY_BASIC_AUTH_FILTER(handler_capture);
     camuri.user_ctx = NULL;
     httpd_register_uri_handler(server, &camuri);
 
     camuri.uri = "/capture_with_flashlight";
-    camuri.handler = handler_capture_with_light;
+    camuri.handler = APPLY_BASIC_AUTH_FILTER(handler_capture_with_light);
     camuri.user_ctx = NULL;
     httpd_register_uri_handler(server, &camuri);
 
     camuri.uri = "/save";
-    camuri.handler = handler_capture_save_to_file;
+    camuri.handler = APPLY_BASIC_AUTH_FILTER(handler_capture_save_to_file);
     camuri.user_ctx = NULL;
     httpd_register_uri_handler(server, &camuri);
 }

code/components/jomjol_fileserver_ota/server_file.cpp

@@ -46,6 +46,7 @@ extern "C" {
 
 #include "Helper.h"
 #include "miniz.h"
+#include "basic_auth.h"
 
 static const char *TAG = "OTA FILE";
 
@@ -1174,7 +1175,7 @@ void register_server_file_uri(httpd_handle_t server, const char *base_path)
     httpd_uri_t file_download = {
         .uri       = "/fileserver*",  // Match all URIs of type /path/to/file
         .method    = HTTP_GET,
-        .handler   = download_get_handler,
+        .handler = APPLY_BASIC_AUTH_FILTER(download_get_handler),
         .user_ctx  = server_data    // Pass server data as context
     };
     httpd_register_uri_handler(server, &file_download);
@@ -1183,7 +1184,7 @@ void register_server_file_uri(httpd_handle_t server, const char *base_path)
     httpd_uri_t file_datafileact = {
         .uri       = "/datafileact",  // Match all URIs of type /path/to/file
         .method    = HTTP_GET,
-        .handler   = datafileact_get_full_handler,
+        .handler = APPLY_BASIC_AUTH_FILTER(datafileact_get_full_handler),
         .user_ctx  = server_data    // Pass server data as context
     };
     httpd_register_uri_handler(server, &file_datafileact);
@@ -1192,15 +1193,15 @@ void register_server_file_uri(httpd_handle_t server, const char *base_path)
     httpd_uri_t file_datafile_last_part_handle = {
         .uri       = "/data",  // Match all URIs of type /path/to/file
         .method    = HTTP_GET,
-        .handler   = datafileact_get_last_part_handler,
+        .handler = APPLY_BASIC_AUTH_FILTER(datafileact_get_last_part_handler),
         .user_ctx  = server_data    // Pass server data as context
     };
     httpd_register_uri_handler(server, &file_datafile_last_part_handle);
 
     httpd_uri_t file_logfileact = {
         .uri       = "/logfileact",  // Match all URIs of type /path/to/file
         .method    = HTTP_GET,
-        .handler   = logfileact_get_full_handler,
+        .handler = APPLY_BASIC_AUTH_FILTER(logfileact_get_full_handler),
         .user_ctx  = server_data    // Pass server data as context
     };
     httpd_register_uri_handler(server, &file_logfileact);
@@ -1209,7 +1210,7 @@ void register_server_file_uri(httpd_handle_t server, const char *base_path)
     httpd_uri_t file_logfile_last_part_handle = {
         .uri       = "/log",  // Match all URIs of type /path/to/file
         .method    = HTTP_GET,
-        .handler   = logfileact_get_last_part_handler,
+        .handler = APPLY_BASIC_AUTH_FILTER(logfileact_get_last_part_handler),
         .user_ctx  = server_data    // Pass server data as context
     };
     httpd_register_uri_handler(server, &file_logfile_last_part_handle);
@@ -1219,7 +1220,7 @@ void register_server_file_uri(httpd_handle_t server, const char *base_path)
     httpd_uri_t file_upload = {
         .uri       = "/upload/*",   // Match all URIs of type /upload/path/to/file
         .method    = HTTP_POST,
-        .handler   = upload_post_handler,
+        .handler = APPLY_BASIC_AUTH_FILTER(upload_post_handler),
         .user_ctx  = server_data    // Pass server data as context
     };
     httpd_register_uri_handler(server, &file_upload);
@@ -1228,7 +1229,7 @@ void register_server_file_uri(httpd_handle_t server, const char *base_path)
     httpd_uri_t file_delete = {
         .uri       = "/delete/*",   // Match all URIs of type /delete/path/to/file
         .method    = HTTP_POST,
-        .handler   = delete_post_handler,
+        .handler = APPLY_BASIC_AUTH_FILTER(delete_post_handler),
         .user_ctx  = server_data    // Pass server data as context
     };
     httpd_register_uri_handler(server, &file_delete);

code/components/jomjol_fileserver_ota/server_ota.cpp

@@ -42,6 +42,7 @@
 
 #include "Helper.h"
 #include "statusled.h"
+#include "basic_auth.h"
 #include "../../include/defines.h"
 
 /*an ota data write buffer ready to write to the flash*/
@@ -690,13 +691,13 @@ void register_server_ota_sdcard_uri(httpd_handle_t server)
     httpd_uri_t camuri = { };
     camuri.method    = HTTP_GET;
     camuri.uri       = "/ota";
-    camuri.handler   = handler_ota_update;
+    camuri.handler = APPLY_BASIC_AUTH_FILTER(handler_ota_update);
     camuri.user_ctx  = (void*) "Do OTA";    
     httpd_register_uri_handler(server, &camuri);
 
     camuri.method    = HTTP_GET;
     camuri.uri       = "/reboot";
-    camuri.handler   = handler_reboot;
+    camuri.handler = APPLY_BASIC_AUTH_FILTER(handler_reboot);
     camuri.user_ctx  = (void*) "Reboot";    
     httpd_register_uri_handler(server, &camuri);
 

code/components/jomjol_flowcontroll/ClassFlowCNNGeneral.cpp

@@ -65,11 +65,11 @@ string ClassFlowCNNGeneral::getReadout(int _analog = 0, bool _extendedResolution
 
     if (CNNType == Digit) {
         for (int i = 0; i < GENERAL[_analog]->ROI.size(); ++i) {
-            if (GENERAL[_analog]->ROI[i]->result_klasse >= 10) {
-                result = result + "N";
+            if ((GENERAL[_analog]->ROI[i]->result_klasse >= 0) && (GENERAL[_analog]->ROI[i]->result_klasse < 10)) {
+                result = result + std::to_string(GENERAL[_analog]->ROI[i]->result_klasse);
             }
             else {
-                result = result + std::to_string(GENERAL[_analog]->ROI[i]->result_klasse);
+                result = result + "N";
             }
         }
         return result;
@@ -78,7 +78,7 @@ string ClassFlowCNNGeneral::getReadout(int _analog = 0, bool _extendedResolution
     if ((CNNType == DoubleHyprid10) || (CNNType == Digit100)) {
         float number = GENERAL[_analog]->ROI[GENERAL[_analog]->ROI.size() - 1]->result_float;
         // NaN?
-        if (number >= 0) {
+        if ((number >= 0) && (number < 10)) {
             // is only set if it is the first digit (no analogue before!)
             if (_extendedResolution) {
                 int result_after_decimal_point = ((int) floor(number * 10)) % 10;
@@ -95,8 +95,15 @@ string ClassFlowCNNGeneral::getReadout(int _analog = 0, bool _extendedResolution
                 else {
                     prev = PointerEvalHybridNew(GENERAL[_analog]->ROI[GENERAL[_analog]->ROI.size() - 1]->result_float, prev, prev);
                 }
-                result = std::to_string(prev);
-                LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "getReadout(dig100)  prev=" + std::to_string(prev));
+
+                // is necessary because a number greater than 9.994999 returns a 10! (for further details see check in PointerEvalHybridNew)
+                if ((prev >= 0) && (prev < 10)) {
+                    result = std::to_string(prev);
+                    LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "getReadout(dig100)  prev=" + std::to_string(prev));
+                }
+                else {
+                    result = "N";
+                }
             }
         }
         else {
@@ -107,7 +114,7 @@ string ClassFlowCNNGeneral::getReadout(int _analog = 0, bool _extendedResolution
         }
 
         for (int i = GENERAL[_analog]->ROI.size() - 2; i >= 0; --i) {
-            if (GENERAL[_analog]->ROI[i]->result_float >= 0) {
+            if ((GENERAL[_analog]->ROI[i]->result_float >= 0) && (GENERAL[_analog]->ROI[i]->result_float < 10)) {
                 prev = PointerEvalHybridNew(GENERAL[_analog]->ROI[i]->result_float, GENERAL[_analog]->ROI[i+1]->result_float, prev);
                 LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "getReadout#PointerEvalHybridNew()= " + std::to_string(prev));
                 result = std::to_string(prev) + result;
@@ -117,7 +124,6 @@ string ClassFlowCNNGeneral::getReadout(int _analog = 0, bool _extendedResolution
                 prev = -1;
                 result = "N" + result;
                 LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "getReadout(result_float<0 /'N')  result_float=" + std::to_string(GENERAL[_analog]->ROI[i]->result_float));
-        
             }
         }
         return result;
@@ -150,6 +156,9 @@ int ClassFlowCNNGeneral::PointerEvalHybridNew(float number, float number_of_pred
         // on first digit is no spezial logic for transition needed
         // we use the recognition as given. The result is the int value of the recognition
         // add precisition of 2 digits and round before trunc
+        // a number greater than 9.994999 is returned as 10, this leads to an error during the decimal shift because the NUMBERS[j]->ReturnRawValue is one digit longer.
+        // To avoid this, an additional test must be carried out, see "if ((CNNType == DoubleHyprid10) || (CNNType == Digit100))" check in getReadout()
+        // Another alternative would be "result = (int) ((int) trunc(round((number+10 % 10)*1000))) / 1000;", which could, however, lead to other errors?
         result = (int) ((int) trunc(round((number+10 % 10)*100)) )  / 100;
 
         LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "PointerEvalHybridNew - No predecessor - Result = " + std::to_string(result) +

code/components/jomjol_flowcontroll/ClassFlowControll.cpp

@@ -26,6 +26,7 @@ extern "C" {
 
 #include "server_help.h"
 #include "MainFlowControl.h"
+#include "basic_auth.h"
 #include "../../include/defines.h"
 
 static const char* TAG = "FLOWCTRL";