Make changes to integrate with MbedTLSv3.5.1 and remove OTA cmake targets

Author: DakshitBabbar

.github/workflows/update-submodules.yml

@@ -8,7 +8,7 @@ on:
         required: true
         default: main
       allowed:
-        description: 'Optional regex pattern passed to `grep` to update only the specified library submodules, e.g. "ota\|jobs" updates only libraries with "ota" or "jobs" in the name.'
+        description: 'Optional regex pattern passed to `grep` to update only the specified library submodules, e.g. "jobs" updates only libraries with "jobs" in the name.'
         required: false
         default: .*
 

CHANGELOG.md

@@ -4,8 +4,10 @@
 
 ### Major Changes
 
-- [#1929](https://github.com/aws/aws-iot-device-sdk-embedded-C/pull/1929) This release includes 202406.01-LTS versions of coreMQTT, corePKCS11, coreHTTP, coreJSON, backoffAlgorithm, AWS IoT Device Shadow, AWS IoT Jobs, AWS IoT Device Defender, AWS IoT Fleet Provisioning and SigV4 libraries. These libraries have gone through code quality checks including verification that no function has a [GNU Complexity](https://www.gnu.org/software/complexity/manual/complexity.html) score greater than 8, checks against deviations from the mandatory rules in the [MISRA coding standard](https://www.misra.org.uk/), static code analysis from [Coverity static analysis](https://scan.coverity.com/) and validation of memory safety through the [CBMC automated reasoning tool](http://www.cs.cmu.edu/~modelcheck/cbmc/).
-- [#1929](https://github.com/aws/aws-iot-device-sdk-embedded-C/pull/1929)  Removes the OTA library and OTA demo . They are planned to be replaced by [aws-iot-core-mqtt-file-streams-embedded-c](https://github.com/aws/aws-iot-core-mqtt-file-streams-embedded-c) in future releases.
+- [#1929](https://github.com/aws/aws-iot-device-sdk-embedded-C/pull/1929) This release brings the following changes:
+  - Includes [202406.01-LTS](https://github.com/FreeRTOS/FreeRTOS-LTS/releases/tag/202406.01-LTS) versions of coreMQTT, corePKCS11, coreHTTP, coreJSON, backoffAlgorithm, AWS IoT Device Shadow, AWS IoT Jobs, AWS IoT Device Defender, AWS IoT Fleet Provisioning and SigV4 libraries.
+  - Removes the OTA library and OTA demo . They are planned to be replaced by [aws-iot-core-mqtt-file-streams-embedded-c](https://github.com/aws/aws-iot-core-mqtt-file-streams-embedded-c) in future releases following [this announcement](https://aws.amazon.com/about-aws/whats-new/2023/12/freertos-modular-composable-ota-libraries/). To learn more about FreeRTOS OTA libraries, visit the [FreeRTOS OTA page](https://freertos.org/freertos-core/over-the-air-updates/index.html). To get started, see the new OTA [reference demo](https://freertos.org/freertos-core/over-the-air-updates/mqtt-ota-agent-orchestrator.html). 
+  - Update readme instructions for demos for better user experience.
 
 ### Minor Changes
 
@@ -16,7 +18,6 @@
 - [#1875](https://github.com/aws/aws-iot-device-sdk-embedded-C/pull/1875) Add a Fleet provisioning demo to get certificate and private key via CreateKeysAndCertificate API.
 - [#1899](https://github.com/aws/aws-iot-device-sdk-embedded-C/pull/1899) Add Tunnelmole as an open source tunneling option in addition to ngrok.
 - [#1901](https://github.com/aws/aws-iot-device-sdk-embedded-C/pull/1901) Add a HTTP demo to generate a pre-signed URL to an S3 object file.
-- Update readme instructions for demos for better user experience.
 
 ## 202211.00 (November 2022)
 

MISRA.md

@@ -36,4 +36,4 @@ Deviations from the MISRA standard are listed below:
 | Rule 11.8 | Required | An OpenSSL API `SSL_set_tlsext_host_name`, which is used in the TLS transport implementation, internally casts a string literal to a `void *` pointer. |
 | Rule 13.4 | Required | A POSIX-specific macro utility `FD_SET` is flagged for this violation. This macro utility, whose implementation is supplied by the system, is used in the transport implementation. |
 | Rule 14.4 | Required | A POSIX-specific macro utility `FD_ZERO` is flagged for this violation. This macro utility, whose implementation is supplied by the system, is used in the transport implementation. |
-| Rule 21.6 | Required | The Standard Library input/output functions for opening and closing files are used by the OpenSSL transport implementation, since the OpenSSL API `PEM_read_X509` to read PEM files takes `FILE *` as an argument. The standard C library file handling functions are also used in POSIX platform implementation of OTA. |
+| Rule 21.6 | Required | The Standard Library input/output functions for opening and closing files are used by the OpenSSL transport implementation, since the OpenSSL API `PEM_read_X509` to read PEM files takes `FILE *` as an argument. |

README.md

@@ -488,13 +488,7 @@ The following creates a job that specifies a Linux Kernel link for downloading.
         --targets arn:aws:iot:us-west-2:<account-id>:thing/<thing-name> \
         --document '{"url":"https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.8.5.tar.xz"}'
 ```
-3. Execute the following command:
-```
-/build/bin/jobs_demo_mosquitto
-```
-This command will provide detailed instructions for running the demo.
-
-4. When prompted, run the demo using this command:
+3. Run the demo using this command:
 ```    
 ./build/bin/jobs_demo_mosquitto \
 -n <thing-name> \

demos/CMakeLists.txt

@@ -26,8 +26,6 @@ check_symbol_exists(fork "unistd.h" HAVE_FORK)
 if(${LIB_RT} STREQUAL "LIB_RT-NOTFOUND")
     set(librt_demos
             "http_demo_s3_download_multithreaded"
-            "ota_demo_core_http"
-            "ota_demo_core_mqtt"
     )
     message( WARNING "rt library could not be found. Demos that use it will be excluded from the default target." )
     foreach(demo_name ${librt_demos})
@@ -46,8 +44,6 @@ if(NOT ${OpenSSL_FOUND})
             "mqtt_demo_basic_tls"
             "mqtt_demo_mutual_auth"
             "mqtt_demo_subscription_manager"
-            "ota_demo_core_http"
-            "ota_demo_core_mqtt"
             "shadow_demo_main"
             "greengrass_demo_local_auth"
     )
@@ -56,16 +52,6 @@ if(NOT ${OpenSSL_FOUND})
         set_target_properties(${demo_name} PROPERTIES EXCLUDE_FROM_ALL true)
     endforeach()
 endif()
-if(NOT ${Threads_FOUND})
-    set(thread_demos
-            "ota_demo_core_http"
-            "ota_demo_core_mqtt"
-    )
-    message( WARNING "Threads library could not be found. Demos that use it will be excluded from the default target." )
-    foreach(demo_name ${thread_demos})
-        set_target_properties(${demo_name} PROPERTIES EXCLUDE_FROM_ALL true)
-    endforeach()
-endif()
 if(NOT HAVE_FORK)
     set(fork_demos
             "http_demo_s3_download_multithreaded"

demos/fleet_provisioning/fleet_provisioning_keys_cert/pkcs11_operations.c

@@ -44,11 +44,11 @@
 /* MbedTLS include. */
 #include "mbedtls/ctr_drbg.h"
 #include "mbedtls/entropy.h"
-#include "mbedtls/entropy_poll.h"
+#include "entropy_poll.h"
 #include "mbedtls/error.h"
 #include "mbedtls/oid.h"
 #include "mbedtls/pk.h"
-#include "mbedtls/pk_internal.h"
+#include "pk_wrap.h"
 #include "mbedtls/sha256.h"
 #include "mbedtls/x509_crt.h"
 #include "mbedtls/x509_csr.h"
@@ -537,10 +537,15 @@ static CK_RV provisionPrivateKey( CK_SESSION_HANDLE session,
     mbedtls_pk_type_t mbedKeyType = MBEDTLS_PK_NONE;
     int mbedResult = 0;
     mbedtls_pk_context mbedPkContext = { 0 };
+    mbedtls_ctr_drbg_context ctr_drbg;
+    mbedtls_entropy_context entropy;
 
     mbedtls_pk_init( &mbedPkContext );
+    mbedtls_entropy_init(&entropy);
+    mbedtls_ctr_drbg_init(&ctr_drbg);
+    mbedtls_ctr_drbg_seed(&ctr_drbg, mbedtls_entropy_func, &entropy, NULL, 0);
     mbedResult = mbedtls_pk_parse_key( &mbedPkContext, ( const uint8_t * ) privateKey,
-                                       privateKeyLength, NULL, 0 );
+                                       privateKeyLength, NULL, 0, mbedtls_ctr_drbg_random, &ctr_drbg );
 
     if( mbedResult != 0 )
     {

demos/fleet_provisioning/fleet_provisioning_keys_cert/pkcs11_operations.h

@@ -27,6 +27,11 @@
 #include <stdlib.h>
 #include <stdbool.h>
 
+/**
+ * @brief To access the private members of the MbedTLS structs
+ */
+ #define MBEDTLS_ALLOW_PRIVATE_ACCESS
+
 /* corePKCS11 include. */
 #include "core_pkcs11.h"
 

demos/fleet_provisioning/fleet_provisioning_with_csr/pkcs11_operations.c

@@ -44,11 +44,11 @@
 /* MbedTLS include. */
 #include "mbedtls/ctr_drbg.h"
 #include "mbedtls/entropy.h"
-#include "mbedtls/entropy_poll.h"
+#include "entropy_poll.h"
 #include "mbedtls/error.h"
 #include "mbedtls/oid.h"
 #include "mbedtls/pk.h"
-#include "mbedtls/pk_internal.h"
+#include "pk_wrap.h"
 #include "mbedtls/sha256.h"
 #include "mbedtls/x509_crt.h"
 #include "mbedtls/x509_csr.h"
@@ -243,11 +243,12 @@ static int extractEcPublicKey( CK_SESSION_HANDLE p11Session,
  * @param[in] pRng Unused.
  * @param[in] pRngContext Unused.
  */
-static int32_t privateKeySigningCallback( void * pContext,
+static int32_t privateKeySigningCallback( mbedtls_pk_context * pContext,
                                           mbedtls_md_type_t mdAlg,
                                           const unsigned char * pHash,
                                           size_t hashLen,
                                           unsigned char * pSig,
+                                          size_t sig_size,
                                           size_t * pSigLen,
                                           int ( * pRng )( void *, unsigned char *, size_t ),
                                           void * pRngContext );
@@ -636,10 +637,15 @@ static CK_RV provisionPrivateKey( CK_SESSION_HANDLE session,
     mbedtls_pk_type_t mbedKeyType = MBEDTLS_PK_NONE;
     int mbedResult = 0;
     mbedtls_pk_context mbedPkContext = { 0 };
+    mbedtls_ctr_drbg_context ctr_drbg;
+    mbedtls_entropy_context entropy;
 
     mbedtls_pk_init( &mbedPkContext );
+    mbedtls_entropy_init(&entropy);
+    mbedtls_ctr_drbg_init(&ctr_drbg);
+    mbedtls_ctr_drbg_seed(&ctr_drbg, mbedtls_entropy_func, &entropy, NULL, 0);
     mbedResult = mbedtls_pk_parse_key( &mbedPkContext, ( const uint8_t * ) privateKey,
-                                       privateKeyLength, NULL, 0 );
+                                       privateKeyLength, NULL, 0, mbedtls_ctr_drbg_random, &ctr_drbg );
 
     if( mbedResult != 0 )
     {
@@ -900,11 +906,12 @@ static int extractEcPublicKey( CK_SESSION_HANDLE p11Session,
 
 /*-----------------------------------------------------------*/
 
-static int32_t privateKeySigningCallback( void * pContext,
+static int32_t privateKeySigningCallback( mbedtls_pk_context * pContext,
                                           mbedtls_md_type_t mdAlg,
                                           const unsigned char * pHash,
                                           size_t hashLen,
                                           unsigned char * pSig,
+                                          size_t sig_size,
                                           size_t * pSigLen,
                                           int ( * pRng )( void *, unsigned char *, size_t ),
                                           void * pRngContext )

demos/fleet_provisioning/fleet_provisioning_with_csr/pkcs11_operations.h

@@ -27,6 +27,11 @@
 #include <stdlib.h>
 #include <stdbool.h>
 
+/**
+ * @brief To access the private members of the MbedTLS structs
+ */
+ #define MBEDTLS_ALLOW_PRIVATE_ACCESS
+
 /* corePKCS11 include. */
 #include "core_pkcs11.h"
 

demos/http/common/src/http_demo_s3_utils.c

@@ -36,6 +36,7 @@
 
 /* MBEDTLS API header. */
 #include "mbedtls/sha256.h"
+#include "mbedtls/compat-2.x.h"
 
 /* OpenSSL transport header. */
 #include "openssl_posix.h"