MQTTv5 Implementation by adituc · Pull Request #316 · FreeRTOS/coreMQTT

adituc · 2025-03-11T10:00:39Z

Upgrading MQTT Library to support v5 features

Description

This change adds the client implementation of MQTT version 5 CONNECT, CONNACK, Outgoing PUBLISH, PUBLISH ACKS, Incoming PUBLISH, SUBSCRIBE, SUBACK, UNSUBSCRIBE, UNSUBACK, DISCONNECT.
Existing data structures and functions are modified, and some new functions are added to serialize and deserialize the packets.
Plaintext Demo for reference ( https://github.com/adituc/FreeRTOS/blob/pooja-main-branch/FreeRTOS-Plus/Demo/coreMQTT_Windows_Simulator/MQTTV5_Plain_Text/DemoTasks/PlaintextMQTTExampleV5.c )

Test Steps

Unit Tests are added for all new and modified features.
Unit Tests for MQTTv5 functions are added in a separate folder in test.

Checklist:

I have tested my changes. No regression in existing tests.
I have modified and/or added unit-tests to cover the code changes in this Pull Request.

Related Issue

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

source/core_mqtt.c

source/core_mqtt_serializer.c

source/core_mqtt.c

source/include/core_mqtt.h

source/core_mqtt.c

source/core_mqtt_serializer.c

source/core_mqtt.c

source/include/core_mqtt_serializer.h

source/core_mqtt.c

source/core_mqtt_serializer.c

source/core_mqtt.c

source/include/core_mqtt.h

…o mqtt-v5

kstribrnAmzn · 2025-06-19T20:45:28Z

.github/workflows/ci.yml

        with:
          path: ./
-          horrid_threshold: 12
+          horrid_threshold: 17


Why did our complexity go up so much?

The complexity increased mainly due to functions like deserializeConnackProperties, which now has a complexity of 17. This function handles parsing of all the possible properties in a CONNACK packet (about 17 different properties). I considered splitting the logic into smaller functions, but since all of it is part of a single, cohesive task (parsing the CONNACK properties), breaking it up would have made the flow harder to follow. So we decided to increase the complexity threshold.

DakshitBabbar · 2025-06-15T18:28:23Z

source/core_mqtt.c

+        ( pSubscriptionList[ iterator ].topicFilterLength == 0U ) )
+    {
+        LogError( ( "Argument cannot be null : pTopicFilter or topicFilterLength cannot be zero. " ) );
+        status = MQTTBadParameter;


We should validate the QoS level with the amxQoS sent by the server in the Connack as well.

DakshitBabbar · 2025-06-15T18:34:54Z

source/core_mqtt.c

+ * @param[in] pContext MQTT Connection context.
+ * @param[in] pSubscriptionList List of MQTT subscription info.


The code is written in a way that this function returns true if the wildcard subscriptions are used but not supported and false if it is valid. Please update the return values documentation.

DakshitBabbar · 2025-06-15T18:40:38Z

source/core_mqtt.c

+                                                 const MQTTSubscribeInfo_t * pSubscriptionList,
+                                                 const size_t iterator )
+{
+    MQTTStatus_t status = MQTTSuccess;


Please create a macro for this constant.

DakshitBabbar · 2025-06-24T06:14:35Z

source/core_mqtt.c

+
+    if( bytesSentOrError != ( int32_t ) totalMessageLength )
+    {
+        status = MQTTSendFailed;


We can create a local variable for pSubscriptionList[ iterator ].pTopicFilter

DakshitBabbar · 2025-06-24T06:29:48Z

source/core_mqtt.c

+
+static MQTTStatus_t validateSharedSubscriptions( const MQTTContext_t * pContext,
+                                                 const MQTTSubscribeInfo_t * pSubscriptionList,
+                                                 const size_t iterator )
+{
+    MQTTStatus_t status = MQTTSuccess;
+    uint16_t topicFilterLength = pSubscriptionList[ iterator ].topicFilterLength;
+    bool isSharedSub = ( topicFilterLength > 7U );


It would be better to use the memchr function here instead of this loop.

DakshitBabbar · 2025-06-24T06:38:42Z

source/core_mqtt_serializer.c

+
+    return status;
+}
+


Please add a logerror statement here.

DakshitBabbar · 2025-06-24T07:25:09Z

source/core_mqtt_serializer.c

+
+
+    if( ( pPropertyBuilder != NULL ) && ( pPropertyBuilder->pBuffer != NULL ) )
+    {
+        propertyLength = pPropertyBuilder->currentIndex;


If the propbuilder is null or the buffer inside it is null then the return statement would be MQTTSuccess. But it should be MQTTBadParameter.

DakshitBabbar · 2025-06-24T09:07:27Z

source/core_mqtt_serializer.c

    {
-        /* MQTT DISCONNECT packets always have the same size. */
-        *pPacketSize = MQTT_DISCONNECT_PACKET_SIZE;
+        /*Reason code.*/


Please add a logerror statement here.

DakshitBabbar · 2025-06-24T09:44:20Z

source/core_mqtt_serializer.c

+        status = MQTTBadParameter;
+    }
+    else if( pDisconnectInfo == NULL )
+    {
+        status = MQTTBadParameter;
+    }
+    else if( maxPacketSize == 0U )
+    {
+        status = MQTTBadParameter;
+    }
+    /*Packet size should not be more than the max allowed by the client.*/
+    else if( ( pPacket->remainingLength + variableLengthEncodedSize( pPacket->remainingLength ) + 1U ) > maxPacketSize )
+    {
+        status = MQTTBadResponse;


We should add the corresponding logerror statements at places where we assign status to MQTTBadParameter/MQTTBadResponse

DakshitBabbar · 2025-06-24T11:04:52Z

source/core_mqtt_serializer.c

+            if( pPropBuffer != NULL )
+            {
+                pPropBuffer->bufferLength = propertyLength;
+                pPropBuffer->pBuffer = pIndex;
+            }
+
+            /*Validate the remaining length.*/
+            if( pPacket->remainingLength != ( propertyLength + variableLengthEncodedSize( propertyLength ) + 1U ) )
+            {
+                status = MQTTBadResponse;
+            }


Shouldn't this block be within the if( status == MQTTSuccess ) block?

tony-josi-aws · 2025-08-05T11:54:27Z

test/cbmc/proofs/DecodeAckProperties/DecodeAckProperties_harness.c

+    __CPROVER_assume( isValidMqttPropBuilder( propBuffer ) );
+
+    __CPROVER_assume( propertyLength >= 0 );
+    __CPROVER_assume( propertyLength <= MAX_PROPERTY_LENGTH );


This assumption seems incorrect; I don't think we have checks to make sure propertyLength is less than MAX_PROPERTY_LENGTH.

This assumption on the maximum length of the properties buffer was made because without this the CBMC proofs were taking too much time. Without this constraint the maximum size of the properties buffer can be ~256 MB ($2^{28}$B).

tony-josi-aws · 2025-08-05T12:03:07Z

test/cbmc/proofs/DeserializeConnackProperties/DeserializeConnackProperties_harness.c

+    __CPROVER_assume( isValidMqttPropBuilder( propBuffer ) );
+
+    __CPROVER_assume( propertyLength >= 0 );
+    __CPROVER_assume( propertyLength <= MAX_PROPERTY_LENGTH );


This assumption seems incorrect; I don't think we have checks to make sure propertyLength is less than MAX_PROPERTY_LENGTH.

tony-josi-aws · 2025-08-05T12:04:58Z

test/cbmc/proofs/DeserializeDisconnectProperties/DeserializeDisconnectProperties_harness.c

+    size_t propertyLength;
+
+    __CPROVER_assume( propertyLength >= 0 );
+    __CPROVER_assume( propertyLength <= MAX_PROPERTY_LENGTH );


This assumption seems incorrect; I don't think we have checks to make sure propertyLength is less than MAX_PROPERTY_LENGTH.

DakshitBabbar changed the base branch from main to mqttv5-dev March 18, 2025 04:45

DakshitBabbar marked this pull request as ready for review March 18, 2025 04:46

DakshitBabbar marked this pull request as draft March 18, 2025 06:06

adituc added 2 commits March 24, 2025 12:18

Modifiying disconnect API

681d7b3

disconnect api changes

d843640

DakshitBabbar reviewed Mar 24, 2025

View reviewed changes

source/core_mqtt.c Outdated Show resolved Hide resolved

source/core_mqtt.c Outdated Show resolved Hide resolved

source/core_mqtt.c Outdated Show resolved Hide resolved

adituc added 4 commits March 25, 2025 12:13

adding comments

7f4e41b

removing v3 flags

c110e31

removing v3 code

6897ce4

adding comments

4781f96

DakshitBabbar reviewed Mar 25, 2025

View reviewed changes

adituc and others added 4 commits March 26, 2025 11:00

removing rc

9a3c4df

Add optional will props builder in Connect

682caac

Remove v3 code

31472df

Add minor updates to code

af9a933

DakshitBabbar reviewed Mar 26, 2025

View reviewed changes

source/core_mqtt.c Outdated Show resolved Hide resolved

source/include/core_mqtt.h Outdated Show resolved Hide resolved

source/core_mqtt.c Outdated Show resolved Hide resolved

source/core_mqtt_serializer.c Show resolved Hide resolved

adituc added 4 commits March 27, 2025 06:48

Move MQTTV5_GetAckPacketSize to serializer.c

c1f5b84

Delete v3 code, modify sendConnectWithoutCopy

8c1833e

Modifying sendPublishWithoutCopy

1fc680a

Modify Disconnect to allow for NULL props

0c5ca3f

DakshitBabbar reviewed Mar 27, 2025

View reviewed changes

adituc added 4 commits March 27, 2025 21:06

Change sendSubscribeWithoutCopy

da1173b

Change MQTTV5_GetAckPacketSize

cd938c0

Change MQTTV5_GetAckPacketSize

4fa21fa

Change reason code

d2480cf

DakshitBabbar reviewed Mar 31, 2025

View reviewed changes

DakshitBabbar reviewed Apr 1, 2025

View reviewed changes

source/core_mqtt.c Outdated Show resolved Hide resolved

Shift API calls to serializer.c

76577b1

DakshitBabbar reviewed Apr 1, 2025

View reviewed changes

source/core_mqtt.c Outdated Show resolved Hide resolved

source/core_mqtt.c Outdated Show resolved Hide resolved

source/core_mqtt.c Outdated Show resolved Hide resolved

source/include/core_mqtt.h Outdated Show resolved Hide resolved

Add minor updates to code

af99d3a

adituc and others added 12 commits June 16, 2025 08:40

Resolve PR Comments

0edda48

Minor change to decodeSubackPropertyLengt()

71d23c1

Update eventcallback stub signature for cbmc proofs

c1bb2e6

Fix Documentation for functions

2b90b3c

Merge branch 'mqtt-v5' of https://github.com/blipbloping/coreMQTT int…

98ebcf0

…o mqtt-v5

Minor updates to fix CBMC proofs.

db78089

Update CBMC proof readme and unwanted code

c8f743e

Fix code to pass CI Checks

6e68afe

Fix formatting

933dbe7

Change parameter names of pointers to start with p*

3796c32

Fix formatting and memory checks

1c5a1da

Fix coverity errors

dd899e1

kstribrnAmzn reviewed Jun 19, 2025

View reviewed changes

adituc added 2 commits June 20, 2025 06:30

Fix unit test warning

52e4720

Fix formatting check and doxygen check

bcf69a8

DakshitBabbar reviewed Jun 24, 2025

View reviewed changes

DakshitBabbar and others added 8 commits June 25, 2025 16:03

Update migration guide for formating fixes

fc06bff

Fix Migration Guide formatting

88fe883

Update doxygen documentation

79e4cb3

Add table of contents to migration guide

26f88c4

Update the doxyfile to remove warnings

2f53ce3

Fix formating and spell checks

595ba80

Clean up cbmc directories

f00e888

Add documentation changes

95d1c4b

tony-josi-aws reviewed Aug 5, 2025

View reviewed changes

AniruddhaKanhere changed the base branch from main to MQTTv5-preview August 20, 2025 20:43

AniruddhaKanhere merged commit a4fad60 into FreeRTOS:MQTTv5-preview Aug 20, 2025
10 of 11 checks passed

		* @param[in] pContext MQTT Connection context.
		* @param[in] pSubscriptionList List of MQTT subscription info.

Conversation

adituc commented Mar 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Test Steps

Checklist:

Related Issue

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

adituc Jun 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

adituc commented Mar 11, 2025 •

edited

Loading

adituc Jun 24, 2025 •

edited

Loading