Merge branch 'master' into fix/attribute-loss-when-updating-a-device

Author: AlvaroVega

.github/workflows/ci.yml

@@ -45,13 +45,12 @@ jobs:
     runs-on: ubuntu-latest
     services:
       mongodb:
-        image: mongo:6.0
+        image: mongo:8.0
         ports:
         - 27017:27017
     strategy:
       matrix:
         node-version:
-          - 14.x
           - 16.x
           - 18.x
     steps:
@@ -72,7 +71,7 @@ jobs:
     needs: unit-test
     services:
       mongodb:
-        image: mongo:6.0
+        image: mongo:8.0
         ports:
         - 27017:27017
     steps:

Changelog

@@ -77,7 +77,9 @@ var config = {
     providerUrl: 'http://192.168.56.1:4041',
     deviceRegistrationDuration: 'P1M',
     defaultType: 'Thing',
-    expressLimit: '1Mb'
+    expressLimit: '1Mb',
+    useCBflowControl: false,
+    storeLastMeasure: false
 };
 
 module.exports = config;

config.js

@@ -7,6 +7,7 @@
 -   [Administration manual](admin.md)
 -   [Deprecated features](deprecated.md)
 -   [Roadmap](roadmap.md)
+-   [Data models](models/models.md)
 
 ## Development documentation
 

doc/README.md

@@ -125,9 +125,9 @@ allowing the computer to interpret the rest of the data with more clarity and de
 ```
 
 Under mixed mode, **NGSI v2** payloads are used for context broker communications by default, but this payload may also
-be switched to **NGSI LD** at group or device provisioning time using the `ngsiVersion` field in the
-provisioning API. The `ngsiVersion` field switch may be added at either group or device level, with the device level
-overriding the group setting.
+be switched to **NGSI LD** at group or device provisioning time using the `ngsiVersion` field in the provisioning API.
+The `ngsiVersion` field switch may be added at either group or device level, with the device level overriding the group
+setting.
 
 #### `server`
 
@@ -306,7 +306,8 @@ added `agentPath`:
 
 #### `types`
 
-This parameter includes additional groups configuration as described into the [Config group API](api.md#config-group-api) section.
+This parameter includes additional groups configuration as described into the
+[Config group API](api.md#config-group-api) section.
 
 #### `service`
 
@@ -415,7 +416,38 @@ IotAgents, as all Express applications that use the body-parser middleware, have
 size that the application will handle. This default limit for ioiotagnets are 1Mb. So, if your IotAgent receives a
 request with a body that exceeds this limit, the application will throw a “Error: Request entity too large”.
 
-The 1Mb default can be changed setting the `expressLimit` configuration parameter (or equivalente `IOTA_EXPRESS_LIMIT` environment variable).
+The 1Mb default can be changed setting the `expressLimit` configuration parameter (or equivalente `IOTA_EXPRESS_LIMIT`
+environment variable).
+
+#### `storeLastMeasure`
+
+If this flag is activated, last measure arrived to Device IoTAgent without be processed will be stored in Device under
+`lastMeasure` field (composed of sub-fields `timestamp` and `measure` for the measure itself, in multi-measure format).
+This flag is overwritten by `storeLastMeasure` flag in group or device. This flag is disabled by default.
+
+For example in a device document stored in MongoDB will be extended with a subdocument named lastMeasure like this:
+
+```json
+{
+    "lastMeasure": {
+        "timestamp": "2025-01-09T10:35:33.079Z",
+        "measure": [
+            [
+                {
+                    "name": "level",
+                    "type": "Text",
+                    "value": 33
+                }
+            ]
+        ]
+    }
+}
+```
+
+#### `useCBflowControl`
+
+If this flag is activated, when iotAgent invokes Context Broker will use [flowControl option](https://github.com/telefonicaid/fiware-orion/blob/master/doc/manuals/admin/perf_tuning.md#updates-flow-control-mechanism). This flag is overwritten by
+`useCBflowControl` flag in group or device. This flag is disabled by default.
 
 ### Configuration using environment variables
 
@@ -479,6 +511,8 @@ overrides.
 | IOTA_DEFAULT_ENTITY_NAME_CONJUNCTION | `defaultEntityNameConjunction`  |
 | IOTA_RELAX_TEMPLATE_VALIDATION       | `relaxTemplateValidation`       |
 | IOTA_EXPRESS_LIMIT                   | `expressLimit`                  |
+| IOTA_STORE_LAST_MEASURE              | `storeLastMeasure`              |
+| IOTA_CB_FLOW_CONTROL                 | `useCBflowControl`              |
 
 Note:
 

doc/admin.md

@@ -775,6 +775,8 @@ The registration of the commands is performed once in the lifetime of the Device
 
 #### Command Execution
 
+##### Based in update (classic way)
+
 Scenario 3 begins with the request for a command from the User to the Context Broker (P1):
 
 ```bash
@@ -825,21 +827,8 @@ Fiware-Correlator: 9cae9496-8ec7-11e6-80fc-fa163e734aab
 }
 ```
 
-The IoT Agent detects the selected attribute is a command, and replies to the Context Broker with the following payload
-(200 OK):
-
-```json
-[
-    {
-        "type": "device",
-        "id": "Dev0001",
-        "switch": {
-            "type": "command",
-            "value": ""
-        }
-    }
-]
-```
+The IoT Agent detects the selected attribute is a command, and replies to the Context Broker with a 204 OK (without
+payload).
 
 This response just indicates that the IoT Agent has received the command successfully, and gives no information about
 the requested information or command execution.
@@ -862,6 +851,100 @@ The Context Broker, forwards the same response to the user, thus replying the or
 At this point, the command has been issued to the IoTAgent and the User doesn't still know what the result of its
 request will be.
 
+##### Based in notification (new way)
+
+A new way to ContextBroker provides a command to a IoTAgent is through notifications. In this case CB notify the command
+to the IotAgent with a request like the following:
+
+```bash
+POST /notify HTTP/1.1
+Host: <target-host>:<northbound_port>
+fiware-service: workshop
+Fiware-ServicePath: /iota2ngsi
+Accept: application/json
+Content-length: 290
+Content-type: application/json; charset=utf-8
+Fiware-Correlator: 9cae9496-8ec7-11e6-80fc-fa163e734aab
+
+{
+    "subscriptionId": "60b0cedd497e8b681d40b58e",
+    "data": [{
+     "id": "123456abcdefg",
+     "type": "switchOnOffExecution",
+     "targetEntityId": {
+         "type": "Text",
+         "value": "Dev0001",
+         "metadata": {}
+     },
+     "targetEntityType": {
+         "type": "Text",
+         "value": "device",
+         "metadata": {}
+     },
+     "execTs": {
+         "type": "DateTime",
+         "value": "2020-05-27T00:00:00.000Z",
+         "metadata": {}
+     },
+     "cmd": {
+         "type": "Text",
+         "value": "switch",
+         "metadata": {}
+     },
+     "params": {
+         "type": "Text",
+         "value": "54, 12",
+         "metadata": {}
+     },
+     "status": {
+         "type": "Text",
+         "value": "FORWARDED",
+         "metadata": {}
+     },
+     "info": {
+         "type": "Text",
+         "value": null,
+         "metadata": {}
+     },
+     "onDelivered": {
+         "type": "Request",
+         "value": {
+         }
+     },
+     "onOk": {
+         "type": "Request",
+         "value": {
+         }
+     },
+     "onError": {
+         "type": "Request",
+         "value": {
+         }
+     },
+     "onInfo": {
+         "type": "Request",
+         "value": {
+         }
+     },
+     "cmdExecution": {
+         "type": "value",
+         "value": true,
+         "metadata": {}
+     },
+     "dateExpiration": {
+         "type": "DateTime",
+         "value": "2020-05-27T20:00:00.000Z",
+         "metadata": {}
+     }
+    }]
+}
+```
+
+In this case relevant fields are just `targetEntityId`, `targetEntityType`, `cmd` and `params`.
+
+The IoT Agent detects the selected attribute is a command, and replies to the Context Broker with a 204 OK (without
+payload).
+
 #### Result reporting
 
 Once the IoT Agent has executed the command or retrieved the information from the device, it reports the results to the
@@ -955,6 +1038,30 @@ The Context Broker replies with all the desired data, in R2 format (200 OK):
 ]
 ```
 
+#### Differences regarding the new commands mode
+
+A new commands flow has been defined (involving also modifications at ContextBroker). As part of that design, commands
+are not sent to IOTAs using NGSIv2 notifications, but the current implementation has some differences regarding the
+desired behaviour, which are described next:
+
+-   Fields others than `targetEntityId`, `targetEntityType`, `cmd` and `params` (i.e. `execTs`, `status`, `info`,
+    `onDelivered`, `onOk`, `onError`, `onInfo`, `cmdExecution` and `dataExpiration`), are not actually used. By the
+    moment they are stored in the commands model (commands collection) but nothing is done with them appart from
+    storing.
+-   The "Result reporting" should not be a "hardwired" behaviour updating the entity associated to the device, but using
+    the corresponding `on*` attribute in the notificaiton (e.g. `onOk` in the case of success). That attribute would
+    typically be a [HATEOAS](https://en.wikipedia.org/wiki/HATEOAS) object like this
+    `"onOk": { "href": "/v2/entities/123456abcdefg/attrs/status?type=switchExecution", "method": "PUT" }`. Moreover, the
+    entity to be updated in that HATEOAS would be the transient entity corresponding to command execuion, not the entity
+    associated to the device.
+
+```
+PUT /v2/entities/123456abcdefg/attrs/status?type=switchExecution
+content-type: text/plain
+
+OK
+```
+
 ### Scenario 3: commands (error)
 
 In Scenario 3, errors can happen asynchronously, out of the main interactions. When the IoTAgent detects an error