Autoprovision functionality not working as expected when configured at device/group level

[kandarp-n]

Issue: Autoprovision functionality not working as expected when configured at device/group level.

Steps to reproduce:

1. Register a device with 'autoprovision' flag set to true:

{
"devices": [
   {
     "device_id":   "sensor001",
     "entity_name": "urn:ngsi-ld:Sensor:sensor001",
     "entity_type": "Sensor",
     "timezone":    "Europe/Berlin",
     "attributes": [
       { "object_id": "temperature", "name":"temperature", "type":"number"}
      ],
      "autoprovision":true
   }
 ]
}

2. Send measurement for this device with an extra measurement attribute. Sample measurement payload:

{
    "temperature":23.4,
    "humidity":40
}

Expected result: As "autoprovision" is "true" for this device, the new attribute "humidity" should get created in the Orion entity with its measurement value.
Actual result: "humidity" attibute is not getting created in Orion entity. HTTP status code 404 is returned. (Note: Measurement of "temperature" is getting updated in Orion though.)

I've identified potential cause of the issue:

iotagent-node-lib/lib/services/ngsi/entities-NGSI-v1.js

Lines 243 to 249 in 8a6cbd1

	if (
	'autoprovision' in typeInformation &&
	/* jshint -W101 */
	typeInformation.autoprovision === undefined
	? typeInformation.autoprovision === true
	: config.getConfig().appendMode === true
	) {

Below fix is required (line# 246)

    if (
        'autoprovision' in typeInformation &&
        /* jshint -W101 */
        typeInformation.autoprovision !== undefined  //This should be !== operator and not ===
            ? typeInformation.autoprovision === true
            : config.getConfig().appendMode === true
    ) {

[kandarp-n]

Am I supposed to raise a pull request instead (because I already found a fix)? I'm new to the open source world, so please advise :)

[fgalan]

iotagent-node-lib/lib/services/ngsi/entities-NGSI-v1.js corresponds to an obsolete version of the API soon to be removed.

Thus, could you configure your IOTA in NGSIv2 mode and test how it goes? Or if the same problem occurs in the NGSIv2 part (I guess iotagent-node-lib/lib/services/ngsi/entities-NGSI-v2.js) it would be great if you could send it a PR with the fix.

Thanks for your feedback!

[AlvaroVega]

@kandarp-n it seems a typo and you has found a fix. Just edit that file in github as proposed and create a new Pull Request then.

[kandarp-n]

iotagent-node-lib/lib/services/ngsi/entities-NGSI-v1.js corresponds to an obsolete version of the API soon to be removed.

Thus, could you configure your IOTA in NGSIv2 mode and test how it goes? Or if the same problem occurs in the NGSIv2 part (I guess iotagent-node-lib/lib/services/ngsi/entities-NGSI-v2.js) it would be great if you could send it a PR with the fix.

Thanks for your feedback!

@fgalan Thanks, after configuring IOTA in NGSIv2 mode, it worked fine. But when I made autoprovision=false, then too new attributes are getting created, which shouldn't get created, right?

[kandarp-n]

@kandarp-n it seems a typo and you has found a fix. Just edit that file in github as proposed and create a new Pull Request then.

@AlvaroVega Thank you for confirmation. I have created a pull request.

[SBlechmann]

iotagent-node-lib/lib/services/ngsi/entities-NGSI-v1.js corresponds to an obsolete version of the API soon to be removed.
Thus, could you configure your IOTA in NGSIv2 mode and test how it goes? Or if the same problem occurs in the NGSIv2 part (I guess iotagent-node-lib/lib/services/ngsi/entities-NGSI-v2.js) it would be great if you could send it a PR with the fix.
Thanks for your feedback!

@fgalan Thanks, after configuring IOTA in NGSIv2 mode, it worked fine. But when I made autoprovision=false, then too new attributes are getting created, which shouldn't get created, right?

I am facing similar issues. I just can't get IOTA to ignore unprovisioned attributes. Have you been able to fix it yet?

[fgalan]

@fgalan Thanks, after configuring IOTA in NGSIv2 mode, it worked fine. But when I made autoprovision=false, then too new attributes are getting created, which shouldn't get created, right?

You are right, that shouldn't be happening... @kandarp-n maybe you could have a look and fix it the same way you fixed the other bug? :)

If confirmed, probably this is happening because we don't have proper unit tests for autoprovision=false cases. It would be great to have them.

[fgalan]

PR #1005 has been merged. However, I understand this issue should be kept open, as some autoprovision=false cases have been notified in the latest comment.

[SBlechmann]

I have tested the new image of the json-iotagent 1.17.1 which uses the lib version 2.15.1 and it worked fine for me now.
-> I created a device group (/iot/services) that does not allow autoprovision. I have sent data the configured mqtt topic / apikey using an unknown deviceid. No new device has been created.

But the error message in the IoT agent looks like:
time=2021-04-08T08:49:39.604Z | lvl=ERROR | corr=n/a | trans=n/a | op=IoTAgentJSON.Utils | from=n/a | srv=XXXX| subsrv=XXXX| msg=MEASURES-001: Couldn't find device data for APIKey [1234] and DeviceId[XXXX] | comp=IoTAgent
The service with apikey 1234 is registered in the service "test" while the error message threw a different service (which I blurred to XXXX) that I use with other devices.
-> Shouldn't this refer to the service (fiware-service, srv) that I have created the group / apikey in?
-> Apparently, the log level for this is ERROR, this might spam the logs. What do you think about changing the log level to WARN or even INFO instead?

It seems that I am not able to use the explicitAttrs option .. but I'm gonna raise another issue for this.

[fgalan]

It seems that I am not able to use the explicitAttrs option .. but I'm gonna raise another issue for this.

So this one can be closed and discussion continues in #1016?

[SBlechmann]

I'm affraid not - I was too quick.

I have done a quick test again while trying something with explicitAttrs and then my first observations were ruined.
So when I tried to send data to an unprovisioned device with an already existing service
{ "commands": [], "lazy": [], "attributes": [], "_id": "6058a2c9cc3fb500a130762d", "resource": "/iot/json", "apikey": "1234", "service": "test", "subservice": "/", "autoprovision": false, "explicitAttrs": true, "__v": 0, "static_attributes": [], "internal_attributes": [], "entity_type": "Sensor:Temperature" }
no new device was created -> that what I explained in my previous post.

Then I created a new service that actually had the same settings with just the apikey changed from 1234 to 12345
{ "commands": [], "lazy": [], "attributes": [], "_id": "606ed29b5c0d990080921b32", "resource": "/iot/json", "apikey": "12345", "service": "test", "subservice": "/", "autoprovision": false, "explicitAttrs": true, "__v": 0, "static_attributes": [], "internal_attributes": [], "entity_type": "Sensor:Temperature" }
and I tried to send data to /json/12345/tsensor00X/attrs a new device was created although I set autoprovision to false.

[SBlechmann]

And still the issue with the raised mongo alarm for the wrong service and the log level.
Shall I create new issues for this?