Not all nodes all added to the monitored items

[steffenbeermann]

Describe the bug
We have a opc publisher configured with an endpoint containing 30398 OpcNodes. If we start the opc publisher, only 6.000 (exactly) are added to the monitored items and you can see this in the diagnostics of the endpoint as well:

 "sentMessagesPerSec": 0.10332211227153337,
    "ingestionDuration": "15:49:08.5165570",
    "ingressDataChanges": 205518,
    "ingressValueChanges": 658158,
    "ingressBatchBlockBufferSize": 16,
    "encodingBlockInputSize": 0,
    "encodingBlockOutputSize": 0,
    "encoderNotificationsProcessed": 205502,
    "encoderNotificationsDropped": 0,
    "encoderIoTMessagesProcessed": 5891,
    "encoderAvgNotificationsMessage": 34.88406043116626,
    "encoderAvgIoTMessageBodySize": 25255.861483619112,
    "encoderAvgIoTChunkUsage": 6.165981807524197,
    "estimatedIoTChunksPerDay": 0,
    "outgressInputBufferCount": 0,
    "outgressInputBufferDropped": 0,
    "outgressIoTMessageCount": 5884,
    "connectionRetries": 0,
    "opcEndpointConnected": true,
    "monitoredOpcNodesSucceededCount": 5832,
    "monitoredOpcNodesFailedCount": 168,
    "ingressEventNotifications": 0,
    "ingressEvents": 0,
    "encoderMaxMessageSplitRatio": 0,
    "ingressDataChangesInLastMinute": 179,
    "ingressValueChangesInLastMinute": 290,
    "ingressHeartbeats": 0,
    "ingressCyclicReads": 0
  },

this is our config:

{
    "Hostname": "opcpublisher",
    "Cmd": [
        "--cl=5",
        "--aa",
        "--pf=/mount/publishednodes.json",
        "--PkiRootPath=/mount/pki",
        "--si=90",
        "--di=600000",
        "--bi=10000",
        "--cfa"
    ],
    "HostConfig": {
        "Binds": [
            "/home/iotedge/opcpublisher:/mount"
        ]
    }

}

If we split the endpoint into multiple DataSetWriterGroups, which only contain a few thousand nodes it all nodes are added to the monitored items. Based on the data we receive, it looks like it really just adds the first 6.000 items in the publishednodes.json and than stops.

With another opc server with 90.000 OpcNodes configured, we saw a similar issue, but there the cutoff was 21.000 signals...

We could not see anything in the logs (log level info).

Tested with version 2.9.8. and with out any issue.

To Reproduce
We tried version 2.9.13 and 2.9.14

• add an endpoint with 30.000 nodes or more nodes
• start publisher and wait for monitored items created
• see diagnostics to only have 6.000 endpoints added.

[alxy]

What opc ua Server do you use? Have you checked if the server has limitations on the Number of momitored items?

[steffenbeermann]

We have different servers, from Kepware to Custom Implementation. But I think this are not server limitations as it is working fine in OPC Publisher Versions 2.9.8 and 2.9.11 as well.

[marcschier]

Could you try version 2.9.15-preview3 and see if it fixes your issue? There was a problem in how the hash of the monitored items was calculated, not taking into account the dataset writer (virtual subscriber), therefore overwriting all nodes (that have duplicate node ids).

[steffenbeermann]

I tried it and it seems have solved the issue a partly. For most endpoints, it is now showing all nodes in good or bad. But for some we still have the issue. E.G. for one endpoint we have 39829 total nodes configured but the diagnostics shows:
"monitoredOpcNodesSucceededCount": 15162,
"monitoredOpcNodesFailedCount": 7838,

so only 23.000 Signals were added. This is a very round number again? So it is possible related?

while on Version 2.9.11 we get with the same publishednodes.json:
"monitoredOpcNodesSucceededCount": 31992,
"monitoredOpcNodesFailedCount": 7838,

If I can provide some additional information, feel free to reach out to me!

[marcschier]

Failed node count seems to be correct, so that is good news :-D. Good nodes, ok, will probably need to understand more about the the server, total is 23000 - as even number it could be a limit on the subscription size that cannot be resolved. However, are you using --npd parameter? I will load up my kepware with 40000 nodes instead of 20000 which I just tested and works and see what it looks like.

[steffenbeermann]

For both Versions we used the config mentioned in the first post, so we don't use the --npd parameter. The Server / publisher works fine with the version 2.9.11. And we noticed the issue with different server from different vendors, so it seems not to be a problem with a specific server.

We noticed that the total number of nodes is not always 23.000 but can be different, depending on the server an the server internal max nodes per subscription as it seems to be always a multiple of the npd setting.

[steffenbeermann]

Tested thew new 2.9.15 version as well and we ran into the same issue.