Performance issue with ReadTags (Slow Reading)

[SimoneMaffei98]

Good morning,
I’m using this library to read variables from a Siemens S7-1513 PLC with firmware version 2.9.
There are about 11,000 variables to read. It’s not an excessive amount, but when I try to read all of them at once using the ReadTags function, the execution takes more than 3 seconds, which is unacceptable considering the expected time should be around 10 ms.

After further investigation, I found that the issue seems to be within the SendS7plusFunctionObject and WaitForNewS7plusReceived functions, which are internally called by ReadValues. Each call takes around 20–30 ms, and since the function splits the read requests into chunks of 100 variables, it's invoked approximately 110 times.

Is it possible to send a single request to retrieve all values at once?

Using Wireshark, I compared the network traffic with another similar application that seems to use the same functions. While we are sending multiple requests to read the 11,000 variables, the other application appears to send a single request and then receives the data in around fifty consecutive packets in just around 15ms. Unfortunately, I couldn’t analyze the content further since the communication was encrypted.

Is it possible to achieve a similar behavior using this library?

Thank you very much.

[SimoneMaffei98]

This is S7CommPlusDriver library Network Resource Monitor:

This is the other library Network Resource Monitor:

[thomas-v2]

The GetMultiVariables method allows only the maximum number with one request, which is a system limit of the CPU and is read after connection setup and used later. If this is 50 and you'll add 51 to the request, then the complete request will fail.

The only option to make this faster I know, is to use Subscriptions up to the limits of the CPU, and if the limits are exceeded then switch for the rest to poll mode and read them via GetMultiVariables. This is the way for example WinCC is doing this. For Subscriptions I've done only basic tests on this type of communication, so it's not ready to use yet.

But the request either for reading or creating subscriptions for 11.000 variables will take in the best way about 121.000 bytes. With a TPDU size of 1024 which is always relevant, this will take a minimum of 120 TDPUs. So this can't be a single request PDU to access this amout of data.

There's also another read method which I've seen, but don't know exactly which advantages it has.

To see how they are doing this, I'd would be nice if you have the option to create an unencrypted Wireshark capture of the communication. This can be done with a TLS man in the middle, or the easier way would be to use an older CPU, or just projecting an older firmware inside the TIA project which doesn't support TLS would be enough.

[jogibear9988]

Is it possible in some way to combine they bytes read from the PLC?
Like I've done it in the SiemensPLcToolbox, where I read different PlcValues combined in one big chunk, if the byte address is 4 or less bytes away (https://github.com/dotnetprojects/DotNetSiemensPLCToolBoxLibrary/blob/ab6cf0c6d597c5ec5676bf30aaf094af117dd8bd/LibNoDaveConnectionLibrary/Communication/PLCConnection.cs#L2416)
SO you get less read requests but read more bytes?
Or are the tags in the protocol accessed via there symbolic name? (I did not look into it deeply....)

[thomas-v2]

I don't think it should be the way, to read absolute addresses when you want to read via symbolic access. If not, some driver could check if the requested variables are accessible via absolute address, and read the absolute address area instead and pick the values from the byte-array later. But if someone moves a variable in the plc around, you may get wrong values if you don't immediately explore the plc completely and recalculate all offsets. What I don't know if it's possible to readout the optimized addressed area (where the opmized variables are located), which you also know via browsing. This would speed up reading a lot, if the variables are at least all inside a single datablock. But with the same disadvantage as for absolute addressing.

[SimoneMaffei98]

Hi Thomas,
the subscriptions seem to be working quite well. However, I have a problem: my 11k variables might not be enough, so I'm trying with 100k. The issue is that a single subscription can handle a maximum of 40k variables, so I need to create more than one.
It doesn't seem possible to do this through the same connection — do you know if there's a way to achieve this without having to create multiple connections?

[jogibear9988]

One subscription can handle 40k variables? If you register them, do you get all the values after the subscription and after that only the changes?

[SimoneMaffei98]

Basically yes, this is what I do:

I think it is something in test mode so you will need to change something in TestWaitForVariableChangeNotifications.

[SimoneMaffei98]

Is there a way to make a bulk request to retrieve all the values at once and then convert them in the code?
100 ms still seems too much to me, and I’d like to achieve something around 10 ms, as I already do with Sharp7.

[jogibear9988]

You could read 40k from a S7 with the old protocol in 10ms? That seems nearly unbelivable

[thomas-v2]

If you enable Put/Get communication in the HW config of the Plc, you can still use Sharp7 to access the data. The datablock must be "not optimized". You can access this areas using absolute addressing also with the new protocol, but I guess it's always slower as the old protocol (due to more protocol overhead and TLS encryption).

[SimoneMaffei98]

I thought I could do it in 10 ms, but I was mistaken. It still takes me around 80 ms, which is actually a very good time (with Sharp7).

However, we would like to avoid absolute access, as it seems Siemens is increasingly moving towards symbolic access.

[thomas-v2]

But there seems to be an option, to read out a complete datablock data area in a whole dataset. I've just checked what TIA is doing when you go online in a datablock view. There is at least a subscription created, where a blob is transmitted with the raw data, without any overhead. But the subscription is created different from the standard variable subscriptions. There is first a Tis (Test and Installation) subscription created, and then another subscription where the first is referenced to. There are many unknown values in this, so I don't think this would be possible to do in the same way.

[thomas-v2]

There's an easy option to read a complete datablock as a whole. if you read with the main id of the datablock, for example to read DB99 contents, use ItemAddress("8A0E0063"). Then you get back a struct where one content is the raw datablock of the whole DB99. If you use the OffsetInfo from browsing the db, you get the optimized address from there to locate the value inside the blob. In the same way you can read whole structs inside the DB, by accessing the LID of the struct itself, and not it's elements.

I don't know if there are any issues with data consistency even for plain datatypes like Ints or Reals. For strings it's always a problem, even when you access them via the single element LID.

I don't know if I want such methods in the library, I believe there can be many problems using this type of access. Maybe a convenient method for easy readout a whole DB and get a bytearrea back. Then it's on the user if he need this style for performance reasons.

[SimoneMaffei98]

This is exactly the solution I was looking for. It's important to bind the value correctly to the variable, but this approach seems to be very fast. Thank you.

[thomas-v2]

Calculating the offsets for simple tags or tags in structs is relatively easy. But if there are arrays, this needs knowledge of the size of an element, which for plain types is also easy. For array of structs you must known the size of one element, to know at which offset the next element starts. Maybe there are some padding bytes added at some point. This needs to be tested, and hopefully it's always the same padding style for all controllers and all TIA versions.