Sending random values to PMU Connection Tester Connectivity and Reporting Rate Issues

[Pratyush-das]

Hi,
I am new to GitHub and this field as well. In case I am doing or asking anything stupid, please guide me.
I am running the RandomPMU code but when I connect with PMU Connection tester, I get the following error:

_C:\Users\pdas\PycharmProjects\test32\venv\Scripts\python.exe "C:/Users/pdas/Documents/vPMU progress/pypmu/examples/randomPMU.py"
2020-06-16 22:34:13,200 INFO [1410] - PMU configuration changed.
2020-06-16 22:34:13,200 INFO [1410] - PMU header changed.
2020-06-16 22:34:13,204 INFO [1410] - Waiting for connection on 127.0.0.1:1410
2020-06-16 22:34:31,896 INFO [1410] - Waiting for connection on 127.0.0.1:1410
2020-06-16 22:34:32,800 INFO [1410] - Connection from 127.0.0.1:56705
2020-06-16 22:34:32,801 WARNING [1410] - **Message not received completely <- (127.0.0.1:56705)

Then if I send "Enable Real-time Data" from the PMU connection tester GUI, I can see the values being sent

2020-06-16 22:38:23,428 DEBUG [1410] - Message sent at [1592339903.428882] -> (127.0.0.1:56705)
2020-06-16 22:38:23,521 DEBUG [1410] - Message sent at [1592339903.521880] -> (127.0.0.1:56705)
2020-06-16 22:38:23,630 DEBUG [1410] - Message sent at [1592339903.630881] -> (127.0.0.1:56705)
2020-06-16 22:38:23,680 DEBUG [1410] - Message sent at [1592339903.680881] -> (127.0.0.1:56705)
2020-06-16 22:38:23,784 DEBUG [1410] - Message sent at [1592339903.784881] -> (127.0.0.1:56705)

But I cannot see the values in the plot or the values in the PMU connection tester GUI. It says "Awaiting Configuration Frame".
Furthermore, I have noticed that the Synchrophasor reporting rate despite being set to 30 frames per second, is varying widely. How that can be tackled ?

[poledna]

Hello,
I tested in my computer and I got the same Warning in python exactly equal to yours. and in the PMUconnection tester:

so i tried something:

in the same place as you've set Enable RealTime Data set Send Config Frame2 and click send. then it started sending data. Like this:

So some explanation:
Reading the logs what has happened:

1. You've created a PMU and changed the it's configuration.
2. Python Waited for connection
   So far so good, only python has worked, then you've connected PMU connection tester.
3. Python receives that there is someone opening a connection and opens it
4. Python receives a unknown message. This here should've been the Config Frame 2(CFG2) request frame, but how python didn't receive the request it didn't send it.
   So here in PMU Connection Tester has a dilemma I have a connection but i cant unpack the data because the data that i need to unpack the data is in the CFG2 frame that never came! So PMU Connection Tester waits until it arrives. The proposed "solution" is to ask to python PMU again the lost data so that PMU Connection Tester can correctly unpack the data. (that what is done when you click on the Send CFG2 button).

This solves the issue momentarily but further investigation will be needed to be done. I'm not exactly sure why it isn't receiving the complete message from PMU connection tester but it is clear that it isn't.

On the matter of reporting rate I suggest checking it again after correct reception of the CFG2 dataframe.

[Pratyush-das]

Thank you poledna for your advice, it worked. I am getting these data now after sending the configuration frame 2. However, the frame rate is not at all constant. It becomes quite low. Any suggestions ?

[poledna]

this is going to be a big answer. and you might want to start drawing this out because there are threads and process spawning.

I don't use as much the PMU module, as I use the PDC module, but for what I've seen I assume the following that if it is computer related, it is because of a sleep and if you disable the randomint and put fixed values it increases the throughput. (try checking this one out and try commenting out line 17 of the code it should increase the throughput)

So what is a PMU? From the code perspective a PMU is at the same time a PDC and PMU, because it has to handle the incomming traffic and the outgoing traffic.

In this example you set a 30fps value, I've made tests on the past and managed to get upwards of 480fps, so why is this happening?

So in the pmu object creation you create itself, then set the debug level, the create the CFG2 set it and set its header. Finally call it's run method.
the run method creates a socket and a Thread ( which has a target acceptor).

pypmu/synchrophasor/pmu.py

Lines 193 to 206 in 66e6c49

	def run(self):
	if not self.cfg1 and not self.cfg2 and not self.cfg3:
	raise PmuError("Cannot run PMU without configuration.")
	# Create TCP socket, bind port and listen for incoming connections
	self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
	self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
	self.socket.bind((self.ip, self.port))
	self.socket.listen(5)
	self.listener = Thread(target=self.acceptor) # Run acceptor thread to handle new connection
	self.listener.daemon = True
	self.listener.start()

The Aceptor Thread creates a buffer and a Process that calls on the pdc_handler that deals with the parsing and sending of data .

pypmu/synchrophasor/pmu.py

Lines 209 to 227 in 66e6c49

	def acceptor(self):
	while True:
	self.logger.info("[%d] - Waiting for connection on %s:%d", self.cfg2.get_id_code(), self.ip, self.port)
	# Accept a connection on the bound socket and fork a child process to handle it.
	conn, address = self.socket.accept()
	# Create Queue which will represent buffer for specific client and add it o list of all client buffers
	buffer = Queue()
	self.client_buffers.append(buffer)
	process = Process(target=self.pdc_handler, args=(conn, address, buffer, self.cfg2.get_id_code(),
	self.cfg2.get_data_rate(), self.cfg1, self.cfg2,
	self.cfg3, self.header, self.buffer_size,
	self.set_timestamp, self.logger.level))
	process.daemon = True
	process.start()

this method (now a separate process) has a extensive parsing and then on line 340 has the following lines:

pypmu/synchrophasor/pmu.py

Lines 340 to 350 in 66e6c49

	if sending_measurements_enabled and not buffer.empty():
	data = buffer.get()
	if isinstance(data, CommonFrame): # If not raw bytes convert to bytes
	if set_timestamp: data.set_time()
	data = data.convert2bytes()
	sleep(delay)
	connection.sendall(data)
	logger.debug("[%d] - Message sent at [%f] -> (%s:%d)",
	pmu_id, time(), address[0], address[1])

specifically on line 347 you see a sleep(delay) from which is instantiated at

pypmu/synchrophasor/pmu.py

Lines 258 to 262 in 66e6c49

	if data_rate > 0:
	delay = 1.0 / data_rate
	else:
	delay = -data_rate

okay. so you have a fixed sleep of in this case 1/30 seconds!, but this value does not consider the time the code took to execute! I assume this is the problem, but testing is necessary. If you take out the sleep you're going to send everything you've got on the buffer as fast as possible, so not a good possibility, the ideal would be to consider the time the code took to execute and subtract from the sleep total. But this has its problems. A better alternative might be to use time.monotonic_ns() and calculate time by this. If you're interested i can write more about this possibility.

going back to randomPMU.py:
If we look at the code you use the method send_data from the pmu.py files, then it creates the dataFrame using the CFG2 you've provided and put it on the buffer.
this buffer is a class buffer it is one of the available buffers on self.client_buffers. This buffer is the same read by the pdc_handler and its contents will be sent when available.

I've made some tests with the PMU on linux:
with the delay ON LINUX!!!

And with the PMU on linux and without the delay above commented delay:

but on Windows, I did find the same problem of maximum 20fps. I'd suggest either change to linux or look into that sleep. another alternative would be pypy, but not sure if it works on windows. If it is obligatory windows then look at that delay, if not give linux a try.

I'm sorry if some ideas got disconnected, i've written this in different times.

[sstevan]

Hey everyone,

Regarding the low reporting rate issue, you should check bugfix-v1.0.1 branch. Several users reported poor performance (and memory leaks) on Raspberry PI devices, so we have noticed that this is caused by flooding the buffer with new measurements while the old ones are still waiting to be sent. Therefore, you might want to introduce a delay before pmu.send(pmu.ieee_data_sample).

We have also considered introducing some kind of delta correction time that will cover time required to pack the frames. This means you can keep reducing delay for some tiny amount until you finally tweak it to get ~30fps.

[Pratyush-das]

My system administrator increased the CPU resource in the windows Virtual Machine I was running it on. I have better frame rates now. Still the difference increases for higher frame rates. For example, for 10 fps its ~9.98, for 25 fps its ~24.23, for 50 fps ~48, for 100 fps ~95. I will try with sleep delay and mention my findings.
Thanks for your guidance.

[Pratyush-das]

time.monotonic_ns()

Hi Yuri,
I was way behind on sorting other problems and forgot about this one. I am not sure how to time the program execution using the time.monotonic_ns(). I randomly (after considering an average delay from the calculated data rate) introduced time delay of 1.3 ms in addition to 1/data_rate. This yields a better result, otherwise, the data rate was always below the configured rate of 30fps.
I put two monotonic times as follows:

while True:
        prog_clk1=time.monotonic_ns()
        #print(pmu.cfg2,type(pmu.cfg2))
        cframe = pmu.cfg2
        config_rr = cframe.get_data_rate()
        pmu_rr = str(config_rr).encode("ascii") # int to bytes e.g. 50 to b'50'
        sock_rr_send.sendto(pmu_rr, (UDP_IP, UDP_PORT3))
        #print(pmu_rr)
        try:
            aware_rr = sock_rr.recv(1024)
            #drr = int(struct.unpack('d', aware_rr[8:16])[0]) # if coming from OPAL
            del_rr = int(aware_rr.decode("utf-8")) # the received value is +1 or -1
            #drr = crr + del_rr
            drr=del_rr
            flag = True
        except socket.timeout:  # to catch the error essentially it will only store drr if there is something to store and set the flag
            pass
        if pmu.clients:
            ## Socket Values
            ## Listen to the socket
            raw = sock_ph.recv(32)
            if First:
                tm_strt = datetime.now()
                First = False
            else:
                pass
            #print(raw)
            header, mag, angle, power = struct.unpack('dddd', raw)
            phasor = (mag, angle)
            Vol_A=raw
            VA = float(mag)
            phi_A = float(angle)
            VB = VA
            phi_B = phi_A+(math.pi) * 2 / 3
            VC = VA
            phi_C = phi_A-(math.pi) * 2 / 3
            phasor_list.append([VA, phi_A, VB, phi_B, VC, phi_C])
            # print(len(phasor_list))
            if len(phasor_list) >= (1000 / config_rr):
                phasor_array = np.array(phasor_list)
                #Y = phasor_array[-1]
                y = np.mean(phasor_array, axis=0)
                time_stp = datetime.now()
                sim_time = (time_stp - tm_strt).total_seconds()
                # print(y, time_stp, sim_time)
                phasor_list = []
                prog_clk2 = time.monotonic_ns()
                runtime_prog = prog_clk2-prog_clk1
                # print(runtime_prog,"ns")
                pmu.send_data(phasors=[(y[0], y[1]), (y[2], y[3]), (y[4], y[5])], analog=[9.91], digital=[0x0001])
                # print([(VA,phi_A),(VB,phi_B),(VC,phi_C),datetime.now()])

But I get 0 ns or 16000000ns. I am not sure how to isolate the delay from the sleep and the inherent delay of the code execution time.
Any insight on this would be great.
Regards,
Pratyush

[poledna]

Ok,
time.monotonic_ns() provides a nice and monotonic time in ns, the important part is that you can use it as a monotonic (non-decreasing time) for increased precision (comparatively)!
So if you remove the sleep delay from the code as i said before you can do this:

TEMPO = 33333333 # in this case TEMPO is 1/30 s  or 33,333,333ns 
FRACSEC_LIST=arange(0,1,1/30)
fracsec=0
last_second = monotonic_ns()
if monotonic_ns() >= last_second + TEMPO:
                last_second += TEMPO
                pmu.send_data(phasors=
                    [(abs(valores[0]),angle(valores[0])),(0,0),(0,0)],
                     analog=[0],
                     digital=[0x0001],
                     freq=60+(valores[1]),
                     dfreq=(valores[2]),
                     soc=timestamp,
                     frasec=FRACSEC_LIST[fracsec])
                if fracsec<30:
                    fracsec = fracsec+1
                if fracsec == 30:
                    fracsec = 0

This is a test case scenario for sending data at precisely 1/30s
Essentially you are putting data only in the buffer WHEN you wish to be send! as there is no delay inside the pdc_handler of the pmu.py file, it will send at the instant it arrives at the buffer (plus a tiny moving around time constant).
As the code varies execution time each iteration, due to other kernel interrupts, this code should consider these changes in each loop and alter the values.
If i understood what you were saying this should do it but if it doesn't, or if i completely misunderstood, don't be afraid to reach out.
PS you might want to stick either to datetime or to time packages, you already check the time spent on the loop with the runtime_prog variable it might be cheaper to the code if you don't use datetime and use one extra variable (1 extra variable should be smaller than an entire package) and if you find that the code isn't as "fast" as you need it to be you can always check out pypy.
ps2 there are other ways to time the code this one i tested in a raspberry pi and got satisfied.

[Pratyush-das]

Hi Yuri,
thank you for the clarification. I am receiving phasor data via s UDP socket once every 1ms. These values form a buffer internally with the OS and the pmu.send tries to send each and every packet from the buffer. So I created a list and append it every time I get a new value and after the list size is 1000/data_rate, I call the pmu.send function and send the latest data. There could be a more efficient method to do this but at least this works, with low coefficient of variation (standard deviation of calculated frame rate/mean of calculated framerate). Please see my code and comment if it makes sense, and how can I incorporate the monotonic_ns here.

from synchrophasor.pdc import Pdc
from synchrophasor.frame import DataFrame
from synchrophasor.frame import ConfigFrame2
import time
from synchrophasor.frame import CommandFrame
import copy
import pickle
import sys
import socket
import struct

"""
PDC will connect to pmu_ip:pmu_port and send request
for header message, configuration and eventually
to start sending measurements.
Here it receives data from PMUs, calculates FPS and Data Rate.
Prints and Saves Configured FPS, actual measured FPS and Data Rate along with phasor measurements
"""

if __name__ == "__main__":

    ############## Measured Reporting Rates send to OPAL #########################

    UDP_IP_OPAL = "10.10.114.21"
    # UDP_PORT1fps_send = 8301
    # sock_rr_send1 = socket.socket(socket.AF_INET,  # Internet
    #                               socket.SOCK_DGRAM)  # UDP
    UDP_PORT1fps_send = 8301

    sock_rr_send1 = socket.socket(socket.AF_INET,  # Internet
                                  socket.SOCK_DGRAM)  # UDP


    #init_time = int(time())
    #time_delay = 5
    pdc = Pdc(pdc_id=7, pmu_ip="10.10.114.22", pmu_port=1410)
    pdc.logger.setLevel("DEBUG")
    pdc.run()  # Connect to PMU
    header = pdc.get_header()  # Get header message from PMU
    config = pdc.get_config()  # Get configuration from PMU
    cnfg_fps_default = config.get_data_rate()
    fps = cnfg_fps_default
    pdc.start()  # Request to start sending measurements
    # print(fps)

    timestamps = []
    alldata = []
    first = True
    i = 0
    # ## Configure socket for RR Value Change status##
    # UDP_IP = "10.10.114.22"
    # UDP_PORT = 603  # whether RR is changed or not, if = 1, RR change initiated else 0
    # sock_ch = socket.socket(socket.AF_INET,  # Internet
    #                         socket.SOCK_DGRAM)  # UDP
    # sock_ch.bind((UDP_IP, UDP_PORT))  # binding is required for receiving not sending
    # sock_ch.settimeout(1 / 100)  # you dont want to wait forever a data that might not come
    # change = 0
    try:
        while True:
            # try:
            #     # listening to rr change signal
            #     change = sock_ch.recv(1024)
            #     change = int(change.decode("utf-8"))
            #     #print(change)
            # except socket.timeout:  # if nothing comes
            #     pass
            # # timing asking each 60s or 1 minute:
            # if  change == 1: #time() >= (init_time + time_delay) or
            #     #init_time += time_delay # this is not a pretty way to do it but i think is clear what is happening
            #
            #     # asking the CFG2:
            #     pdc.pmu_socket.sendto(CommandFrame(pdc.pdc_id, 'cfg2').convert2bytes(), pdc.pmu_address)
            #     change = 0
                #print("Config frame received")
            # config = pdc.get_config()
            data = pdc.get()  # Keep receiving data
            if type(data) == ConfigFrame2:
                # config = pdc.get_config()  # version="cfg2"
                cnfg_fps = data.get_data_rate()
                print("config frame received and data rate is %f" %cnfg_fps)

            if type(data) == DataFrame:
                data = data.get_measurements()
                # print(type(data),sys.getsizeof(data)) # Type and size of object in bytes
                # i+=1
                timestamps.append(data['time'])
                if first:
                    timestrt = data["time"]  # unix timestamp; gives time in seconds
                    cnfg_fps = copy.copy(cnfg_fps_default)
                    first = False
                else:
                    timestp = data["time"]
                    acctual_fps = 1 / (timestp - timestrt)
                    data_rate = ((sys.getsizeof(data)) * 8) / (1024 * 1024 * (timestp - timestrt))  # in Mbps
                    timestrt = timestp
                    # print(cnfg_fps, "fps",acctual_fps,"fps",data_rate,"Mbps")
                    print("Configured FPS = %s ; Actual FPS = %f, Data Rate = %f Mbps" % (
                        cnfg_fps, acctual_fps, data_rate))
                    fps = acctual_fps
                    timestamp = time.time()
                    dev_id = 91
                    msg_id = i
                    msg_len = 16
                    send_opal = struct.pack('=hihdd', dev_id, msg_id, msg_len, timestamp, fps)
                    sock_rr_send1.sendto(send_opal, (UDP_IP_OPAL, UDP_PORT1fps_send))  # send actual fps to OPAL
                    i = i + 1

                alldata.extend((data, fps, cnfg_fps))
            if not data:
                #pdc.quit()  # Close connection
                print("Data not coming. Warning !!")
                # print(data,type(data))
                # break
    except ConnectionResetError:
        # pdc.stop()
        # pdc.quit()
        timestring = time.strftime("%Y%m%d-%H%M")
        with open("sync1_%s.pickle"%timestring, "wb+")as handle:
            pickle.dump(alldata, handle)
        print("A file has been written....")

[Pratyush-das]

I tried this, but obviously the if loop is not entered

## Import necessary libraries
import math
import numpy as np
import socket
import struct
from synchrophasor.frame import ConfigFrame2
from synchrophasor.pmu import Pmu
import time
# from synchrophasor.frame import ConfigFrame2
# from time import time
from synchrophasor.frame import CommandFrame
from datetime import datetime
import time

##
"""
randomPMU will listen on ip:port for incoming connections.
After request to start sending measurements - random
values and values from OPAL RT model. It will also receive RR from OPAL RT 
and configure frame rate accordingly.
"""



if __name__ == "__main__":

    ## Configure socket for Phasor data ##

    UDP_IP = "10.10.114.22"
    UDP_PORT = 8201 #UDP phasor values 32 bytes (V,phi,P)
    sock_ph = socket.socket(socket.AF_INET,  # Internet
                         socket.SOCK_DGRAM)  # UDP
    sock_ph.bind((UDP_IP, UDP_PORT))
    print("socket bound, waiting for data...")

    ## Configure socket for RR Value ##

    UDP_PORT2 = 600  # OPAL sends RR to this port
    sock_rr = socket.socket(socket.AF_INET,  # Internet
                            socket.SOCK_DGRAM)  # UDP
    sock_rr.bind((UDP_IP, UDP_PORT2))  # binding is required for receiving not sending
    sock_rr.settimeout(1 / 10000)  # you dont want to wait forever a data that might not come
    flag = False

    ## Configure socket for RR Value send to CA algorithm ##
    UDP_PORT3 = 9901
    sock_rr_send = socket.socket(socket.AF_INET,  # Internet
                                  socket.SOCK_DGRAM)  # UDP
    ## PMU configuration ##

    pmu = Pmu(ip="10.10.114.22", port=1410) #original port config
    ## connect to another VM 10.10.114.23
    #pmu = Pmu(ip="10.10.114.22", port=1410)
    pmu.logger.setLevel("DEBUG")

    cfg = ConfigFrame2(1,  # PMU_ID
                       1000000,  # TIME_BASE
                       1,  # Number of PMUs included in data frame
                       "PDAS_bus2",  # Station name
                       1410,  # Data-stream ID(s)
                       (True, True, True, True),  # Data format - POLAR; PH - REAL; AN - REAL; FREQ - REAL;
                       3,  # Number of phasors(was 3)
                       1,  # Number of analog values
                       1,  # Number of digital status words
                       ["VA", "VB", "VC", "ANALOG1", "BREAKER 1 STATUS",
                        "BREAKER 2 STATUS", "BREAKER 3 STATUS", "BREAKER 4 STATUS", "BREAKER 5 STATUS",
                        "BREAKER 6 STATUS", "BREAKER 7 STATUS", "BREAKER 8 STATUS", "BREAKER 9 STATUS",
                        "BREAKER A STATUS", "BREAKER B STATUS", "BREAKER C STATUS", "BREAKER D STATUS",
                        "BREAKER E STATUS", "BREAKER F STATUS", "BREAKER G STATUS"],  # Channel Names
                       [(0, "v"), (0, "v"),
                        (0, "v")],  # Conversion factor for phasor channels - (float representation, not important)
                       [(1, "pow")],  # Conversion factor for analog channels
                       [(0x0000, 0xffff)],  # Mask words for digital status words
                       50,  # Nominal frequency
                       1,  # Configuration change count
                       25)  # Rate of phasor data transmission) 50 default here

    pmu.set_configuration(cfg)
    pmu.set_header("Random PMU: Modified by Pratyush")

    pmu.run()

    ############## C37.118 Reporting Rates #########################

    rr_list = [5, 10, 25, 50, 100]  # list of available RR as per standard
    First = True
    phasor_list = []
    flag = False
    while True:
        #print(pmu.cfg2,type(pmu.cfg2))
        cframe = pmu.cfg2
        config_rr = cframe.get_data_rate()
        pmu_rr = str(config_rr).encode("ascii") # int to bytes e.g. 50 to b'50'
        sock_rr_send.sendto(pmu_rr, (UDP_IP, UDP_PORT3))
        #print(pmu_rr)
        try:
            aware_rr = sock_rr.recv(1024)
            #drr = int(struct.unpack('d', aware_rr[8:16])[0]) # if coming from OPAL
            del_rr = int(aware_rr.decode("utf-8")) # the received value is +1 or -1
            #drr = crr + del_rr
            drr=del_rr
            flag = True
        except socket.timeout:  # to catch the error essentially it will only store drr if there is something to store and set the flag
            pass
        if pmu.clients:
            ## Socket Values
            ## Listen to the socket

            raw = sock_ph.recv(32)
            if First:
                tm_strt = datetime.now()
                First = False
            else:
                pass
            #print(raw)
            header, mag, angle = struct.unpack('ddd', raw)
            phasor = (mag, angle)
            Vol_A=raw
            VA = float(mag)
            phi_A = float(angle)
            VB = VA
            phi_B = phi_A+(math.pi) * 2 / 3
            VC = VA
            phi_C = phi_A-(math.pi) * 2 / 3
            phasor_list.append([VA, phi_A, VB, phi_B, VC, phi_C])
            # print(len(phasor_list))
            if len(phasor_list) >= (1000 / config_rr):
                phasor_array = np.array(phasor_list)
                y = np.mean(phasor_array, axis=0)
                time_stp = datetime.now()
                sim_time = (time_stp - tm_strt).total_seconds()
                # print(y, time_stp, sim_time)
                phasor_list = []
                ## fine tuning reporting time
                TEMPO = (10**9)/config_rr  # in this case TEMPO is 1/30 s  or 33,333,333ns
                FRACSEC_LIST = np.arange(0, 1, 1 / config_rr)
                fracsec = 0
                last_second = time.monotonic_ns()
                if time.monotonic_ns() >= last_second + TEMPO:
                    last_second += TEMPO
                    pmu.send_data(phasors=[(y[0], y[1]), (y[2], y[3]), (y[4], y[5])], analog=[9.91], digital=[0x0001],
                                  freq=50,frasec=FRACSEC_LIST[fracsec])

                    if fracsec < config_rr:
                        fracsec = fracsec + 1
                    if fracsec == config_rr:
                        fracsec = 0

                # print([(VA,phi_A),(VB,phi_B),(VC,phi_C),datetime.now()])
            #if len(phasor_list) > (1000 / config_rr):

        if flag: # if it received something it changes the RR
            pmu.set_data_rate(drr)
            #time.sleep(1/ 1000)
            flag=False


    pmu.join()

[poledna]

That was an example didn't think you'd actually use it, as plug and play. I have no idea which if it is obvious it wouldn't enter, I'd suggest that you'd create a fluxogram or some activity diagrams to check the code out. i see now that you make some averages with 1000/data_rate so your datarate would be minimum of 10fps and maximum of 200fps. phasor_list = [] could be changed to phasor_list.clear() to have the same effect, but actually if you are using a numpy array it would be better for you to pre-allocate the data as you already know its size by the datarate, with some zeros and then with the iterator determine the moment to make averages. But this is for some time improvements, are you having troubles with time? Also I'm having some trouble understanding where lies the issue right now. Again i suggest to you point out exactly which and were is the problem. I glanced on this: phi_B = phi_A+(math.pi) * 2 / 3, which algorithm are you using to determine the phase A? Now i see that you're using the same magnitude of phase A to phases B and C only with +-2pi/3, so essentially this is still a monophase PMU, you could theoretically send only one phasor of tension if you only have one by changing the config frame. Also FRACSEC_LIST and TEMPO could be changed only when there is a change on the DataRate.

[Pratyush-das]

Hi, I was freaking out because I tried several methods but had some delay creeping in. Upon investigation I found out that the socket data has some sort of delay. I have no clue why.
https://stackoverflow.com/questions/63632159/python-udp-socket-unknown-delay
If you have time, I would really appreciate if you could share your expert opinion about it. I have zero knowledge about OS buffers etc.

[poledna]

So I don't have enough reputation to add a comment in SO and it isn't a fully fledged answer, so i suggest you check out the package arrival at the NIC using Wireshark to test if the packages aren't arriving all at 16ms intervals

[Pratyush-das]

So I don't have enough reputation to add a comment in SO and it isn't a fully fledged answer, so i suggest you check out the package arrival at the NIC using Wireshark to test if the packages aren't arriving all at 16ms intervals

Thanks Yuri, yes i have checked with WireShark and the packets are 1ms interval. I also printed monotonic_ns() values with each data packet, and learnt that 14-16 packets are sent at once and then after 14-16ms delay the next 14-16 are sent. I have no idea why the socket.recv() is behaving this way. The buffer is 32bytes so why is it holding 15*32bytes? I feel so out of my depth here. Any guidance would be hugely helpful for me . Thanks. I still have no reply in SO.

[poledna]

I see that you've updated in SO, and got some comments, I'm not able to replicate your issue in my ubuntu, I've tried your code you've posted in SO with only minor changes as follows:

# receiver2.py
import socket
import time
"""
just get the raw values from UDP socket every 1ms
The sender sends it with that temporal resolution

"""

UDP_IP = ""
UDP_PORT = 8208 #UDP phasor values 32 bytes (V,phi,P)
sock_ph = socket.socket(socket.AF_INET,  # Internet
                     socket.SOCK_DGRAM)  # UDP
sock_ph.bind((UDP_IP, UDP_PORT))
print("socket bound, waiting for data...")

while True:
    time_before_raw = time.monotonic_ns()
    raw = sock_ph.recv(32) #I am receiving 32 bytes data
    time_after_raw = time.monotonic_ns()
    print((time_after_raw-time_before_raw),raw,len(raw))

and as sender:

# sender.py
import socket
from time import sleep,monotonic_ns

TEMPO = 1e6
send = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
last_second = monotonic_ns()
iteration_count = 0
while iteration_count < 1000:
    if monotonic_ns() >= last_second + TEMPO:
        last_second += TEMPO
        send.sendto(bytes(32), ('127.0.0.1', 8208))       
        iteration_count += 1

this configuration executed both at the same time on two terminals provided the 1ms recv datarate. I'm sorry i can't help more, it may be windows. I'd suggest trying this combination of codes in your computer just to see if the data arrives at the correct time in localhost (one in each terminal). But it might not still work. Really weird. Sorry i can't be of more help. But if does work then the plot thickens. Anyway i think SO is the best place to discuss