Half of ADC data is missing

[ekurtgl]

When I start framing with mmwave studio and this module, the recorded openradar .bin file is only half of the size of the mmwave file. 50% of ADC samples are missing. Is there a solution for this problem? The data do not look good with the missing samples. Thanks in advance!

[felipeparralejo]

Maybe you are not converting from binary to int16? If you read from the binary file obtained with mmWave Studio you can use NumPy: np.frombuffer(data, dtype=np.int16) where data is your frame

[ekurtgl]

I don't think that is the problem. The acquired data have half the size of that is recorded with mmWave Studio. The issue can easily be reproduced by making two recordings: one with mmWave Studio, one with OpenRadar module. OpenRadar recorded data will have half the size of that is recorded with mmWave Studio.

[felipeparralejo]

This seems weird because I usually make recordings with mmWave Studio and also with OpenRadar, and the data have exactly the same size. I save the frames in Python using Pickle, and the file size obtained is exactly equal to the file recorded with mmWave Studio. Try reinstalling your OpenRadar module.

With respect to my previous answer, I think that originally the OpenRadar module did not convert the binary data from uint16 to int16, which is necessary. Check your data type after recording a frame and use np.frombuffer(frame, dtype=np.int16) if needed.

Hope this helps

[ekurtgl]

I save the data in the same way: np.frombuffer(frame, dtype=np.int16), but into a .bin file. Are you using real or complex-valued stream? I use complex adc data format in my configuration. I am guessing this module might be optimized for real-valued streams. I will definitely try to implement your method. Thank you for the suggestion.

[felipeparralejo]

I use a complex 1x stream. I have just realized that some radar parameters are hard coded in the adc.py file from the OpenRadar module (Ctrl+Click on the DCA1000 module will lead to the file). Parameters like the number of ADC samples, Rx and Tx antennas and the number of chirp loops are written in the dictionary ADC_PARAMS. I changed those values to the ones I configured my device with. Try having a look at this.

You can see what I am talking about here https://github.com/PreSenseRadar/OpenRadar/blob/master/mmwave/dataloader/adc.py (line 45)

[ekurtgl]

I was using 2x complex stream and changed adc.py accordingly, but will have a look it again and try with 1x complex data as well. Thank you so much for your suggestion!

[ccdcq]

Hello, has your problem been solved? I would like to ask what your solution is, thank you

[loeens]

I think I have found the issue what is causing this and will create a PR probably tomorrow.

The core issue is that the implemented solution only works for when the ADC data from one frame fills out complete UDP packets and thus the boundaries between frames are within in packets.

Let me explain:

Within the read() function there are two while-loops, one to find the beginning of a new frame and one to read the rest of the frame data.

The first while loop in read()
The first while-loop reads a packet and finds the packet which contains the start of a new frame with the following condition:

if byte_count % BYTES_IN_FRAME_CLIPPED == 0:

The second while loop in read()
The second while-loop in the read() function is supposed to read the rest of the frame data as soon the first packet of the frame is found. It relies on the same assumption, that frame boundaries are never within a packet. Also it works with the same condition as finding the first packet of a frame to detect the last packet of a frame:

if byte_count % BYTES_IN_FRAME_CLIPPED == 0:

Example
For simplicity let's assume that each frame contains 3072 Bytes.
3072 // 1456 = 2
3072 % 1456 = 160
So this means each frame stretches over 3 UDP packets, while two packets are filled completely and (if it is the first frame) the third packet is only filled with 160 Bytes.
BYTES_IN_FRAME_CLIPPED = 2.912 in this case.

• Frame 1: The first packet is read which yields byte_count = 0, so the condition correctly returns true and the frame data is read by the second loop. The 3rd packet yields byte_count = 2912, which again makes the second condition become true and the frame is returned.

• Frame 2: The next packet is read which has byte_count = 4368, but this fails the condition. The next packet is read, which has byte_count = 5824, so this is assumed to be the first packet of the next frame. It isn't though, besides we have skipped the frame with the byte_count = 4368, creating an offset. The last packet of the frame is identified with byte_count = 8736.

• Frame 3:The next packet is read which has byte_count = 10192, but this fails the condition. The next packet is read, which has byte_count = 11648, so this is assumed to be the first packet of the next frame. The last packet of the frame is identified with byte_count = 14560

So through the offset the data is corrupted, although apparently very slightly, I could only see it in the range profile plots that something was off. Furthermore packets are skipped.
Apart from the check condition the main issue lies in the assumption that frame boundaries aren't within packets. For this reason not only packets are skipped, but data is also lost from every last packet, which contains the start of the next frame. This combined data loss presumably leads to the 50% decreased frame rate caused by the reader needing to wait for data from chirps from the next frame.

Edit: Sorry I clicked the "Submit" button too early.