PyGPSClient logs have message dropouts

[pat-2010]

I am saving a parsed log file. It has NEMA and RTCM messages. Intermittently the data drops out (averages about once every 2 minutes). It typically returns within about 3 seconds. It drops out in the middle of a message cycle and starts up with the GNRMC message that starts the data for a new second. I get around 33 messages per second, depending on the number of satellites in view. They always follow a sequence of GNRMC, GNGGA, multiple GNGSA, multiple GxGSV and a GNGST. The RTCM messages 1004 and 1012 come in at a bit more than 1 per second. I've looked at multiple cases of dropouts and they appear to start randomly in the middle of the 33 message sequence. Both NMEA and RTCM messages drop out. I have not observed any partial messages.

Using pygpsclient 1.4.22.

Here's a clip from the log file. Note the time jumps from 20:49:39 to 20:49:42

<NMEA(GQGSV, numMsg=1, msgNum=1, numSV=1, svid_01=3, elv_01=9.0, az_01=301, cno_01=38, signalID=1)>
<NMEA(GQGSV, numMsg=1, msgNum=1, numSV=1, svid_01=3, elv_01=9.0, az_01=301, cno_01=41, signalID=6)>
<NMEA(GNGST, time=20:49:39, rangeRms=8.9, stdMajor=0.025, stdMinor=0.02, orient=154.0, stdLat=0.01, stdLong=0.01, stdAlt=0.018)>
<NMEA(GNRMC, time=20:49:42, status=A, lat=38.857116665, NS=N, lon=-121.28045184, EW=W, spd=0.011, cog=, date=2024-10-13, mv=, mvEW=, posMode=F, navStatus=V)>
<NMEA(GNGGA, time=20:49:42, lat=38.857116665, NS=N, lon=-121.28045184, EW=W, quality=5, numSV=12, HDOP=0.76, alt=71.605, altUnit=M, sep=-26.73, sepUnit=M, diffAge=1.0, diffStation=372)>

And a second case

<NMEA(GNGST, time=19:49:45, rangeRms=19.0, stdMajor=0.96, stdMinor=0.62, orient=5.7, stdLat=0.39, stdLong=0.26, stdAlt=0.74)>
<NMEA(GNRMC, time=19:49:46, status=A, lat=38.8571101867, NS=N, lon=-121.28045642, EW=W, spd=0.018, cog=, date=2024-10-13, mv=, mvEW=, posMode=D, navStatus=V)>
<NMEA(GNGGA, time=19:49:46, lat=38.8571101867, NS=N, lon=-121.28045642, EW=W, quality=2, numSV=12, HDOP=0.49, alt=73.642, altUnit=M, sep=-26.73, sepUnit=M, diffAge=, diffStation=133)>
<NMEA(GNGSA, opMode=A, navMode=3, svid_01=21, svid_02=31, svid_03=9, svid_04=28, svid_05=2, svid_06=16, svid_07=26, svid_08=4, svid_09=3, svid_10=, svid_11=, svid_12=, PDOP=0.96, HDOP=0.49, VDOP=0.82, systemId=1)>
<NMEA(GNRMC, time=19:49:48, status=A, lat=38.8571102317, NS=N, lon=-121.28045633, EW=W, spd=0.013, cog=, date=2024-10-13, mv=, mvEW=, posMode=D, navStatus=V)>
<NMEA(GNGGA, time=19:49:48, lat=38.8571102317, NS=N, lon=-121.28045633, EW=W, quality=2, numSV=12, HDOP=0.49, alt=73.683, altUnit=M, sep=-26.73, sepUnit=M, diffAge=, diffStation=133)>

Steps to reproduce the behaviour:
I check data logging with the parsed option. Then select the usb/uart option to open the link to the gps receiver.

Expected Behaviour

A log file with all the data.

Desktop:
Running on a Raspberry pi 5. 64 bit Debian version 12 (Bookworm). USB connection to simpleRTK2B budget receiver. Using Python 3.11.2.

GNSS/GPS Device:

<UBX(MON-VER, swVersion=b'EXT CORE 1.00 (0fa0ae)\x00\x00\x00\x00\x00\x00\x00\x00', hwVersion=b'00190000\x00\x00', extension_01=b'ROM BASE 0x118B2060\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', extension_02=b'FWVER=HPG 1.32\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', extension_03=b'PROTVER=27.31\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', extension_04=b'MOD=ZED-F9P\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', extension_05=b'GPS;GLO;GAL;BDS\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', extension_06=b'SBAS;QZSS\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')>

Additional context

I have used u-center and recorded the NMEA messages with no dropouts.

[semuadmin]

Hi @pat-2010

Can you just confirm that the following:

1. You're saving the logs in parsed rather than binary format?
2. You're saving the logs to the RPI's SD card? What category (max write speed) of SD card are you using?
3. What navigation solution interval is your F9P set to? Is it the default 1000ms (i.e. 1Hz)?
4. Roughly how big are your data log files, in kB? Do the 'dropouts' occur early in the log file or only when the log reaches a certain size?

When you say you've used u-center to record logs successfully, I presume you mean on a separate Windows machine? What happens if you use PyGPSClient's datalogging on that same Windows platform - do you still see 'dropouts'? If not, i'm inclined to think this may simply be an SD write latency issue on the RPi, but I will look into it.

Out of interest, have you tried using the gnssstreamer CLI utility to record datalogs? e.g.

gnssstreamer --port /dev/tty.usbmodem1201 --baudrate 38400 --timeout 3 --format 16 --verbosity 2 --clioutput 5 --output datalog.log

[semuadmin]

FYI I'm away from my usual base at the moment and can't exactly reproduce your configuration, but just tried a quick test using a NEO-M8N with an RPi4 Model B 4MB running Python 3.12 with a SanDisk Ultra (140 MB/s) 64MB SD card and am not seeing the same issue - here's an excerpt from the log covering a 7 minute period:

IDENTITY                    TIME               INCREMENT
<NMEA(GNGGA	time=08:31:28	08:31:28	
<NMEA(GNGGA	time=08:31:29	08:31:29	1s
<NMEA(GNGGA	time=08:31:30	08:31:30	1s
<NMEA(GNGGA	time=08:31:31	08:31:31	1s
<NMEA(GNGGA	time=08:31:32	08:31:32	1s
<NMEA(GNGGA	time=08:31:33	08:31:33	1s
<NMEA(GNGGA	time=08:31:34	08:31:34	1s
<NMEA(GNGGA	time=08:31:35	08:31:35	1s
<NMEA(GNGGA	time=08:31:36	08:31:36	1s
<NMEA(GNGGA	time=08:31:37	08:31:37	1s
<NMEA(GNGGA	time=08:31:38	08:31:38	1s
<NMEA(GNGGA	time=08:31:39	08:31:39	1s
<NMEA(GNGGA	time=08:31:40	08:31:40	1s
<NMEA(GNGGA	time=08:31:41	08:31:41	1s
<NMEA(GNGGA	time=08:31:42	08:31:42	1s
<NMEA(GNGGA	time=08:31:43	08:31:43	1s
<NMEA(GNGGA	time=08:31:44	08:31:44	1s
<NMEA(GNGGA	time=08:31:45	08:31:45	1s
<NMEA(GNGGA	time=08:31:46	08:31:46	1s
<NMEA(GNGGA	time=08:31:47	08:31:47	1s
<NMEA(GNGGA	time=08:31:48	08:31:48	1s
<NMEA(GNGGA	time=08:31:49	08:31:49	1s
<NMEA(GNGGA	time=08:31:50	08:31:50	1s
<NMEA(GNGGA	time=08:31:51	08:31:51	1s
<NMEA(GNGGA	time=08:31:52	08:31:52	1s
<NMEA(GNGGA	time=08:31:53	08:31:53	1s
<NMEA(GNGGA	time=08:31:54	08:31:54	1s
<NMEA(GNGGA	time=08:31:55	08:31:55	1s
<NMEA(GNGGA	time=08:31:56	08:31:56	1s
<NMEA(GNGGA	time=08:31:57	08:31:57	1s
<NMEA(GNGGA	time=08:31:58	08:31:58	1s
<NMEA(GNGGA	time=08:31:59	08:31:59	1s
<NMEA(GNGGA	time=08:32:00	08:32:00	1s
<NMEA(GNGGA	time=08:32:01	08:32:01	1s
<NMEA(GNGGA	time=08:32:02	08:32:02	1s
<NMEA(GNGGA	time=08:32:03	08:32:03	1s
<NMEA(GNGGA	time=08:32:04	08:32:04	1s
<NMEA(GNGGA	time=08:32:05	08:32:05	1s
<NMEA(GNGGA	time=08:32:06	08:32:06	1s
<NMEA(GNGGA	time=08:32:07	08:32:07	1s
<NMEA(GNGGA	time=08:32:08	08:32:08	1s
<NMEA(GNGGA	time=08:32:09	08:32:09	1s
<NMEA(GNGGA	time=08:32:10	08:32:10	1s
<NMEA(GNGGA	time=08:32:11	08:32:11	1s
<NMEA(GNGGA	time=08:32:12	08:32:12	1s
<NMEA(GNGGA	time=08:32:13	08:32:13	1s
<NMEA(GNGGA	time=08:32:14	08:32:14	1s
<NMEA(GNGGA	time=08:32:15	08:32:15	1s
<NMEA(GNGGA	time=08:32:16	08:32:16	1s
<NMEA(GNGGA	time=08:32:17	08:32:17	1s
<NMEA(GNGGA	time=08:32:18	08:32:18	1s
<NMEA(GNGGA	time=08:32:19	08:32:19	1s
<NMEA(GNGGA	time=08:32:20	08:32:20	1s
<NMEA(GNGGA	time=08:32:21	08:32:21	1s
<NMEA(GNGGA	time=08:32:22	08:32:22	1s
<NMEA(GNGGA	time=08:32:23	08:32:23	1s
<NMEA(GNGGA	time=08:32:24	08:32:24	1s
<NMEA(GNGGA	time=08:32:25	08:32:25	1s
<NMEA(GNGGA	time=08:32:26	08:32:26	1s
<NMEA(GNGGA	time=08:32:27	08:32:27	1s
<NMEA(GNGGA	time=08:32:28	08:32:28	1s
<NMEA(GNGGA	time=08:32:29	08:32:29	1s
<NMEA(GNGGA	time=08:32:30	08:32:30	1s
<NMEA(GNGGA	time=08:32:31	08:32:31	1s
<NMEA(GNGGA	time=08:32:32	08:32:32	1s
<NMEA(GNGGA	time=08:32:33	08:32:33	1s
<NMEA(GNGGA	time=08:32:34	08:32:34	1s
<NMEA(GNGGA	time=08:32:35	08:32:35	1s
<NMEA(GNGGA	time=08:32:36	08:32:36	1s
<NMEA(GNGGA	time=08:32:37	08:32:37	1s
<NMEA(GNGGA	time=08:32:38	08:32:38	1s
<NMEA(GNGGA	time=08:32:39	08:32:39	1s
<NMEA(GNGGA	time=08:32:40	08:32:40	1s
<NMEA(GNGGA	time=08:32:41	08:32:41	1s
<NMEA(GNGGA	time=08:32:42	08:32:42	1s
<NMEA(GNGGA	time=08:32:43	08:32:43	1s
<NMEA(GNGGA	time=08:32:44	08:32:44	1s
<NMEA(GNGGA	time=08:32:45	08:32:45	1s
<NMEA(GNGGA	time=08:32:46	08:32:46	1s
<NMEA(GNGGA	time=08:32:47	08:32:47	1s
<NMEA(GNGGA	time=08:32:48	08:32:48	1s
<NMEA(GNGGA	time=08:32:49	08:32:49	1s
<NMEA(GNGGA	time=08:32:50	08:32:50	1s
<NMEA(GNGGA	time=08:32:51	08:32:51	1s
<NMEA(GNGGA	time=08:32:52	08:32:52	1s
<NMEA(GNGGA	time=08:32:53	08:32:53	1s
<NMEA(GNGGA	time=08:32:54	08:32:54	1s
<NMEA(GNGGA	time=08:32:55	08:32:55	1s
<NMEA(GNGGA	time=08:32:56	08:32:56	1s
<NMEA(GNGGA	time=08:32:57	08:32:57	1s
<NMEA(GNGGA	time=08:32:58	08:32:58	1s
<NMEA(GNGGA	time=08:32:59	08:32:59	1s
<NMEA(GNGGA	time=08:33:00	08:33:00	1s
<NMEA(GNGGA	time=08:33:01	08:33:01	1s
<NMEA(GNGGA	time=08:33:02	08:33:02	1s
<NMEA(GNGGA	time=08:33:03	08:33:03	1s
<NMEA(GNGGA	time=08:33:04	08:33:04	1s
<NMEA(GNGGA	time=08:33:05	08:33:05	1s
<NMEA(GNGGA	time=08:33:06	08:33:06	1s
<NMEA(GNGGA	time=08:33:07	08:33:07	1s
<NMEA(GNGGA	time=08:33:08	08:33:08	1s
<NMEA(GNGGA	time=08:33:09	08:33:09	1s
<NMEA(GNGGA	time=08:33:10	08:33:10	1s
<NMEA(GNGGA	time=08:33:11	08:33:11	1s
<NMEA(GNGGA	time=08:33:12	08:33:12	1s
<NMEA(GNGGA	time=08:33:13	08:33:13	1s
<NMEA(GNGGA	time=08:33:14	08:33:14	1s
<NMEA(GNGGA	time=08:33:15	08:33:15	1s
<NMEA(GNGGA	time=08:33:16	08:33:16	1s
<NMEA(GNGGA	time=08:33:17	08:33:17	1s
<NMEA(GNGGA	time=08:33:18	08:33:18	1s
<NMEA(GNGGA	time=08:33:19	08:33:19	1s
<NMEA(GNGGA	time=08:33:20	08:33:20	1s
<NMEA(GNGGA	time=08:33:21	08:33:21	1s
<NMEA(GNGGA	time=08:33:22	08:33:22	1s
<NMEA(GNGGA	time=08:33:23	08:33:23	1s
<NMEA(GNGGA	time=08:33:24	08:33:24	1s
<NMEA(GNGGA	time=08:33:25	08:33:25	1s
<NMEA(GNGGA	time=08:33:26	08:33:26	1s
<NMEA(GNGGA	time=08:33:27	08:33:27	1s
<NMEA(GNGGA	time=08:33:28	08:33:28	1s
<NMEA(GNGGA	time=08:33:29	08:33:29	1s
<NMEA(GNGGA	time=08:33:30	08:33:30	1s
<NMEA(GNGGA	time=08:33:31	08:33:31	1s
<NMEA(GNGGA	time=08:33:32	08:33:32	1s
<NMEA(GNGGA	time=08:33:33	08:33:33	1s
<NMEA(GNGGA	time=08:33:34	08:33:34	1s
<NMEA(GNGGA	time=08:33:35	08:33:35	1s
<NMEA(GNGGA	time=08:33:36	08:33:36	1s
<NMEA(GNGGA	time=08:33:37	08:33:37	1s
<NMEA(GNGGA	time=08:33:38	08:33:38	1s
<NMEA(GNGGA	time=08:33:39	08:33:39	1s
<NMEA(GNGGA	time=08:33:40	08:33:40	1s
<NMEA(GNGGA	time=08:33:41	08:33:41	1s
<NMEA(GNGGA	time=08:33:42	08:33:42	1s
<NMEA(GNGGA	time=08:33:43	08:33:43	1s
<NMEA(GNGGA	time=08:33:44	08:33:44	1s
<NMEA(GNGGA	time=08:33:45	08:33:45	1s
<NMEA(GNGGA	time=08:33:46	08:33:46	1s
<NMEA(GNGGA	time=08:33:47	08:33:47	1s
<NMEA(GNGGA	time=08:33:48	08:33:48	1s
<NMEA(GNGGA	time=08:33:49	08:33:49	1s
<NMEA(GNGGA	time=08:33:50	08:33:50	1s
<NMEA(GNGGA	time=08:33:51	08:33:51	1s
<NMEA(GNGGA	time=08:33:52	08:33:52	1s
<NMEA(GNGGA	time=08:33:53	08:33:53	1s
<NMEA(GNGGA	time=08:33:54	08:33:54	1s
<NMEA(GNGGA	time=08:33:55	08:33:55	1s
<NMEA(GNGGA	time=08:33:56	08:33:56	1s
<NMEA(GNGGA	time=08:33:57	08:33:57	1s
<NMEA(GNGGA	time=08:33:58	08:33:58	1s
<NMEA(GNGGA	time=08:33:59	08:33:59	1s
<NMEA(GNGGA	time=08:34:00	08:34:00	1s
<NMEA(GNGGA	time=08:34:01	08:34:01	1s
<NMEA(GNGGA	time=08:34:02	08:34:02	1s
<NMEA(GNGGA	time=08:34:03	08:34:03	1s
<NMEA(GNGGA	time=08:34:04	08:34:04	1s
<NMEA(GNGGA	time=08:34:05	08:34:05	1s
<NMEA(GNGGA	time=08:34:06	08:34:06	1s
<NMEA(GNGGA	time=08:34:07	08:34:07	1s
<NMEA(GNGGA	time=08:34:08	08:34:08	1s
<NMEA(GNGGA	time=08:34:09	08:34:09	1s
<NMEA(GNGGA	time=08:34:10	08:34:10	1s
<NMEA(GNGGA	time=08:34:11	08:34:11	1s
<NMEA(GNGGA	time=08:34:12	08:34:12	1s
<NMEA(GNGGA	time=08:34:13	08:34:13	1s
<NMEA(GNGGA	time=08:34:14	08:34:14	1s
<NMEA(GNGGA	time=08:34:15	08:34:15	1s
<NMEA(GNGGA	time=08:34:16	08:34:16	1s
<NMEA(GNGGA	time=08:34:17	08:34:17	1s
<NMEA(GNGGA	time=08:34:18	08:34:18	1s
<NMEA(GNGGA	time=08:34:19	08:34:19	1s
<NMEA(GNGGA	time=08:34:20	08:34:20	1s
<NMEA(GNGGA	time=08:34:21	08:34:21	1s
<NMEA(GNGGA	time=08:34:22	08:34:22	1s
<NMEA(GNGGA	time=08:34:23	08:34:23	1s
<NMEA(GNGGA	time=08:34:24	08:34:24	1s
<NMEA(GNGGA	time=08:34:25	08:34:25	1s
<NMEA(GNGGA	time=08:34:26	08:34:26	1s
<NMEA(GNGGA	time=08:34:27	08:34:27	1s
<NMEA(GNGGA	time=08:34:28	08:34:28	1s
<NMEA(GNGGA	time=08:34:29	08:34:29	1s
<NMEA(GNGGA	time=08:34:30	08:34:30	1s
<NMEA(GNGGA	time=08:34:31	08:34:31	1s
<NMEA(GNGGA	time=08:34:32	08:34:32	1s
<NMEA(GNGGA	time=08:34:33	08:34:33	1s
<NMEA(GNGGA	time=08:34:34	08:34:34	1s
<NMEA(GNGGA	time=08:34:35	08:34:35	1s
<NMEA(GNGGA	time=08:34:36	08:34:36	1s
<NMEA(GNGGA	time=08:34:37	08:34:37	1s
<NMEA(GNGGA	time=08:34:38	08:34:38	1s
<NMEA(GNGGA	time=08:34:39	08:34:39	1s
<NMEA(GNGGA	time=08:34:40	08:34:40	1s
<NMEA(GNGGA	time=08:34:41	08:34:41	1s
<NMEA(GNGGA	time=08:34:42	08:34:42	1s
<NMEA(GNGGA	time=08:34:43	08:34:43	1s
<NMEA(GNGGA	time=08:34:44	08:34:44	1s
<NMEA(GNGGA	time=08:34:45	08:34:45	1s
<NMEA(GNGGA	time=08:34:46	08:34:46	1s
<NMEA(GNGGA	time=08:34:47	08:34:47	1s
<NMEA(GNGGA	time=08:34:48	08:34:48	1s
<NMEA(GNGGA	time=08:34:49	08:34:49	1s
<NMEA(GNGGA	time=08:34:50	08:34:50	1s
<NMEA(GNGGA	time=08:34:51	08:34:51	1s
<NMEA(GNGGA	time=08:34:52	08:34:52	1s
<NMEA(GNGGA	time=08:34:53	08:34:53	1s
<NMEA(GNGGA	time=08:34:54	08:34:54	1s
<NMEA(GNGGA	time=08:34:55	08:34:55	1s
<NMEA(GNGGA	time=08:34:56	08:34:56	1s
<NMEA(GNGGA	time=08:34:57	08:34:57	1s
<NMEA(GNGGA	time=08:34:58	08:34:58	1s
<NMEA(GNGGA	time=08:34:59	08:34:59	1s
<NMEA(GNGGA	time=08:35:00	08:35:00	1s
<NMEA(GNGGA	time=08:35:01	08:35:01	1s
<NMEA(GNGGA	time=08:35:02	08:35:02	1s
<NMEA(GNGGA	time=08:35:03	08:35:03	1s
<NMEA(GNGGA	time=08:35:04	08:35:04	1s
<NMEA(GNGGA	time=08:35:05	08:35:05	1s
<NMEA(GNGGA	time=08:35:06	08:35:06	1s
<NMEA(GNGGA	time=08:35:07	08:35:07	1s
<NMEA(GNGGA	time=08:35:08	08:35:08	1s
<NMEA(GNGGA	time=08:35:09	08:35:09	1s
<NMEA(GNGGA	time=08:35:10	08:35:10	1s
<NMEA(GNGGA	time=08:35:11	08:35:11	1s
<NMEA(GNGGA	time=08:35:12	08:35:12	1s
<NMEA(GNGGA	time=08:35:13	08:35:13	1s
<NMEA(GNGGA	time=08:35:14	08:35:14	1s
<NMEA(GNGGA	time=08:35:15	08:35:15	1s
<NMEA(GNGGA	time=08:35:16	08:35:16	1s
<NMEA(GNGGA	time=08:35:17	08:35:17	1s
<NMEA(GNGGA	time=08:35:18	08:35:18	1s
<NMEA(GNGGA	time=08:35:19	08:35:19	1s
<NMEA(GNGGA	time=08:35:20	08:35:20	1s
<NMEA(GNGGA	time=08:35:21	08:35:21	1s
<NMEA(GNGGA	time=08:35:22	08:35:22	1s
<NMEA(GNGGA	time=08:35:23	08:35:23	1s
<NMEA(GNGGA	time=08:35:24	08:35:24	1s
<NMEA(GNGGA	time=08:35:25	08:35:25	1s
<NMEA(GNGGA	time=08:35:26	08:35:26	1s
<NMEA(GNGGA	time=08:35:27	08:35:27	1s
<NMEA(GNGGA	time=08:35:28	08:35:28	1s
<NMEA(GNGGA	time=08:35:29	08:35:29	1s
<NMEA(GNGGA	time=08:35:30	08:35:30	1s
<NMEA(GNGGA	time=08:35:31	08:35:31	1s
<NMEA(GNGGA	time=08:35:32	08:35:32	1s
<NMEA(GNGGA	time=08:35:33	08:35:33	1s
<NMEA(GNGGA	time=08:35:34	08:35:34	1s
<NMEA(GNGGA	time=08:35:35	08:35:35	1s
<NMEA(GNGGA	time=08:35:36	08:35:36	1s
<NMEA(GNGGA	time=08:35:37	08:35:37	1s
<NMEA(GNGGA	time=08:35:38	08:35:38	1s
<NMEA(GNGGA	time=08:35:39	08:35:39	1s
<NMEA(GNGGA	time=08:35:40	08:35:40	1s
<NMEA(GNGGA	time=08:35:41	08:35:41	1s
<NMEA(GNGGA	time=08:35:42	08:35:42	1s
<NMEA(GNGGA	time=08:35:43	08:35:43	1s
<NMEA(GNGGA	time=08:35:44	08:35:44	1s
<NMEA(GNGGA	time=08:35:45	08:35:45	1s
<NMEA(GNGGA	time=08:35:46	08:35:46	1s
<NMEA(GNGGA	time=08:35:47	08:35:47	1s
<NMEA(GNGGA	time=08:35:48	08:35:48	1s
<NMEA(GNGGA	time=08:35:49	08:35:49	1s
<NMEA(GNGGA	time=08:35:50	08:35:50	1s
<NMEA(GNGGA	time=08:35:51	08:35:51	1s
<NMEA(GNGGA	time=08:35:52	08:35:52	1s
<NMEA(GNGGA	time=08:35:53	08:35:53	1s
<NMEA(GNGGA	time=08:35:54	08:35:54	1s
<NMEA(GNGGA	time=08:35:55	08:35:55	1s
<NMEA(GNGGA	time=08:35:56	08:35:56	1s
<NMEA(GNGGA	time=08:35:57	08:35:57	1s
<NMEA(GNGGA	time=08:35:58	08:35:58	1s
<NMEA(GNGGA	time=08:35:59	08:35:59	1s
<NMEA(GNGGA	time=08:36:00	08:36:00	1s
<NMEA(GNGGA	time=08:36:01	08:36:01	1s
<NMEA(GNGGA	time=08:36:02	08:36:02	1s
<NMEA(GNGGA	time=08:36:03	08:36:03	1s
<NMEA(GNGGA	time=08:36:04	08:36:04	1s
<NMEA(GNGGA	time=08:36:05	08:36:05	1s
<NMEA(GNGGA	time=08:36:06	08:36:06	1s
<NMEA(GNGGA	time=08:36:07	08:36:07	1s
<NMEA(GNGGA	time=08:36:08	08:36:08	1s
<NMEA(GNGGA	time=08:36:09	08:36:09	1s
<NMEA(GNGGA	time=08:36:10	08:36:10	1s
<NMEA(GNGGA	time=08:36:11	08:36:11	1s
<NMEA(GNGGA	time=08:36:12	08:36:12	1s
<NMEA(GNGGA	time=08:36:13	08:36:13	1s
<NMEA(GNGGA	time=08:36:14	08:36:14	1s
<NMEA(GNGGA	time=08:36:15	08:36:15	1s
<NMEA(GNGGA	time=08:36:16	08:36:16	1s
<NMEA(GNGGA	time=08:36:17	08:36:17	1s
<NMEA(GNGGA	time=08:36:18	08:36:18	1s
<NMEA(GNGGA	time=08:36:19	08:36:19	1s
<NMEA(GNGGA	time=08:36:20	08:36:20	1s
<NMEA(GNGGA	time=08:36:21	08:36:21	1s
<NMEA(GNGGA	time=08:36:22	08:36:22	1s
<NMEA(GNGGA	time=08:36:23	08:36:23	1s
<NMEA(GNGGA	time=08:36:24	08:36:24	1s
<NMEA(GNGGA	time=08:36:25	08:36:25	1s
<NMEA(GNGGA	time=08:36:26	08:36:26	1s
<NMEA(GNGGA	time=08:36:27	08:36:27	1s
<NMEA(GNGGA	time=08:36:28	08:36:28	1s
<NMEA(GNGGA	time=08:36:29	08:36:29	1s
<NMEA(GNGGA	time=08:36:30	08:36:30	1s
<NMEA(GNGGA	time=08:36:31	08:36:31	1s
<NMEA(GNGGA	time=08:36:32	08:36:32	1s
<NMEA(GNGGA	time=08:36:33	08:36:33	1s
<NMEA(GNGGA	time=08:36:34	08:36:34	1s
<NMEA(GNGGA	time=08:36:35	08:36:35	1s
<NMEA(GNGGA	time=08:36:36	08:36:36	1s
<NMEA(GNGGA	time=08:36:37	08:36:37	1s
<NMEA(GNGGA	time=08:36:38	08:36:38	1s
<NMEA(GNGGA	time=08:36:39	08:36:39	1s
<NMEA(GNGGA	time=08:36:40	08:36:40	1s
<NMEA(GNGGA	time=08:36:41	08:36:41	1s
<NMEA(GNGGA	time=08:36:42	08:36:42	1s
<NMEA(GNGGA	time=08:36:43	08:36:43	1s
<NMEA(GNGGA	time=08:36:44	08:36:44	1s
<NMEA(GNGGA	time=08:36:45	08:36:45	1s
<NMEA(GNGGA	time=08:36:46	08:36:46	1s
<NMEA(GNGGA	time=08:36:47	08:36:47	1s
<NMEA(GNGGA	time=08:36:48	08:36:48	1s
<NMEA(GNGGA	time=08:36:49	08:36:49	1s
<NMEA(GNGGA	time=08:36:50	08:36:50	1s
<NMEA(GNGGA	time=08:36:51	08:36:51	1s
<NMEA(GNGGA	time=08:36:52	08:36:52	1s
<NMEA(GNGGA	time=08:36:53	08:36:53	1s
<NMEA(GNGGA	time=08:36:54	08:36:54	1s
<NMEA(GNGGA	time=08:36:55	08:36:55	1s
<NMEA(GNGGA	time=08:36:56	08:36:56	1s
<NMEA(GNGGA	time=08:36:57	08:36:57	1s
<NMEA(GNGGA	time=08:36:58	08:36:58	1s
<NMEA(GNGGA	time=08:36:59	08:36:59	1s
<NMEA(GNGGA	time=08:37:00	08:37:00	1s
<NMEA(GNGGA	time=08:37:01	08:37:01	1s
<NMEA(GNGGA	time=08:37:02	08:37:02	1s
<NMEA(GNGGA	time=08:37:03	08:37:03	1s
<NMEA(GNGGA	time=08:37:04	08:37:04	1s
<NMEA(GNGGA	time=08:37:05	08:37:05	1s
<NMEA(GNGGA	time=08:37:06	08:37:06	1s
<NMEA(GNGGA	time=08:37:07	08:37:07	1s
<NMEA(GNGGA	time=08:37:08	08:37:08	1s
<NMEA(GNGGA	time=08:37:09	08:37:09	1s
<NMEA(GNGGA	time=08:37:10	08:37:10	1s

I'll do further tests when I'm back at base.

[pat-2010]

Hi, 1- Logs are in parsed mode. The samples i put in Git-Hub were directly from the log file 2- saving logs to a WD SN740 256gGB SSD on a raspberry pi M.2 hat. Disk latency shouldn't be an issue. 3- 1Hz 4- Log files are about 3 megabytes. The plot below is the duration of the dropout vs time for one of the cases. It's actually the time difference between GNRMC messages when it was longer than 1 second, This was about 15 log files combined. I have seen times where it is a bit more regular. edit: trying to past in the plot which didn't make it here from the email  Yes,  u-center was run from my desktop. From there I don't have long enough cables to get the gps antenna to a location where I can get RTK fix. I saw your other email with no dropouts. I do have a laptop with linux. I'll try putting PyGPSClient on it and see if the log works there. I was also running the pi5 headless from my desktop so perhaps something there was interfering with the PI 5 operations.  Thanks for looking at this so quickly. Let me do some experimentation on my end to either find or rule out equipment problems. Pat On 10/16/2024 12:29:06 AM, SEMU Admin ***@***.***> wrote: Hi @pat-2010 [https://github.com/pat-2010] Can you just confirm that the following: * You're saving the logs in parsed rather than binary format? * You're saving the logs to the RPI's SD card? * What navigation solution interval is your F9P set to? Is it the default 1000ms (i.e. 1Hz)? * Roughly how big are your data log files, in kB? Do the 'dropouts' occur early in the log file or only when the log reaches a certain size? When you say you've used u-center to record logs successfully, I presume you mean on a separate Windows machine? What happens if you use PyGPSClient's datalogging on that same Windows platform - do you still see 'dropouts'? If not, i'm inclined to think this may simply be an SD write latency issue on the RPi, but I will look into it. — Reply to this email directly, view it on GitHub [#157 (comment)], or unsubscribe [https://github.com/notifications/unsubscribe-auth/BBHNPP5XNHGYSWNOSE5ZHO3Z3YIT7AVCNFSM6AAAAABQADKBPOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMJVHE2DQOBVGQ]. You are receiving this because you were mentioned.Message ID: ***@***.***>

[pat-2010]

I haven't tried using the gnssstreamer CLI utility. I'm trying to sort out some performance issues with my GPS system, so I want to have the RTCM messages as well as the NMEA messages. Unfortunately ArduSimple (manufacturer of my GPS receiver board) do not have a user's group so I can't see if others have had similar issues.

[semuadmin]

I haven't tried using the gnssstreamer CLI utility. I'm trying to sort out some performance issues with my GPS system, so I want to have the RTCM messages as well as the NMEA messages.

Fair enough. As a general observation, unless you particularly need GUI functionality, I'd generally advocate using CLI tools like gnssstreamer (which supports RTCM RTK by the way) on low end platforms like the RPi (particularly if you're running headless), and - where appropriate - logging binary UBX data rather than NMEA (the former is far more concise), but PyGPSClient's datalogging should work regardless - it's just a straight file write. Things may get a bit dicey if you're running a full cohort of NMEA and RTCM messages at very high solution rates (e.g. 20Hz), though the limiting factor in those cases is very often the TX buffer on the receiver itself rather than any downstream data logger. If the 'dropouts' are consistently periodic, that would tend to point to another running process causing contention of some kind - do you have any other daemon processes running on the RPi (gpsd is a common candidate)?

Unfortunately ArduSimple (manufacturer of my GPS receiver board) do not have a user's group so I can't see if others have had similar issues.

You'll appreciate I'm not in a position to provide direct support for proprietary GNSS gear, but Ardusimple's technical support is normally pretty good, as is the u-blox community forum. Have you tried raising your performance issues with either of them?

[pat-2010]

<<If the 'dropouts' are consistently periodic, that would tend to point to another running process causing contention of some kind - do you have any other daemon processes running on the RPi (gpsd is a common candidate)?>>
Nothing else running on the raspberry pi other than VNC to run the pi headless. The dropouts are not periodic. Here's data from the beginning of one run. This shows only the GNGGA messages that were more than 1 second away from the last GNGGA message. So on the first line, the message was 4 seconds from the last GNGGA (indicating 3 dropped GNGGA messages) and preceeding this message was 4:39 without any dropout.

  | message type |   Time | Delta from last message | time since last dropout

<NMEA | GNGGA | time=18:40:55 | 4 | 04:39
<NMEA | GNGGA | time=18:41:52 | 2 | 00:57
<NMEA | GNGGA | time=18:42:22 | 2 | 00:30
<NMEA | GNGGA | time=18:42:56 | 3 | 00:34
<NMEA | GNGGA | time=18:43:18 | 3 | 00:22
<NMEA | GNGGA | time=18:43:39 | 3 | 00:21
<NMEA | GNGGA | time=18:44:06 | 4 | 00:27
<NMEA | GNGGA | time=18:44:36 | 3 | 00:30
<NMEA | GNGGA | time=18:51:56 | 2 | 07:20
<NMEA | GNGGA | time=18:52:57 | 3 | 01:01
<NMEA | GNGGA | time=18:54:04 | 3 | 01:07
<NMEA | GNGGA | time=18:54:45 | 2 | 00:41
<NMEA | GNGGA | time=18:55:21 | 3 | 00:36
<NMEA | GNGGA | time=18:55:52 | 3 | 00:31
<NMEA | GNGGA | time=18:56:27 | 3 | 00:35
<NMEA | GNGGA | time=18:57:06 | 3 | 00:39
<NMEA | GNGGA | time=18:57:27 | 2 | 00:21
<NMEA | GNGGA | time=18:57:41 | 2 | 00:14
<NMEA | GNGGA | time=18:58:06 | 3 | 00:25
<NMEA | GNGGA | time=18:58:27 | 3 | 00:21
<NMEA | GNGGA | time=18:58:52 | 3 | 00:25
<NMEA | GNGGA | time=18:59:19 | 3 | 00:27
<NMEA | GNGGA | time=18:59:55 | 3 | 00:36
<NMEA | GNGGA | time=19:00:20 | 3 | 00:25
<NMEA | GNGGA | time=19:00:42 | 3 | 00:22
<NMEA | GNGGA | time=19:01:08 | 3 | 00:26
<NMEA | GNGGA | time=19:01:34 | 3 | 00:26
<NMEA | GNGGA | time=19:01:50 | 3 | 00:16
<NMEA | GNGGA | time=19:02:05 | 2 | 00:15
<NMEA | GNGGA | time=19:02:27 | 3 | 00:22
<NMEA | GNGGA | time=19:02:45 | 2 | 00:18
<NMEA | GNGGA | time=19:03:02 | 3 | 00:17
<NMEA | GNGGA | time=19:03:25 | 3 | 00:23
<NMEA | GNGGA | time=19:03:44 | 3 | 00:19
<NMEA | GNGGA | time=19:04:08 | 3 | 00:24
<NMEA | GNGGA | time=19:04:38 | 3 | 00:30

<<<You'll appreciate I'm not in a position to provide direct support for proprietary GNSS gear, but Ardusimple's technical support is normally pretty good, as is the u-blox community forum. Have you tried raising your performance issues with either of them?>>>

Yes, I've started discussions with Ardusimple tech support. They pointed to the correction service as one of the potential problems, which is why I'm trying to get a complete log of NMEA and RTCM messages covering the period where I am seeing unexpected accuracy variations. Thanks for pointing me to the u-blox community forum. I'll do some research there.

[semuadmin]

Are you confident the F9P is actually outputting the 'missing' NMEA messages? Are you seeing them in the PyGPSClient console, or for example via a standard terminal utility like screen e.g.

screen -L /dev/ttyACM0

[pat-2010]

On 10/16/2024 11:58:14 AM, SEMU Admin ***@***.***> wrote: Are you confident the F9P is actually outputting the 'missing' NMEA messages? Are you seeing them in the PyGPSClient console, or for example logging the raw NMEA using a standard terminal utility like screen e.g. screen -L /dev/ttyACM0 Pat Hascall:  RTCM messages are missing as well.  I have only been using PyGPSClient and it's log files. Given the missing RTCM messages I don't expect it is a problem with F9P output. But I will do a log file for an hour or so and see what happens.The problem is intermittent and can at times there can be many minutes between data dropouts. Makes it hard to look at data that comes once per second and spot it. I will pipe it to a file and examine it there. — Reply to this email directly, view it on GitHub [#157 (comment)], or unsubscribe [https://github.com/notifications/unsubscribe-auth/BBHNPP5CXWRT3Y7PWZ6O53TZ32ZMHAVCNFSM6AAAAABQADKBPOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMJXGY3TSOBXGM]. You are receiving this because you were mentioned.Message ID: ***@***.***>

[pat-2010]

Status: I can't get the problem to repeat. I checked my old data and for the 5 setups I did using PyGPSclient I had dropouts in every setup. Today I collected data for 1 1/2 hours without a single dropout. The only difference was that my computer was in my house and not on the back patio and the GPS antenna was in a different location. Also experimented with a different computer (laptop using linux) and didn't see any dropouts there either.

So tomorrow I will replicate exactly what I did before with the computer and antenna in the same physical location. I am using wifi to connect to the NTRIP server (but I think I have a good signal on my patio).

[semuadmin]

The only difference was that my computer was in my house and not on the back patio and the GPS antenna was in a different location.

Hmmmm - sounds like a back patio problem 😉

[pat-2010]

I believe the problem is somewhere in my system and not in PyGPSClient. Today I used my pi in a setup matching exactly that when I originally saw the problem. The dropped messages returned. I then replaced the pi with a laptop runningPyGPSClient on linux. No messages were dropped. I'm giving up trying to figure out why the pi drops messages sometimes but not always. I'm collecting enough data even with the drop outs for my purposes. Thanks for your help.

[semuadmin]

Hi @pat-2010

Sorry this one remains a puzzler. AFAICS there's nothing in the PyGPSClient datalogging code that would differentiate between different Python platforms or storage media, so I'm inclined to agree it's a local problem in this case. PyGPSClient has been tested extensively on a variety of RPi platforms, but that's not to say there aren't improvements that could be made.

Out of interest, which specific M2 hat model are you using? I think I have a few old M2 SSDs lying around back at base - I may get the same M2 hat and see if I can reproduce the problem. I'll keep this issue open for now.

Also, when you say "64 bit Debian version 12 (Bookworm)", can I assume you mean the stock version of Raspberry Pi OS 64-bit that's available from e.g. RPi Imager? Are you running the Debian OS on the M2 drive or on a separate SD drive?

[pat-2010]

Good morning, I'm going to try one more experiment. I was using the RPi 5 headless using RealVNC. I'll take a monitor/keyboard out to the setup to see if the RealVNC load on the RPi and wifi could be contributing to the problem. I'm using the Raspberry Pi M.2 HAT+.  On 10/17/2024 11:38:32 PM, SEMU Admin ***@***.***> wrote: Hi @pat-2010 [https://github.com/pat-2010] Sorry this one remains a puzzler. AFAICS there's nothing in the PyGPSClient datalogging code that would differentiate between different Python platforms or storage media, so I'm inclined to agree it's a local problem in this case. PyGPSClient has been tested extensively on a variety of RPi platforms, but that's not to say there aren't improvements that could be made. Out of interest, which specific M2 hat model are you using? I think I have a few old M2 SSDs lying around back at base - I may get an M2 hat and see if I can reproduce the problem. I'll keep this issue open for now. — Reply to this email directly, view it on GitHub [#157 (comment)], or unsubscribe [https://github.com/notifications/unsubscribe-auth/BBHNPP4HOVYQHNAORM77JW3Z4CUGPAVCNFSM6AAAAABQADKBPOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMRRGU2DIOJVGA]. You are receiving this because you were mentioned.Message ID: ***@***.***>

[pat-2010]

Well, I didn't expect that result. I set the RPi up headless with VNC. Ran for 5 minutes, turned off VNC for 5 minutes then turned it back on again. With VNC on there were no dropouts. When I turned VNC off I saw dropouts. These dropouts were periodic, so sound like a background process contention problem like you said earlier. With VNC back on the dropouts stopped. Just the opposite of what I expected. I know that there were some compatibility issues on the RPi 4-5 when the RPi went to Bookworm with a Wayland environment vs the older X-11. I have an older RPi400 that has the older operating system installed. When I have the time I'll break it out and see what happens.

Yes the RPi is running the stock version (Bookworm) loaded to SD card using the RPi imager then moved to the M2 drive. The RPi boots from the M2 drive. I realized that when you edit the comment it doesn't show up in my email. So from now on I'll come to github to get the latest comment updates.