Skip to content

Commit af831cd

Browse files
committed
python -m georinex.read
python -m georinex.time use: set[str] python -m georinex.loc python -m georinex.rinex2hdf5 update test syntax
1 parent d5de43c commit af831cd

File tree

18 files changed

+341
-341
lines changed

18 files changed

+341
-341
lines changed

README.md

Lines changed: 26 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -85,7 +85,7 @@ Normally you'd use the `-p` option with single files to plot, if not converting.
8585
```
8686
* Batch convert RINEX to NetCDF4 / HDF5 (this example for RINEX 2 OBS):
8787
```sh
88-
rnx2hdf5 ~/data "*o" -o ~/data
88+
python -m georinex.rinex2hdf5 ~/data "*o" -o ~/data
8989
```
9090
in this example, the suffix `.nc` is appended to the original RINEX filename: `my.15o` => `my.15o.nc`
9191

@@ -102,7 +102,7 @@ import georinex as gr
102102
```
103103

104104
Uses speculative time preallocation `gr.load(..., fast=True)` by default.
105-
Set `fast=False` or `georinex_read -strict` to fall back to double-read strict (slow) preallocation.
105+
Set `fast=False` or CLI option `python -m georinex.read -strict` to fall back to double-read strict (slow) preallocation.
106106
Please open a GitHub issue if this is a problem.
107107

108108
### Time limits
@@ -285,29 +285,33 @@ To convert ECEF to Latitude, Longitude, Altitude or other coordinate systems, us
285285

286286
Read location from NetCDF4 / HDF5 file can be accomplished in a few ways:
287287

288-
* using `georinex_loc` script, which loads and plots all RINEX and .nc files in a directory
288+
* `python -m georinex.loc` to load and plot all RINEX and .nc files in a directory
289289
* using `xarray`
290+
290291
```python
291292
obs = xarray.open_dataset('my.nc)
292293

293294
ecef = obs.position
294295
latlon = obs.position_geodetic # only if pymap3d was used
295296
```
296297
* Using `h5py`:
298+
297299
```python
298300
with h5py.File('my.nc') as f:
299301
ecef = h['OBS'].attrs['position']
300302
latlon = h['OBS'].attrs['position_geodetic']
301303
```
302304

303-
## Converting to Pandas DataFrames
305+
## Convert to Pandas DataFrames
306+
304307
Although Pandas DataFrames are 2-D, using say `df = nav.to_dataframe()` will result in a reshaped 2-D DataFrame.
305308
Satellites can be selected like `df.loc['G12'].dropna(0, 'all')` using the usual
306309
[Pandas Multiindexing methods](http://pandas.pydata.org/pandas-docs/stable/advanced.html).
307310

308311
## Benchmark
309312

310313
An Intel Haswell i7-3770 CPU with plain uncompressed RINEX 2 OBS processes in about:
314+
311315
* [6 MB file](ftp://data-out.unavco.org/pub/rinex/obs/2018/021/ab140210.18o.Z): 5 seconds
312316
* [13 MB file](ftp://data-out.unavco.org/pub/rinex/obs/2018/021/ab180210.18o.Z): 10 seconds
313317

@@ -316,66 +320,64 @@ This processing speed is about within a factor of 2 of compiled RINEX parsers, w
316320
OBS2 and NAV2 currently have the fast pure Python read that has C-like speed.
317321

318322
### Obs3
323+
319324
OBS3 / NAV3 are not yet updated to new fast pure Python method.
320325

321-
Done on 5 year old Haswell laptop:
326+
On Haswell laptop:
327+
322328
```sh
323-
time georinex_read tests/CEDA00USA_R_20182100000_23H_15S_MO.rnx.gz -u E
329+
time python -m georinex.read tests/CEDA00USA_R_20182100000_23H_15S_MO.rnx.gz -u E
324330
```
325331

326332
> real 48.6 s
327333

328334
```sh
329-
time georinex_read tests/CEDA00USA_R_20182100000_23H_15S_MO.rnx.gz -u E -m C1C
335+
time python -m georinex.read tests/CEDA00USA_R_20182100000_23H_15S_MO.rnx.gz -u E -m C1C
330336
```
331337

332338
> real 17.6 s
333339

334340
### Profiling
341+
335342
using
343+
336344
```sh
337345
conda install line_profiler
338346
```
347+
339348
and `ipython`:
349+
340350
```ipython
341351
%load_ext line_profiler
342352

343353
%lprun -f gr.obs3._epoch gr.load('tests/CEDA00USA_R_20182100000_23H_15S_MO.rnx.gz', use='E', meas='C1C')
344354
```
345-
shows that `np.genfromtxt()` is consuming about 30% of processing time, and `xarray.concat` and xarray.Dataset` nested inside `concat` takes over 60% of time.
346-
347355

356+
shows that `np.genfromtxt()` is consuming about 30% of processing time, and `xarray.concat` and xarray.Dataset` nested inside `concat` takes over 60% of time.
348357
349358
## Notes
350359

351360
* RINEX 3.03 specification: ftp://igs.org/pub/data/format/rinex303.pdf
352361
* RINEX 3.04 specification (Dec 2018): ftp://igs.org/pub/data/format/rinex304.pdf
353362
* RINEX 3.04 release notes: ftp://igs.org/pub/data/format/rinex304-release-notes.pdf
354363

355-
356-
- GPS satellite position is given for each time in the NAV file as
357-
Keplerian parameters, which can be
358-
[converted to ECEF](https://ascelibrary.org/doi/pdf/10.1061/9780784411506.ap03).
359-
- <https://downloads.rene-schwarz.com/download/M001-Keplerian_Orbit_Elements_to_Cartesian_State_Vectors.pdf>
360-
- <http://www.gage.es/gFD>
364+
* GPS satellite position is given for each time in the NAV file as Keplerian parameters, which can be [converted to ECEF](https://ascelibrary.org/doi/pdf/10.1061/9780784411506.ap03).
365+
* https://downloads.rene-schwarz.com/download/M001-Keplerian_Orbit_Elements_to_Cartesian_State_Vectors.pdf
366+
* http://www.gage.es/gFD
361367

362368
### Number of SVs visible
369+
363370
With the GNSS constellations in 2018, per the
364371
[Trimble Planner](https://www.gnssplanning.com/)
365372
the min/max visible SV would be about:
366373

367374
* Maximum: ~60 SV maximum near the equator in Asia / Oceania with 5 degree elev. cutoff
368375
* Minimum: ~6 SV minimum at poles with 20 degree elev. cutoff and GPS only
369376

370-
371377
### RINEX OBS reader algorithm
372378

373-
1. read overall OBS header (so we know what to expect in the rest of the OBS file)
374-
2. fill the xarray.Dataset with the data by reading in blocks --
375-
another key difference from other programs out there, instead of
376-
reading character by character, I ingest a whole time step of text
377-
at once, helping keep the processing closer to CPU cache making it
378-
much faster.
379+
1. read overall OBS header (so we know what to expect in the rest of the OBS file)
380+
2. fill the xarray.Dataset with the data by reading in blocks -- another key difference from other programs out there, instead of reading character by character, I ingest a whole time step of text at once, helping keep the processing closer to CPU cache making it much faster.
379381

380382
### Data
381383

@@ -397,11 +399,10 @@ UNAVCO RINEX 2 data:
397399
* OBS: ftp://data-out.unavco.org/pub/rinex/obs/
398400
* NAV: ftp://data-out.unavco.org/pub/rinex/nav/
399401

400-
401402
### Hatanaka compressed RINEX .crx
403+
402404
The compressed Hatanaka `.crx` or `.crx.gz` files are supported seamlessly via `crx2rnx`.
403405
These are distinct from the supported `.rnx`, `.gz`, or `.zip` RINEX files.
404406

405-
Hatanaka, Y. (2008), A Compression Format and Tools for GNSS Observation
406-
Data, Bulletin of the Geospatioal Information Authority of Japan, 55, 21-30.
407+
Hatanaka, Y. (2008), A Compression Format and Tools for GNSS Observation Data, Bulletin of the Geospatioal Information Authority of Japan, 55, 21-30.
407408
(available at http://www.gsi.go.jp/ENGLISH/Bulletin55.html)

setup.cfg

Lines changed: 1 addition & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
[metadata]
22
name = georinex
3-
version = 1.16.0
3+
version = 1.16.1
44
author = Michael Hirsch, Ph.D.
55
author_email = [email protected]
66
description = Python RINEX 2/3 NAV/OBS reader with speed and simplicity.
@@ -39,14 +39,6 @@ package_dir=
3939
[options.packages.find]
4040
where=src
4141

42-
[options.entry_points]
43-
console_scripts =
44-
georinex_read = georinex.__main__:georinex_read
45-
georinex_time = georinex.__main__:georinex_time
46-
georinex_plot = georinex.__main__:georinex_plot
47-
georinex_loc = georinex.__main__:georinex_loc
48-
rinex2hdf5 = georinex.__main__:rinex2hdf5
49-
5042
[options.extras_require]
5143
tests =
5244
pytest>=3.9

0 commit comments

Comments
 (0)