Skip to content

Commit 10f98d2

Browse files
RemingtonRohelRemingtonRohel
authored
Merge pull request #28 from SuperDARNCanada/develop
Release: v0.5.0
2 parents 5096906 + febae48 commit 10f98d2

24 files changed

+1725
-333
lines changed

‎.gitignore‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
# C extensions
2+
*.so
3+
4+
# Byte-compiled / optimized / DLL files
5+
__pycache__/
6+
7+
# Built files
8+
target/
9+

‎CONTRIBUTING.md‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
# Developer Guidelines
2+
3+
Any contributions are welcome! Below is a brief description of the project structure.
4+
5+
## Structure
6+
7+
### `src/record.rs`
8+
This file contains the `Record` trait, which defines a set of functions that specific DMAP formats must implement.
9+
For example, `read_file(infile: &PathBuf) -> Result<Vec<Self>, DmapError>` is defined in the `Record` trait, and handles
10+
reading in records from a file at the specified path. This function is generic, in that it doesn't know what type of records
11+
(RAWACF, FITACF, etc.) are expected. Also, since it is a trait function, you can only use it through a struct which implements
12+
the trait. For example, the `FitacfRecord` struct defined in `src/formats/fitacf.rs` implements the `Record` trait, and so
13+
you can call `FitacfRecord::read_file(...)` to read a FITACF file, but you couldn't invoke `Record::read_file(...)`.
14+
15+
### `src/types.rs`
16+
This file defines necessary structs and enums for encapsulating basic types (`i8`, `u32`, `String`, etc.) into
17+
objects like `DmapField`, `DmapScalar`, `DmapVec`, etc. that abstract over the supported underlying types.
18+
For instance, when reading scalar from a DMAP file, the underlying data type is inferred from the `type` field in the
19+
scalar's metadata, so it can't be known beforehand. This requires some encapsulating type, `DmapScalar` in this case,
20+
which contains the metadata of the field and has a known size for the stack memory.
21+
22+
This file defines the `Fields` struct, which is used to hold the names and types of the required and optional
23+
scalar and vector fields for a type of DMAP record (RAWACF, FITACF, etc.).
24+
25+
This file defines the `DmapType` trait and implements it for supported data types that can be in DMAP records, namely
26+
`u8`, `u16`, `u32`, `u64`, `i8`, `i16`, `i32`, `i64`, `f32`, `f64`, and `String`. The implementation of the trait for
27+
these types only means that other types, e.g. `i128`, cannot be stored in DMAP records.
28+
29+
Lastly, functions for parsing scalars and vectors from a byte buffer are defined in this file.
30+
31+
### `src/formats`
32+
This directory holds the files that define the DMAP record formats: IQDAT, RAWACF, FITACF, GRID, MAP, SND, and the generic DMAP.
33+
If you are defining a new DMAP format, you will need to make a new file in this directory following the structure of the
34+
existing files. Essentially, you define the scalar and vector fields, both required and optional, and the groups of vector
35+
fields which must have identical dimensions, then call a macro to autogenerate the struct code for you.
36+
37+
### `src/compression.rs`
38+
This file contains the automatic bz2 detection function. If more compression types are to be supported, this is where you would
39+
put them.
40+
41+
### `src/io.rs`
42+
This file contains the function `bytes_to_file`, which handles writing a `Vec<u8>` to `AsRef<Path>`. If the path
43+
ends in `.bz2`, the function will compress first with bz2.
44+
45+
### `tests`
46+
In `tests.rs`, integration tests for reading and writing all file types are present. Small example files
47+
are contained in `tests/test_files`.
48+
49+
### `benches/io_benchmarking.rs`
50+
This file contains benchmarking functions for checking the performance of the basic read functions.

‎COPYING‎

Lines changed: 165 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,165 @@
1+
GNU LESSER GENERAL PUBLIC LICENSE
2+
Version 3, 29 June 2007
3+
4+
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
5+
Everyone is permitted to copy and distribute verbatim copies
6+
of this license document, but changing it is not allowed.
7+
8+
9+
This version of the GNU Lesser General Public License incorporates
10+
the terms and conditions of version 3 of the GNU General Public
11+
License, supplemented by the additional permissions listed below.
12+
13+
0. Additional Definitions.
14+
15+
As used herein, "this License" refers to version 3 of the GNU Lesser
16+
General Public License, and the "GNU GPL" refers to version 3 of the GNU
17+
General Public License.
18+
19+
"The Library" refers to a covered work governed by this License,
20+
other than an Application or a Combined Work as defined below.
21+
22+
An "Application" is any work that makes use of an interface provided
23+
by the Library, but which is not otherwise based on the Library.
24+
Defining a subclass of a class defined by the Library is deemed a mode
25+
of using an interface provided by the Library.
26+
27+
A "Combined Work" is a work produced by combining or linking an
28+
Application with the Library. The particular version of the Library
29+
with which the Combined Work was made is also called the "Linked
30+
Version".
31+
32+
The "Minimal Corresponding Source" for a Combined Work means the
33+
Corresponding Source for the Combined Work, excluding any source code
34+
for portions of the Combined Work that, considered in isolation, are
35+
based on the Application, and not on the Linked Version.
36+
37+
The "Corresponding Application Code" for a Combined Work means the
38+
object code and/or source code for the Application, including any data
39+
and utility programs needed for reproducing the Combined Work from the
40+
Application, but excluding the System Libraries of the Combined Work.
41+
42+
1. Exception to Section 3 of the GNU GPL.
43+
44+
You may convey a covered work under sections 3 and 4 of this License
45+
without being bound by section 3 of the GNU GPL.
46+
47+
2. Conveying Modified Versions.
48+
49+
If you modify a copy of the Library, and, in your modifications, a
50+
facility refers to a function or data to be supplied by an Application
51+
that uses the facility (other than as an argument passed when the
52+
facility is invoked), then you may convey a copy of the modified
53+
version:
54+
55+
a) under this License, provided that you make a good faith effort to
56+
ensure that, in the event an Application does not supply the
57+
function or data, the facility still operates, and performs
58+
whatever part of its purpose remains meaningful, or
59+
60+
b) under the GNU GPL, with none of the additional permissions of
61+
this License applicable to that copy.
62+
63+
3. Object Code Incorporating Material from Library Header Files.
64+
65+
The object code form of an Application may incorporate material from
66+
a header file that is part of the Library. You may convey such object
67+
code under terms of your choice, provided that, if the incorporated
68+
material is not limited to numerical parameters, data structure
69+
layouts and accessors, or small macros, inline functions and templates
70+
(ten or fewer lines in length), you do both of the following:
71+
72+
a) Give prominent notice with each copy of the object code that the
73+
Library is used in it and that the Library and its use are
74+
covered by this License.
75+
76+
b) Accompany the object code with a copy of the GNU GPL and this license
77+
document.
78+
79+
4. Combined Works.
80+
81+
You may convey a Combined Work under terms of your choice that,
82+
taken together, effectively do not restrict modification of the
83+
portions of the Library contained in the Combined Work and reverse
84+
engineering for debugging such modifications, if you also do each of
85+
the following:
86+
87+
a) Give prominent notice with each copy of the Combined Work that
88+
the Library is used in it and that the Library and its use are
89+
covered by this License.
90+
91+
b) Accompany the Combined Work with a copy of the GNU GPL and this license
92+
document.
93+
94+
c) For a Combined Work that displays copyright notices during
95+
execution, include the copyright notice for the Library among
96+
these notices, as well as a reference directing the user to the
97+
copies of the GNU GPL and this license document.
98+
99+
d) Do one of the following:
100+
101+
0) Convey the Minimal Corresponding Source under the terms of this
102+
License, and the Corresponding Application Code in a form
103+
suitable for, and under terms that permit, the user to
104+
recombine or relink the Application with a modified version of
105+
the Linked Version to produce a modified Combined Work, in the
106+
manner specified by section 6 of the GNU GPL for conveying
107+
Corresponding Source.
108+
109+
1) Use a suitable shared library mechanism for linking with the
110+
Library. A suitable mechanism is one that (a) uses at run time
111+
a copy of the Library already present on the user's computer
112+
system, and (b) will operate properly with a modified version
113+
of the Library that is interface-compatible with the Linked
114+
Version.
115+
116+
e) Provide Installation Information, but only if you would otherwise
117+
be required to provide such information under section 6 of the
118+
GNU GPL, and only to the extent that such information is
119+
necessary to install and execute a modified version of the
120+
Combined Work produced by recombining or relinking the
121+
Application with a modified version of the Linked Version. (If
122+
you use option 4d0, the Installation Information must accompany
123+
the Minimal Corresponding Source and Corresponding Application
124+
Code. If you use option 4d1, you must provide the Installation
125+
Information in the manner specified by section 6 of the GNU GPL
126+
for conveying Corresponding Source.)
127+
128+
5. Combined Libraries.
129+
130+
You may place library facilities that are a work based on the
131+
Library side by side in a single library together with other library
132+
facilities that are not Applications and are not covered by this
133+
License, and convey such a combined library under terms of your
134+
choice, if you do both of the following:
135+
136+
a) Accompany the combined library with a copy of the same work based
137+
on the Library, uncombined with any other library facilities,
138+
conveyed under the terms of this License.
139+
140+
b) Give prominent notice with the combined library that part of it
141+
is a work based on the Library, and explaining where to find the
142+
accompanying uncombined form of the same work.
143+
144+
6. Revised Versions of the GNU Lesser General Public License.
145+
146+
The Free Software Foundation may publish revised and/or new versions
147+
of the GNU Lesser General Public License from time to time. Such new
148+
versions will be similar in spirit to the present version, but may
149+
differ in detail to address new problems or concerns.
150+
151+
Each version is given a distinguishing version number. If the
152+
Library as you received it specifies that a certain numbered version
153+
of the GNU Lesser General Public License "or any later version"
154+
applies to it, you have the option of following the terms and
155+
conditions either of that published version or of any later version
156+
published by the Free Software Foundation. If the Library as you
157+
received it does not specify a version number of the GNU Lesser
158+
General Public License, you may choose any version of the GNU Lesser
159+
General Public License ever published by the Free Software Foundation.
160+
161+
If the Library as you received it specifies that a proxy can decide
162+
whether future versions of the GNU Lesser General Public License shall
163+
apply, that proxy's public statement of acceptance of any version is
164+
permanent authorization for you to choose that version for the
165+
Library.

‎Cargo.toml‎

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,14 @@
11
[package]
2-
name = "dmap"
3-
version = "0.4.0"
2+
name = "darn-dmap"
3+
version = "0.5.0"
44
edition = "2021"
55
rust-version = "1.63.0"
6+
authors = ["Remington Rohel"]
7+
description = "SuperDARN DMAP file format I/O"
8+
repository = "https://github.com/SuperDARNCanada/dmap"
9+
license = "LGPL-3.0-or-later"
10+
keywords = ["SuperDARN", "dmap", "I/O"]
11+
categories = ["parser-implementations", "science"]
612

713
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
814

@@ -13,8 +19,8 @@ name = "dmap"
1319
crate-type = ["cdylib", "rlib"]
1420

1521
[dependencies]
16-
pyo3 = { version = "0.22.5", features = ["extension-module", "indexmap", "abi3-py38"] }
17-
numpy = "0.22.0"
22+
pyo3 = { version = "0.26.0", features = ["extension-module", "indexmap", "abi3-py38"] }
23+
numpy = "0.26.0"
1824
indexmap = "2.3.0"
1925
itertools = "0.13.0"
2026
rayon = "1.10.0"

‎README.md‎

Lines changed: 21 additions & 40 deletions
Original file line numberDiff line numberDiff line change
@@ -1,49 +1,30 @@
1-
# Dmap
1+
A library for SuperDARN DMAP file I/O
2+
=====================================
23

3-
Rust tools for SuperDARN DMAP file format operations.
4+
[![github]](https://github.com/SuperDARNCanada/dmap)&ensp;[![crates-io]](https://crates.io/crates/darn-dmap)&ensp;[![docs-rs]](crate)
5+
6+
[github]: https://img.shields.io/badge/github-8da0cb?style=for-the-badge&labelColor=555555&logo=github
7+
[crates-io]: https://img.shields.io/badge/crates.io-fc8d62?style=for-the-badge&labelColor=555555&logo=rust
8+
[docs-rs]: https://img.shields.io/badge/docs.rs-66c2a5?style=for-the-badge&labelColor=555555&logo=docs.rs
49

510
This project exposes both Rust and Python APIs for handling DMAP I/O.
6-
I/O can be conducted either on byte buffers, or directly to/from files.
11+
I/O can be conducted either directly to/from files or byte buffers.
712

813
The SuperDARN DMAP file formats are all supported (IQDAT, RAWACF, FITACF, GRID, MAP, and SND)
9-
as well as a generic DMAP format that is unaware of any required fields or types
10-
(e.g. char, int32) for any fields.
14+
as well as a generic DMAP format that is unaware of any required fields or types (e.g. char, int32) for any fields.
1115

12-
## Developer Guidelines
16+
## Installation
1317

14-
### `src/record.rs`
15-
This file contains the `Record` trait, which defines a set of functions that specific DMAP formats must implement.
16-
For example, `read_file(infile: &PathBuf) -> Result<Vec<Self>, DmapError>` is defined in the `Record` trait, and handles
17-
reading in records from a file at the specified path. This function is generic, in that it doesn't know what type of records
18-
(RAWACF, FITACF, etc.) are expected. Also, since it is a trait function, you can only use it through a struct which implements
19-
the trait. For example, the `FitacfRecord` struct defined in `src/formats/fitacf.rs` implements the `Record` trait, and so
20-
you can call `FitacfRecord::read_file(...)` to read a FITACF file, but you couldn't invoke `Record::read_file(...)`.
18+
### Rust
19+
1. Add the crate to your dependencies in your `Cargo.toml` file
20+
2. Add `use dmap::*;` to your imports.
2121

22-
### `src/types.rs`
23-
This file defines necessary structs and enums for encapsulating basic types (`i8`, `u32`, `String`, etc.) into
24-
objects like `DmapField`, `DmapScalar`, `DmapVec`, etc. that abstract over the supported underlying types.
25-
For instance, when reading scalar from a DMAP file, the underlying data type is inferred from the `type` field in the
26-
scalar's metadata, so it can't be known beforehand. This requires some encapsulating type, `DmapScalar` in this case,
27-
which contains the metadata of the field and has a known size for the stack memory.
22+
### Python
23+
This package is registered on PyPI as `darn-dmap`, you can install the package with your package manager.
2824

29-
This file defines the `Fields` struct, which is used to hold the names and types of the required and optional
30-
scalar and vector fields for a type of DMAP record (RAWACF, FITACF, etc.).
31-
32-
This file defines the `DmapType` trait and implements it for supported data types that can be in DMAP records, namely
33-
`u8`, `u16`, `u32`, `u64`, `i8`, `i16`, `i32`, `i64`, `f32`, `f64`, and `String`. The implementation of the trait for
34-
these types only means that other types, e.g. `i128`, cannot be stored in DMAP records.
35-
36-
Lastly, functions for parsing scalars and vectors from a byte buffer are defined in this file.
37-
38-
### `src/formats`
39-
This directory holds the files that define the DMAP record formats: IQDAT, RAWACF, FITACF, GRID, MAP, SND, and the generic DMAP.
40-
If you are defining a new DMAP format, you will need to make a new file in this directory following the structure of the
41-
existing files. Essentially, you define the scalar and vector fields, both required and optional, and the groups of vector
42-
fields which must have identical dimensions, then call a macro to autogenerate the struct code for you.
43-
44-
### `tests`
45-
In `tests.rs`, integration tests for reading and writing all file types are present. Small example files
46-
are contained in `tests/test_files`.
47-
48-
### `benches/io_benchmarking.rs`
49-
This file contains benchmarking functions for checking the performance of the basic read functions.
25+
### From source
26+
If you want to build from source, you first need to have Rust installed on your machine. Then:
27+
1. Clone the repository: `git clone https://github.com/SuperDARNCanada/dmap`
28+
2. Run `cargo build` in the repository directory
29+
3. If wanting to install the Python API, create a virtual environment and source it, then install `maturin`
30+
4. In the project directory, run `maturin develop` to build and install the Python bindings. This will make a wheel file based on your operating system and architecture that you can install directly on any compatible machine.

‎pyproject.toml‎

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ build-backend = "maturin"
44

55
[project]
66
name = "darn-dmap"
7-
version = "0.4.0"
7+
version = "0.5.0"
88
requires-python = ">=3.8"
99
authors = [
1010
{ name = "Remington Rohel" }
@@ -13,11 +13,25 @@ classifiers = [
1313
"Programming Language :: Python",
1414
"Programming Language :: Rust"
1515
]
16+
description = "SuperDARN DMAP file format I/O"
17+
readme = "python/README.md"
1618
dependencies = ["numpy<3"]
19+
license = "LGPL-3.0-or-later"
20+
21+
[project.urls]
22+
Repository = "https://github.com/SuperDARNCanada/dmap"
1723

1824
[tool.maturin]
25+
python-source = "python"
26+
module-name = "dmap.dmap_rs"
1927
bindings = "pyo3"
2028
profile = "release"
2129
compatibility = "manylinux2014"
2230
auditwheel = "repair"
2331
strip = true
32+
33+
[project.optional-dependencies]
34+
dev = [
35+
"pytest",
36+
"ruff",
37+
]

0 commit comments

Comments
 (0)