v0.7.0

Features

• Compression with bz2 possible even when returning bytes objects and not writing direct to file.
• bz2 input required for write functions as a bool, supersedes file extension for flagging bzip2 compression (i.e. even if file doesn't end in .bz2, if the input is true it will still compress the data before writing).

v0.6.0

Features

• New reading mode which only reads metadata fields from files. Use with mode="metadata" in Python functions, or Record::read_file_metadata() function in Rust.
• New benchmarks for reading bzipped files and reading metadata functions.

v0.5.0

This release moves to a mixed Python/Rust project, featuring a more user-friendly Python API with integration tests on the API. It also bumps pyo3 to 0.26.0, which adds support up to Python 3.14. Also, the Rust API has been changed slightly, removing the convenience functions like write_fitacf and instead using write_to_file from the Record trait.

Python API

The functions below show the generic read/write ability of this package. Additionally, there are specific functions for each SuperDARN file type (iqdat, rawacf, fitacf, grid, map, and snd). These are structured identically, but conduct more tests to determine that the data is compliant with the specific file format (i.e. all required fields are present, no additional fields are present, fields have the expected data type, etc.).

def read_dmap(
    source: Union[str, bytes], mode: str = "lax"
) -> Union[dict, list[dict], tuple[list[dict], Optional[int]]]:
    """
    Reads in DMAP data from `source`.

    Parameters
    ----------
    source: Union[str, bytes]
        Where to read data from. If input is of type `str`, this is interpreted as the path to a file.
        If input is of type `bytes`, this is interpreted as the raw data itself.
    mode: str
        Mode in which to read the data, either "lax" (default), "strict", or "sniff".
        In "lax" mode, all valid records will be returned in a tuple along with the byte index of `source` where the
        corruption starts.
        In "strict" mode, any corruption in the data will raise an error.
        In "sniff" mode, `source` must be a path, and only the first record will be read.

    Returns
    -------
    If `mode` is `lax`, returns `tuple[list[dict], Optional[int]]`, where the first element is the records which were parsed,
    and the second is the byte index where `source` was no longer a valid record of type `fmt`.
    If `mode` is `strict`, returns `list[dict]` which is the parsed records.
    If `mode` is `sniff`, returns `dict`, which is the first record.
    """

def write_dmap(
    source: list[dict], outfile: Union[None, str] = None
) -> Union[None, bytes]:
    """
    Writes DMAP data from `source` to either a `bytes` object or to `outfile`.

    Parameters
    ----------
    source: list[dict]
        list of DMAP records as dictionaries.
    outfile: Union[None, str]
        If `None`, returns the data as a `bytes` object. If this is a string, then this is interpreted as a path
        and data will be written to the filesystem. If the file ends in the `.bz2` extension, the data will be
        compressed using bzip2.
    """

Rust API

This can be seen by cloning the repo and running cargo doc to generate the HTML documentation. You can search for the darn-dmap crate on https://docs.rs/.

v0.4.0

Features

• Auto-detection of bz2 compression using "BZh" magic signature at the start of the stream
• Easier importing of crate contents, removing any nested modules. E.g. use dmap::FitacfRecord instead of use dmap::formats::fitacf::FitacfRecord
• Significant number of unit tests added.

Release: v0.3.0

Features

File "sniffing", where only the first record is read from disk. Works on both compressed and non-compressed files.

v0.2.1

• Adds new functions read_[type]_bytes_lax that allow for read_[type]_lax-style reading straight from bytes input.

Release: v0.2.0

Creates new lax reading functions per DDWG direction, and exposes I/O functions to/from memory instead of just files.

Features

• New functions read_[type]_lax that don't error on corrupted records. Instead, they return a tuple of the good records, plus the byte corresponding to the start of the first bad record, if it exists.
• New Python API functions read_[type]_bytes, write_[type]_bytes that interface with memory (in this case, Python bytes objects) rather than files.
• Internal refactor using macro_rules to greatly reduce code repetition.
• More integration tests, covering generic and lax I/O functions.

Release: v0.1.6

Adds in changes that were meant for v0.1.5, but were unwittingly removed.

Features

• Support for numpy 2+
• MSRV (minimum supported Rust version) 1.74.1
• Updated the permissions so that files are appended to with the write_[type]() functions.
• Expose new read_[type] functions (e.g., read_rawacf()) in Rust library, for convenience
• Adds more conversion traits to/from DmapScalar, DmapVec, and DmapField for user convenience. Essentially, you can convert the types that go into any of these infallibly (i.e. they implement From<T>), and convert back fallibly (i.e. TryFrom<Dmap[...]> for T is implemented for types within Dmap[...]).
• Better error messages for API functions, detailing only the first error and listing all records that have errors.

Release: v0.1.5

Features

• Support for numpy 2+
• Updated the permissions so that files are appended to with the write_[type]() functions.
• Expose new read_[type] functions (e.g., read_rawacf()) in Rust library, for convenience
• Adds more conversion traits to/from DmapScalar, DmapVec, and DmapField for user convenience. Essentially, you can convert the types that go into any of these infallibly (i.e. they implement From<T>), and convert back fallibly (i.e. TryFrom<Dmap[...]> for T is implemented for types within Dmap[...]).
• Better error messages for API functions, detailing only the first error and listing all records that have errors.

Release: v0.1.4

Patch Release

Brings the optional fields for file types into line with pydarnio v1.3.