diff --git a/.readthedocs.yml b/.readthedocs.yml index 6b3674a7..08bc28ac 100644 --- a/.readthedocs.yml +++ b/.readthedocs.yml @@ -8,8 +8,7 @@ build: # Content here largely copied from Iris # see : https://github.com/SciTools/iris/pull/4855 post_checkout: - # The SciTools/iris repository is shallow i.e., has a .git/shallow, - # therefore complete the repository with a full history in order + # In case the repository is shallow, complete with a full history in order # to allow setuptools-scm to correctly auto-discover the version. - git fetch --unshallow - git fetch --all diff --git a/README.md b/README.md index fd8c8598..236571f6 100644 --- a/README.md +++ b/README.md @@ -4,6 +4,10 @@ Generic NetCDF data in Python. +Represents raw netCDF data in Python, with no structural assumptions or interpretations, +and provides facilities to inspect and manipulate it with complete freedom. + +### Also Provides fast data exchange between analysis packages, and full control of storage formatting. @@ -70,7 +74,7 @@ enable_lockshare(iris=True) ncdata = from_nc4(input_path) for var in ncdata.variables.values(): if "coords" in var.attributes: - var.attributes.rename("coords", "coordinates") + var.attributes.rename("coords", "coordinates") cubes = to_iris(ncdata) ``` @@ -93,20 +97,21 @@ ncdata = from_nc4("file1.nc") keys = ["air_", "surface"] # Explicitly add dimension names, to include all the dimension variables -keys += + list(ncdata.dimensions) +keys += list(ncdata.dimensions) # Identify the wanted variables select_vars = [ var for var in ncdata.variables.values() - if any(key in var.name for key in keys) + if any(var.name.startswith(key) for key in keys) ] # Add any referenced coordinate variables -for var in list(select_vars): - var = ncdata.variables[varname] - for coordname in var.attributes.get("coordinates", "").split(" "): - select_vars.append(ncdata.variables[coordname]) +for var in select_vars: + coordnames = var.avals.get("coordinates") + if coordnames: + for coordname in coordnames.split(" "): + select_vars.append(ncdata.variables[coordname]) # Replace variables with only the wanted ones ncdata.variables.clear() @@ -115,11 +120,3 @@ ncdata.variables.addall(select_vars) # Save to_nc4(ncdata, "pruned.nc") ``` - - -# Older References in Iris - * Iris issue : https://github.com/SciTools/iris/issues/4994 - * planning presentation : https://github.com/SciTools/iris/files/10499677/Xarray-Iris.bridge.proposal.--.NcData.pdf - * in-Iris code workings : https://github.com/pp-mo/iris/pull/75 - - diff --git a/docs/Makefile b/docs/Makefile index bb4252cf..a5c0035c 100644 --- a/docs/Makefile +++ b/docs/Makefile @@ -19,7 +19,7 @@ allapi: sphinx-apidoc -Mfe -o ./details/api ../lib/ncdata towncrier: - towncrier build --yes + if [ -e changelog_fragments/*.rst ]; then towncrier build --yes; fi # Tweaked "make html", which restores the changelog state after docs build. diff --git a/docs/change_log.rst b/docs/change_log.rst index f193f2a7..dd3b9ca2 100644 --- a/docs/change_log.rst +++ b/docs/change_log.rst @@ -11,7 +11,7 @@ We intend to follow `PEP 440 `_, or (older) `SemVer `_ versioning principles. This means the version string has the basic form **"major.minor.bugfix[special-types]"**. -Current release version is at **"v0.2"**. +Current release version is at : |version| This is a complete implementation, with functional operational of all public APIs. The code is however still experimental, and APIs are not stable @@ -25,6 +25,55 @@ Summary of key features by release number. .. towncrier release notes start +v0.3.0 +~~~~~~ +Added handy utilities; made attribute access easier; reworked documentation. + +Features +^^^^^^^^ + +- Added the ability to extract a sub-region by indexing/slicing over dimensions. + The :class:`ncdata.NcData` objects can be indexed with the ``[]`` operation, or over + specifed dimensions with the :meth:`~ncdata.NcData.slicer` method. + This is based on the new :meth:`~ncdata.utils.index_by_dimensions()` utility method + and :class:`~ncdata.utils.Slicer` class. + See: :ref:`utils_indexing` (`ISSUE#68 `_) +- Added the :func:`~ncdata.utils.rename_dimension` utility. + This provides a "safe" dimension rename, which also replaces + the name in all variables which use it. (`ISSUE#87 `_) +- Added the ".avals" property as an easier way of managing attributes: + This provides a simple "name: value" map, bypassing the NcAttribute objects and converting values to and from simple Python equivalents. + This effectively replaces the older 'set_attrval' and 'get_attrval', which will eventually be removed. + See: :ref:`attributes_and_avals` (`ISSUE#117 `_) +- Make :meth:`~ncdata.iris.to_iris` use the full iris load processing, + instead of :meth:`iris.fileformats.netcdf.loader.load_cubes`. + This means you can use load controls such as callbacks and constraints. (`ISSUE#131 `_) +- Provide exact == and != for datasets and variables, by just calling the difference utilities. + This can be inefficient, but is simple to understand and generally useful. + See: :ref:`equality_testing` (`ISSUE#166 `_) + + +Documentation changes +^^^^^^^^^^^^^^^^^^^^^ + +- Added a `userguide page `_ summarising all the utility features in :mod:`ncdata.utils`. (`ISSUE#161 `_) +- Made all docs examples into doctests; add doctest CI action. (`ISSUE#136 `_) + + +Bug Fixes +^^^^^^^^^ + +- Fixed a bug in dataset comparison, where variables with missing or unbroadcastable data arrays could cause errors rather than generating difference messages. (`ISSUE#153 `_) + + +Developer and Internal changes +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +- Switch to towncrier for whats-new management. (`ISSUE#116 `_) +- Added regular linkcheck gha. (`ISSUE#123 `_) +- @valeriupredoi added test for Zarr conversion to Iris cubes. (`ISSUE#145 `_) + + v0.2.0 ~~~~~~ Overhauled data manipulation APIs. Expanded and improved documentation. diff --git a/docs/changelog_fragments/116.dev.txt b/docs/changelog_fragments/116.dev.txt deleted file mode 100644 index f3bda00a..00000000 --- a/docs/changelog_fragments/116.dev.txt +++ /dev/null @@ -1 +0,0 @@ -Switch to towncrier for whats-new management. diff --git a/docs/changelog_fragments/117.feat.rst b/docs/changelog_fragments/117.feat.rst deleted file mode 100644 index a58d2d32..00000000 --- a/docs/changelog_fragments/117.feat.rst +++ /dev/null @@ -1,3 +0,0 @@ -Added the ".avals" property as an easier way of managing attributes: -This provides a simple "name: value" map, bypassing the NcAttribute objects and converting values to and from simple Python equivalents. -This effectively replaces the older 'set_attrval' and 'get_attrval', which will eventually be removed. diff --git a/docs/changelog_fragments/123.dev.rst b/docs/changelog_fragments/123.dev.rst deleted file mode 100644 index 053d2cd9..00000000 --- a/docs/changelog_fragments/123.dev.rst +++ /dev/null @@ -1 +0,0 @@ -Added regular linkcheck gha. diff --git a/docs/changelog_fragments/131.feat.rst b/docs/changelog_fragments/131.feat.rst deleted file mode 100644 index 1fa54a2b..00000000 --- a/docs/changelog_fragments/131.feat.rst +++ /dev/null @@ -1,3 +0,0 @@ -Make :meth:`~ncdata.iris.to_iris` use the full iris load processing, -instead of :meth:`iris.fileformats.netcdf.loader.load_cubes`. -This means you can use load controls such as callbacks and constraints. diff --git a/docs/changelog_fragments/136.dev.rst b/docs/changelog_fragments/136.dev.rst deleted file mode 100644 index ed285cac..00000000 --- a/docs/changelog_fragments/136.dev.rst +++ /dev/null @@ -1 +0,0 @@ -Made all docs examples into doctests; add doctest CI action. diff --git a/docs/changelog_fragments/145.dev.rst b/docs/changelog_fragments/145.dev.rst deleted file mode 100644 index 02433cbb..00000000 --- a/docs/changelog_fragments/145.dev.rst +++ /dev/null @@ -1 +0,0 @@ -@valeriupredoi added test for Zarr conversion to Iris cubes. diff --git a/docs/changelog_fragments/153.bug.rst b/docs/changelog_fragments/153.bug.rst deleted file mode 100644 index 29bb28a6..00000000 --- a/docs/changelog_fragments/153.bug.rst +++ /dev/null @@ -1 +0,0 @@ -Fixed a bug in dataset comparison, where variables with missing or unbroadcastable data arrays could cause errors rather than generating difference messages. diff --git a/docs/changelog_fragments/161.doc.rst b/docs/changelog_fragments/161.doc.rst deleted file mode 100644 index 3579c040..00000000 --- a/docs/changelog_fragments/161.doc.rst +++ /dev/null @@ -1 +0,0 @@ -Added a `userguide page `_ summarising all the utility features in :mod:`ncdata.utils`. diff --git a/docs/changelog_fragments/166.feat.rst b/docs/changelog_fragments/166.feat.rst deleted file mode 100644 index d80c79d7..00000000 --- a/docs/changelog_fragments/166.feat.rst +++ /dev/null @@ -1,3 +0,0 @@ -Provide exact == and != for datasets and variables, by just calling the difference utilities. -This can be inefficient, but is simple to understand and generally useful. -See: :ref:`equality_testing` diff --git a/docs/changelog_fragments/68.feat.rst b/docs/changelog_fragments/68.feat.rst deleted file mode 100644 index 1d8fed08..00000000 --- a/docs/changelog_fragments/68.feat.rst +++ /dev/null @@ -1,6 +0,0 @@ -Added the ability to extract a sub-region by indexing/slicing over dimensions. -The :class:`ncdata.NcData` objects can be indexed with the ``[]`` operation, or over -specifed dimensions with the :meth:`~ncdata.NcData.slicer` method. -This is based on the new :meth:`~ncdata.utils.index_by_dimensions()` utility method -and :class:`~ncdata.utils.Slicer` class. -See: :ref:`utils_indexing` \ No newline at end of file diff --git a/docs/changelog_fragments/87.feat.rst b/docs/changelog_fragments/87.feat.rst deleted file mode 100644 index 2d6149cd..00000000 --- a/docs/changelog_fragments/87.feat.rst +++ /dev/null @@ -1,3 +0,0 @@ -Added the :func:`~ncdata.utils.rename_dimension` utility. -This provides a "safe" dimension rename, which also replaces -the name in all variables which use it.