Skip to content

Commit 647d2bc

Browse files
Merge pull request #265 from NCAS-CMS/polish_README
polish README
2 parents 49f5ba8 + 5f83704 commit 647d2bc

File tree

1 file changed

+17
-7
lines changed

1 file changed

+17
-7
lines changed

README.md

Lines changed: 17 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -35,25 +35,31 @@ Python versions supported: 3.10, 3.11, 3.12, 3.13. Fully compatible with `numpy
3535

3636
This package provides
3737

38-
1. the class `Active`, which is a shimmy to NetCDF4 (and HDF5) storage via kerchunk metadata and the zarr indexer. It does not however, use zarr for the actual read.
38+
1. the class `Active`, which is a shimmy to NetCDF4 (and HDF5) via a [`Pyfive.File`](https://github.com/NCAS-CMS/pyfive) file object
3939
2. The actual reads are done in the methods of `storage.py` or `reductionist.py`, which are called from within an `Active.__getitem__`.
4040

41-
Example usage is in the file `tests/test_harness.py`, but it's basically this simple:
41+
Example usage is in the test files, depending on the case:
42+
43+
- [`tests/test_harness.py`](https://github.com/NCAS-CMS/PyActiveStorage/blob/main/tests/test_harness.py)
44+
- [`test_real_s3.py`](https://github.com/NCAS-CMS/PyActiveStorage/blob/main/tests/test_real_s3.py)
45+
- [`test_real_https.py`](https://github.com/NCAS-CMS/PyActiveStorage/blob/main/tests/test_real_https.py)
46+
47+
but it's basically this simple:
4248

4349
```python
44-
active = Active(self.testfile, "data")
45-
active.method = "mean"
46-
result = active[0:2, 4:6, 7:9]
50+
active = Active(file.Path | Pyfive.Dataset, ncvar="some_var")
51+
active._version = 2
52+
result = active.mean[0:2, 4:6, 7:9]
4753
```
4854

49-
where `result` will be the mean of the appropriate slice of the hyperslab in `var`.
55+
where `result` will be the mean of the appropriate slice of the hyperslab in `some_var` variable data.
5056

5157
There are some (relatively obsolete) documents from our exploration of zarr internals in the docs4understanding, but they are not germane to the usage of the Active class.
5258

5359
## Storage types
5460

5561
PyActiveStorage is designed to interact with various storage backends.
56-
The storage backend is specified using the `storage_type` argument to `Active` constructor.
62+
The storage backend is automatically detected, but can still be specified using the `storage_type` argument to the `Active` constructor.
5763
There are two main integration points for a storage backend:
5864

5965
#. Load netCDF metadata
@@ -77,6 +83,10 @@ To load metadata, netCDF files are opened using `s3fs`, with `h5netcdf` used to
7783
Chunk reductions are implemented in `activestorage.reductionist`, with each operation resulting in an API request to the Reductionist server.
7884
From there on, `Active` works as per normal.
7985

86+
### HTTPS-compatible on an NGINX server
87+
88+
The same infrastructure as for S3, but the file is passed in as an `https` URI.
89+
8090
## Testing overview
8191

8292
We have written unit and integration tests, and employ a coverage measurement tool - Codecov, see PyActiveStorage [test coverage](https://app.codecov.io/gh/NCAS-CMS/PyActiveStorage) with current coverage of 87%; our Continuous Integration (CI) testing is deployed on [Github Actions](https://github.com/NCAS-CMS/PyActiveStorage/actions), and we have nightly tests that run the entire testing suite, to be able to detect any issues introduced by updated versions of our dependencies. Github Actions (GA) tests also test the integration of various storage types we currently support; as such, we have dedicated tests that test Active Storage with S3 storage (by creating and running a MinIO client from within the test, and deploying and testing PyActiveStorage with data shipped to the S3 client).

0 commit comments

Comments
 (0)