Thr3mi L1C reader #2908

pepephillips · 2024-09-24T14:06:07Z

Closes #xxxx
Tests added
Fully documented

…to thr3mi_dev Merge branch 'main' of https://github.com/pytroll/satpy into map_dev To take on board latest changes from satpy main before 3MI reader PR

pepephillips · 2024-09-24T14:10:58Z

i just looked at the commits and there is an old MAP L1C reader commit in there. The MAP L1C readers were since removed and obviously not part of the final commit, only the new 3MI readers are there.

codecov · 2024-09-24T14:13:50Z

Codecov Report

❌ Patch coverage is 95.89744% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 96.32%. Comparing base (ff96b89) to head (3f09755).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
satpy/readers/thr3mi_l1c_nc.py	92.50%	6 Missing ⚠️
satpy/tests/reader_tests/test_thr3mi_l1c_nc.py	98.26%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2908      +/-   ##
==========================================
- Coverage   96.32%   96.32%   -0.01%     
==========================================
  Files         463      465       +2     
  Lines       58172    58367     +195     
==========================================
+ Hits        56035    56222     +187     
- Misses       2137     2145       +8

Flag	Coverage Δ
behaviourtests	`3.63% <0.00%> (-0.02%)`	⬇️
unittests	`96.41% <95.89%> (-0.01%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Merge branch 'main' of https://github.com/pytroll/satpy into thr3mi_dev

Merge branch 'thr3mi_dev' of https://github.com/pepephillips/satpy into thr3mi_dev

ameraner · 2025-02-15T14:37:00Z

pre-commit.ci autofix

for more information, see https://pre-commit.ci

coveralls · 2025-02-15T14:57:41Z

Pull Request Test Coverage Report for Build 17614285017

Details

189 of 197 (95.94%) changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.002%) to 96.392%

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
satpy/tests/reader_tests/test_thr3mi_l1c_nc.py	114	116	98.28%
satpy/readers/thr3mi_l1c_nc.py	75	81	92.59%

Totals
Change from base Build 17593031136:	-0.002%
Covered Lines:	56613
Relevant Lines:	58732

💛 - Coveralls

…erage for exception handling

pepephillips · 2025-02-25T14:38:08Z

@mraspaud, this is a very basic 3mi L1C reader for the native 3mi normalised reflectances in the 1D data format. As a result, scene.show() will not work as the data is still 1D (but I can hopefully update that later). However, once resampled, you can plot the projected data ok using show(). I suspect my coding is too basic and I hope someone can improve on it downstream.... I hope to include conversion to reflectance/radiance and colour composites (including polarised composites) too.

ameraner

Hi Pepe, thanks again for the reader. Two suggestions inline to fix an import warning and a broken docstring link.

ameraner · 2025-09-10T14:40:55Z

satpy/readers/thr3mi_l1c_nc.py

+
+import xarray as xr
+
+from satpy.readers.netcdf_utils import NetCDF4FileHandler


this needs a new import after the recent refactoring:

Suggested change

from satpy.readers.netcdf_utils import NetCDF4FileHandler

from satpy.readers.core.netcdf import NetCDF4FileHandler

ameraner · 2025-09-10T14:45:38Z

satpy/readers/thr3mi_l1c_nc.py

+
+This is an initial draft version. Note that as the 3MI uses 1D arrays, the data has to be
+resampled first before using the "show()" function. Working on a conversion to a 2D array at input to fix this.
+


the reference to the doc is missing

Suggested change

.. _EPS-SG 3MI Level 1C Product Format Specification: https://user.eumetsat.int/s3/eup-strapi-media/EPS_SG_3_MI_Level_1_C_Product_Format_Specification_web_1e7e9e67b2.pdf # noqa: E501

ameraner · 2025-09-10T14:49:40Z

satpy/readers/thr3mi_l1c_nc.py

+            if i_overlap == 0:
+                variable_old = variable.copy(deep=True)
+            if i_overlap > 0:
+                #variable_old = variable.copy(deep=True)


debug line to be removed probably

Suggested change

#variable_old = variable.copy(deep=True)

ameraner · 2025-09-10T14:52:24Z

satpy/etc/readers/thr3mi_l1c_nc.yaml

+    Reader for EUMETSAT EPS-SG 3MI Multi-Angle Polarimeter Level 1C Radiance files in NetCDF4 format per PFS
+  sensors: [3mi]
+  reader: !!python/name:satpy.readers.yaml_reader.FileYAMLReader
+


a couple more fields are required nowadays

Suggested change

supports_fsspec: true

status: Beta

pepephillips · 2025-09-12T09:56:38Z

Hi Andrea, Thanks loads for the suggestions, I'll get these changes done. Cheers, Pepe

________________________________ From: Andrea Meraner ***@***.***> Sent: Wednesday, September 10, 2025 16:48 To: pytroll/satpy ***@***.***> Cc: Pepe Phillips ***@***.***>; Author ***@***.***> Subject: Re: [pytroll/satpy] Thr3mi L1C reader (PR #2908) WARNING: This email originated outside the organisation. Treat links and attachments with caution. @ameraner requested changes on this pull request. Hi Pepe, thanks again for the reader. Two suggestions inline to fix an import warning and a broken docstring link.

________________________________ In satpy/readers/thr3mi_l1c_nc.py<#2908 (comment)>:

+The ``3mi_l1c_nc`` reader reads EPS-SG 3MI L1C image data in netCDF format. The format is explained

+in the `EPS-SG 3MI Level 1C Product Format Specification`_. Details of format and test data can be +found at: +https://user.eumetsat.int/resources/user-guides/metop-sg-3-mi-l1b-and-l1c-data-guide + +This is an initial draft version. Note that as the 3MI uses 1D arrays, the data has to be +resampled first before using the "show()" function. Working on a conversion to a 2D array at input to fix this. + +""" + +import logging +from datetime import datetime + +import xarray as xr + +from satpy.readers.netcdf_utils import NetCDF4FileHandler this needs a new import after the recent refactoring: ⬇️ Suggested change

-from satpy.readers.netcdf_utils import NetCDF4FileHandler +from satpy.readers.core.netcdf import NetCDF4FileHandler

________________________________ In satpy/readers/thr3mi_l1c_nc.py<#2908 (comment)>:

+# but WITHOUT ANY WARRANTY; without even the implied warranty of

+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with satpy. If not, see <http://www.gnu.org/licenses/>. +"""EUMETSAT EPS-SG Multi-view, Multi-channel, Multi-polarisation Imager (3MI) Level 1C products reader. + +The ``3mi_l1c_nc`` reader reads EPS-SG 3MI L1C image data in netCDF format. The format is explained +in the `EPS-SG 3MI Level 1C Product Format Specification`_. Details of format and test data can be +found at: +https://user.eumetsat.int/resources/user-guides/metop-sg-3-mi-l1b-and-l1c-data-guide + +This is an initial draft version. Note that as the 3MI uses 1D arrays, the data has to be +resampled first before using the "show()" function. Working on a conversion to a 2D array at input to fix this. + the reference to the doc is missing ⬇️ Suggested change - + +.. _EPS-SG 3MI Level 1C Product Format Specification: https://user.eumetsat.int/s3/eup-strapi-media/EPS_SG_3_MI_Level_1_C_Product_Format_Specification_web_1e7e9e67b2.pdf # noqa: E501 — Reply to this email directly, view it on GitHub<#2908 (review)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AP77JHZE2Q6OEMVVY22FULD3SA23RAVCNFSM6AAAAABOYMV5KGVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZTEMBWGUYTANBWGE>. You are receiving this because you authored the thread.Message ID: ***@***.***> Any email message from EUMETSAT is sent in good faith but shall neither be binding nor construed as constituting a commitment by EUMETSAT, except where provided for in a written agreement or contract or if explicitly stated in the email. Please note that any views or opinions presented in this email are solely those of the sender and do not necessarily represent those of EUMETSAT. This message and any attachments are intended for the sole use of the addressee(s) and may contain confidential and privileged information. Any unauthorised use, disclosure, dissemination or distribution (in whole or in part) of its contents is not permitted. If you received this message in error, please notify the sender and delete it from your system.

…correct PFS reference, correct netcdfFileHandler import and removed redundant debug line

ameraner · 2025-11-10T10:34:09Z

satpy/readers/thr3mi_l1c_nc.py

+    @property
+    def ssp_lon(self):
+        """Return subsatellite point longitude."""
+        # This parameter is not applicable to 3MI
+        return None


this is not strictly needed so it can be removed for simplicity

pepephillips added 4 commits May 23, 2024 14:30

prototype MAP L1C readers

ccb567a

Syncing with upstream changes

9b0b489

v1 reader for 3MI L1C with basic functionality

93c0541

Merge branch 'thr3mi_dev' of https://github.com/pepephillips/satpy in…

ed528e3

…to thr3mi_dev Merge branch 'main' of https://github.com/pytroll/satpy into map_dev To take on board latest changes from satpy main before 3MI reader PR

pepephillips requested review from djhoese and mraspaud as code owners September 24, 2024 14:06

pepephillips and others added 6 commits February 4, 2025 10:38

Merge branch 'pytroll:main' into thr3mi_dev

965a51a

Collecting updates since September 2024

a802f39

Merge branch 'main' of https://github.com/pytroll/satpy into thr3mi_dev

removed map.yaml as was not needed

e761395

needed for sync

6f39755

Merge branch 'thr3mi_dev' of https://github.com/pepephillips/satpy into thr3mi_dev

Merge branch 'main' of https://github.com/pytroll/satpy into thr3mi_dev

a515e4f

improved test coverage, PEP8 corrections and few typos in yaml

bc112ff

[pre-commit.ci] auto fixes from pre-commit.com hooks

5934a89

for more information, see https://pre-commit.ci

pepephillips and others added 5 commits February 18, 2025 15:28

improved test coverage and edits to coding style from pre-commit

f3bc56e

resolved merge conflict on thr3mi_l1c_nc.yaml

ddd5b84

Merge branch 'pytroll:main' into thr3mi_dev

2fcc7d0

corrected issues from pre-commit ruff and improved unit test coverage

54c0157

more corrected issues from pre-commit ruff and improved unit test cov…

c1aeb1a

…erage for exception handling

pepephillips added 2 commits May 8, 2025 15:11

Merge branch 'pytroll:main' into thr3mi_dev

88f6e83

Merge branch 'pytroll:main' into thr3mi_dev

a6fa587

ameraner requested changes Sep 10, 2025

View reviewed changes

ameraner reviewed Sep 10, 2025

View reviewed changes

Merge branch 'pytroll:main' into thr3mi_dev

13efe60

thr3mi_l1c_nc.yaml includíng new fieilds, thr3mi_l1c_nc.py including …

3f09755

…correct PFS reference, correct netcdfFileHandler import and removed redundant debug line

ameraner reviewed Nov 10, 2025

View reviewed changes


		import xarray as xr

		from satpy.readers.netcdf_utils import NetCDF4FileHandler

	from satpy.readers.netcdf_utils import NetCDF4FileHandler
	from satpy.readers.core.netcdf import NetCDF4FileHandler


		This is an initial draft version. Note that as the 3MI uses 1D arrays, the data has to be
		resampled first before using the "show()" function. Working on a conversion to a 2D array at input to fix this.



	.. _EPS-SG 3MI Level 1C Product Format Specification: https://user.eumetsat.int/s3/eup-strapi-media/EPS_SG_3_MI_Level_1_C_Product_Format_Specification_web_1e7e9e67b2.pdf # noqa: E501

Thr3mi L1C reader #2908

Are you sure you want to change the base?

Thr3mi L1C reader #2908

Uh oh!

Conversation

pepephillips commented Sep 24, 2024

Uh oh!

pepephillips commented Sep 24, 2024

Uh oh!

codecov bot commented Sep 24, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

ameraner commented Feb 15, 2025

Uh oh!

coveralls commented Feb 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Test Coverage Report for Build 17614285017

Details

💛 - Coveralls

Uh oh!

pepephillips commented Feb 25, 2025

Uh oh!

ameraner left a comment

Choose a reason for hiding this comment

Uh oh!

ameraner Sep 10, 2025

Choose a reason for hiding this comment

Uh oh!

ameraner Sep 10, 2025

Choose a reason for hiding this comment

Uh oh!

ameraner Sep 10, 2025

Choose a reason for hiding this comment

Uh oh!

ameraner Sep 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pepephillips commented Sep 12, 2025 via email

Uh oh!

ameraner Nov 10, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codecov bot commented Sep 24, 2024 •

edited

Loading

coveralls commented Feb 15, 2025 •

edited

Loading

ameraner Sep 10, 2025 •

edited

Loading