cfgrib does not recognise channelNumber in ECMWF synthetic satellite products #405
Description
What happened?
When opening a grib2 file from ECMWF model version 49r1 which includes messages with different channels of cloudy reflectance and cloudy brightness temperature from simulated satellite images, cfgrib only identifies one channel from cloudy reflectance and one from cloudy brightness temperature from each time step. The other channels seems to not be read or to be discarded. My guess is that cfgrib is unable to recognize different channels because it does not seem to consider channelNumber as a GRIB-entity.
What are the steps to reproduce the bug?
-
Download a synthetic satellite data from ECMWF (so far only test data), assumes access to ECMWF MARS service. Here is an exemple of mars request (from https://confluence.ecmwf.int/display/FCST/Simulated+Satellite+Data+update):
retrieve,
channel=1/2/3/4/7,
class=od,
date=2024-10-24,
expver=79,
ident=71,
instrument=210,
param=260512,
step=0/to/144/by/3,
stream=oper,
time=00:00:00,
type=ssd,
target="fci_vis.grib" -
Open in python with xarray and cfgrib:
ds=xr.open_dataset('fci_vis.grib.grib', engine='cfgrib')
and expand the only variable in the dataset:
ds['cdrfl'].
Only data from one channel is shown (the other channels seems not to be read or discarded). No metadata field like GRIB_channelNumber is shown.
- Compare with grib_dump:
All channels are shown as separate messages with grib_dump, here is an example of output for a message:
GRIB {
Satellite remote sensing products (formerly Space products) (grib2/tables/33/0.0.table)
discipline = 3;
editionNumber = 2;
European Centre for Medium-Range Weather Forecasts (common/c-11.table)
centre = 98;
subCentre = 0;
Start of forecast (grib2/tables/33/1.2.table)
significanceOfReferenceTime = 1;
dataDate = 20241028;
dataTime = 1200;
Operational products (grib2/tables/33/1.3.table)
productionStatusOfProcessedData = 0;
Missing (grib2/tables/33/1.4.table)
typeOfProcessedData = 255;
Satellite Channel Data (grib2/grib2LocalSectionNumber.98.table)
grib2LocalSectionNumber = 24;
Operational archive (mars/class.table)
marsClass = 1;
Simulated satellite data (mars/type.table)
marsType = 87;
Atmospheric model (mars/stream.table)
marsStream = 1025;
experimentVersionNumber = 0079;
channelNumber = 7;
numberOfDataPoints = 90857;
There is no appended list (grib2/tables/33/3.11.table)
interpretationOfNumberOfPoints = 0;
Latitude/longitude (Also called equidistant cylindrical, or Plate Carree) (grib2/tables/33/3.1.table)
gridDefinitionTemplateNumber = 0;
Earth assumed spherical with radius of 6 371 229.0 m (grib2/tables/33/3.2.table)
shapeOfTheEarth = 6;
Ni = 377;
Nj = 241;
iScansNegatively = 0;
jScansPositively = 0;
jPointsAreConsecutive = 0;
alternativeRowScanning = 0;
latitudeOfFirstGridPointInDegrees = 72;
longitudeOfFirstGridPointInDegrees = 355;
latitudeOfLastGridPointInDegrees = 42;
longitudeOfLastGridPointInDegrees = 42;
iDirectionIncrementInDegrees = 0.125;
jDirectionIncrementInDegrees = 0.125;
gridType = regular_ll;
NV = 0;
Analysis or forecast at a horizontal level or in a horizontal layer at a point in time for simulated (synthetic) (grib2/tables/33/4.0.table)
productDefinitionTemplateNumber = 32;
Quantitative products (grib2/tables/33/4.1.3.table)
parameterCategory = 1;
Cloudy reflectance (%) (grib2/tables/33/4.2.3.1.table)
parameterNumber = 31;
#-READ ONLY- parameterUnits = %;
#-READ ONLY- parameterName = Cloudy reflectance;
Missing (grib2/tables/33/4.3.table)
typeOfGeneratingProcess = 255;
generatingProcessIdentifier = 158;
Hour (grib2/tables/33/4.4.table)
indicatorOfUnitForForecastTime = 1;
stepUnits = h;
forecastTime = 144;
stepRange = 144;
NB = 1;
satelliteSeries = 334;
satelliteNumber = 71;
instrumentType = 210;
scaleFactorOfCentralWaveNumber = 0;
scaledValueOfCentralWaveNumber = 621100;
shortName = cdrfl;
name = Cloudy reflectance;
cfName = unknown;
#-READ ONLY- cfVarName = cdrfl;
#-READ ONLY- modelName = IFS;
#-READ ONLY- modelVersion = unknown;
numberOfValues = 90857;
packingType = grid_ccsds;
ccsdsFlags = 14;
ccsdsBlockSize = 32;
ccsdsRsi = 128;
A bit map does not apply to this product (grib2/tables/33/6.0.table)
bitMapIndicator = 255;
bitmapPresent = 0;
values(90857) = {
0.100552, 0.0930603, 0.0969818, 0.101712, 0.119809,
0.104611, 0.0875672, 0.0662354, 0.0633362, 0.0571869,
0.0493439, 0.0516937, 0.0566834, 0.059018, 0.058606,
0.0576905, 0.0583161, 0.0591553, 0.0635803, 0.0658081,
0.0656098, 0.0579651, 0.0547302, 0.0553253, 0.0536316,
0.0469483, 0.0463684, 0.056546, 0.0682495, 0.0775421,
0.0904663, 0.102597, 0.10835, 0.111401, 0.112759,
0.105389, 0.0929077, 0.0782288, 0.128735, 0.199628,
0.268475, 0.357922, 0.445798, 0.498853, 0.446347,
0.406583, 0.353665, 0.279599, 0.202069, 0.16908,
0.139752, 0.108151, 0.100186, 0.0924805, 0.0869416,
0.0922669, 0.0979737, 0.101407, 0.105954, 0.113889,
0.122388, 0.114697, 0.103009, 0.0978211, 0.092511,
0.086789, 0.0807312, 0.0821198, 0.0784577, 0.0664337,
0.0557984, 0.0846985, 0.104749, 0.111325, 0.101895,
0.126202, 0.150388, 0.150952, 0.151288, 0.154645,
0.166013, 0.17738, 0.183667, 0.193036, 0.206158,
0.221143, 0.245435, 0.270581, 0.29025, 0.306577,
0.319516, 0.322476, 0.329877, 0.321622, 0.293515,
0.265195, 0.244885, 0.228986, 0.213239, 0.224957
... 90757 more values
}
#-READ ONLY- maximum = 0.839856;
#-READ ONLY- minimum = 0.0300263;
#-READ ONLY- average = 0.323615;
#-READ ONLY- standardDeviation = 0.181049;
#-READ ONLY- skewness = 0.414087;
#-READ ONLY- kurtosis = -0.67145;
#-READ ONLY- isConstant = 0;
#-READ ONLY- numberOfMissing = 0;
#-READ ONLY- getNumberOfValues = 90857;
}
Version
0.9.14.1
Platform (OS and architecture)
Ubuntu 20.04.6 LTS
Relevant log output
>>> import xarray as xr
>>> ds=xr.open_dataset('fci_vis.grib.grib', engine='cfgrib')
>>> ds
<xarray.Dataset> Size: 18MB
Dimensions: (step: 49, latitude: 241, longitude: 377)
Coordinates:
time datetime64[ns] 8B ...
* step (step) timedelta64[ns] 392B 0 days 00:00:00 ... 6 days 00:00:00
* latitude (latitude) float64 2kB 72.0 71.88 71.75 ... 42.25 42.12 42.0
* longitude (longitude) float64 3kB -5.0 -4.875 -4.75 ... 41.75 41.88 42.0
valid_time (step) datetime64[ns] 392B ...
Data variables:
cdrfl (step, latitude, longitude) float32 18MB ...
Attributes:
GRIB_edition: 2
GRIB_centre: ecmf
GRIB_centreDescription: European Centre for Medium-Range Weather Forecasts
GRIB_subCentre: 0
Conventions: CF-1.7
institution: European Centre for Medium-Range Weather Forecasts
history: 2024-11-04T16:15 GRIB to CDM+CF via cfgrib-0.9.1...
>>> ds['cdrfl']
<xarray.DataArray 'cdrfl' (step: 49, latitude: 241, longitude: 377)> Size: 18MB
[4451993 values with dtype=float32]
Coordinates:
time datetime64[ns] 8B ...
* step (step) timedelta64[ns] 392B 0 days 00:00:00 ... 6 days 00:00:00
* latitude (latitude) float64 2kB 72.0 71.88 71.75 ... 42.25 42.12 42.0
* longitude (longitude) float64 3kB -5.0 -4.875 -4.75 ... 41.75 41.88 42.0
valid_time (step) datetime64[ns] 392B ...
Attributes: (12/29)
GRIB_paramId: 260512
GRIB_dataType: ssd
GRIB_numberOfPoints: 90857
GRIB_stepUnits: 1
GRIB_stepType: instant
GRIB_gridType: regular_ll
... ...
GRIB_name: Cloudy reflectance
GRIB_shortName: cdrfl
GRIB_units: %
long_name: Cloudy reflectance
units: %
standard_name: unknown
ds['cdrfl'].attrs
{'GRIB_paramId': 260512, 'GRIB_dataType': 'ssd', 'GRIB_numberOfPoints': 90857, 'GRIB_stepUnits': 1, 'GRIB_stepType': 'instant', 'GRIB_gridType': 'regular_ll', 'GRIB_uvRelativeToGrid': 0, 'GRIB_NV': 0, 'GRIB_Nx': 377, 'GRIB_Ny': 241, 'GRIB_cfName': 'unknown', 'GRIB_cfVarName': 'cdrfl', 'GRIB_gridDefinitionDescription': 'Latitude/longitude', 'GRIB_iDirectionIncrementInDegrees': 0.125, 'GRIB_iScansNegatively': 0, 'GRIB_jDirectionIncrementInDegrees': 0.125, 'GRIB_jPointsAreConsecutive': 0, 'GRIB_jScansPositively': 0, 'GRIB_latitudeOfFirstGridPointInDegrees': 72.0, 'GRIB_latitudeOfLastGridPointInDegrees': 42.0, 'GRIB_longitudeOfFirstGridPointInDegrees': 355.0, 'GRIB_longitudeOfLastGridPointInDegrees': 42.0, 'GRIB_missingValue': 3.4028234663852886e+38, 'GRIB_name': 'Cloudy reflectance', 'GRIB_shortName': 'cdrfl', 'GRIB_units': '%', 'long_name': 'Cloudy reflectance', 'units': '%', 'standard_name': 'unknown'}Accompanying data
No response
Organisation
No response