Reflectance standard name variations

[mraspaud]

Question

In our field (weather satellites), we often work with the toa_bidirectional_reflectance. Our understanding of that quantity is that it is the top of atmosphere radiance measured by the satellite instrument normalised by the solar irradiance, taking into account the solar zenith angle (see here for the formula for example).

However, reading files with these data, it is quite often so that we have arrays where the solar zenith angle normalisation (needed for the full fledged Reflectance) has not been applied, and thus we are looking for a possible standard name to propose to this community, but we can't agree on the naming.

Suggestions for standard names we have so far include:

• toa_bidirectional_reflectance_multiplied_by_cosine_solar_zenith_angle
• toa_bidirectional_pseudoreflectance
• toa_bidirectional_zenith_equivalent_reflectance

Would any of these be appropriate? are they following the expected forms for standard names? Are there extra guidelines (on top the already available standard name guidelines here) we can use to converge to a consensus?

[JonathanGregory]

Dear Martin @mraspaud

I think your first suggestion is best, because it includes the two existing standard names toa_bidirectional_reflectance and solar_zenith_angle, making it rather self-explanatory. Following the guidelines and existing standard names, we wouldn't say multiplied; instead it would be product_of_cosine_solar_zenith_angle_and_toa_bidirectional_reflectance (putting the factors in alphabetical order). Would this be all right?

We could say cosine_of_, like square_of_ and product_of_, but on the other hand we don't have of_ with ln_ and log10_ standard names. I think it's clear without of_. This will be the first standard name containing a trigonometric function!

In your reference there's π in the numerator, but there isn't in wikipedia. Can you explain this? I guess it's something to do with solid angle. I notice that the help for toa_bidirectional_reflectance says "does not include any integration over solid angle". Also the canonical unit is given as 1, whereas wikipedia says sr-1. I think the standard name description ought to be more informative. There may be subtle differences. I am not at all familiar with these quantities, as you can no doubt tell!

Best wishes

Jonathan

[djhoese]

I was discussing this with a coworker and he made a good point and I'm wondering if there are examples of this in CF:

The product_of_ and multiplied_by variants are being used to describe the simpler quantity by modifying a more complicated quantity. That is, the usual series of calculations here would be:

radiance -> "pseudoreflectance" -> reflectance

So we're taking the standard name of the reflectance (toa_bidirectional_reflectance) and back-stepping one of the calculations that was applied to get that reflectance quantity (/ cos(SZA)) so that we can describe the prior "pseudo" quantity. It seems odd. I can't think of another case like this but it's like saying a quantity is sqrt(x^2) rather than just x.

[mraspaud]

@JonathanGregory thanks a lot for your input!
product_of_cosine_solar_zenith_angle_and_toa_bidirectional_reflectance is working fine for us I think, I will of course consult with my colleagues before we make an official request for the standard name.

Regarding your question, my understanging is that the reflectance in our domain (earth-observation satellite remote-sensing) make the assumption of isotropic diffusion, and thus we can integrate over the half sphere. This assumption is of course wrong most of the times, but since we usually don't really know what we are looking at (what is the surface made of? what is the content of the atmosphere on this light path?), it's a compromise we make. Again, just my understanding of the question.

[JonathanGregory]

David @djhoese and Martin @mraspaud, thanks for your comments.

it's like saying a quantity is sqrt(x^2) rather than just x

That's a good point. It's quite often the case that there's more than one way you can describe something, which is what standard names do. We try to find a description which is simple, clear, and as consistent as possible with usual terminology (although that's less important than being self-explanatory). I am not sure I've understood this quantity. Would it be correct to describe it as the ratio of the shortwave flux (received at the instrument, per m2 perpendicular to the beam) to the TOA incoming shortwave flux (per m2 of Earth's surface)?

Regarding your question, my understanding is that the reflectance in our domain (earth-observation satellite remote-sensing) make the assumption of isotropic diffusion, and thus we can integrate over the half sphere

Ah, I see. Is that perhaps why it's called "bidirectional reflectance", depends on two angles and is dimensionless, whereas the object described by wikipedia is the "bidirectional reflectance distribution function", which depends on four angles and is in srad-1?

[gerritholl]

Hi @JonathanGregory . A good and widely cited reference on terminology is this paper:

Schaepman-Strub, G., M. E. Schaepman, T. H. Painter, S. Dangel, and J. V. Martonchik. 2006. "Reflectance Quantities in Optical Remote Sensing-Definitions and Case Studies.", Remote Sensing of Environment 103 (1): 27-42. doi:10.1016/j.rse.2006.03.002, https://www.sciencedirect.com/science/article/pii/S0034425706001167

Following Schaepman-Strub et al. (2006), toa_bidirectional_reflectance is not a measurable quantity, but can only be estimated using assumptions (as Martin was already writing). The quantity that can be almomst unambiguously calculated from measurements is hemispherical–conical reflectance: radiation reflects from all sources and enters the field of view of the detector (a cone). If we neglect the difference between hemispherical-conical reflectance and conical-conical reflectance (i.e. we neglect any daytime irradiance not coming from our own Sun), then this quantity can be unambiguously calculated from measurements. Following this, the correct name might be product_of_cosine_solar_zenith_angle_and_hemispherical_conical_reflectance or
product_of_cosine_solar_zenith_angle_and_hemispherical_conical_toa_reflectance or product_of_cosine_solar_zenith_angle_and_conical_conical_reflectance or product_of_cosine_solar_zenith_angle_and_conical_conical_toa_reflectance. However, in practice, the terms hemispherical-conical reflectance or conical-conical reflectance are rarely used, and the calibrated quantity is normally called toa_bidirectional_reflectance, even though calibration to toa_bidirectional_reflectance involves isotropy assumptions but hemispherical_conical_reflectance would not.

[pdebuyl]

Hello @JonathanGregory , all,

I have some extra ideas about this (after some discussion in a satpy meeting). The back-stepping mentioned by @djhoese makes sense for my intuition. What is measured is the toa radiance within the instrument's spectral band (say L). Then, L is normalized by the incoming solar irradiance (in the same spectral band), and that is always between 0 and 1. But we also know that L will be proportional to the cosine of the solar zenith angle and can divide by it to neutralize (or try to as atmospheric effects are still left out) the geometry dependence.

From this point of view, we take the ratio of "toa_outgoing_radiance_per_unit_wavelength" by "solar_irradiance_per_unit_wavelength". But that maybe a bit long. Note: those quantities are both defined in CF :-)

In GOES files for instance, you can find the following quantities:

            Rad:standard_name = "toa_outgoing_radiance_per_unit_wavenumber" ;
            esun:standard_name = "toa_shortwave_irradiance_per_unit_wavelength" ;

I hope that this helps to move forward for this issue.

[djhoese]

So the psuedo-reflectance being discussed here is equal to toa_outgoing_radiance_per_unit_wavenumber / toa_shortwave_irradiance_per_unit_wavelength?

Edit: Or are there other factors/constants or variables involved?

[pdebuyl]

There are factors. One can look at https://github.com/pytroll/satpy/blob/main/satpy/readers/abi_l1b.py#L133 for instance. Note that the

Units of Rad: "mW m-2 sr-1 (cm-1)-1"

Units of esun: "W m-2 um-1"

There is one "pi" for the angular part, but also the fact that the solar irradiance needs to be scaled by the square of the earth-sun distance (in astronomic units, so that it is unitless). Also, one needs to check the "per unit wavelength" and "per unit wavenumber" to convert them properly :-)

[JonathanGregory]

Dear all

We agreed that the quantity we want to name equals the TOA bidirectional reflectance multiplied by the cosine of the solar zenith angle.

Earlier, I asked whether the TOA bidirectional reflectance is the ratio of the shortwave flux (received at the instrument, per m2 perpendicular to the beam) to the TOA incoming shortwave flux (per m2 of Earth's surface).

The help for toa_bidirectional_reflectance says "bidirectional_reflectance depends on the angles of incident and measured radiation". I think the angle of incidence is the solar zenith angle. What happens to the angle of reflection? The help says "toa_bidirectional_reflectance ... does not include any integration over solid angle." What does that mean? Is it correct? Is the instrument measuring radiation from one direction (i.e. a radiance in W m-2 srad-1), or an entire hemisphere (i.e. a radiative flux in W m-2)? Perhaps this is the same as asking whether the Earth fills the entire hemisphere of view of the instrument (because it's in low orbit) or appears as a remote source in one direction (because it's in high orbit)?

@pdebuyl writes that the quantity we want to name is calculated as the ratio of toa_outgoing_radiance_per_unit_wavelength to solar_irradiance_per_unit_wavelength. This raises the same question about angles, because outgoing radiance (in W m-2 srad-1) has a direction, whereas solar irradiance is a flux (in W m-2). The ratio of these two quantities wouldn't be dimensionless.

As you see, I'm not clear about these concepts yet. More explanation and insight would be useful. 😃

Best wishes

Jonathan

[mraspaud]

I like @pdebuyl 's proposal, but I'll let the scientists here answer the unit question.

[gerritholl]

The instrument measures radiation from a small solid angle (a cone) that is much smaller than the solid angle for the entire Earth disc. The radiation from one direction would be, in the limit, zero. We integrate over the solid angle and it becomes dimensionless.

Borrowing an image from Schaepman-Scrub et al. (2006):

From an entire hemisphere would be the bihemispherical case, but what we have in practice is the biconical case (in practice equal to hemispherical-conical). Both the nominator and denominator integrate over the solid angle so the ratio is again dimensionless.

[JonathanGregory]

Thanks for the diagram, but I'm sorry, I don't understand this yet, not having the background. Could we talk about it?

[pdebuyl]

Hello all, update on the units:

1. I checked the units of Rad for a thermal band (above, in "mW m-2 sr-1 (cm-1)-1"). The visible bands have Rad in "W m-2 sr-1 um-1" so the unit issue is moot for ABI data;
2. The units for FCI visible bands in the product user guide are consistently in "mW / (m2 cm-1)" and for ABI in "W m-2 um-1" (with a 1/sr appearing consistently in the radiance and not in the incident energy).
3. EUMETSAT's FCI product user guide refers specifically to Schaepman-Strub "BRF / Bidirectional Reflectance Factor" and states "BRF = pi BRDF". (I believe that there is then a missing pi in the document). The BRF then includes the cos(SZA) correction, which suggests that this quantity (the BRF) is indeed CF's toa_bidirectional_reflectance as it "includes a factor to account for the cosine of the solar zenith angle" (from CF standard names).

So, CF's toa_bidirectional_reflectance is "Rad * pi * d**2 * cos(SZA) / esun" (d is here unitless).

Dropping the cosine, we have "Rad * pi * d**2 / esun" or in full words "toa_outgoing_radiance_per_unit_wavelength times divided by the solar_irradiance_per_unit_wavelength. If a scalar distance_from_sun is included in the dataset, the quantity solar_irradiance_per_unit_wavelength can be considered distance corrected.

So we have "ratio of product of toa_outgoing_radiance_per_unit_wavelength and pi by solar_irradiance_per_unit_wavelength". Now, there is the issue of conical vs directional but I believe that we are in fact in "case 1" (so, directional, as is the canonical BRDF in the paper) unless there is a specific geometric information (i.e. conical angle of the measurement or whatever, which we don't have in the imager measurements. If there is such a geometric consideration it is taken into account in the radiance calibration and not visible to the data users).

[pdebuyl]

@gerritholl @mraspaud @djhoese @JonathanGregory and @pdebuyl had a video meeting about the topic.

Here is the summary:

1. We agree that the quantity is correctly described by the definitions above (i.e. ratio of product of pi and radiance with solar irradiance, or equally TOA BRF times cos(SZA)
2. The name proposed by Pierre is long and remains confusing. Some CF names exist that state "integrated over an hemisphere" or similar, to be thought about.
3. There is currently a confusion about the definition. For instance, the GOES L1 product guide quantity "kappa_0 times radiance" is the same as we are discussing but is called "reflectance factor". The units can't be used to distinguish between all these quantities (cos(SZA) is unitless).
4. There is a purpose: storing data for imaging purposes (sometimes the cos(SZA) is not used for imaging and sometimes other atmospheric corrections are applied).
5. There is a discussion about the fact that the intermediate quantities in CF are defined "per unit wavelength": toa_outgoing_radiance_per_unit_wavelength and solar_irradiance_per_unit_wavelength. In principle, those are all integrated by the spectral response function (pre-processing for solar irradiance and physically by the sensor for the radiance). This cancels out in the data as far as we are aware.
6. There is the question on whether the pi should be 2 pi, to be checked!
7. Next step: reach out to space agencies about this to see which names would be best suited and/or whether they actually have a reference name for this quantity.

Options:

1. ratio_of_product_of_toa_outgoing_radiance_per_unit_wavelength_and_pi_by_solar_irradiance_per_unit_wavelength
2. product_of_cosine_solar_zenith_angle_and_toa_bidirectional_reflectance
3. Other?

[pnuu]

How about oblique_toa_bidirectional_reflectance? Option 2 has been scaled with the cosine of zenith angle to make it equivalent to zero SZA. If we remove that scaling, we get slanted, or oblique, "variant".

[JonathanGregory]

@pnuu. Thanks for your suggestion. Although that's shorter, and "oblique" is a relevant word if you know what the quantity is, I feel that this wouldn't be a self-explanatory standard name for those who aren't familiar with these terms. Pierre's two options do have that advantage.

[mraspaud]

I agree that "oblique_toa_bidirectional_reflectance" is not as self-explanatory as the other two, but it comes with the advantage of shortness. Looking at the existing "surface_direct_shortwave_hemispherical_reflectance" for example, quite a lot is explained in the definition of the standard name, without the standard name itself being long. Moreover, my understanding is that to get a fully featured name, one could use long-name instead of standard name, and that the standard name is just that, a "standard", some term the community has agreed upon to refer to a given quantity. And as far as I know, there is no other toa reflectance than bidirectional and this new one we want to introduce now in the satellite community, so I don't think we need many variations in the future (a reason for which we would need a longer, more descriptive, standard name). Hence I am rooting for brevity in this standard name, as it will make it less prone to errors/confusion/cognitive overload when looking at the dataset metadata.

So in summary, I like "oblique_toa_bidirectional_reflectance", and unless it is scientifically incorrect, I would then propose that as a candidate.
"ratio_of_product_of_toa_outgoing_radiance_per_unit_wavelength_and_pi_by_solar_irradiance_per_unit_wavelength" could then instead be used as a basis for a definition :)

[pdebuyl]

Hello, I have to admit that despite "my" propositions carrying more specific information, there are other standard names where brevity is preferred, leaving the full definition for the standard names table.

A variation on toa_bidirectional_reflectance makes sense and I have no better proposition than @pnuu

[JonathanGregory]

Thank you, everyone, for the useful discussion yesterday, and your helpful explanation to me of the concepts involved. Thank you, Pierre @pdebuyl, for the above summary. I would add a couple of points:

• Regarding option (1), as you note, the spectral dependence is removed by integration, but even if it wasn't it would cancel out. Therefore I think the ratio-type name would be something like ratio_of_(some phrase meaning the hemispherically integrated)_toa_outgoing_radiance_to_solar_irradiance. That is, per_unit_wavelength (twice) can be omitted. Also note that the construction contains "to" i.e. ratio_of_X_to_Y. Although per_unit_wavelength is not mentioned in the name, the quantity can still be a function of wavelength or frequency, specified by a coordinate variable or scalar coordinate variable, and this possibility should be mentioned in the description. The default would be for the spectral integral, which is what you want. That's consistent with the usual convention of standard names that if some aspect is not qualified, it implies everything i.e. a comprehensive default.

• As we discussed at the start of the meeting yesterday, the quantity you want applies for the actual Earth–sun distance by default, whereas the quantity named toa_bidirectional_reflectance applies at 1 AU by default. This will need to be explained in the description for sure, and possibly we might need to add some word(s) to the standard name, because otherwise the similarity of the two names could be misleading (if we choose the product-type name).

Cheers, Jonathan

[gerritholl]

The shorter names radiance_factor and reflectance_factor have been raised. Definitions exist for those.

From the OSIRIS-REx Asteroid Sample Return Mission Algorithm Descriptions- Spectral Analysis, compiled by Driss Takir, Beth Clark, Christian Drouet d’Aubigny, Jian-Yang Li, Carl Hergenrother, Josh Emery, and Bonnie Buratti, available from the University of Arizona, sourcing Hapke, B. 1993. Theory of Reflectance and Emittance Spectroscopy. Cambridge University Press, https://doi.org/10.1017/CBO9781139025683:

Radiance Factor (RADF) is the ratio of the bidirectional reflectance of a surface to that of a perfectly diffuse surface illuminated at i = 0 (the Sun!), rather than at the same angle of illumination.

From the NPL-led Measurement report CEOS WGCV pilot comparison of techniques and instruments used for the vicarious calibration of land surface imaging through a ground reference standard test site 2009, by Irina Behnert, Andrew Deadman, Nigel Fox, Peter Harris, Selime Gürol, Hilal Özen, Martin Bachmann, Yannick Boucher, Sophie Lachérade, available from NPL:

radiance factor

ratio of radiance of a surface element in a given direction to that of a perfectly reflecting Lambertian diffuser identically irradiated

The same source also defines:

reflectance factor

ratio of radiant power reflected by a surface element into a given solid angle to that of a perfectly reflecting Lambertian surface element

And Schaepman-Strub et al. (2006) define:

The reflectance factor is the ratio of the radiant flux reflected by a surface to that reflected into the same reflected-beam geometry and wavelength range by an ideal (lossless) and diffuse (Lambertian) standard surface, irradiated under the same condition

I'm not sure if those definitions are consistent and if they match our observand.

[JonathanGregory]

Dear @gerritholl, @pdebuyl et al.

Thanks for these. I'm not keen on radiance_factor and reflectance_factor as CF standard names because they're not self-explanatory. "Factor" would not describe the quantity to a non-expert.

By asking an LLM I have just found The principles of field spectroscopy, which gives some definitions very explicitly in terms that I can understand, as physicist but not a spectroscopy expert.

On the basis of this, I'd suggest the following edition of @pdebuyl's first option: ratio_of_toa_radiant_exitance_to_solar_irradiance, having learned that "radiant exitance" is the term for hemispherically integrated outgoing radiance. That's not too long. What do you think?

I have left out per_unit_wavelength because it would appear in both numerator and denominator and therefore cancels out. The quantity could be a function of wavelength (or frequency), if specified as a coordinate, and by default the numerator and denominator would be the spectral integrals.

Of course radiant_exitance may not be a well-known term to a non-expert, but it seems to be well-defined e.g. by wikipedia. We can't use the word "spectral" in standard names in the way it appears to be used in spectroscopy, because it means per_unit_wavelength or per_unit_frequency. These take different units, so the standard names cannot be the same.

Best wishes

Jonathan

[pdebuyl]

Hello @JonathanGregory , I can't find the term exitance in the book "The principles of field spectroscopy". It is also not used by the community.

The other issue is that the exitance is a flux, whereas the radiometers of the type we are discussing measure radiances, so it could not apply.

Simplifying the "per unit..." as you suggest would thus result in ratio_of_product_of_toa_outgoing_radiance_and_pi_by_solar_irradiance. Up to a factor, this is actually a BDRF, but according to the Schaepman-Strub, this is a BRF (or toa_bidirectional_reflectance_factor times cos(SZA)). The quantity is very close to a reflectance factor.

I am not sure that we can find an exact meaning expression that is at the same time brief.

Contenders:

• ratio_of_product_of_toa_outgoing_radiance_and_pi_by_solar_irradiance
• oblique_toa_bidirectional_reflectance

[JonathanGregory]

Dear Pierre @pdebuyl

Maybe I have misunderstood the previous discussion, but I thought the idea of the numerator is that it's a hemispherically integrated radiance. That's why it contains π, though it's unexplained why the factor isn't 2π Regardless of the 2, I suppose this factor is a solid angle. Hence the numerator is a flux (W m-2), like the irradiance in the denominator, as is necessary to make the ratio dimensionless. Have I got something wrong here?

Best wishes

Jonathan

[JonathanGregory]

The American Meterological Society glossary contains radiant exitance but implies that it's an outdated term. It says

Radiant flux per unit area leaving a plane surface; no longer used as a substitute for emittance. In current usage the equivalent term flux density is preferred, or irradiance is adapted to include emission from a surface; units are watts per square meter (W m-2).

It seems to me that it would be confusing to use irradiance to mean outgoing radiation as well as incoming, although this comment suggests doing that. Perhaps a more promising alternative is emittance, which it mentions first. Its entry for that says

(Or emissive power.) Emitted irradiance.

Is "emittance" a term that you are aware of, @pdebuyl @mraspaud @gerritholl? If I've understand this correctly, it is a word which means the hemispherically integrated outgoing radiance. Using this word, the standard name of this issue would be ratio_of_toa_shortwave_emittance_to_solar_irradiance, with both numerator and denominator understood to mean spectral integrals unless a coordinate of wavelength or frequency is supplied. That's rather simple and self-explanatory, I would say. But would it work for you? The description of the term would describe its relationship to toa_bidirectional_reflectance.