Clarify the prohibition of numerical factors and offsets in `units`

[JonathanGregory]

[Deleted]

[JonathanGregory]

In Discussion 457 @matdeslandes asked a question about Section 3.1.3. He wasn't sure whether the prohibition of the UDUNITS syntax for additive offsets and multiplicative factors might also apply to the SI prefixes. In order to clarify this, he suggests starting a new paragraph. For symmetry, I think we should start a third paragraph for the packing attributes, which are again a different subject, and I suggest inserting a few more words to indicate these are three different things. The text would then appear as follows, with new words in bold:

UDUNITS recognises the SI prefixes shown in Table 3.1 for decimal multiples and submultiples of units, and allows them to be applied to non-SI units as well.

UDUNITS also offers a syntax of its own for indicating arbitrary scale factors and offsets to be applied to a unit. (Note that this is different from the scale factors and offsets used for converting between units, as discussed for temperature in Section 3.1.2, "Temperature units".) This UDUNITS syntax for arbitrary transformation of units is not supported by the CF standard, except for the case of specifying reference time (Section 4.4, "Time Coordinate").

The application of any scale factors or offsets to the data values, without altering the units, should be indicated by the scale_factor and add_offset attributes. Use of these attributes for data packing, which is their most important application, is discussed in detail in Section 8.1, "Packed Data".

Does anyone have a concern about modifying the text to correct this as a defect? If not, it will be accepted and I will make a PR in three weeks' time, on 6th March.

[DocOtak]

Oh no! I'll need to fix some of my data that was using that udunits syntax...

I support the clarification, perhaps with an example of what is not allowed or link to relevant sections of udunits docs (should they exist).

[JonathanGregory]

As @larsbarring has pointed out, it's not easy to grasp what UDUNITS permits, and probably it's been elaborated since that sentence was first written in the earliest versions of CF. I believe the intention is to disallow the appearance of numbers in the units as scale factors or offsets of units. Numbers are allowed only as dimensionless units. The syntax of UDUNITS is currently described by https://docs.unidata.ucar.edu/udunits/current/udunits2lib.html#Grammar. In those terms, I think that INT or REAL are not allowed in the units string in combination with ID, which is a named unit or special symbol e.g. units of 1000 kg, 10% and kg @ 10.0 are not allowed. I think it would be OK to insert a statement like that in order to correct a defect, so long as no-one disagrees with my interpretation (and memory) of what this prohibition was intended to mean.

As usual, there are other things we could clarify, but they would be more than correcting a defect, I feel, since they've never been clear. For instance, we allow numbers as dimensionless units, such as 1 or 1e3, but do we permit any number as a dimensionless unit, such as 42 or 3.14159? Sect 3.3.1 on dimensionless units doesn't prohibit them. But please don't answer that in this issue! 😃 It should be the subject of a separate non-trivial issue if we want to consider changing it.

[larsbarring]

The syntax of UDUNITS is currently described by https://docs.unidata.ucar.edu/udunits/current/udunits2lib.html#Grammar.

This is not exactly true, the published grammar is not complete and in some details not correct. Here is an attempt to provide a more detailed specification following the general style of the published one. It optimistically assumes that two PRs have been ingested in the code.

I think that INT or REAL are not allowed in the units string in combination with ID, which is a named unit or special symbol e.g. units of 1000 kg, 10% and kg @ 10.0 are not allowed.

In fact numbers are allowed:

$ udunits2 -h
Usage:
.... ....
    <XML_file>  XML database file. Default is "/usr/share/udunits/udunits2.xml".
$ udunits2 -H "12" -W ""
    12 1                        ## Note the unit "1" for dimensionless
$ udunits2 -H "kilopond" -W "N"
    1 kilopond = 9.80665 N
    x/N = 9.80665*(x/kilopond)
$ udunits2 -H "9.8 N" -W "kilopond"
    9.8 N = 0.999322 kilopond
    x/kilopond = 0.101972*(x/N)
$ udunits2 -H "pi" -W ""
    3.14159265358979 1            ## In UDUNITS "pi" is a unit (that in itself is dimensionless) 
$ udunits2 -H "2 pi" -W "circle"
    2 pi = 1 circle
    x/circle = 0.5*(x/pi)
$ udunits2 -H "circle/2" -W "pi"
    1 circle/2 = 1 pi              ## In UDUNITS "pi" is a unit 
    x/pi = (x/(circle/2))
$ udunits2 -H "circle/180" -W ""
    0.0349065850398866 rad
$ udunits2 -H "circle/180" -W "degree"
    1 circle/180 = 2 degree
    x/degree = 2*(x/(circle/180))
$ udunits2 -H "circle 180^-1" -W "degree"
    1 circle 180^-1 = 2 degree
    x/degree = 2*(x/(circle 180^-1))

But I agree, this is not something we should have in CF.

[JonathanGregory]

Thanks for clarification and further information, Lars, and also for your hard work on improving UDUNITS documentation. Yes, I meant that the CF text is intended to mean that "INT or REAL are not allowed in the units string in combination with ID", although (as you say) they are allowed by UDUNITS.

Including a clarification, as requested by Barna @DocOtak, the modified text would be

UDUNITS recognises the SI prefixes shown in Table 3.1 for decimal multiples and submultiples of units, and allows them to be applied to non-SI units as well.

UDUNITS also offers a syntax of its own for indicating arbitrary scale factors and offsets to be applied to a unit. This UDUNITS syntax for arbitrary transformation of units is not supported by the CF standard, except for the case of specifying reference time (Section 4.4, "Time Coordinate"). The syntax that is prohibited by CF allows dimensionless numbers to be included in units as multiplicative factors before the name of a unit and as additive offsets to a named unit (following @ or various keywords). Note that this is different from the scale factors and offsets used for converting between units, as discussed for temperature in Section 3.1.2, "Temperature units".

The application of any scale factors or offsets to the data values, without altering the units, should be indicated by the scale_factor and add_offset attributes. Use of these attributes for data packing, which is their most important application, is discussed in detail in Section 8.1, "Packed Data".

and in the conformance document we need a requirement for Sect 3.1

The units string must not contain literal numbers (integer or real) as factors or offsets to a named unit. In terms of the UDUNITS grammar, CF does not allow INT or REAL in a string containing ID, except for the specification of a reference time in the units of time coordinates.

I should not have used this issue, because this defect isn't a trivial typo etc. I didn't read the instructions that I had written myself. Sorry! I will therefore rename this issue and start a new empty one for truly trivial defects.

[JonathanGregory]

I have created a PR #636 with the above changes - not exactly as above, but very similar. I discovered there was a related statement at the end of Sect 3.1.1. Since that is repetition, I've deleted it, and instead inserted a reference to Sect 3.1.3 at the end of the preamble of Sect 3.1.

No concerns were raised about this change, which is a defect correction to clarify but not change the convention, and no comments have been made since 18th Feb. Therefore I think it's unnecessary to reset the clock, and I presume this change can be accepted if no-one raises any concerns before 9th March. However, please say if you need time to consider the wording of the PR in detail.

Thanks to @cofinoa's magic, the rendered version with this change applied can be seen at https://jonathangregory.github.io/cf-conventions/branch/issue620/

[davidhassell]

In terms of the link:$$https://docs.unidata.ucar.edu/udunits/current/udunits2lib.html#Grammar$$[UDUNITS Unit Grammar], CF does not allow INT or REAL in a string containing ID, except for the specification of a reference time in the units of time coordinates.

I had a look at that link, but found that I still didn't know what this conformance requirement is! It may just be me (in which case, that's fine :)), but is it possible to describe this in more friendly manner?

[JonathanGregory]

The "user-friendly" statement is the preceding sentence, "The units string must not contain literal numbers (integer or real) as factors or offsets to a named unit of measure." Is that clear? I added the "In terms of the link ..." sentence to be precise about what this means in terms of the UDUNITS syntax, in case a conformance checker is referring to that syntax. Should we omit the "In terms of ..." sentence, or explain why it's there (for instance, as I just have)?