UTC & leap seconds (again! again!!!)

[pvanlaake]

Topic for discussion

A Wise Man once said “If you think you know all about Time, you haven’t thought long enough”. Hence I beg your forgiveness for bringing up the UTC / leap seconds issue after the extended discussion (#304) and the recent inclusion in CF 1.12 (not to mention my missing the PR).

First of all, great job on the revamped section 4.4 and happy to see some new “calendars” added.

That immediately brings me to the first issue: Neither UTC nor TAI are calendars, they are clocks. So rather than than saying for utc “A Gregorian calendar with leap seconds as prescribed by UTC”, can this be changed to “Coordinated Universal Time (UTC) which includes leap seconds and where dates are represented using the Gregorian calendar”? And likewise for tai.

In the description of both calendars I would add that time zone offsets are not allowed and that a time zone indication, when given in a datetime, must be 0.

UTC

More concerning, from my perspective, is the definition of utc. As defined, it starts in 1958, but UTC did not exist until 1960 and was only officially adopted in 1963 (it was based on that same epoch as TAI, which is 1958-01-01 00:00:00 UT). The issue of leap seconds in UTC is not meaningfully addressed in the new text. Of course, there is section 4.4.3, but that really only deals with the consequences of having to consider leap seconds. I have seen the arguments being made in issue #542 but it feels unbalanced to have the detailed explanation of figure 4.1 but no basic information on leap seconds (such as where to find an authoritative list of the 27 leap seconds since 1972) or how it differs from the other calendars.

Further, prior to 1972 there were no leap seconds. The divergence of nearly 10 seconds between UTC and UT that accumulated between 1958-01-01 and 1971-12-31 was applied through three mechanisms: (1) a shifting in the length of the UTC second, at various rates over the period 1960-1971; (2) frequent small increments of multiples of 50ms; and finally (3) an anomalous jump of 0.107758 SI seconds to make the difference exactly 10 seconds at the onset of 1972-01-01. In other words, prior to 1972, the second used by UTC is not equal to the SI second. The introduction of the leap second was made exactly to align the UTC second with the SI second and to make time shifts to align the UTC clock with the celestial reality less burdensome (all of this before the advent of modern computer and communication infrastructure). You may find all the details here.

It is then a bit of a mystery to me why CF would want to dive head-first into this murky 1960-1971 business that many Wise Men have tried to obliterate through the introduction of the leap second. Is there a known use case for data sets using UTC that span the early years of that clock?

I would propose to side-step all of this complication by changing to definition of utc to begin at 1972-01-01 00:00:00, being the instant when UTC started using the SI second and leap seconds were introduced, with no time coordinates prior to that instant allowed.

units_metadata

I do not understand the purpose of this argument. The utc calendar was introduced to enable data producers that need second accuracy to record their observations correctly. When leap seconds are of concern, then why would the data producer use the standard or proleptic_gregorian calendar? Having this need for accuracy but not knowing whether or not the data collection equipment properly records and processes leap seconds does not seem like a real-world combination to me.

Why is there reference to the julian calendar here? Leap seconds are defined in the context of UTC and that uses the Gregorian calendar exclusively.

Finally, why an attribute units_metadata with a compound structure? Wouldn’t a simple attribute leap_seconds suffice?

[ChrisBarker-NOAA]

I'm all for making the docs more precise and clear, which I think is what much of this is about.

But is there any of this that might cause actual problems with using the new calendar?

To a few points:

“A Gregorian calendar with leap seconds as prescribed by UTC”, can this be changed to “Coordinated Universal Time (UTC) which includes leap seconds and where dates are represented using the Gregorian calendar”? And likewise for tai.

Sure -- that seems a bit more correct.

In the description of both calendars I would add that time zone offsets are not allowed and that a time zone indication, when given in a datetime, must be 0.

I don't think that's wise. As you point out about above, UTC is not a calendar per se -- a calendar specified days, and the time within in that is kind of a separate thing. However, for CF, they are a single concept a "calendar" is the full spec for how to interpret (and produce) datetime strings. So the difference between, say the standard calendar and utc is leap seconds, that's it -- there is just as much validity to specifying a timezone offset with the UTC calendar as any other. In that case, the time is not UTC, but it is relative to UTC -- so that's fine.

([UTC] was based on that same epoch as TAI, which is 1958-01-01 00:00:00 UT).

exactly, so it's valid from that date, yes? what's the problem here?

The issue of leap seconds in UTC is not meaningfully addressed in the new text. ... but no basic information on leap seconds (such as where to find an authoritative list of the 27 leap seconds since 1972) or how it differs from the other calendars.

I think that was pretty deliberate -- we try not to re-write specifications that others are responsible for. That being said, a bit more detail, and in particular a link to an external source with the full explanation would be great -- can you propose one?

It is then a bit of a mystery to me why CF would want to dive head-first into this murky 1960-1971 business that many Wise Men have tried to obliterate through the introduction of the leap second.

I'm guessing that was out of ignorance ....

I would propose to side-step all of this complication by changing to definition of utc to begin at 1972-01-01 00:00:00, being the instant when UTC started using the SI second and leap seconds were introduced, with no time coordinates prior to that instant allowed.

That sounds good to me -- but I'll let others that may know better weight in.

The goal here is that people can use a UTC timestamp, convert that to a CF-style "time-since" form, (and vice versa) and clearly define what that means. If that's not possible pre-1972, then yes, we shouldn't allow it.

When leap seconds are of concern, then why would the data producer use the standard or proleptic_gregorian calendar? Having this need for accuracy but not knowing whether or not the data collection equipment properly records and processes leap seconds does not seem like a real-world combination to me.

The issue is not that data producers don't know whether leap seconds are included in their timestamps -- the problem is that most of the software out there is not leap-second aware. So if a producer knows they have proper UTC timestamps, but their software (and that of their users) doesn't support leap seconds, then they need to use the standard calendar.

Practicality beats purity, and all that.

Why is there reference to the julian calendar here? Leap seconds are defined in the context of UTC and that uses the Gregorian calendar exclusively.

agreed -- we should remove that.

Finally, why an attribute units_metadata with a compound structure? Wouldn’t a simple attribute leap_seconds suffice?

because units_metadata was already introduced for temperature -- so keeping things a bit more consistent.

However, now that you mention is, I see in section 3.1.2:

"A variable must not have a units_metadata attribute if it has no units attribute or if its units do not involve a temperature unit."

That needs an update.

[pvanlaake]

Hi Chris, thanks for your reply. A few responses to your comments:

there is just as much validity to specifying a timezone offset with the UTC calendar as any other

Here I have to disagree. The U stands for Universal, UTC time is the same all over the world at a single instant, irrespective of where you are. Bangkok, London and Denver all have the same UTC time at the same instant.

UTC is managed by BIPM so I would suggest a reference to https://www.bipm.org/en/work-programme/time as the authority on UTC and TAI would suffice, perhaps with the addition that this is technical background information. On that page is a link to "time department database" where lots of reference material is available. Under "resources", then "time scales" are a number of files, including a list of the 27 leap seconds that have been applied. Relationship between TAI and UTC is instructive in the context of this discussion. The first few lines read:

 RELATIONSHIP BETWEEN TAI AND UTC
 ------------------------------------------------------------------------------- 
 Limits of validity(at 0h UTC)       TAI - UTC  
 
 1961  Jan.  1 - 1961  Aug.  1     1.422 818 0s + (MJD - 37 300) x 0.001 296s
       Aug.  1 - 1962  Jan.  1     1.372 818 0s +        ""
 1962  Jan.  1 - 1963  Nov.  1     1.845 858 0s + (MJD - 37 665) x 0.001 123 2s
 1963  Nov.  1 - 1964  Jan.  1     1.945 858 0s +        ""
 1964  Jan.  1 -       April 1     3.240 130 0s + (MJD - 38 761) x 0.001 296s
       April 1 -       Sept. 1     3.340 130 0s +        ""
       Sept. 1 - 1965  Jan.  1     3.440 130 0s +        ""
 1965  Jan.  1 -       March 1     3.540 130 0s +        ""
       March 1 -       Jul.  1     3.640 130 0s +        ""
       Jul.  1 -       Sept. 1     3.740 130 0s +        ""
       Sept. 1 - 1966  Jan.  1     3.840 130 0s +        ""
 1966  Jan.  1 - 1968  Feb.  1     4.313 170 0s + (MJD - 39 126) x 0.002 592s
 1968  Feb.  1 - 1972  Jan.  1     4.213 170 0s +        ""
 1972  Jan.  1 -       Jul.  1    10s            
       Jul.  1 - 1973  Jan.  1    11s
 1973  Jan.  1 - 1974  Jan.  1    12s	
...

So in 1961-01-01 the difference between TAI and UTC was 1.422818 SI seconds and increasing at a rate of 0.001296 SI seconds per day. That rate was adjusted three more times before 1972 and in addition there were steps of multiples of 0.05 SI seconds. Given that there were still only 86400 seconds in a day, the UTC second did not align with the SI second.

([UTC] was based on that same epoch as TAI, which is 1958-01-01 00:00:00 UT).

exactly, so it's valid from that date, yes? what's the problem here?

The values may have been coincident on that day, but they immediately started drifting apart.

Best, Patrick

[JonathanGregory]

Dear Patrick

Thanks for your comments and the discussion.

• You suggest we should change the first sentence in Sect 4.4.2 defining the utc calendar from "A Gregorian calendar with leap seconds as prescribed by UTC" to "UTC, which includes leap seconds and where dates are represented using the Gregorian calendar." (We don't need to spell out what "UTC" stands for in this sentence, because it is well-known and was anyway already defined earlier in Sect 4.4.2.) Actually, I don't see a difference of meaning between these two versions. Maybe the word "prescribed" is unclear? We could change "with leap seconds as prescribed by UTC" to "with the leap seconds that are included in UTC." Would that be better?

• I agree that it would be helpful to provide a link to Relationship between TAI and UTC. Thanks for suggesting it. We could also link Leap_Second.dat. Both of these files are available for download from BIPM under the CC BY-ND 4.0 licence, I see.

• You're right that we elided the complicated history of adjustments to UTC betwen its invention and 1972. I hadn't grasped that they had "rubber seconds" (as explained in wikipedia more concisely than by BIPM). Because of this, I think you're right that we should prohibit utc before 1972. In CF, a second is always an SI second.

• Like Chris, I don't think it's problematic to use time-zones with utc and tai, because these are CF calendars, not time-zones. A CF calendar defines an ordered set of valid datetimes with integer seconds, as stated in Sect 1.3 and at the start of Sect 4.4.2. We could remind the reader of this in the paragraph defining utc. It's important to be aware of, because the leap seconds are defined in UTC +0 and therefore occur in different hours when there's a non-zero time-zone offset. We should mention this at the end of Sect 4.4.1 as well.

• Chris, thanks for noticing that in Sect 3.1.2 we need to change "A variable must not have a units_metadata attribute if it has no units attribute or if its units do not involve a temperature unit." We overlooked it there. We did remember to update the conformance document, I'm glad to say, which now reads, "A variable must not have a units_metadata attribute if it has no units attribute, or if its units do not involve a temperature unit or a reference time unit."

• We included the julian calendar for completeness in the treatment of leap seconds, because it was easy to do, although there's unlikely to be a CF use-case. The Orthodox churches and some Berber communities (see e.g. Amazigh New Year) use the Julian calendar. Since they don't differ from everyone else in clock time, that means their version of the Julian calendar (in the CF sense) must include UTC leap seconds. Postscript: Acknowledging that leap seconds are unlikely to be needed with the Julian calendar, we could make units_metadata="leap_seconds: none" the default (instead of unknown) for calendar="julian". Then we need not recommend including units_metadata for that calendar. Do you think that would make more sense?

Best wishes and happy New Year (whenever it exactly occurs for you!)

Jonathan

[pvanlaake]

Dear Jonathan,

Happy New Year to you and all other collaborators on the Gregorian calendar! (For those of you on the Julian calendar, do not hesitate to fault me if I forget about your New Year in 9 days' time.)

I remain rather uncomfortable with some of the arguments made in previous comments. As I understand these arguments, they are based more on ease-of-use and every-day practice rather than standards and principles. UTC is one thing and one thing only. Time zones are referenced to UTC so to call a calendar utc and then allow, say, -6 to indicate that the time is actually local time 6 hours behind UTC comes across as a CF-specific quirk that will confuse people more than that it will help them. The fact that in Boulder, CO, there may have occurred an instant of time with the datetime 2015-06-30 17:59:60 -6 doesn't turn Mountain Daylight Time into UTC, so why create that impression through the calendar utc? (This example is not incidental. The conventions document has used an example from the UDUNITS documentation since at least version 1.7 which refers to Mountain Daylight Time, as observed in Boulder, CO, where, probably totally by coincidence, UCAR is located. Perhaps tellingly, the reference to MDT has disappeared in the most recent conventions text. So the explicit reference to daylight savings time has disappeared (and a good thing too!) but now we have the leap second.)

The first use-case that was made for a calendar using UTC would very unlikely be using the local time as reported by a local computer synchronising with an NTP server, and which might result in a datetime as indicated above. And that is probably the case with the large majority of data producers using the CF conventions. In climate and weather forecasting science, practitioners tend to know their clocks, and how to convert from their local time as reported by their instruments and processing environment into UTC. So, again, why the extra whistles and bells?

It should also be considered that utc as currently defined may lead to a confusion between a data producer using utc + timezone, and a data user interpreting that as UTC. Buyer beware!

The CF calendar utc uses the UTC clock and the Gregorian calendar: two separate things. The fact that these two separate entities are conflated into a single CF calendar is the root of the issue we are discussing now.

Best,
Patrick

[JonathanGregory]

Dear Patrick

Thanks for your further thoughts. You comment that the arguments "are based more on ease-of-use and every-day practice rather than standards and principles." I think that's partly true. Perhaps you've seen that the preceding discussions on the issue of leap seconds were extraordinarily long and difficult. It seemed to me that these arguments were often about what things ought to mean "in principle". Because of their different understanding of these principles, the participants in the debate could not agree, and perhaps sometimes could not understand one another properly.

In this case, we could not achieve an consensus about all the principles involved. This is a very unusual outcome for CF discussions, but it's not a show-stopper, because CF is a convention i.e. an agreement, rule, practice or custom. Even though we couldn't agree on what things ought to mean, we did manage to agree some workable conventions. You're not alone in being uncomfortable about some of our choices. That's regrettable, but perhaps can't be helped.

I think the important question to consider is whether there are any hazards with these conventions that are likely to cause data-producers to write incorrect metadata, or data-users to misinterpret the metadata. If there are, we should either clarify the standards document, or consider changing the conventions. This is consistent with one of the principles for design (Sect 1.2):

The conventions should minimise the possibility for mistakes by data-writers and data-readers.

You wrote about some possible hazards before, and I have responded with possible remedies. Thanks for those points. Here, you're suggesting that time zones in utc are a hazard:

It should also be considered that utc as currently defined may lead to a confusion between a data producer using utc + timezone, and a data user interpreting that as UTC.

Timezones have always been allowed in CF datetimes. This convention was iinherited from COARDS, which contains the example in Mountain Daylight Time which you refer to. As COARDS says, that example comes from the UDUNITS documentation. Another of the CF principles is

Conventions are provided to allow data-producers to describe the data they wish to produce, rather than attempting to prescribe what data they should produce.

If you have a data-logger which records local time and is leap-second-aware, the datetimes would most conveniently be expressed in CF using the utc calendar with a timezone offset in the units.

In the example you give, you might have a time coordinate of 64801 with units="seconds since 2015-6-30 0:0:0 -6" and calendar="utc". The convention says, "Subtracting the time zone offset from a given datetime converts it to the equivalent datetime with zero time zone offset." Therefore seconds since 2015-6-30 0:0:0 -6 means the same as seconds since 2015-6-30 6:0:0 in UTC. 64801 seconds since 2015-6-30 6:0:0 is the leap second 2015-6-30 23:59:60 in UTC, hence 2015-6-30 17:59:60 -6.

Where would the confusion arise? Do you think the data-user might ignore the timezone in the units, simply because the calendar is called utc? In that case, maybe we should adopt a different name for this calendar? What about civil, as a suggestion? Wikipedia, Royal Museums Greenwich and Time and Date all use "civil time" to mean local time which differs by an offset from UTC.

Best wishes

Jonathan

[ChrisBarker-NOAA]

I’m on vacation, so brief, but a few notes. Here I have to disagree. The U stands for Universal, UTC time is the same

all over the world at a single instant, irrespective of where you are. Bangkok, London and Denver all have the same UTC time at the same instant.

Of course, but the offset is with respect to UTC, and it is provided as part of the string, so there is no confusion about what it means. And this isn’t a CF-specific oddity — it’s part of the ISO 8601. As you say, CF is indeed focused on “on ease-of-use and every-day practice rather than standards and principles.” “Practicality beats purity” as the Python folks say. Indeed, that’s why UTC has not been supported in CF until now— very little software (and none of the most commonly used tools) support it properly. Note: we should try to use “time zone offset”, or “offset”, rather than “time zone”. Everyone in this discussion knows the difference, but when written down, we should make sure to minimize confusion. One of the practical challenges is that a LOT of people assume UTC means “time at the prime meridian”, I.e. with no time zone offset applied, which it does, but it’s also more specific than that, but that distinction is lost on many people. That rate was adjusted three more times before 1972 …

The values may have been coincident on that day, but they immediately

started drifting apart. Ah, thanks - then I agree— 1972+ makes better sense then.

What about civil, as a suggestion?

I think that would be even more confusing. UTC +- offset is precise and clear and very commonly used.

…

-CHB

[chris-little]

@JonathanGregory I strongly recommend a Best Practice of data creators/servers using UTC timestamps without timezones, as recommended by various restrictive profiles of ISO8601, such as IETF RFC3339. Timezone notation should really be a function of client software that "knows where it is". Timezones, and daylight saving times, can change at the whim of politicians and others. Environmental Data really ought to have a global context. And of course ISO8601 has evolved, and people often reference deprecated versions as they may be freely available unlike the costly current standard.

To quote the W3C: "It is usually desirable to normalize timestamp values to UTC ..." and "When in doubt, use UTC for serializing, storing, and exchanging date and time values."

And too much backward compatability will in the long run build a legacy of technnical debt (aka "rat's nest" or "can of worms")

HTH, Chris

[pvanlaake]

I believe we may be converging on an outcome that is both practical and standards-based; maybe even both practical and pure?

UTC +- offset is precise and clear and very commonly used

Correct, but that is local time or the common understanding of civil time as Jonathan pointed out. So rather than call it utc, I second Jonathan's suggestion to call it the civil calendar (I had been toying with the same idea when walking my dog a few days ago but came up with the local calendar name).

Rather than tossing out UTC as a calendar option, I would propose the following:

• utc calendar, starting 1972-01-01 on the Gregorian calendar, at that instant 10 seconds behind TAI. Time zone must be 00:00 if given. This satisfies the requirements of many data producers publishing global data sets or satellite imagery or basically anything that is not specifically tailored to a small geography.
• An additional civil (or local) calendar, based on UTC and the Gregorian or Julian calendar but using a time zone offset (and possibly supporting daylight savings time (on the data producer side!) through the correct use of tz offsets). Personally I prefer the local name as that conveys the fact that the data may be tied to a specific geography and requiring adjustment to align with other time series.

Would something like that work?

As a final note on standards versus practice: The CF conventions are far more influential than you make it sound. The decisions reached on issues such as calendars reverberate throughout the larger CF community and materialize in data sets that hang around for decades. The netcdf library itself was modified in response to CF conventions. I'm just putting in my 2 cents to propose something that may still be valid and logical in a few decades` time.

[davidhassell]

Hi,

On the 1958-1972 issue: I think we should keep the 1958 epoch. This change in approach before/after 1972 was indeed considered during the creation of the new text, and the wording we used was deliberately chosen to allow for any frequency of leap second application (e.g. daily), and an individual adjustment being any amount of seconds (e.g. negative amounts and non-integer amounts). I think that what we have now is entirely consistent with the messy situation that occurred in 1958-1972, so there is no need to change the minimum valid time. If you can deal with leap seconds applied after 1972, then dealing with the situation before 1972 (should have such data) is logically no different. (Whilst it currently seems unlikely, the leap second situation could get more messy again in the future, in which case the conventions will still hold, and we'dbe glad we didn't arbitrarily remove part of the valid time period.)

On the timezone/calendar-name issue: To me, the use of a timezone offset is merely an encoding choice in the units attribute string value in the dataset, just as choosing "seconds since" or "days since" is your units string is. Therefore I see no reason disallow their use, and no reason to change the CF calendar name. Also, if we changed the CF utc calendar name, surely we'd have to change the tai calendar name, too? (which I don't think we want!).

Cheers,
David

[larsbarring]

This is more of a general comment than a response to any of the previous comments in this discussion.

I think that we have to careful about what words we use in relation to all this. For example what do we mean by the word "calendar" (as a general concept) and when do we mean calendar as a CF construct. Is "time"/"timestamp" different from "date"/"datestamp" and from "datetime"/"datetimestamp"? Are they just the same, even across diffent calendars? What is a "clock", is it just a physical mechanism providing ticks and tocks at regular intervals,? And then what is a "regular interval" if you have two clocks?

These are maybe silly questions. But at the same time, CF is trying to make several different "clocks" and "calendars" come together into one concept; the CF calendar construct as a representation of time coordinates. From my own perspective, I think that Chris Little's presentation⁽¹⁾ at the 2024 CF Conventions Workshop was very useful, and in this context particularly viewgraphs 17-19.

I think that CF would be well served by general discussion about all this. For example, there seems to be a strong push towards having the "datetimestamp" labelling (strings such as "2025-01-07 12:00:00Z") as a vehicle for unifying points in time across different calendars. This can be a very useful and pragmatic solution that is legitimate and accepted in many situations. But in other situations such a solution cannot be justified or even make it invalid, whereas other pragmatic solutions exist. Trying to fold any such pragmatic solution into the CF Conventions is doomed to cause trouble and limit its usefulness.

(1) Little, C. (2024, September 17). OGC Temporal Domain Working Group Activities. 2024 CF Workshop, Swedish Meteorological and Hydrological Institute (SMHI), Norrköping, Sweden. https://doi.org/10.5281/zenodo.14194152.

[JonathanGregory]

Dear Lars

You're right that we have to be careful about words. In the world as a whole, the words you mention may be used in various ways. If there is inconsistency, we can't help it. Within the CF convention, however, we can be careful to use our words consistently. I hope that within the convention document we always mean "calendar" and "datetime" in the senses with which they're defined in Sect 1.3. On briefly looking, I believe that "date" separately always refers to just the year-month-day part of the datetime. A "time" coordinate represents a datetime, of course, not just a time of day; there's a lot in Sect 4.4 about it! In the standards document, "stamp" occurs only once, in Appendix B; we could delete it. The word "clock" occurs only once, in Sect 4.4.1; we could replace it. The phrase "regular interval" occurs only once, in App J, where it refers to distance rather than time.

Best wishes

Jonathan

[larsbarring]

Dear Jonathan,

I was more thinking of the different discussions, both here and in all the preceding discussions and issues on time/calendars/etc over the years. It is very easy to misunderstand or simply miss a point just because different understanding of concepts behind words. I am sure that we all have done it. Hence my general view is that CF has not done a very good job in what I mentioned as: "CF is trying to make several different "clocks" and "calendars" come together into one concept; the CF calendar construct as a representation of time coordinates." But this is a huge and difficult topic as we all are aware of. My point is that in order to make some progress in disentangling all this, it is useful to look at what others are doing. Hence my reference to Chris's presentation as food for thought.

[ChrisBarker-NOAA]

From my own perspective, I think that Chris Little's presentation⁽¹⁾ at

the 2024 CF Conventions Workshop was very useful, and in this context particularly viewgraphs 17-19.

I think that CF would be well served by general discussion about all this.

Is there a freely available summary of the new ISO 34000 ?

https://www.iso.org/standard/77019.html

Or do any of you have access to a copy? It seems that could be helpful to reference.

…

-CHB

[ChrisBarker-NOAA]

If you can deal with leap seconds applied after 1972, then dealing with the situation before 1972 (should have such data) is logically no

different. IIUC ( and I may not) it IS logically different — that is, rather than applying a “leap second” integer or not, at particular times, there was a slight adjustment to the size of every second in a range (did someone call them rubber seconds?). Which is a different approach, and would require different code to handle. @pvanlaake: could you please confirm? My concern about the addition of a UTC Calendar is that it is an attractive nuisance -- a lot of people *think* they are using UTC, when they are not (quite) -- and the software they are using may not support it properly. Allowing pre-1972 may make it even more likely for not-really-UTC data to be presented as UTC.

[davidhassell]

Ah, interesting. I though the changes were applied daily, rather than every second. Clarification would be very useful!

However, applying an increment annually, daily, hourly, even every second, is logically the same, surely?

[pvanlaake]

As I mentioned in my initial post in this thread, the second used by UTC in the period 1961 - 1972 is not the SI second. The table in my reply to Chris shows the rate at which the UTC second differs from the SI second used by TAI, expressed as a total amount per day but applied to the length of the second. The whole point of introducing leap seconds as of 1972 was to make UTC run on the SI second, which it previously didn't.

However, applying an increment annually, daily, hourly, even every second, is logically the same, surely?

Surely. As surely as applying an increment every second would make a mockery of claiming to use UDUNITS. And I do recall seeing some discussion about sub-second precision...

[davidhassell]

Hi Patrick,

I'm sorry, but I don't understand the UDUNITS analogy - could you expand (are you agreeing or disagreeing :))?

One of the reasons that the original discussions took so long was trying to understand the relationship between UTC time as stored in a CF dataset and elapsed time in the world (real or idealised) being measured.

Consider the difference between the UTC datestamps 1961-03-01 0:0:0 and 1961-03-01 0:0:1 (a rubber second), and the difference between the UTC datestamps 2016-12-31 0:0:0 and 2017-01-01 0:0:0 (includes the most recently applied leap second). Non-leap-second-aware software would calculate these differences as 1 SI second and 1 SI day respectively. Leap-second-aware software would give the differences as 0.999999985 SI seconds (a reduction of 15 ppb) and 1.000011574074074 SI days (86401 SI seconds), respectively.

I hope this illustrates why I think that the pre-1972 era is logically no different to the post-1972 era. Messier, for sure, but not really different!

Cheers,
David

[pvanlaake]

Hi David,

The "rubber second" is compressed (of shorter duration) relative to the SI second. So it is not a question of "applying an increment every second" - there is no such increment, it is a rate of divergence (my terminology) baked into the second that the UTC clock pre-1972 used. There is a computational solution, for sure, but at a significant cost of complicating code. And again, the leap second was introduced precisely because the pre-1972 system was considered too cumbersome.

Your suggestion has a very awkward fit with the concept of "second" as used otherwise in the conventions. Throughout the #304 thread, it is brought up on a few occasions that the "second" in CF is the SI second.

Is there a compelling need for CF support of the period 1961-1972? (1958-1960 is a real no-no because there was no UTC during that period and there are no published clock rates or step increments.) Can you identify a single data set that has true UTC datetimes that cover that period? Would it not be better to exclude this period until someone expresses that compelling need?

[davidhassell]

Hi Patrick,

Firstly, I'm more than happy to restrict ourselves to after 1972. It will also prevent anyone from the unlikely event of re-writing existing data from before 1972 using the utc calendar (which is almost certainly not what they want to do!).

This doesn't change my assertion that the text is "not wrong" for the pre-1972 period, but I think we're splitting hairs on different conceptual views of the same thing, so I'll stop going on about it, now :).

With regards use cases, as I recall the original use case was for a tai calendar in the remote sensing community, but we felt you can't introduce TAI with dealing with the concept of leap seconds, which we felt was best done by introducing a corresponding utc calendar to properly define it.

Cheers,
David

[ChrisBarker-NOAA]

“”” Non-leap-second-aware software would calculate these differences as 1 SI second and 1 SI day respectively. Leap-second-aware software would give the differences as 0.999999985 SI seconds “”” Well, “rubber seconds” aren’t the same as leap seconds, though they serve a similar purpose. With Leap seconds, seconds are always an integer. So we *could* say the “fully UTC compliant software could manage that” it can’t be handled with integer seconds. In fact, it looks like you’d need finer than microseconds, so pretty problematic. (And single precision float wouldn’t work either). And in any case when mapping between calendars, you couldn’t do it at all if the otgg bf er calendar only gandkk k ed integer seconds … So yes, it’s still a well defined transform, but from a practical perspective it’s a pretty different beast. Again, I’m concerned about the attractive nuisance — is there ANY software that can handle this correctly? So wouldn’t it be better to disallow it? Hmm — thinking now, it might actually work better to use the standard calendar in that case. You could use unaware software to transform back and forth, and it would be lossless, unlike leap seconds, where that almost works, except when there is an actual leap second, as most (all) non-leap second aware software doesn’t allow a 60th second.

[pvanlaake]

it might actually work better to use the standard calendar

This is also my thought. Allowing the inclusion of 23:59:60 into a datetime of a standard, proleptic_gregorian or julian calendar with units_metadata = "leap_seconds:utc" would solve many of the issues we have been discussing.

That opens up the issue of older software not being able to properly read newer data sets but that is a different discussion. (Software tends to follow new conventions. Should older software, i.e. of the kind that is no longer kept up-to-date, be actively "safe-guarded"? That is a rather unusual arrangement in the computing world and really holding back these conventions.)

[larsbarring]

CF is usually enhanced, expanded or changed based on concrete use cases. Over the years it has been argued, and accepted, that there are enough use cases to include the UTC and TAI calendar, where the latter was more of an complement to the UTC calendar.

However, concrete use cases are missing, even though there have been many references to that the satellite folks need it. And to that there are high-resolution measurement that may need this kind of precision. When I asked a well-placed EUMETSAT person I got the response that they reset their clocks at midnight (we did not go into any further details, so this is a bit of hearsay). And the only recent concrete high-frequency data collection example was from David @davidhassell, where they used UTC but was not even aware of the concept of leap seconds (David please correct me if I have mis-interpreted something). Way back I was involved in field measurements using sonic anemometers where the raw 20 Hz output was stored and the system clock was in the measurement hardware. But the final data was then reduced to 5- or 10-min covariances (etc), meaning that the second precision was not needed anymore. Similar systems today I imagine would either use NTP or GPS as clock.

All in all, without concrete uses cases we are not on firm ground, and I would really like to have at least one concrete use case to help us here.

[JonathanGregory]

Dear all

I think David is correct in asserting that the current text is not wrong, because in Sect 4.4.3 we say, for integer-second datetimes, "a time coordinate value expressed in seconds equals the number of valid (integer-second) datetimes after (not including) the reference datetime in the units up to (and including) the datetime that the time coordinate represents." That is, the time coordinate value is not the length of the time interval in SI seconds between the two instants. It is actually the count of the number of valid integer-second datetimes between them. These two quantities are equal except when a leap second intervenes in calendars where leap seconds are not valid datetimes, most importantly the standard calendar.

In that sense, rubber seconds are less problematic than leap seconds because, as Chris remarked, the standard calendar has the correct set of datetimes for that situation. There are always 86400 integer-second datetimes in a day in UTC between 1960 and 1972, so the encoding and decoding of time-coordinates by counting integer datetimes is easy!

However, the current text isn't adequate in Section 4.4.2, because we don't mention that seconds aren't SI seconds in UTC before 1972, or about the 0.1 s and 0.2 s jumps. As a curiosity, how did they deal with these in datetimes? If 0.1 s was inserted at the end of a day, for instance, did the clock jump from 23:59:60.1 to 0:0:0?

The simplest solution is to disallow the utc calendar before 1972, as Patrick suggested. I think we must also insert a further comment, however, about time-coordinates before 1972-1-1 0:0:0 UTC in all real-world calendars except tai, which is precise, and allowed back to 1958.

In other calendars, before 1972, when real-world datetimes are encoded as time-coordinates, the difference between two time-coordinates is in general not exactly equal to the length of the time-interval between them in SI seconds. Obviously it can't be, before the atomic clock was invented. The further back you go, the less accurate and universal the second was. For me, this underlines that the correct general intepretation of a real-world time-coordinate is as an encoded datetime. This encoding is additionally useful because, since 1972, and except for leap seconds, the difference between two time-coordinates equals the length of the intervening interval, but that's not the fundamental definition. Of course, in idealised model calendars, it's always exactly true.

Best wishes

Jonathan

[pvanlaake]

Recapping the discussion of the past weeks to see where we stand on the different issues:

• There appears to be a consensus to restrict the utc calendar to start at the datetime "1972-01-01 00:00:00".
• An argument has been made to not allow time zone offsets on either utc or tai calendars because the clocks used by these calendars do not use the concept of time zones. Where a time is expressed with a time zone offset from UTC it is "civil" or "local" time. Using utc as the name of a CF calendar suggests that datetimes are in "real" UTC. Perhaps have a "real" utc calendar and an additional civil or local calendar? These arguments are not supported by several contributors so the issue is still open (from my perspective anyway).
• Concrete use cases should guide the definition of CF calendars. (An existing data set using TAI with a time zone offset comes to mind...)

Perhaps for CF-2:

• Base the description of the entire section 4.4 on ISO 34000 and/or OGC terminology.

[JonathanGregory]

Dear Patrick and Antonio

While I appreciate your concern and your thinking about it, I have to say that I disagree with changing the approach that we adopted in CF1.12. You're probably not surprised to hear me say that! It took an immense amount of work and discussion by quite a few people over many years, as you've probably seen, to devise something that every agreed to accept, and to describe it clearly in the document. We can now deal with the use-cases we know about, including data already archived, so we don't need to reconfigure the arrangements.

During this discussion, thanks to Patrick's comments, we've identified some changes we can adopt without radical overhaul, including:

• Adding more information about the relationship between TAI and UTC, including links to BIPM documents.

• Starting the valid datetimes in the utc calendar from 1972 instead of 1958.

• Renaming the utc calendar to avoid the possible misunderstanding that it can only be used with UTC+0 times. Karl and I have both suggested utc_based. We don't have a use-case for a UTC-based CF calendar that does not allow time zone offsets. They're optional anyway.

Some other smaller clarifications in the text would also be good, arising from this discussion.

As you strongly suspect, if I have to choose between practicality and purity for the CF conventions, I choose practicality. CF is a practical tool. That's not to imply that purity is useless. On the contrary, a carefully thought-out logical self-consistent framework (some characteristics of "pure", according to my way of thinking), is usually a simple convention to understand, implement and interpret. We always have abstractions in mind. That's why CF has a data model, for instance.

I think what we have put in CF1.12 is logical, self-consistent and simple, except for the awkward part that comes from dealing with existing data. If we were starting from scratch, we would distinguish between standard calendars for timelines with and without leap seconds—but we aren't starting from scratch.

Sorry if that's annoying. That's not my intention. Happy weekend

Jonathan

[ChrisBarker-NOAA]

I agree with Jonathan here. A couple points:

based on:

Specification of time standards:
A time_standard Attribute: This attribute would explicitly state whether the dataset uses UTC, TAI, or another standard.
Expanding the units_metadata Attribute: This would include additional metadata about leap second handling (e.g., units_metadata="UTC: with leap seconds").

Then:

Your suggestion provides an elegant solution to maintain backward compatibility while offering flexibility for advanced use cases

well, not entirely backward compatible, or if we do make sure it is, then we lose the flexibility:

• To be fully backward compatible, any of the previously-defined calendars wouldn't be allowed to use the units_metadata attribute (or only with one allowed value, which is the same thing)

units_metadata = "leap_seconds: unknown"

ouch! leap_seconds are not "unknown" in the older calendars -- they are known not to be used. (in practice, there may be some non-compliant files with leap. seconds in the field, but ....) [well, standard is weird -- there may be leap seconds in the original timestamps, but the encoding is known not to be computed with leap-seconds, and no timestamps with an actual leap-second (eg 61st second) is allowed.]

NOTE: that there was a long discussion a while back about adding a "started-from-utc-timestamps-but-encoded-with-non-leap second-aware-software" -- that wasn't done because (credit to Jonathan for his persistence) that's exactly what the "standard" calendar IS.

I can see the logical appeal of a separate calendar and clock that could be combined in various ways to allow a matrix of combinations. But as you point out in your point (2) -- many (most) of the combinations make no sense -- it would be a very sparse matrix indeed. So even without the legacy -- explicitly naming out the entries in that matrix that actually make sense is perfectly fine.

As we know from this discussion: In practice, a CF "calendar" is a particular combination of a calendar and a clock -- which is totally fine. Perhaps there's a bit of addition to the text that might make this more clear?

[pvanlaake]

Backwards compatibility is always restrictive compared to a current specification, it is a service to past efforts but unavoidable. And in this case that does not mean that defined calendars cannot be extended with new functionality, it only means that data sets written with the old calendar specification are still valid in the new specification albeit with a default setting for any functionality that was not previously available.

ouch! leap_seconds are not "unknown" in the older calendars -- they are known not to be used

False and ouch back to you. The conventions state that "unknown means that the data-writer did not know or did not record whether the leap seconds exist in the timeline, nor how they are treated if they did exist".

[well, standard is weird -- there may be leap seconds in the original timestamps, but the encoding is known not to be computed with leap-seconds, and no timestamps with an actual leap-second (eg 61st second) is allowed.]

How to square this with the Zen of Python? "Simply is better than complex" does come to mind but not in the affirmative.

it would be a very sparse matrix indeed

That's way more Zen: "Sparse is better than dense". Apart from the fact that I never spoke in terms of a matrix, what is the problem with limiting the allowable combinations? The conventions do it all over the map already. And that is exactly what the conventions are about: agreeing on what is allowable and on how to describe it.

There should be one– and preferably only one –obvious way to do it.
Although that way may not be obvious at first unless you’re Dutch.

I hope that's the last I have to hear about the Zen of Python in this discussion.

[ChrisBarker-NOAA]

Backwards compatibility is always restrictive compared to a current specification, it is a service to past efforts but unavoidable. And in this case that does not mean that defined calendars cannot be extended with new functionality, it only means that data sets written with the old calendar specification are still valid in the new specification albeit with a default setting for any functionality that was not previously available.

It means that the old spec needs to mean the same thing. So we can't allow folks to add a new attribute to an existing calendar that changes the meaning -- (older) software that doesn't look for the new attribute should still work.

[well, standard is weird -- there may be leap seconds in the original timestamps, but the encoding is known not to be computed with leap-seconds, and no timestamps with an actual leap-second (eg 61st second) is allowed.]

How to square this with the Zen of Python? "Simply is better than complex" does come to mind but not in the affirmative.

no one is saying that this is ideal, or how we would design it from scratch -- it's a mess, but it's the mess we have.

it would be a very sparse matrix indeed
That's way more Zen: "Sparse is better than dense".

Right -- my point is that it's not helpful to support a matrix if, in fact, that matrix is very sparse -- the "linear" version is just fine.

Apart from the fact that I never spoke in terms of a matrix, what is the problem with limiting the allowable combinations?

Nothing -- that's my point -- we DO need to limit the allowable combinations to the point that it's not really helpful to define it as though there were many options:

If we had N calendars and M clocks that could be combined in a variety of ways, that would be the way to do it.
But in fact, most calendar/clocks have a 1:1 relationship, with a few (one?) a 1:3 relationship:

That is, "utc" and "tai" are only useful with the gregorian calendar, yes?

However, if I misunderstand, and you are only proposing that when we talk about it, we make the distinction between a calendar and a clock -- then sure. That's a fine idea -- propose away some better text! 

[davidhassell]

Hello,

Chris said "a CF "calendar" is a particular combination of a calendar and a clock". Indeed, and this has always been the case in CF, even if the "clock" part has been implicit. I don't think that redesigning the whole encoding of calendars (into calendar and clock) will help us resolve the current conceptual questions over leap seconds, so if we were to have that discussion I suggest we have it elsewhere :-)

Edit: The following example was written with a typographical error ("09:00-7" was meant, but "16:00-7" was written) . See https://github.com/orgs/cf-convention/discussions/400#discussioncomment-11891114 for the corrected version. I have left the original here, struck through, because of the subsequent comments pointing out the mistake!

If I'm reporting daily observations from a weather station in Boulder Colorado U.S. at 09:00 Mountain Time (7 hours behind UTC), then I might have a time coordinate that is encoded as either

• data: 0, 1, 2, units = days since 2025-01-01 16:00-7

or

• data: 0, 1, 2, units = days since 2025-01-01 16:00

Surely both of these encode the same datetimes: 2025-01-01 16:00Z, 2025-01-02 16:00Z, 2025-01-03 16:00Z. The one with the non-zero offset merely highlights that the observations were taken when civil time was 9 a.m. in Boulder. I would say that there is no ambiguity, here.

Given that, I don't agree that there is any likelihood of time offset confusion with the new calendars.

I agree that it may be a bit less than clear what the utc calendar is from that name, but just because the name looks like UTC (which is not a calendar outside of CF). I don't like utc_based, because the standard calendar might also be UTC-based! I propose standard_utc, alongside standard_tai. These, I think, would make it clear that these are "standard" calendars with, relative to the un-suffixed standard calendar, specific treatments.

Cheers,
David

[pvanlaake]

Surely both of these encode the same datetimes

No, they do not. Considering that the conventions state that "the default time zone offset is zero" they are, in fact, 7 hours apart. It appears there may be some time zone offset confusion here.

I propose standard_utc, alongside standard_tai.

This starts to look like the N x M matrix that Chris wants to avoid. Because now you also need to define julian_utc and julian_tai.

[Armin-RS]

Hi @davidhassell,

in my email programme, an email which arrived at 13:29 local time (which is Europe/Berlin or Central European Time (CET) in my case) has this time-stamp in the header:
"20 Jan 2025 13:29:22 +0100".

So the time-stamp itself is in local time and the "+0100" time-zone designator (belonging to CET) gives the relation to UTC, i.e. the email arrived at 12:29 UTC.

So "2025-01-01 16:00Z" is "2025-01-01 16:00 UTC", while "2025-01-01 16:00 -7" is "2025-01-01 23:00 UTC".

Cheers,
Armin

[pvanlaake]

Hi @Armin-RS,

"2025-01-01 16:00 -7" is "2025-01-10 09:00 UTC"

2025-01-01 23:00Z

[Armin-RS]

Hi @pvanlaake, yes, of course you're right. Maybe I'm too used to living in a positive time-zone offset country.

I've corrected my post to avoid further confusion.

[davidhassell]

Hi @pvanlaake and @Armin-RS,

No, they do not. Considering that the conventions state that "the default time zone offset is zero" they are, in fact, 7 hours apart. It appears there may be some time zone offset confusion here.

Oops. I made a bit a bad mistake in my example - sorry, and and thanks for putting me right :) However, that doesn't change my argument. I'll have another go ...

"If I'm reporting daily observations from a weather station in Boulder Colorado U.S. at 09:00 Mountain Time (7 hours behind UTC), then I might have a time coordinate that is encoded as either

• data: 0, 1, 2, units = days since 2025-01-01 09:00-7

or

• data: 0, 1, 2, units = days since 2025-01-01 16:00

Both of these encode the same datetimes: 2025-01-01 16:00Z, 2025-01-02 16:00Z, 2025-01-03 16:00Z. The one with the non-zero offset merely highlights that the observations were taken when civil time was 9 a.m. in Boulder. I would say that there is no ambiguity, here."

How's that?

Because now you also need to define julian_utc and julian_tai.

I don't think so. The current utc calendar is defined as a variant of the standard calendar, and no others, because that is the use case. As Jonathan has described previously in this thread, the julian and proleptic_gregorian calendars get a mention only because in their past usage we don't know what the leap second situation was, so they need to get the units_metadata treatment. So, if someone wants a julian_utc calendar, then they can't right now - regardless of what we call the current utc calendar - and if they wanted it in a future version they would have to raise an issue in the normal manner to create this new option.

[davidhassell]

A short one - I've just noticed that the original 2018 discussion named new gregorian_utc and gregorian_tai calendars. I had forgotten that :-) This was in the days before gregorian was deprecated in favour of standard ...

[larsbarring]

Based on my experience from previous CF debates on this topic I am very hesitant to engage in this. Thus I have not followed this discussion closely, only now and then read up on the progress. However, there have been a few comments that I would like to comment on (in no particular order):

• Patrick @pvanlaake asked:

  Is it of no or minor concern that a new entry in these conventions is incongruous with an international standard, even if the definition of the new entry is otherwise correct?

  To which my answer is: this would certainly be an unfortunate outcome that do not bode well for CF, and any steps in that direction I would find extremely difficult to support.

• Jonathan @JonathanGregory wrote:

  ,,, an immense amount of work and discussion by quite a few people over many years, as you've probably seen, to devise something that every agreed to accept, and to describe it clearly in the document ...

  To which I would like to remind that, while I did not here CF github oppose to the proposal that was accepted accepted last autumn, I made it very clear in a rather extensive offline email conversation with Jonathan and David (@davidhassell) that I did not support the proposal, but also stated that I would not "officially block" (if there is such a thing) it here in the forum. But to state that it was "...something that every agreed to accept..." is to draw it a bit too far.

The reason why I did not want to oppose the proposal was that back then it received support from knowledgeable people, and I simply didn't (and still don't) have time or inclination to engage deeply in this endless debate where we are just going round in circles without any progress. To me there is too much confusion and misunderstanding about basic concepts, terminology, names and words here. On the contrary I find the OCG Abstract Conceptual Model for time to be both clear and consistent and written in reasonably simple language. It is available from this web page, and from this direct link.

[JonathanGregory]

I'm sorry you think I misrepresented your position, Lars. For the record, it was to acknowledge your reservations that I wrote "agreed to accept" rather than "agreed". In general, according to my understanding, "consensus", which is what we always aim at, means something we can all agree to accept. It doesn't always mean that we all agree that every aspect is the best decision.

[ChrisBarker-NOAA]

Because now you also need to define julian_utc and julian_tai.

Actually, that was my point -- there is no use case for julian_utc or juliam_tai

Why in the world would anyone want to use the Julian calendar with that level of precision in modern times??

But this brings up an idea for names -- "gregorian_utc" and "gregorian_tai" ?

( I guess a bit like how we use a naming scheme like that in standard names ...)

Lars -- I can't recall what your concerns were, but I think we're close here. What's still on the table:
now:

1. What name to use -- there's concern that simply "utc" isn't ideal, because it's not exactly the same in definition as utc -- that doesn't bother me, but to keep quoting teh zen of python (sorry, Patrick!) -- explicit is better than implicit, so if we can find a better name -- great!

2. Whether to allow timezone offsets -- I think that only Patrick is worried about that -- but maybe with a new name, he'll be OK with it ?

Have I missed siomething?

[davidhassell]

I agree, Chris, I suggested similar names in https://github.com/orgs/cf-convention/discussions/400#discussioncomment-11891114 (noting in https://github.com/orgs/cf-convention/discussions/400#discussioncomment-11894036 that "standard" is preferred to "gregorian", meaning that it might be better to add a suffix to the preferred name, rather than a deprecated one)

[JonathanGregory]

I think that your (1) and (2) are linked, Chris, since Patrick's objection to utc as a name is partly that UTC does not have time zones.

I recall the proposal of calendar names like standard_utc. In CF1.12 we adopted units_metadata leap_seconds to make the distinction instead.

[JonathanGregory]

Dear all

The CF committee discussed this matter today. We decided it would be helpful to convene an online group to talk about it, because we think that would be more effective than discussion in writing. The group should consider:

• Whether there are any flaws in the CF1.12 convention for time coordinates (sec 4.4). That means conceptual mistakes that we made in formulating it, such that data written according to the convention might not be interpretable correctly and unambiguously. We're not aware of any flaws (in this strict sense), but if there are any we must rectify them.

• Whether there are any respects in which CF1.12 is not interoperable with ISO and OGC time standards, and how we can make the correspondence clearer where it is useful to do so. We don't expect it to include all their concepts, nor that they should include all of CF's, because each standard has a distinct set of applications.

• Where the text of CF1.12 should be clarified or the substance improved. Some possibilities have already been discussed here, such as to exclude datetimes before 1972 from the CF utc calendar.

If the group identifies changes for consideration, they will be proposed and discussed on GitHub as usual.

If you'd like to join this group, please say so here within the next three weeks (before 18th February). Thanks

Jonathan

[pvanlaake]

Dear Jonathan,

Thank you and the Committee for your support for this. I would like to join the group.

Best,
Patrick

[larsbarring]

Please include me. / Lars

[sethmcg]

I'm in!

[cofinoa]

I would like to join the group.

[davidhassell]

I would like to join the group.

[davidhassell]

Thanks, Jonathan. I would like to join the group.

[ChrisBarker-NOAA]

please add me, too -- I'll join if I can.

[JonathanGregory]

Thanks to the above for expressing interest in this discussion. I have sent you a poll to find a date. If there's anyone else who would be interested, please say so.

[JonathanGregory]

Dear @chris-little

I'm replying to your comment above, where you wrote, "I strongly recommend a Best Practice of data creators/servers using UTC timestamps without timezones, as recommended by various restrictive profiles of ISO8601." The group which convened to continue this discussion viva voce agrees with you! As a result of our discussions, we hope to have revised text of the calendar section before the CF workshop, and definitely in time for CF 1.13. Among other things, we agreed:

• That CF 1.13 disallow the use of time-zone offsets in the reference date-time of the units of a time coordinate variable in the utc and tai calendars. This is a backward-incompatible change (in the generous interpretation of principle 9 of Sect 1.2), but we don't think it matters because these calendars were newly introduced in CF 1.12 and they probably have not yet been used in archived data.

• In the other calendars (which all existed before CF 1.12) numeric time-zone offsets are allowed (as they always have been), but we propose recommending in CF 1.13 that time-zones should not be used. This is because it's easy to make mistakes with them, especially with the sign and in allowing for seasonal changes in clocks. (Principle 7 in Sect 1.2 is "The conventions should minimise the possibility for mistakes by data-writers and data-readers.") A time-zone offset is simply a number of hours to be added or subtracted when converting between time coordinates and date-times. Exactly the same can be achieved by modifying the time coordinates i.e. converting them to the zulu time-zone. Note that named time-zones e.g. GMT and MDT are never allowed. CF 1.12 says so in 4.4.1, but we could say it more strongly.

Best wishes

Jonathan

[ChrisBarker-NOAA]

... we propose recommending in CF 1.13 that time-zones should not be used.

I must have missed that meeting. I disagree. I think offset are totally appropriate when used for data that are already "local time" , e.g. have an offset.

This is because it's easy to make mistakes with them, especially with the sign and in allowing for seasonal changes in clocks. (Principle 7 in Sect 1.2 is "The conventions should minimise the possibility for mistakes by data-writers and data-readers.")

Exactly -- so why invite mistakes by requiring people to convert when the save the data??? I"m found it's a helpful principle to record data the way it come in, without doing any conversion -- use the units they data come in as, and use the timezone offset the data comes in with.

I work with this issue a LOT -- and in my work, folks usually want data in "local time" -- and it a whole lot easier (and less error prone) to use an offset than to have to convert when writing the data and then convert back.

Personally, I think using an offset with the UTC calendar is also reasonable -- what differs the UTC calendar from the standard calendar is the issue of leap seconds, not timezone offsets. But I do understand that principle, and the confusion around what "UTC" means so I'm not going to argue that one.

Note that named time-zones e.g. GMT and MDT are never allowed.

Totally agree with that.

Anyway, I believe this is a discussion for a hackfest, yes?

[pvanlaake]

I must have missed that meeting

Seeing that you commented on the section on "agreements reached" in the minutes of that meeting, I am afraid you are having a momentary memory lapse here. The points that Jonathan flagged are pretty much verbatim taken from that section.

[ChrisBarker-NOAA]

I am afraid you are having a momentary memory lapse here

All too likely :-(

Though I don't think I would have agreed to not recommending time zone offsets in any context at all.

I guess it comes down the final choice of words.

This isn't a blocker for me in any case.