Skip to content

[libc++] Enable experimental tzdb on Apple platforms #122010

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

[libc++] Enable experimental tzdb on Apple platforms #122010

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
d335a22
[libc++] Enable experimental tzdb on Apple platforms
ldionne Jan 7, 2025
ecef594
[WIP] Fix some tests for binary leap second parsing
ldionne Jan 8, 2025
957fca1
WIP: Try using the binary tzdb unconditionally on all platforms
ldionne Jan 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion libcxx/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ option(LIBCXX_ENABLE_MONOTONIC_CLOCK
#
# TODO TZDB make the default always ON when most platforms ship with the IANA
# database.
if(${CMAKE_SYSTEM_NAME} MATCHES "Linux")
if(${CMAKE_SYSTEM_NAME} MATCHES "Linux|Darwin")
set(ENABLE_TIME_ZONE_DATABASE_DEFAULT ON)
else()
set(ENABLE_TIME_ZONE_DATABASE_DEFAULT OFF)
Expand Down
2 changes: 2 additions & 0 deletions libcxx/docs/ReleaseNotes/20.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,8 @@ Improvements and New Features
optimized, resulting in a performance improvement of up to 2x for trivial element types (e.g., `std::vector<int>`),
and up to 3.4x for non-trivial element types (e.g., `std::vector<std::vector<int>>`).

- Experimental support for ``std::chrono::tzdb`` has now been implemented on Apple platforms.

Deprecations and Removals
-------------------------

Expand Down
104 changes: 67 additions & 37 deletions libcxx/src/experimental/tzdb.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,8 @@ namespace chrono {
_LIBCPP_WEAK string_view __libcpp_tzdb_directory() {
#if defined(__linux__)
return "/usr/share/zoneinfo/";
#elif defined(__APPLE__)
return "/usr/share/zoneinfo/";
#else
# error "unknown path to the IANA Time Zone Database"
#endif
Expand Down Expand Up @@ -94,14 +96,23 @@ static void __skip(istream& __input, string_view __suffix) {
}

static void __matches(istream& __input, char __expected) {
if (std::tolower(__input.get()) != __expected)
std::__throw_runtime_error((string("corrupt tzdb: expected character '") + __expected + '\'').c_str());
_LIBCPP_ASSERT_INTERNAL(std::islower(__expected), "lowercase characters only here!");
char __c = __input.get();
if (std::tolower(__c) != __expected)
Comment on lines +100 to +101
Copy link
Member

@mordante mordante Jan 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
char __c = __input.get();
if (std::tolower(__c) != __expected)
if (char __c = __input.get(); std::tolower(__c) != __expected)

The same for the hunk below. I would suggest to commit these two hunks directly to main as an NFC patch.

std::__throw_runtime_error(
(string("corrupt tzdb: expected character '") + __expected + "', got '" + __c + "' instead").c_str());
}

static void __matches(istream& __input, string_view __expected) {
for (auto __c : __expected)
if (std::tolower(__input.get()) != __c)
std::__throw_runtime_error((string("corrupt tzdb: expected string '") + string(__expected) + '\'').c_str());
for (auto __c : __expected) {
_LIBCPP_ASSERT_INTERNAL(std::islower(__c), "lowercase strings only here!");
char __actual = __input.get();
if (std::tolower(__actual) != __c)
std::__throw_runtime_error(
(string("corrupt tzdb: expected character '") + __c + "' from string '" + string(__expected) + "', got '" +
__actual + "' instead")
.c_str());
}
}

[[nodiscard]] static string __parse_string(istream& __input) {
Expand Down Expand Up @@ -621,16 +632,18 @@ static void __parse_tzdata(tzdb& __db, __tz::__rules_storage_type& __rules, istr
}
}

// This function parses the leap-seconds "binary file" compiled from the .list file
// by the zic compiler. That format is widely available as it comes by default with
// the IANA Time Zone Database.
//
// The format looks like:
//
// # Leap YEAR MON DAY 23:59:60 + S
// Leap 1972 Jun 30 23:59:60 + S
// Leap 1972 Dec 31 23:59:60 + S
// Leap 1973 Dec 31 23:59:60 + S
//
static void __parse_leap_seconds(vector<leap_second>& __leap_seconds, istream&& __input) {
// The file stores dates since 1 January 1900, 00:00:00, we want
// seconds since 1 January 1970.
constexpr auto __offset = sys_days{1970y / January / 1} - sys_days{1900y / January / 1};

struct __entry {
sys_seconds __timestamp;
seconds __value;
};
vector<__entry> __entries;
[&] {
while (true) {
switch (__input.peek()) {
Expand All @@ -648,27 +661,50 @@ static void __parse_leap_seconds(vector<leap_second>& __leap_seconds, istream&&
continue;
}

sys_seconds __date = sys_seconds{seconds{chrono::__parse_integral(__input, false)}} - __offset;
chrono::__matches(__input, "leap");
Copy link
Member

@mordante mordante Jan 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The specs allow items to be abbreviated to the shortest unique name. So you need to test the first character to be 'l' and then chrono::__skip(__input, "ink");

chrono::__skip_mandatory_whitespace(__input);

year __year = chrono::__parse_year_value(__input);
chrono::__skip_mandatory_whitespace(__input);

month __month = chrono::__parse_month(__input);
chrono::__skip_mandatory_whitespace(__input);
seconds __value{chrono::__parse_integral(__input, false)};

day __day = chrono::__parse_day(__input);
chrono::__skip_mandatory_whitespace(__input);

hours __hour(chrono::__parse_integral(__input, /* leading zeros allowed */ true));
chrono::__matches(__input, ':');

minutes __minute(chrono::__parse_integral(__input, /* leading zeros allowed */ true));
chrono::__matches(__input, ':');

seconds __second(chrono::__parse_integral(__input, /* leading zeros allowed */ true));
chrono::__skip_mandatory_whitespace(__input);

// Now create a timestamp from everything we parsed
year_month_day __date = __year / __month / __day;
seconds __time = __hour + __minute + __second;
sys_seconds __timestamp = sys_seconds(sys_days(__date)) + __time;

// Finally, parse the value of the leap second (1s or -1s)
string __sign = __parse_string(__input);
seconds __value;
if (__sign == "+")
__value = seconds{1};
else if (__sign == "-")
__value = seconds{-1};
else
std::__throw_runtime_error(("corrupt tzdb: invalid leap second sign " + __sign).c_str());

chrono::__skip_line(__input);

__entries.emplace_back(__date, __value);
__leap_seconds.emplace_back(std::__private_constructor_tag{}, __timestamp, __value);
}
}();
// The Standard requires the leap seconds to be sorted. The file
// leap-seconds.list usually provides them in sorted order, but that is not
// guaranteed so we ensure it here.
ranges::sort(__entries, {}, &__entry::__timestamp);

// The database should contain the number of seconds inserted by a leap
// second (1 or -1). So the difference between the two elements is stored.
// std::ranges::views::adjacent has not been implemented yet.
(void)ranges::adjacent_find(__entries, [&](const __entry& __first, const __entry& __second) {
__leap_seconds.emplace_back(
std::__private_constructor_tag{}, __second.__timestamp, __second.__value - __first.__value);
return false;
});

// Ensure the leap seconds are sorted properly.
ranges::sort(__leap_seconds, {}, &leap_second::date);
}

void __init_tzdb(tzdb& __tzdb, __tz::__rules_storage_type& __rules) {
Expand All @@ -680,13 +716,7 @@ void __init_tzdb(tzdb& __tzdb, __tz::__rules_storage_type& __rules) {
ranges::sort(__tzdb.zones);
ranges::sort(__tzdb.links);
ranges::sort(__rules, {}, [](const auto& p) { return p.first; });

// There are two files with the leap second information
// - leapseconds as specified by zic
// - leap-seconds.list the source data
// The latter is much easier to parse, it seems Howard shares that
// opinion.
chrono::__parse_leap_seconds(__tzdb.leap_seconds, ifstream{__root / "leap-seconds.list"});
chrono::__parse_leap_seconds(__tzdb.leap_seconds, ifstream{__root / "leapseconds"});
}

#ifdef _WIN32
Expand Down
35 changes: 14 additions & 21 deletions libcxx/test/libcxx/time/time.zone/time.zone.db/leap_seconds.pass.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@
scoped_test_env env;
[[maybe_unused]] const std::filesystem::path dir = env.create_dir("zoneinfo");
const std::filesystem::path tzdata = env.create_file("zoneinfo/tzdata.zi");
const std::filesystem::path leap_seconds = env.create_file("zoneinfo/leap-seconds.list");
const std::filesystem::path leap_seconds = env.create_file("zoneinfo/leapseconds");

std::string_view std::chrono::__libcpp_tzdb_directory() {
static std::string result = dir.string();
Expand Down Expand Up @@ -65,32 +65,25 @@ static void test_exception(std::string_view input, [[maybe_unused]] std::string_
}

static void test_invalid() {
test_exception("0", "corrupt tzdb: expected a non-zero digit");

test_exception("1", "corrupt tzdb: expected whitespace");

test_exception("1 ", "corrupt tzdb: expected a non-zero digit");

test_exception("5764607523034234880 2", "corrupt tzdb: integral too large");
test_exception("0", "corrupt tzdb: expected character 'l' from string 'leap', got '0' instead");
test_exception("Leap x", "corrupt tzdb: expected a digit");
test_exception("Leap 1970 J", "corrupt tzdb month: invalid name");
test_exception("Leap 1970 Jan 1 23:59:60 x", "corrupt tzdb: invalid leap second sign x");
Comment on lines +69 to +71
Copy link
Member

@mordante mordante Jan 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we need to test with different abbreviations of leap and different cases and with some invalid values, like "Leak".

}

static void test_leap_seconds() {
using namespace std::chrono;

// Test whether loading also sorts the entries in the proper order.
const tzdb& result = parse(
R"(
2303683200 12 # 1 Jan 1973
2287785600 11 # 1 Jul 1972
2272060800 10 # 1 Jan 1972
86400 9 # 2 Jan 1900 Dummy entry to test before 1970
1 8 # 2 Jan 1900 Dummy entry to test before 1970

# Fictional negative leap second
2303769600 11 # 2 Jan 1973

# largest accepted value by the parser
5764607523034234879 12
const tzdb& result = parse(R"(
Leap 1973 Jan 1 23:59:60 + S
Leap 1972 Jul 1 23:59:60 + S
Leap 1972 Jan 1 23:59:60 + S
Leap 1900 Jan 2 23:59:60 + S # 2 Jan 1900 Dummy entry to test before 1970
Leap 1900 Jan 2 00:00:01 + S # 2 Jan 1900 Dummy entry to test before 1970

Leap 1973 Jan 2 23:59:60 - S # Fictional negative leap second
Leap 32767 Jan 1 23:59:60 + S # Largest year accepted by the parser
)");

assert(result.leap_seconds.size() == 6);
Expand Down
15 changes: 10 additions & 5 deletions libcxx/test/libcxx/time/time.zone/time.zone.db/rules.pass.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,10 @@
// ADDITIONAL_COMPILE_FLAGS: -I %{libcxx-dir}/src/experimental/include

#include <chrono>
#include <cstdio>
#include <fstream>
#include <string>
#include <string_view>
#include <string>
#include <variant>

#include "assert_macros.h"
Expand Down Expand Up @@ -96,7 +97,7 @@ static void test_invalid() {
test_exception("R r 0 mix", "corrupt tzdb: expected whitespace");
test_exception("R r 0 1", "corrupt tzdb: expected whitespace");

test_exception("R r 0 1 X", "corrupt tzdb: expected character '-'");
test_exception("R r 0 1 X", "corrupt tzdb: expected character '-', got 'X' instead");

test_exception("R r 0 1 -", "corrupt tzdb: expected whitespace");

Expand All @@ -106,13 +107,17 @@ static void test_invalid() {

test_exception("R r 0 1 - Ja +", "corrupt tzdb weekday: invalid name");
test_exception("R r 0 1 - Ja 32", "corrupt tzdb day: value too large");
test_exception("R r 0 1 - Ja l", "corrupt tzdb: expected string 'last'");
test_exception(
"R r 0 1 - Ja l",
std::string{"corrupt tzdb: expected character 'a' from string 'last', got '"} + (char)EOF + "' instead");
test_exception("R r 0 1 - Ja last", "corrupt tzdb weekday: invalid name");
test_exception("R r 0 1 - Ja lastS", "corrupt tzdb weekday: invalid name");
test_exception("R r 0 1 - Ja S", "corrupt tzdb weekday: invalid name");
test_exception("R r 0 1 - Ja Su", "corrupt tzdb on: expected '>=' or '<='");
test_exception("R r 0 1 - Ja Su>", "corrupt tzdb: expected character '='");
test_exception("R r 0 1 - Ja Su<", "corrupt tzdb: expected character '='");
test_exception(
"R r 0 1 - Ja Su>", std::string{"corrupt tzdb: expected character '=', got '"} + (char)EOF + "' instead");
test_exception(
"R r 0 1 - Ja Su<", std::string{"corrupt tzdb: expected character '=', got '"} + (char)EOF + "' instead");
test_exception("R r 0 1 - Ja Su>=+", "corrupt tzdb: expected a non-zero digit");
test_exception("R r 0 1 - Ja Su>=0", "corrupt tzdb: expected a non-zero digit");
test_exception("R r 0 1 - Ja Su>=32", "corrupt tzdb day: value too large");
Expand Down
5 changes: 3 additions & 2 deletions libcxx/test/libcxx/time/time.zone/time.zone.db/version.pass.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,10 @@
// This is not part of the public tzdb interface.

#include <chrono>
#include <cstdio>
#include <fstream>
#include <string>
#include <string_view>
#include <string>

#include "assert_macros.h"
#include "concat_macros.h"
Expand Down Expand Up @@ -60,7 +61,7 @@ static void test_exception(std::string_view input, [[maybe_unused]] std::string_
}

int main(int, const char**) {
test_exception("", "corrupt tzdb: expected character '#'");
test_exception("", std::string{"corrupt tzdb: expected character '#', got '"} + (char)EOF + "' instead");
test_exception("#version", "corrupt tzdb: expected whitespace");
test("#version \t ABCD", "ABCD");
test("#Version \t ABCD", "ABCD");
Expand Down
Loading