Skip to content

feat: add timestamp_to_datetime macro #860

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
isaacharrisholt wants to merge 2 commits into
base: main
Choose a base branch
from
Open

feat: add timestamp_to_datetime macro #860

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
383ebb3
feat: add timestamp_to_datetime macro
isaacharrisholt Dec 3, 2025
95bebbd
fix: use datetime.timezone instead of pytz
isaacharrisholt Dec 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions airbyte_cdk/sources/declarative/interpolation/macros.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,24 @@ def timestamp(dt: Union[float, str]) -> Union[int, float]:
return str_to_datetime(dt).astimezone(pytz.utc).timestamp()


def timestamp_to_datetime(ts: Union[int, float, str]) -> datetime.datetime:
"""
Converts a Unix timestamp to a datetime object with UTC timezone.

Usage:
"{{ timestamp_to_datetime(1658505815) }}"

:param ts: Unix timestamp (in seconds) to convert to datetime
:return: datetime object in UTC timezone
"""
try:
ts_value = float(ts)
except (TypeError, ValueError) as exc:
raise ValueError(f"Invalid timestamp value: {ts}") from exc

return datetime.datetime.fromtimestamp(ts_value, tz=datetime.timezone.utc)

Comment on lines +74 to +90
Copy link
Contributor

@coderabbitai coderabbitai bot Dec 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Document timestamp_to_datetime behavior for Xero millisecond timestamps; consider exception wrapping

The function explicitly documents "Unix timestamp (in seconds)" and passes float(ts) to datetime.fromtimestamp. However, the Xero C#.NET format /Date(1764104832903+0000)/ contains milliseconds. If a connector author extracts 1764104832903 from Xero data and passes it directly here, they'll get an OverflowError instead of a clear, actionable error message.

Consider one of these approaches:

  • Explicit seconds-only: Enhance the docstring and add a clear example showing how to convert Xero milliseconds to seconds before calling this macro (e.g., divide by 1000), or
  • Auto-detect milliseconds: Detect and handle millisecond inputs (e.g., by checking if the value exceeds a reasonable seconds threshold and dividing by 1000), though this introduces ambiguity for legitimate large timestamps.

As a follow-up polish (optional), consider wrapping datetime.fromtimestamp's potential OverflowError and OSError in ValueError to provide a consistent exception contract, similar to your float(ts) handling.

🤖 Prompt for AI Agents 
airbyte_cdk/sources/declarative/interpolation/macros.py around lines 74-90: The
timestamp_to_datetime docstring and implementation assume seconds but callers
(e.g., Xero) may pass millisecond values causing OverflowError; update the
docstring to clearly state the function expects seconds and give an example
showing dividing milliseconds by 1000, and modify the implementation to either
(preferred) detect millisecond inputs (if ts numeric and > 10**11 treat as ms
and divide by 1000) or (alternative) leave behavior unchanged but add an
explicit check that raises a ValueError with a clear message if the numeric
value is unreasonably large; also wrap datetime.fromtimestamp calls in
try/except to catch OverflowError and OSError and re-raise as ValueError with a
descriptive message including the original value.


def str_to_datetime(s: str) -> datetime.datetime:
"""
Converts a string to a datetime object with UTC timezone
Expand Down Expand Up @@ -222,6 +240,7 @@ def generate_uuid() -> str:
now_utc,
today_utc,
timestamp,
timestamp_to_datetime,
max,
min,
day_delta,
Expand Down
33 changes: 33 additions & 0 deletions unit_tests/sources/declarative/interpolation/test_macros.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
[
("test_now_utc", "now_utc", True),
("test_today_utc", "today_utc", True),
("test_timestamp_to_datetime", "timestamp_to_datetime", True),
("test_max", "max", True),
("test_min", "min", True),
("test_day_delta", "day_delta", True),
Expand Down Expand Up @@ -176,6 +177,38 @@ def test_timestamp(test_name, input_value, expected_output):
assert actual_output == expected_output


@pytest.mark.parametrize(
"test_name, input_value, expected_output",
[
(
"test_seconds_int",
1646006400,
datetime.datetime(2022, 2, 28, 0, 0, tzinfo=datetime.timezone.utc),
),
(
"test_seconds_float",
1646006400.5,
datetime.datetime(2022, 2, 28, 0, 0, 0, 500000, tzinfo=datetime.timezone.utc),
),
(
"test_seconds_string",
"1646006400",
datetime.datetime(2022, 2, 28, 0, 0, tzinfo=datetime.timezone.utc),
),
],
)
def test_timestamp_to_datetime(test_name, input_value, expected_output):
timestamp_to_datetime_fn = macros["timestamp_to_datetime"]
actual_output = timestamp_to_datetime_fn(input_value)
assert actual_output == expected_output


def test_timestamp_to_datetime_invalid_value():
timestamp_to_datetime_fn = macros["timestamp_to_datetime"]
with pytest.raises(ValueError):
timestamp_to_datetime_fn("invalid-timestamp")


def test_utc_datetime_to_local_timestamp_conversion():
"""
This test ensures correct timezone handling independent of the timezone of the system on which the sync is running.
Expand Down
Loading