datetime does not support timestamps after 2038 on 32-bit

[bmwiedemann]

Bug report

cpython on 32-bit platforms cannot handle dates beyond 2038-01-19

python3.10 -c 'import datetime; datetime.datetime.fromtimestamp(3000000000)'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
OverflowError: timestamp out of range for platform time_t

Your environment

• CPython versions tested on: 3.8, 3.9, 3.10
• Operating system and architecture: openSUSE-Tumbleweed 20230110 i586

This comes from
https://github.com/python/cpython/blob/8e9d08b0/Modules/_datetimemodule.c#L4982
https://github.com/python/cpython/blob/8e9d08b0/Python/pytime.c#L173

These could use __time64_t if available or we compile everything with __TIMESIZE=64

[sobolevn]

It works on my machine (macOS 10.14) on main:

(.venv) ~/Desktop/cpython  main ✔                                                    
» ./python.exe -c 'import datetime; datetime.datetime.fromtimestamp(3000000000)' 

(.venv) ~/Desktop/cpython  main ✔                                                    
» echo $?                                                                        
0

The same for 3.9:

Python 3.9.15 (main, Nov 22 2022, 17:18:20) 
[Clang 11.0.0 (clang-1100.0.33.16)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import datetime; datetime.datetime.fromtimestamp(3000000000)
datetime.datetime(2065, 1, 24, 8, 20)

I don't say that this is not a bug, I say that this is totally system-specific.

[mcepl]

It works on my machine (macOS 10.14) on main:

Is your system 32bit? I doubt it with Mac.

[sobolevn]

No, it is 64 bit

[mcepl]

No, it is 64 bit

Then your comment is unfortunately irrelevant.

[sobolevn]

@mcepl yeap, that's what I said :)

I don't say that this is not a bug, I say that this is totally system-specific.

[bmwiedemann]

I guess, this could be solved outside of python by rebuilding our 32-bit Linuxes with
-D_TIME_BITS=64 -D_FILE_OFFSET_BITS=64

So that their (or glibc's) time_t would have 8 bytes.

[calimeroteknik]

I came across this issue as well on my raspberry pi's Cortex A7 (32-bit CPU). Two remarks:

This works until year 10000 everywhere (incl. 32bit), buying us some extra time:

rasberrypi ~ $ python3.10 -c 'import datetime; print(datetime.datetime.utcfromtimestamp(0)+datetime.timedelta(seconds=3000000000))'
2065-01-24 05:20:00

The Y10K problem here is still a limitation if you want to handle timestamps in the (distant) future. I know they said 2000 was far away and it came, I know they said 2038 was far away and now it's at our door, but Y10K feels sufficiently far away for representing the current time of reality (it will cause problems in Y30K science-fiction — and yes, I'm not too happy when the SF program that ran fine on my laptop crashes on my raspberry pi).

I found a heterogenous mix of errors when going past year 9999:

anyplatform ~ $ python3.10 -c 'import datetime; datetime.datetime.utcfromtimestamp(0)+datetime.timedelta(seconds=300000000000)'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
OverflowError: date value out of range

anyplatform ~ $ python3.10 -c 'import datetime; datetime.datetime.utcfromtimestamp(0)+datetime.timedelta(seconds=100000000000000)'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
OverflowError: days=1157407407; must have magnitude <= 999999999

anyplatform ~ $ python3.10 -c 'import datetime; datetime.datetime.utcfromtimestamp(0)+datetime.timedelta(seconds=300000000000000)'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
OverflowError: Python int too large to convert to C int

Okay, enough goofing around, let's get to the second remark:
Perl suffers from none of this.

raspberrypi ~ $ perl -le 'print scalar localtime 300000000000000'
Thu Jul 21 06:20:00 9508591

None of the Y10K bug, D1G bug ( a Python original) or "C int" shenanigans that we saw with Python above.
It doesn't seem to mind that the platform is 32bit, or about any -D_SOMETHING_BITS either.

Maybe Perl is enough of a low bar that it's not egregious to say that Python's time library is antiquated, buggy, and in need of an urgent fix. Important: I'm not demanding anyone volunteer to fix this; however I do seek unmitigated agreement that it's indeed Python's datetime that's broken on 32bit, not anything else.

[bmwiedemann]

however I do seek unmitigated agreement that it's indeed Python's datetime that's broken on 32bit, not anything else.

AFAIU, the problem originates in glibc's time_t implementation that currently defaults to 32-bit on 32-bit archs and is not trivial to switch for compatibility reasons. It can be solved at the level of distributions/glibc outside of python. But as shown by perl, solutions could be made within python.