-
-
Notifications
You must be signed in to change notification settings - Fork 19.3k
Open
Labels
Arrowpyarrow functionalitypyarrow functionalityBugDtype ConversionsUnexpected or buggy dtype conversionsUnexpected or buggy dtype conversionsNeeds DiscussionRequires discussion from core team before further actionRequires discussion from core team before further actionReduction Operationssum, mean, min, max, etc.sum, mean, min, max, etc.
Description
When using .sum() on duration[ns][pyarrow] we get a Timedelta, but all other units give datetime.timedelta.
import datetime as dt
ser = pd.Series([dt.timedelta(seconds=1)], dtype="duration[ns][pyarrow]")
print(type(ser.iloc[0]), type(ser.sum()))
# <class 'pandas.Timedelta'> <class 'pandas.Timedelta'>
ser = pd.Series([dt.timedelta(seconds=1)], dtype="duration[us][pyarrow]")
print(type(ser.iloc[0]), type(ser.sum()))
# <class 'pandas.Timedelta'> <class 'datetime.timedelta'>
ser = pd.Series([dt.timedelta(seconds=1)], dtype="duration[ms][pyarrow]")
print(type(ser.iloc[0]), type(ser.sum()))
# <class 'pandas.Timedelta'> <class 'datetime.timedelta'>
ser = pd.Series([dt.timedelta(seconds=1)], dtype="duration[s][pyarrow]")
print(type(ser.iloc[0]), type(ser.sum()))
# <class 'pandas.Timedelta'> <class 'datetime.timedelta'>I suspect we want to make all of the sums be Timedeltas. I haven't investigated other reductions, but suspect we'll find similar behavior.
Metadata
Metadata
Assignees
Labels
Arrowpyarrow functionalitypyarrow functionalityBugDtype ConversionsUnexpected or buggy dtype conversionsUnexpected or buggy dtype conversionsNeeds DiscussionRequires discussion from core team before further actionRequires discussion from core team before further actionReduction Operationssum, mean, min, max, etc.sum, mean, min, max, etc.