Skip to content

Enhancement: Merge two time series and apply a function to overlapping datesΒ #26035

@MattConflitti

Description

@MattConflitti

Enhancement description
A standard way to take two dataframes that have a DateTimeIndex and do a time series join where same dates will be together on the same day and allow me to apply a function to it in order to choose which value to take if more than one fall on the same day.

If this functionality doesn't exist, can we open an enhancement request?

Expected Behavior
Time Series 1:

Date | Value
------------
1-1-2019 | 1
1-2-2019 | 2
1-3-2019 | 3

Time Series 2:

Date | Value
------------
1-3-2019 | 4
1-4-2019 | 5
1-5-2019 | 6

Then,

#default to taking 'right' time series value on overlapped days
ts1.time_series_join(ts2, overlap_func='right')

or

#apply lambda applies to overlapped days
ts1.time_series_join(ts2, overlap_func=lambda x,y: max(x,y))

which would return (defaulting to a left join behavior as far as keeping the index goes):

Date | Value
------------
1-1-2019 | 1
1-2-2019 | 2
1-3-2019 | 4

I haven't seen anything like this on my data processing adventures and think this would be a great feature. A similar feature is implemented in SparkTS for RDDs. Right now, I have resorted to doing some gross looping and utilizing the .at[] selector to update overlapping values.

Thanks!

Metadata

Metadata

Assignees

No one assigned

    Labels

    Needs DiscussionRequires discussion from core team before further actionReshapingConcat, Merge/Join, Stack/Unstack, Explode

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions