Standardize abstract series handling

[bertini97]

Is your feature request related to a problem? Please describe.

I'm currently maintaining a Timeseries <: AbstractDiffEqArray object in my scripts that sometimes collides with what is already provided. For example, DiffEqArray has this opaque name, it's not interpolable, etc. Interpolation objects in SciMLBase use different fields than the InterpolationData in OrdinaryDiffEq.

I feel like there is a need for a standard way to handle (time) series in the SciML ecosystem. ReservoirComputing.jl could use something like this to handle its output, and I'm sure there are other packages that would need it.

Describe the solution you’d like

My proposal is the following.

• Introduce an AbstractSeries <: AbstractVectorOfArray that expects two fields, u, t. Done by AbstractDiffEqArray.

• Rename AbstractDiffEqArray to something more generic like AbstractSeries.

• Require defining interpolable(::Type) = false/true for types derived, and define a fallback that errors

  (ts::AbstractSeries)(t::Number) = error("Interpolation not defined for $(typeof(ts))")

• What if often used in SciML for interpolation is having a field in the type that holds interpolation information: a cache that usually contains u, t plus some required parameters. Standardize interpolation cache creation by creating AbstractSeriesInterpolation that expects at least the fields u, t, replacing AbstractDiffEqInterpolation.

• Provide a unified package for creating the caches from u, t, merging DataInterpolation and the routines already inside SciMLBase (for Linear and Hermite interpolations). Define also methods for creating interp objects from u, t, du that will be used by DifferentialEquation, to decouple further.

• In DifferentialEquations have interpolationData be <: AbstractSeriesInterpolation and have the standard fields.

• Provide MakieExt routines to plot the series object appropriately, similar to what is already in place.

Optionally:

• Provide SubSeries <: AbstractSeries object and seriesview(s::AbstractSeries, idxs...) and maybe seriesview(s::AbstractSeries, tspan) to create it. It acts like a view but for the series object. This is something I would need often, as I'm always working with large series. It would also streamline plotting.

Additional context

I described my problems in more details in SciML/OrdinaryDiffEq.jl#2750 and SciML/RecursiveArrayTools.jl#415. I intend to merge everything into this issue here, and hopefully describe what I would like fully.

I can't work on this alone in terms of time, expertise and sheer size. Everytime I want to open a PR to change a component, i find that it's interconnected with dozens of other packages. If you find this useful and are a developer of SciML, please let me know what could be a start to this.

[ChrisRackauckas]

Introduce an AbstractSeries <: AbstractVectorOfArray in place of DiffEqArray that expects two fields, u, t.

Isn't that exactly AbstractDiffEqArray?

https://docs.sciml.ai/RecursiveArrayTools/stable/AbstractVectorOfArrayInterface/

Maybe it needs a better name but that already exists.

Introduce AbstractInterpolableSeries <: AbstractSeries that also supports the field interp, and define a method
Standardize interpolation objects by creating AbstractSeriesInterpolation that expects at least the fields u, t, replacing AbstractDiffEqInterpolation.
Make the DifferentialEquations packages use this new framework, leveraging the new constructors. Have interpolationData be <: AbstractSeriesInterpolation and have the standard fields.
Provide MakieExt routines to plot the series object appropriately, similar to what is already in place.
Provide SubSeries <: AbstractSeries object and seriesview(s::AbstractSeries, idxs...) and maybe seriesview(s::AbstractSeries, tspan) to create it. It acts like a view but for the series object. This is something I would need often, as I'm always working with large series. It would also streamline plotting.

Reasonable.

Provide the way of creating this interpolation object from u, t, merging DataInterpolation and the routines already inside SciMLBase (for Linear and Hermite interpolations). Define also methods for creating interp objects from u, t, du.

Probably not there, probably separate library. Optimization.jl doesn't need it. NonlinearSolve doesn't need it. Etc.

[ChrisRackauckas]

I can't work on this alone in terms of time, expertise and sheer size. Everytime I want to open a PR to change a component, i find that it's interconnected with dozens of other packages. If you find this useful and are a developer of SciML, please let me know what could be a start to this.

For reference, working on core interfaces is harder than any functionality of packages. I mentioned this in a video awhile ago:

https://youtu.be/fQJrEQIRPZU?si=hzYc4VlNfWFSUH4i&t=3137

I do think this kind of change will need to be done by a core dev because moving something at this level is moving a lot of pieces at the same time.

[bertini97]

Thank you for looking into this.

Isn't that exactly AbstractDiffEqArray?

Yes, my bad, I am even using it right now. Maybe I got confused by the name. I'll update the issue.

Probably not there, probably separate library. Optimization.jl doesn't need it. NonlinearSolve doesn't need it. Etc.

Agreed.

I do think this kind of change will need to be done by a core dev because moving something at this level is moving a lot of pieces at the same time.

I'll ask on the slack. I can do single PRs if guided.

Note

In SciML/RecursiveArrayTools.jl#415 you said

I'd make the interpolation optional based on having the call defined, and leave it open to interpretation how to implement it, and have a function to query if something can interpolate.

Do you think this is a better approach than having AbstractInterpolableArray that expects the interp field?

[ChrisRackauckas]

Trait approaches are generally more genrealizable so I'd do that instead of having a deep abstract hierarchy.