Skip to content

Make the resamplers take into account out-of-order samples #47

@leandro-lucarella-frequenz

Description

What's needed?

The resampling actor might receive samples from a stream out of order (i.e. it might receive a sample with a lower timestamp after a sample with a higher timestamp). It should be able to deal with it.

Proposed solution

Sort samples in the internal ring buffer by timestamp before evicting any existing samples instead of blindly evicting the first sample that was added.

Use cases

We decided not to guarantee the ordering of samples in a stream the resampler can consume to make the system more real-time, so a source won't have to care about re-sorting samples, it can just send them as soon as they arrive. The resampler already have to deal with faulty sources (not incoming data for example -> intra/extrapolation).

Alternatives and workarounds

Guarantee that sources provide in-order samples.

Additional context

Note that this means the resampling actor doesn't take timeseries as input, as a timeseries is by definition sorted by timestamp.

Metadata

Metadata

Assignees

No one assigned

    Labels

    part:data-pipelineAffects the data pipelinetype:enhancementNew feature or enhancement visitble to users

    Type

    No type

    Projects

    Status

    To do

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions