Implement an exponentially weighted moving rate #124507
Conversation
|
The maths used here is explained in section two of the attached document. |
This is intended to be used to efficiently calculate a write load metric for use by the auto-sharding algorithm which favours more recent loads.
PeteGillinElastic
force-pushed
the
ES-10037-ewmr
branch
from
3df375a to
7296435
Compare
PeteGillinElastic
added
:Data Management/Stats
elasticsearchmachine
added
the
Team:Data Management
|
Pinging @elastic/es-data-management (Team:Data Management) |
| throw new IllegalArgumentException("lambda must be non-negative but was " + lambda); | ||
| } | ||
| if (startTimeInMillis <= 0.0) { | ||
| throw new IllegalArgumentException("lambda must be non-negative but was " + startTimeInMillis); |
There was a problem hiding this comment.
startTime must be ....
There was a problem hiding this comment.
Oops, well spotted.
parkertimmins
left a comment
parkertimmins
left a comment
There was a problem hiding this comment.
Other than a minor exception message fix, this looks good to me. The thread-safety looks correct and I think it accurately reflects the math in the linked pdf.I feel less confident to speak about that math itself; though what I did read made sense. Really cool analysis and work. I particularly like the work to making it numerically stable.
|
Thanks @parkertimmins ! |
|
I belatedly noticed that the code which is going to use this actually measures times using |
This is intended to be used to efficiently calculate a write load metric for use by the auto-sharding algorithm which favours more recent loads. ES-10037 #comment Core algorithm added in elastic#124507
This is intended to be used to efficiently calculate a write load metric for use by the auto-sharding algorithm which favours more recent loads. ES-10037 #comment Core algorithm added in elastic#124507
3 participants
This is intended to be used to efficiently calculate a write load metric for use by the auto-sharding algorithm which favours more recent loads.
The maths used here is explained in section two of the attached document.
The unit test serves the additional function of illustrating the behaviour of the EWMR in a way which is hopefully easy to understand. In particular,
testEwmr_longSeriesWithStepChangeInRate_fitsHalfLife_contrastWithZeroLambdaillustrates the issue we have seen in a number of recent support cases, and shows how the new thing (which a suitably chosenlambda) compares with the current metric (which effectively haslambda = 0).