Add a Kalman filter for smoothing noisy data

[mitra42]

Cheap sensors are typically noisy

To lower that noise, Smoothing is already an option on Sensor_uint16 (which also makes it available on subclasses, Analog (Soil, Battery etc))

It uses a simple smooth (note smooth is a logarithmical value e.g. smooth=1 will get you a new value half way from the old value to the new reading. and smooth=n gets a value, 1/(2^n) from old value to new.

Note value is 2^n times the reading, which is intentional, since noisy readings are often small numbers.

uint16_t Sensor_Uint16::convert(uint16_t v) {
  uint16_t vv;
  if (smooth) {
    vv = output->value - (output->value >> smooth) + v;
  } else {
    vv = v;
  }
  return(vv); // Set the value in the OUT object and send
}

An alternative is a Kalman filter. When the expected value is the same as the previous value, then this is almost the same as the smoothing above, except it calculates a value for how noisy readings are, K and then applies a smoothing e.g. if V is old value and v is reading then V' (new value) is ...

V' = V+K*(v-V') e.g. if K = 1 then V' = v and if K=0 then V'=V

K can be calculated by taking two readings quickly enough that the value shouldn't change, and saying K = min(v1,v2) / max(v1,v2)

I could imagine doing this calibration step either intensively at some point (not immediately on startup, allow sensor to stabilize).

A simple way to do this might be .... (and there could be better ways).

• For first n readings, take a pair of readings, and accumulate the difference.
• Then divide accumlated difference by n to get the expected variance
• And Value / (Value + Variance) gets K (in range 0..1)
• And then apply the formula above on each new reading.

This would need thinking through if doing deep sleep between readings, as would need to store this variance somewhere (possibly in Spiffs or RTC)