How save average and standard deviation of data stream everyone minute in the csv

[espruino-discuss2]

Posted at 2022-09-22 by user149667

New to both Bangle and JavaScript.
I am currently trying to save all the HRM, accelerometer and magnetometer output's individual average and standard deviation once in minute. The code below is giving me Interpreter error: LOW_MEMORY.

Please help.

Bangle.setCompassPower(1);
filename = "trolololo.csv";

var allData = require("Storage").open(filename, "a");
Bangle.setHRMPower(1);

const buffer = {
  start: null,
  end: null,
  data: []
};
const maxSamples = 100;

const computeBufferAverage = () => {
  let sum = [], sum2 = [], mean = [], std = [];
  const nrows = maxSamples;
  const ncols = buffer.data[0].length;
  for(let i=0;i<ncols;i++) {
    sum[i] = 0;
    sum2[i] = 0;
    mean[i] = 0;
    std[i] = 0;
  }

  for (const data of buffer.data) {
    for (let i=0; i<ncols; i++) {
      sum[i] += data[i];
      sum2[i] += data[i]*data[i];
    }
  }

  for (let i=0; i<ncols; i++) {
    mean[i] = sum[i] / nrows;
    std[i] = Math.sqrt(nrows*sum2[i] - Math.pow(sum[i], 2))/(nrows-1);
  }

  return {mean: mean, std: std};
};

Bangle.on('HRM-raw', function(hrm) {
    const hrmRaw = [
      "HRM:",
      hrm.raw,
      hrm.filt,
      hrm.bpm,
      hrm.confidence
    ];
  const compass = Bangle.getCompass();
  const accel = Bangle.getAccel();

  if (buffer.data.length === 0) {
    buffer.start = Math.floor(Date.now());
  }
  if (buffer.data.length < maxSamples) {
    buffer.data.push([
      a.x, a.y,a.z,c.x,c.y,c.z,c.dx,c.dy,c.dz,hrm.raw,hrm.filt,hrm.bpm,hrm.confidence
    ]);
  } else {
    buffer.end = Math.floor(Date.now());

    const result = computeBufferAverage();
    const str = [
      buffer.start,
      buffer.end,
      result.mean.join(","),
      result.std.join(",")
    ].join(",");
    print(str+"\n");
    allData.write(str);
    allData.write("\n");
    buffer.data.length = 0;
  }

});

[espruino-discuss2]

Posted at 2022-09-22 by @thyttan

trolololo.csv I see, I made an instruction here.

---

Posted at 2022-09-23 by @gfwilliams

@thyttan I'm not sure I understand?

Is this a Bangle.js 1 or 2?

I think the issue is you're storing large amounts of data in RAM with your buffer.data.push([. If you keep adding stuff to a normal array it's not stored very efficiently.

Right now it seems you have 100 samples, but for each sample you store 13 bits of data (which can sometimes require up to 26 vars) - you can be using 2600 vars. Normally it wouldn't be a huge deal as Bangle.js 2 has 12000 in total, but for the v1 it could cause problems.

There is some info at https://www.espruino.com/Performance#array-buffers-are-the-most-efficient-way-to-store-data about how to store in an arraybuffer.

You can use a DataView to store different types of data in one big array: https://www.espruino.com/Reference#l_DataView_DataView

But... if you used a Typed array for each data entry (eg a.x/a.y/etc) then you can use built-in methods like E.sum and E.variance (https://www.espruino.com/Reference#l_E_variance) to work out mean and standard deviation really quickly.

---

Posted at 2022-09-23 by @thyttan

I only tried to joke regarding the file name @user149667 used in the example code. But it was somewhat ruined by the video being embedded rather than just being linked to 🙃

---

Posted at 2022-09-23 by user149667

Thank you very much @gfwilliams
I am using BangleJs2. I will use the arraybuffer and share the code very soon.
Thanks again.

---

Posted at 2022-09-23 by user149667

Lol I just realised hehe.

---

Posted at 2022-09-26 by @gfwilliams

@thyttan ahh! That makes a lot more sense now :)