clearTimeout and WARNING: Out of memory while appending to array

[espruino-discuss3]

Posted at 2014-07-12 by Hardware_Hacks

Hi All,

I am running the following GPS code:

Serial4.setup(9600,{tx:C10,rx:C11});

var SleepDuration = null;
var SearchDuration = null;

function Start_GPS(){
  
  console.log("Starting GPS...");
  //After 120 seconds disable the GPS regardless
  SearchDuration = setTimeout(function (e) { Stop_GPS(); }, 60000);
 
  //Set enable pin mode
  pinMode(A9,'output');
  
  //Set enable pin to high
  digitalWrite(A9,1);
  
  //load gps
    var gps = require("GPS").connect(Serial4, function(data) {
    console.log(data);
    //Check if we have a fix
      if(data.fix>0)
      {
        Stop_GPS();
        console.log("Send SMS:" + data.lat + "|" + data.lon);
      }
  });
}

function Stop_GPS(){
  clearTimeout(SearchDuration);
  digitalWrite(A9,0);
  console.log("Disabling GPS...");
}

function onInit(){
SleepDuration = setTimeout(function (e) {onInit(); }, 120000);
console.log("Starting up device & Setting timers...");
Start_GPS();

}

onInit();

I'm seeing an error popup every so often (not every time the code runs) as shown below:

ERROR: Error processing Serial data handler - removing it.
Execution Interrupted during event processing.
Uncaught Error: Unknown Timeout
 at line 1 col 29
{clearTimeout(SearchDuration);digitalWrite(A9,0);console.log...
                             ^
in function "Stop_GPS" called from line 1 col 44
...a);if(data.fix>0){Stop_GPS();console.log("Send SMS:"+data.la...
                               ^
in function "c" called from line 2 col 81
...,altitude:parseFloat(a[9])})}c=b.line.indexOf("\n")}}
                               ^
in function called from system
Starting up device & Setting timers...
Starting GPS...

After leaving the code to execute I am eventually hitting:

"lat":NaN,"lon":NaN,"fix":0,"ERROR: Out of Memory!
satellit":0}
WARNING: Unable to create string as not enough memory
ERROR: Error processing Serial data handler - removing it.
Execution Interrupted during event processing.
at line 1 col 18
{console.log(data);if(data.fix>0){Stop_in function called from system
at line 2 col 81
...,altitude:parseFloat(a[9])})}c=b.line.indexOf("\n")}}
                               ^
in function called from system
Disabling GPS...
Starting up device & Setting timers...

And even later on:

Execution Interrupted during event processing.
at line 1 col 144
...tr(3,3)){var a=a.split(","),d=a[2].indexOf("."),e=a[4].index...
                               ^
in function called from system
Execution Interrupted during event processing.
at line 1 col 144
...tr(3,3)){var a=a.split(","),d=a[2].indexOf("."),e=a[4].index...
                               ^
in function called from system
WARNING: Out of memory while appending to array
WARNING: Out of memory while appending to array
WARNING: Out of memory while appending to array
WARNING: Out of memory while appending to array
WARNING: Out of memory while appending to array
WARNING: Out of memory while appending to array
WARNING: Out of memory while appending to array
 

Just wondering if anyone can see whats going wrong here? I assume it might be to do with the setTimeouts and then clearing them?

[espruino-discuss3]

Posted at 2014-07-13 by @gfwilliams

Will check properly on but it seems that one problem might be that you're not really ever getting rid of the gps module, so for every fix you'll try and stop the interval again. Could it be that a9 isn't actually stopping the gps?

I'd maybe add a hasHadFix variable and explicitly ignore any fixes other than the first.

---

Posted at 2014-07-13 by @gfwilliams

Also in searchduration you're clearing a timeout which won't exist (because it will have just timed out... That is probably what is giving you your timeout errors...

---

Posted at 2014-07-13 by Hardware_Hacks

Thanks for the pointers Gordon I'll take a look.

A9 is just an enable pin which disables/enables the GPS running when you change the output. It's worked fine on other devices this way so I would assume your first point and second post are more to do with it.

---

Posted at 2014-07-13 by Hardware_Hacks

I just have to let the timeout run its duration it seems. If I try to clear the timeout when I get a fix it seems to produce an error every other (e.g. all even times/2times) time it searches.

Serial4.setup(9600,{tx:C10,rx:C11});

var SleepDuration = null;
var SearchDuration = null;
var foundFix = 0;
function Start_GPS(){
  
  console.log("Starting GPS...");
  //Reset fix found
  foundFix=0;
  
  //After 120 seconds disable the GPS regardless
  SearchDuration = setTimeout(function (e) { Stop_GPS(); }, 60000);
 
  //Set enable pin mode
  pinMode(A9,'output');
  
  //Set enable pin to high
  digitalWrite(A9,1);
  
  //load gps
    var gps = require("GPS").connect(Serial4, function(data) {
    console.log(data);
    //Check if we have a fix
      if(data.fix>0)
      {
        console.log("Send SMS:" + data.lat + "|" + data.lon);
        foundFix=1;
        Stop_GPS();
      }
  });
}

function Stop_GPS(){
  var gps="";
  digitalWrite(A9,0);
  console.log("Disabling GPS...");
}

function onInit(){
//SleepDuration = setTimeout(function (e) {onInit(); }, 360000);
console.log("Starting up device & Setting timers...");
setInterval(function (e) { Start_GPS(); }, 360000);
Start_GPS();
}

onInit();

---

Posted at 2014-07-14 by @gfwilliams

Could it be that Start_GPS is getting called twice, so is then overwriting the value from SearchDuration? In code that I write I tend to explicitly check and clear the interval before I set it.

Also, I'd try and call require('GPS').connect only once - as otherwise every hour it'll add another GPS handler and will probably run out of memory - even if you set gps to "", the handler will still be receiving data from Serial4.

Just a quick JS thing too - if you say var gps in a function, that'll create a local variable rather than using a global one. So for example if you write var gps = ...; in a function, when you leave the function the reference to gps will be lost (although it'll still keep working).

Maybe try:

Serial4.setup(9600,{tx:C10,rx:C11});
var SleepDuration = null;
var SearchDuration = null;
var foundFix = 0;
var gps = undefined;

function Start_GPS(){
  
  console.log("Starting GPS...");
  //Reset fix found
  foundFix=0;
  
  //After 120 seconds disable the GPS regardless
  if (SearchDuration) clearTimeout(SearchDuration);
  SearchDuration = setTimeout(function (e) { SearchDuration=undefined; Stop_GPS(); }, 60000);
 
  //Set enable pin mode
  pinMode(A9,'output');
  
  //Set enable pin to high
  digitalWrite(A9,1);
}

function Stop_GPS(){
  if (SearchDuration) {
    clearTimeout(SearchDuration);
    SearchDuration = undefined;
  }
  digitalWrite(A9,0);
  console.log("Disabling GPS...");
}

function onInit(){
  //load gps
    gps = require("GPS").connect(Serial4, function(data) {
    console.log(data);
    //Check if we have a fix
      if(data.fix>0)
      {
        console.log("Send SMS:" + data.lat + "|" + data.lon);
        foundFix=1;
        Stop_GPS();
      }
  });
console.log("Starting up device & Setting timers...");
setInterval(function (e) { Start_GPS(); }, 360000);
Start_GPS();
}
onInit();

---

Posted at 2014-07-14 by Hardware_Hacks

@gfwilliams ran the code for a couple of hours and I hit:

{
  "time":"18:54:21",
  "lat":NaN,"lon":NaN,"fix":0,"ERROR: Out of Memory!
satellit":3}
ERROR: Error processing Serial data handler - removing it.
Execution Interrupted during event processing.
at line 1 col 18
{console.log(data);if(data.fix>0){consoin function called from system
Disabling GPS...
Starting GPS...

Is there any way I can trace where the memory is leaking from?

---

Posted at 2014-07-15 by @gfwilliams

Hmm... You could try writing trace() which should dump any memory immediately accessible from root.

It's a really odd one because it looks like console.log is causing the error, but it really shouldn't allocate much memory.

The only thing I can think of is that the GPS module doesn't limit the amount of characters it stores in a line, so if you got a few thousand characters without a line break, that could cause a problem.

---

Posted at 2014-07-15 by Hardware_Hacks

So I assume best way is to check the differences between two traces() at any time?

If so this is what I found:
http://www.diffchecker.com/nt8yotd5

"line" mentioned at line 85/86 is in the underlying GPS module @gfwilliams created for Espruino I think?

http://www.espruino.com/modules/GPS.js

Edit:
Removed NoFix as it wasn't actually needed/being used. "line" still seems to keep growing up to 1500+ and then I get the out of memory.

---

Posted at 2014-07-16 by @gfwilliams

That's a really handy website :) Yes, diffing is the way to go.

So yes, it looks like it's an issue with that module.

What is the contents of 'gps.line' when it gets so big? Espruino is relying on a newline being sent by the module, but it might not be sending one for some reason?

I'll add some sanity check in there, but I'd hoped that I wouldn't have to...

---

Posted at 2014-07-16 by @gfwilliams

Right - I've just updated the GPS module to (hopefully) limit the size of gps.line to 80 characters... So just reprogramming Espruino again should fix your memory issues.

I'd still be interested to see what was in gps.line though...

---

Posted at 2014-07-16 by @gfwilliams

Just been running the simple GPS example here, and this happened to me. It seems there's actually a proper memory leak somewhere (almost certainly an internal Espruino one).... I'll see if I can find out what it is - exactly one variable per call of the callback is getting lost.

---

Posted at 2014-07-16 by @gfwilliams

Wow, that was bad. There was a leak in String.split.

I've fixed it now (so it'll be fixed in 1v68), and you'll be able to pick up the latest build via www.espruino.com/binaries/git/commits/f4de82e93f6efc911299b30293128c6b51999d20 in about an hour.

Sorry about all that - I was pretty sure there wasn't an issue with something so simple (in fact I remember looking for a memory leak in GPS before so I'm amazed I didn't find this one at the same time).

---

Posted at 2014-07-16 by Hardware_Hacks

No problem you helped me plenty with the code nice to see that we've found something else to contribute to the project :)