Easy javascript question - calling a property of type function by name?

[espruino-discuss]

Posted at 2015-04-14 by DrAzzy

Say I have an object with a number of properties of the function type.

var obj={}
obj.test1=function(a){console.log("test1 called"+a);};
obj.test2=function(a){console.log("test2 called"+a);};

Now, I want a function that takes a string and an argument. If the object has that property, call the function with the argument. If not, return an error. Is there any way to do this without eval, or making scratch variables that waste memory?

function jstest(funname,argument)
if (obj.hasOwnProperty(funname))
			{
				console.log(obj. //what do I do here? if I call jstest("test1",a), I want this line to do console.log(obj.test1(a)), if called with "test2" instead, obj.test2(a), etc. How do I go between 
			} else { //handle error }

The context is that I am writing a webserver code for Espruino which will serve most file extensions off the SD card, but if it sees certain extensions (indicating that the page should be dynamically generated), it will see if there's a function to generate that page, and call it if so.

My thinking was that I'd have an object like this, so if I got a request for status.json, I'd call the jsonPage.status() function, if I got a request for asdf.json, I'd call jsonPage.asdf() - or if that didn't exist, I'd return a 404. But what I'm puzzled about is how to call the function without using eval(), which I've been criticized for overusing in the past - I have no idea how else to do it! Even using eval is really awkward, since I'd need to do var temp=eval("jsonPage."+funname); temp(argument); - this means making a copy, which is wasteful of memory...

jsonPage = {};
jsonPage.status=function(path,query){return "status goes here";};
jsonPage.history=function(path,query){return "history goes here";};

There's got to be a simple way to do this, but I just don't know what it is, nor what to call this to get results from google.

[espruino-discuss]

Posted at 2015-04-14 by alexanderbrevig

What about

var obj={};
obj.test1=function(a){console.log("test1­ called"+a);};

function jstest(funname,argument) {
  if (obj[funname] !== undefined) { //or if (typeof obj[funname] === 'function') ??
    obj[funname](argument);
  }
}

jstest("test1", "test");

Not tested.

[gfwilliams]

Posted at 2015-04-14 by @gfwilliams

Also if (funname in obj) ... ?

[espruino-discuss]

Posted at 2015-04-15 by @allObjects

@drazzy, cool thing your are up to... #webserver

you can go even further... as you know, require() can load dynamically - at runtime - module from the card and there is also a way to unload it afterwards.

Depending on the extension, you first check for a statically loaded function, if it is not there, go for a module, and if there is no module loaded, throw the error... 404.

What about below code (used long names help with documentation):

var requestHandler = 
{ handlers: // handlers for extension and file|page (=function) name:
  { json:
    { status: { function() { // handles status.json
        // status function code accepting with 'arguments' - either specified or as array
      }
    , history: function() { // handles history.json
        // history function code ...
      }
    , etc: function() { // handles etc.json
        ...
      }
    , _: function() // default handler for json extension using file(content) as parm
        ...
      }
    } // /json
  , xml:
    { status: function() { // handles status.xml
        // status function code ...
      }
    , history: function() { // handles history.xml
        // history function code ... 
      }
    , etc: function() { // handles etc.xml
        ...
      }
    , _: function() // default handler for xml extension using file(content) as parm
        ...
      }
    } // /xml
  } // handlers
, statistics: { // houskeeping for to make it really 'smart'/controlled: 
                // managing handlers like virtual memory managment
                // or old (non-virtual) drop and reload of exec code.
                // detect limit on load (see below) and purge 
                // least used stuff... you can reload later anyway
     ...
  }
, load: function(extension,functionName,funktion) { // load a handler function
    var handlers = this.handlers[extension];
    if (!handlers) { handlers = (this.handlers[extension] = {}); }
    if (handlers[functionName]) { console.log("function " + functionName + " already loaded"); }
    handler[functionName] = funktion; // load it anyway?
  }
, purge: function(extension,functionName) { // purge handler function to free memory
    var handlers = this.handlers[extension];
    if (handlers && handlers[functionName]) { 
      delete handlers[functionName]);
      if (handlers.length === 0) { delete this.handlers[extension]; } }
  }
, handle: function(path,fileDotExtension,parms,response) { // request ...
    // request may be url w/ query string and/or body from http request
    // path / uri can be normalized by replacing / with _ and 
    //  become part of the file and thus function name (for now not considered)
    var  fileDotExtensionElements.split("."), file = fdees[0], ext = fdees[1];
    var handlers = this.handlers[ext], funktion;
    if (handlers) { // if you do not purge handler extensions and start with set of 
                    // predefined handler extensions, then you can take this set as 
                    // the criteria to detect invalid / unimplemented extensions...
                    // or put this instead into the stats / housekeeping / control
      var functionName = (file.charAt === "_")  ? "_" : file; // do some defaulting:
        // do default _ function for files beginning w/ _, for example .jpg, etc
      funktion = handlers[functionName]);
      if (!funktion && (funktion = require(functionName + "_"+ ext + ".js")) { 
        // function not in handler but found a module (module name syntax to be defined)
        this.load(ext,functionName,funktion); }
    }
    if (funktion) { // function available, finally do something...
      funktion(parms,....); // ...invoke (or you may also use call or apply)
    } else {
      console.log("no handler for extension " + ext + " and file " + file); 
      // you may want to put also something on the response.
    }
  }  
}; // /handler

Are you also thinking of a file type / extension that can include references to other files and
is recursively interpreted (with this handler) to allow page / response composition? ...for example as a nested structure of divs of a streamed out html page?

The handler as module could also have ways to go after 'load code from EPROM'. Btw, if managed code is eval-ed, it is not THAT a bad thing to do...

PS: I'm currently in a time/resource bind, otherwise the code above would be tested... ;-)

[gfwilliams]

Posted at 2015-04-15 by @gfwilliams

Just to overcomplicate matters, you could actually serve up a kind of poor-mans JSP, by loading the file and searching for <%= and %>, then running eval on what's inside :)

[espruino-discuss]

Posted at 2015-04-15 by @allObjects

The context is that I am writing a webserver code for Espruino which will serve most file extensions off the SD card, but if it sees certain extensions (indicating that the page should be dynamically generated), it will see if there's a function to generate that page, and call it if so.

@drazzy, is that the requirement?

[espruino-discuss]

Posted at 2015-04-15 by @joakim

I like the require() idea of @allObjects, to store dynamic pages as modules on the SD card, load them on demand and unload from memory when done. Could be made to be pretty flexible, with the modules exporting properties and methods to be used by the webserver.

One could even create a dynamic lightweight routing table in memory by loading each of the "route modules" one by one, storing its parameters + a reference to its file (not the callback itself) in the routing table and then unloading the module from memory. On lookup, the server would load the matching route's module, run its callback, and unload it.

The route modules could live in e.g. /routes for simple automatic retrieval. Their exports object could look something like hapi.js' routing definitions, even having the server support stuff like parameters in path definitions.

[espruino-discuss]

Posted at 2015-04-15 by @allObjects

@joakim, it all depends on the requirements... but thanks for your kind words. @drazzy's current implementation can be seen at the very beginning of this post - ESP8266 server only serves page once, sometimes twice. His needs are obviously more than satisfied with a simple dedicated implementation using switch-case to handle just 3 options.

Currently I'm deep into node.js / express / ... (and other heavy things - as whole frameworks solutions compared to (small) modules) where things are not that resource constraint... and that leads me (sometimes to go astray) to more generic than just pragmatic solutions. It is an adjustment from client and server side JS to micro controller JS.

[espruino-discuss]

Posted at 2015-04-15 by DrAzzy

Thanks all for the suggestions and thoughts. I've obviously gotten the original question sorted out.

I have a bunch of Pico's coming, and I figure many of them will be connected to the internet doing stuff. The natural choice for internet interfacing is HTTP - as it's standard, simple, the tools for it are well developed, and basic client software is ubiquitous. I figure if it needs to react to conditions communicated over WiFi, that's an http server. Since I'm going to need something like this on multiple projects, I figured I should make something generic.

So I thought about requirements:

1. Serve (dynamically generated) files containing information generated on the Espruino
2. Accept requests that change settings or initiate process changes on the Espruino
3. Execute arbitrary javascript code included as a query option (likely not enabled under normal circumstances).
4. Serve static files off an SD card, enabling it to serve up a web control panel without help.
5. Minimize memory overhead associated with the webpage serving code, to leave as much as possible available for the application.

@gordon - I absolutely do not want to go down the path of creating my own markup system for pages - that makes the Espruino do a whole lot more work, when you can just serve a static page with some JS that loads the status page and puts the values into the right places - which also has the benefit of the big pages only having to be served once. Not only is it more work, I think parsing files like that is the Wrong Way.

@allObjects - Definitely right call on the "handlers" object to contain json/etc handlers - that also lets me genericize the code that calls the handler.

I'm not sure what the memory use ramifications of using require() to load the handlers. I'd hope that the handlers would be small enough that that's not a problem, but I could easily have it call code stored on eeprom/SD card if I had to, and add or remove handlers at will (for example, we could dynamically add handlers in response to other events...). However, at this point, it doesn't seem necessary or useful.

Serial4.setup(9600, { rx: C11, tx : C10 });


var http = require("http");
var wifi = require("ESP8266WiFi").connect(Serial4, function(err) {
  
  if (err) throw err;
  wifi.reset(function(err) {
    if (err) throw err;
    console.log("Connecting to WiFi");
    wifi.connect(wifi.config.ssid,wifi.config.pass, function(err) {
      if (err) throw err;
      console.log("Connected");
      wifi.getIP(function(err,ip){wifi.config.ip=ip;});
      setTimeout(wifi.userinit.bind(wifi),15000);
    });
  });
});
wifi.config={ssid:"TwilightZone", pass:"snip", port:80};
wifi.fpfx="html"; //file prefix for serving files off SD card;

wifi.userinit= function() { //set up the server. 
	console.log("setting up server on "+this.config.ip+":"+this.config.port);
	this.server=require('http').createServer(this.onRequest.bind(this)).listen(this.config.port);
};
wifi.onRequest=function (req, res) {
	var par=url.parse(req.url,true);
	var q=par.query; 
	var nam=par.pathname; 
	var l=nam.indexOf("/");
	nam=nam.slice(l);
	var rd=this.procreq(nam,q);
	res.writeHead(rd.code,rd.head?rd.head:{'Content-Type': 'text/plain'});
	if (!rd.file) {
		res.write(rd.body);
        res.end();
	} else {
		rd.file.pipe(res);
        res.end();
	}
};

wifi.procreq = function (path,query) {
	var paths=path.split(".");
	var rd={};
	rd.code=404;
	rd.body="";
	// code goes here
    console.log(paths[1]);
	if (paths[1] in this.handler) {
		if (paths[0].slice(1) in this.handler[paths[1]])
			rd=this.handler[paths[1]][paths[0].slice(1)](path,query);
		} else if ("_" in this.handler[paths[1]]){
			rd=this.handler[paths[1]]["_"](path,query);
		} else { rd.body="Handler does not support this file.";}
	else {
		var f = E.openFile(wifi.fpfx+"/"+path, "r");
		if (f==undefined) {
			rd.body="File "+path+" not found";
		} else {
			rd.code=200;
			rd.file=f;
		}	
	}
	return rd;
};
wifi.handler={};
wifi.handler.json={};
wifi.handler.json.status= function (path,query) {
	 return {code:200,body:'{gtg:true,dtf:false,missiles:["armed","armed","repair","mothballed"]}'};
};
wifi.handler.json._ = function (path,query) {
	 return {code:404,body:"Invalid json data requested: "+path};
};
wifi.handler.run={};
wifi.handler.run.code= function (path,query) {
	 try {
	 	return {code:200,body:+eval(query.code)}; //danger! This is about as insecure as it gets!
	 } catch(err) {
	 	return {code:500,body:"Error thrown: "+err};
	 }
};

[espruino-discuss]

Posted at 2015-04-15 by @allObjects

@drazzy, is above code working to your expectations? I try to understand and get a bit confused about the sequence in which things get called. Last but not least the 15s timeout on line 16 throws me off. (The double-require of http - once in line 5 and the second time in 25 are probably just a casual thing...).

To capture the ready state, Promise help. When challenged with that, I though chose a more lightweight approach and has retry option built in - and called it Tasker - Task synchronizer - kind of a 'purposed' Promise. It helped me to detangle the callbacks, and also get rid of timeouts that most of the time take annoying long but are never long enough when it comes to hick0-ups.

As you posted in a different conversation, the callback-mania can become a throw-off and a challenge. May be it is a bit here too... So many things have to get ready - including dependencies have to be considered - before being able to 'moving on' - and you know only within (nested) callbacks when that is the case (for example, you need the ip address, which you can get when successfully connected, and then you can create the server...).

I also wonder why you put all into one single object: wifi. Having the things separated would allow to exchange the 'http connection' with what ever is available C33K, ESP8266, WIZ550S2E,... On the other hand - as pointed out for your case - your gearing up for managing swarms of Picos that connect with ESP8266.

Bottom line, the code you present, is straight forward.

[gfwilliams]

Posted at 2015-04-16 by @gfwilliams

Looks good - just some ideas:

• A simple way to make the arbitrary code execution a bit more secure is to hash it with a hidden value: if (hash(receivedCode+"MyRandomPhrase")==receivedHash) eval(receivedCode). There's already the sha256 implementation in Espruino that would handle that I guess.
• If you ever did want to execute code off the SD, I wouldn't use require. I recently sorted out new Function() so it'll take arguments, so you can do: return (new Function("wifi", myCode))(wifi) which'll execute your code in its own scope, but with a variable called wifi.
• For some things you might want to stream something to the output from code that is larger than the RAM you have available (see 'Transferring large amounts of data' in http://www.espruino.com/Internet) - Simple example is where you have a big Uint8Array of historical data, but turning that into comma-separated text increases the size by 3-4 times. It might make sense to pass the http response into the handler - you could always detect whether the handler returns anything - if it does you just send it as you do now, but if it returns undefined (or null?) you just leave the http response alone and assume that the handler has taken ownership of it.

[espruino-discuss]

Posted at 2015-04-16 by @joakim

@gordon Why would require()ing a module from SD and executing its handler/callback be a bad idea? My thinking was that loading code on-demand and unloading it when done would be better than having everything in memory at all times. One would be able to have a lot more dynamic pages than would otherwise fit in Espruino's memory. Each module would of course have to be small enough to fit in memory when loaded.

Or were you thinking of latency, energy consumption or some other factor?

[gfwilliams]

Posted at 2015-04-16 by @gfwilliams

Loading on demand is fine, but require() loads the code as a module, so all of the module gets cached (in case you use it a second time) and you have to manually clear it with Modules.removeCached(...).

If you just load it with new Function(..) or eval then when the function is done executing, all the variables referenced by it will get freed.