Architecture: WiFi generic object interface

[gfwilliams]

Posted at 2015-09-28 by @gfwilliams

As requested by @Kolban.

Still not sure on a name, apart from network? As a starting point:

wifi.setPower(onOrOff, function(err) { ... })

Initialise WiFi, or turn it off. That could be handy?

wifi.connect(ssid, key, [options], function(err) { ... });

Connect to the given access point. options could be an object extra stuff like the security type.

The callback is called with err==null on success. Could maybe also return the IP?

wifi.getAPs(function(err, aps) { ... });

Call the callback with a list of access points, of the form aps = [ { ssid, enc, signal_strength, mac_address } ].

The callback is called with err==null on success.

wifi.getConnectedAP(function(err, ap) { ... });

Call the callback with the name of the currently connected access point. The callback is called with err==null on success.

wifi.createAP(ssid, key, channel, enc, function(err) { ... })

Create an access point with the given ssid, key, channel, and encoding. Encoding can be 0, undefined, "open", "wep", "wpa_psk", "wpa2_psk" or "wpa_wpa2_psk".

Example: wifi.createAP("ESP123","HelloWorld",5,"wpa2_psk",print)

wifi.getConnectedDevices(function(err, devices) { ... });

If if AP mode (with wifi.createAP), call the callback with the second argument as an array of { ip, mac } objects - one for each connected device.

wifi.getIP(function(err, ip) { ... });

Call the callback with the current address details: {ip:string, mac:string, ...?}

The callback is called with err==null on success.

[espruino-discuss3]

Posted at 2015-09-28 by @allObjects

2cts: I would not call the parm in the callback err. I'd rather call it somthing like status or info or ...

If returning 0 (zero), or null, or undefined as the only ok values, that works with easy check of negation. If something bad happened, an error object should be returned if there is more information than just a scalar. Returning a simple scalar for error and having to go back and ask for the error is an option, but it costs memory, if the error is not already an anyway-available status property.

Furthermore, I'd prefer an object over a list of parameters.

My comments should be taken with the grain of Espruino salt, because for a resource tight environment other considerations than just object-orientation apply.

---

Posted at 2015-09-28 by Kolban

@gfwilliams This is perfect ... exactly what I hope for from an architect. We can now start designing an implementation for this and see what gets uncovered as that progresses.

@allObjects Your thoughts are vital ... this is community stuff. I think I also lean towards and object with properties over parameters ... however ... I also am keen that if an object is passed in that is missing some required "property" that the implementation makes that known rather than breaks or quietly does nothing.

I am also of an opinion that callback functions that are passed an "error" indication ... that the error parameter be the last parameter in the callback function's formal parameter list. So instead of:

function(err, ap)

we would have

function(ap, err)

Of course if there is some existing standard or convention in Espruino, I would follow that and then if I felt strongly, I'd raise a work item request that we address the standard or convention as a whole.

---

Posted at 2015-09-28 by @gfwilliams

Furthermore, I'd prefer an object over a list of parameters.

Yes, although I think in some cases (args that are always there) it could be preferable to include them? Maybe not though - I'm open to changes.

Of course if there is some existing standard or convention in Espruino, I would follow that

Yeah, it's Node.js. Generally node.js callbacks all have error as the first argument (I think so you can have varying numbers of arguments - including none - and still find the error).

Also, I'd just add:

wifi.setIP(object, callback)

Where object can have ip, mac, subnet, gateway and dns in it (if some are left out they're kept as-is). Basically it matches the object that you get from getIP.

If you just call setIP() it does DHCP - at least that's what WiFi and Ethernet do right now. Not sure if it's worth adding a specific DHCP function though?

---

Posted at 2015-09-29 by MichaelPralow

for the question of api-style (error first/last, object yes/no) i vote for error first and object-style, it's just easier to follow the node.js culture here

assuming espruino is designed to be single-threaded and mostly asynchronous, than node.js seems to be a good (perfect?) style

---

Posted at 2015-09-29 by Kolban

@MichaelPralow Awesome ... community choices are (to me) the most important guidance. I am still ignorant on Node.JS so it is incumbent on me to go study that. However if y'all are saying that the defacto story is error code first ... then that has to be the right answer.

A github work item (#589) has been created to track the low level design and implementation.

espruino/Espruino#589

---

Posted at 2015-09-29 by tve

Some comments:
wifi.setPower(onOrOff, function(err) { ... }) sounds like TX power level to me? We need a function to set the wifi mode, I believe, this could include: none (=off), station, ap, station+ap. Or do you want just on/off and then drive the mode by using connect/disconnect/createAP/deleteAP ?

For wifi.connect(ssid, key, [options], function(err) { ... }); I would prefer to see callbacks that get the current state. That allows the app to show whether it's waiting to associate or waiting to get a DHCP lease. Unless you want to roll all that into a global wifi state callback.

wifi.getAPs(function(err, aps) { ... }); I believe this is usually called scan, no? Or put differently, do you expect getAPs to return a list of already-known APs or do you expect it to initiate a scan? If it initiates a scan, can the callback be called multiple times as APs are discovered or only once when the scan finishes?

Why does wifi.getAPs(function(err, aps) { ... }); need a callback as opposed to just returning the info?

wifi.createAP(ssid, key, channel, enc, function(err) { ... }) I see a problem with the channel, which is that in sta+ap mode the ap has to be on the same channel as the sta. Maybe allowing channel 0 to stand for any can make this work.

Additional thoughts:

Being able to set the IP address and associated network parameters (gateway and mask) is needed.

A call is needed to set the hostname, which may be used in DHCP to update DNS and in other local host discovery protocols. The hostname needs to be set before connecting to an AP...

We need calls to disconnect and to shut down the AP.

How about a call to get RSSI and current data rate? Alternatively, getConnectedAP could return that info.

I wonder how wifi and low-power modes should interact. It would be good to have a wifiPowerMode call with an extensible set of values, starting with full power and something mapping to PS-Poll (using the DTIM intervals to power down Wifi). This way there's at least some portable way to state that it's desirable to use less power at the expense of throughput and latency.

One question with the above is what the state at boot should be. Systems like the esp8266 save the wifi config in flash and can automatically connect at boot time. Also, I'd really like to support connecting the IDE to the esprunio via TCP/Wifi and then the espruino has to connect at boot to something on its own (unless that's expected to be driven by startup JS files stored in flash).

---

Posted at 2015-09-30 by Kolban

@tve Good thoughts ... since there are lots of questions and lots of details ... Ive created a Wiki page we can use to categorize our questions and comments. See the following:

https://github.com/espruino/Espruino/wiki/Generic-WiFi-Design

Feel free to update the page with your own comments and thoughts.

---

Posted at 2015-09-30 by @gfwilliams

Thanks - I reckon it's important to design things to be flexible enough that they can cope with different forms of WiFi/Ethernet in the future - also that functionality can be added without breaking existing code in the future.

Some of the names are choices because I'm trying to break existing code as little as possible. I try hard not to piss off existing users by making non-backwards-compatible changes. Sometimes that means some things aren't perfect though.

Just looking at createAP makes me think it should take an object, where all but the AP name is pretty much optional.

We need a function to set the wifi mode

Good plan - much nicer than setPower

Why does wifi.getAPs(function(err, aps) { ... }); need a callback as opposed to just returning the info?

Because it's something that takes time, and function calls shouldn't really block? As you suggested above, this is really a scan. Maybe it could be changed, but it is a function that exists now and which is in use by existing Espruino users.

Being able to set the IP address and associated network parameters (gateway and mask) is needed.

Yep, that's where setIP comes in. We could add the ability to set hostname in that as well

We need calls to disconnect and to shut down the AP.

Is this not just setMode you described above?

Connect at boot

Yes, it could look at what's in the variables even before the interpreter starts. USB HID does that at the moment.

Personally I'd say the user should connect themselves in onInit - but having the ability to program the device by WiFi without any of that being involved seems like it's worth it :)

But let's discuss on the Wiki

---

Posted at 2015-09-30 by @gfwilliams

Just to add to this, I notice in the Wiki there's a 'let's look at how everyone else does this' up the top.

Please also look at how Espruino handles this right now. There's:

• ESP8266 (see bottom of page for API)
• CC3000
• WIZnet Ethernet
• GSM/GPRS libraries.

They're all relatively similar but not the same - this would be a great opportunity to bring all those properly in line. However plenty of people use Espruino as it is, right now - lets not make changes that force them to rewrite their code for no good reason.

---

Posted at 2015-10-01 by Kolban

I'm thinking that's the good thing about all this pre-amble discussion. If we discuss the proposed specification for the interface up front and iron it all out (as best we can) before we implement and then ship ... that will make sure we are all in agreement.

As I understand it, if we build this new interface ... that will be a new interface that will not have been coded to previously? It will be a generic interface that all existing boards could choose to implement and at that point, a "WiFi" implementation would then be normalized. I could also imagine that we could choose that the existing implementations remain exactly the same so that there would be TWO ways that WiFi could be used ... one would be the way that exists for boards that already exist ... and the other would be a "common" mechanism that new boards would comply against.

I don't personally consider the ESP8266 support "released" so as yet, all ESP8266 board users are still alpha/beta testers and should expect flux.

---

Posted at 2015-10-01 by @gfwilliams

It's an option, but I'd like to only have 'one true interface' if at all possible. It's hard to fit everything in Flash on something like the Original Espruino Board as it is - and having 2 ways of doing the same thing just wasted space and confuses people.

ESP8266 board users are still alpha/beta testers and should expect flux.

Absolutely - I meant existing boards like the Pico and the Original Espruino. There are people developing commercial devices with those - changing stuff under their feet will just annoy them.

---

Posted at 2015-10-01 by Kolban

In my real-life job, when changes are being suggested that might impact folks upstream or downstream we call them "stakeholders" and try and get them involved early and often in things that might impact them. Perhaps if you know of folks who are "important" consumers (and that is an extremely subjective concept) we can make them aware of this work item and see if they would be interested in commenting or getting involved in the decisions? If they say that the impact of changes is not acceptable ... then that's a major consideration. If they say that proposed changes are welcomed ... they may even have important contributions to include.

Maybe drop them a note pointing to this thread and/or the issue?

---

Posted at 2015-10-01 by @gfwilliams

To be honest @AlexanderBrevig is the main one that springs to mind.

Unfortunately there are a lot of people who never come to the forum or even get in touch with me at all - I see their Web IDEs downloading modules from the Espruino website but that's it, as no information gets reported back to me :(

---

Posted at 2015-10-01 by MichaelPralow

this raises an important question for me - is it possible to work with specific module versions (entangled with firmware if needed) ?

from

var sensor = require("ADNS5050").connect(SPI3, B6); 

to

var sensor = require("ADNS5050", "1.0.2").connect(SPI3, B6); // or '1.0.+' or '1.+' or no version = latest

would be perfect to get more stable code and important hints to the exact cause of "never changed code, but now it explodes" problems

if the version is part of the require api, some sort of header informations or even a module.versions file per project does not matter, it could even be near/identical to node.js/NPM with a package.info file which contains even more nice informations for building, packaging, etc. (e.g. which espruino firmware version this javacode depends on...)

combine that with some "building info" e.g. print out of used firmware/module versions (inside WEB IDE or running espruino as well) :-)

---

Posted at 2015-10-01 by profra

Good post... it would be very useful... e.g. case ESP8266-01 on shim with different versions of AT commands... surely will come next versions... :-)

---

Posted at 2015-10-01 by @gfwilliams

@profra there are already two distinct modules for the two firmware versions available :(

@MichaelPralow yes, it's a good point. It's been asked before - and might well be worth another post about nice ways to handle this - especially since the modules don't have version numbers at the moment.

For now, as you can type a URL into require, you can actually just pull a module right off GitHub. For instance you could look at the history of something on GitHub, and could then just require a specific version of that file with:

var g = require("https://github.com/espruino/EspruinoDocs/blob/411be7a72bc55b4a51d05f3df5bc8fa80017f3b0/devices/PCD8544.js");

The URL comes from clicking <> by the commit you were interested in.

---

Posted at 2015-10-01 by MichaelPralow

@gfwilliams thx for the info, ... i was about to ask you to add an example or tutorial, but i can and will do that myself in the next days

---

Posted at 2015-10-01 by @gfwilliams

@MichaelPralow that's awesome - thanks!

You might also want to make a note that there's an option in the IDE for whether (and how) the modules you load off the net get minified. The ones that are usually loaded when you type require('module') have been pre-minified.

---

Posted at 2015-10-01 by profra

@gfwilliams I know, I was the initiator of this new version :-) ... but solution described by @MichaelPralow would be nice, elegant and universal...

---

Posted at 2015-10-01 by @allObjects

Next step to this pulling of a version is adding a config/mapping thing to the IDE to not have it in the code itself...

---

Posted at 2015-10-02 by the1laz

I think that aiming for separate config for versioning straight away would be better. Considering how widely used the npm approach is, and that espruino is using the same commonjs/require() method of using modules, I think that it would be best to not deviate from that. Also, I think keeping versioning out of Espruino runtime and only in the module-loading code would have a lot of advantages (simpler, doesn't take up flash/ram, no issues with modules on sd cards, new users don't have to deal with it).

---

Posted at 2015-10-02 by tve

I looked through the various libraries linked off the wiki page. What strikes me is that other than the particle.io one none handle the fact that wifi can come up automatically at power-up. They all seem to think that the JS code needs to drive the wifi activity when I think that that's not a good idea for several reasons.

The first is power. In the case of the esp8266 the SDK can initiate a connection before Espruino even starts to run. Depending on the state is was in (e.g. sleep) it shouldn't have to get a fresh association.

The second is that things do get complicated quickly. For example, do you really want the logic to handle state transitions and to retry to connect if the AP is lost in JS? That quickly adds up to quite some code.

Or if power-save sleep results is the loss of a connection, do you want to code the logic to receive a callback and then reconnect in JS?

I would much prefer an interface that prescribes the desired state and that lets some lower device dependent code make it happen and "keep it so" even through restarts. So instead of "connect" I would prefer a "station" call that makes Espruino a station on the specified network (the params are mostly the same). Similarly an "access_point" call that makes Espruino an AP.

How is that different? The "station" call might take some parameters to control how aggressively to reconnect when the AP is lost but the biggest difference is in the semantics. In one case the effect of a call is persistent, in the other it's not.

Maybe this can be implemented in layers so a lower layer with the primitives is available as well, but that does add code and complexity.

---

Posted at 2015-10-02 by @gfwilliams

We're going properly off-topic here! New thread?

---

@the1laz how does node.js handle requesting certain versions? Is it all done via npm - or can you require("[email protected]")?

The issue for me at the moment is that there is no versioning on the modules. I'm not 100% sure there should be either as Git commits maybe make more sense... Someone could easily come up with a historic modules plugin for the Web IDE that let you choose from a list of modules, and then let you choose the git commit from a list (which could also show date and commit message) or maybe even add your own URL if you wanted to do some development work.

---

On topic again :)

@tve I totally agree here. Espruino's about trying to bring the simplicity of scripting on desktop to an MCU, so dealing with WiFi disconnect shouldn't be something you have to worry about.

Making the connection persistent (at least between soft resets) is vital if there's going to be any OTA programming too.

Only thing I see with connection at startup is it's very likely that someone will do function onInit() { require('http').get("http://foo/i_am_alive); }.

In that case, what do you do? When the ESP8266 boots, it might be trying to connect, but it almost certainly won't have connected when onInit runs.

... and you can't delay onInit - it's very likely someone might want to display Connecting... on an LCD or something.

So you could delay executing any pending network requests until you have a connection I guess? It all gets a bit complicated.

Also, connecting to a WiFi network shouldn't really involve a write to flash memory (but it would have to if it were to be persistent?).

I'd be tempted to say:

• After connection, WiFi should stay connected between soft resets
• On a reboot, WiFi isn't connected and needs to be enabled
• (possibly) if the user does a save() while WiFi is connected, it could auto-connect at power-on?

---

Posted at 2015-10-02 by tve

Sounds like there need to be a couple of simple config options around what to do on reset/reboot 'cause I would definitely want it to always reconnect and you seem to want something different. I think I can cook a proposal up...

For the connection at boot it might be handy to have a onInitialConnect callback that can be used to trigger stuff when a connection is first established after a restart/reset. This would be different from a onStateChange callback that is called everytime something happens and makes more sense to display status or so.

---

Posted at 2015-10-02 by @gfwilliams

and you seem to want something different...

I'd suggest WiFi.connect({ ...., connectAtBoot:true}) or something similar? Then it's not overcomplicating the API.

... but I still think you should have to save() after you type that in order for it to take effect. It makes the implementation a lot easier as you can then just create an init handler for the WiFi that checks for a WiFi config JsVar and does what it suggests.

• part of me wonders whether that part of it (handling reconnects and connect at startup) is something that could be platform independent, but I might be complicating things.

it might be handy to have a onInitialConnect callback

Yes, I think that's sensible. Possible as events on this WiFi object?

---

Posted at 2015-10-02 by tve

I'm not so sure about the save() call. I would like to use Espruino without wires, also for development. That means I need to store the wifi settings persistently and unaffected by whatever latest sketch I may have loaded. If I have to save() and that changes the wifi settings, then I have to be careful to have all this wifi init boilerplate in every sketch? That's not very appealing, is it?

---

Posted at 2015-10-02 by @gfwilliams

All I meant was:

WiFi.connect({ ...., connectAtBoot:true});
save();

Not something like:

function onInit() {
 WiFi.connect({ ...., connectAtBoot:true}, function() {
  // do my stuff
 });
}
save();

I really don't think that WiFi.connect should itself save information to flash - not only is it a massive pain to implement without it also saving all current information to flash, I'm not sure it's all that helpful.

---

Posted at 2015-10-02 by tve

OK, I guess it comes down to preference/opinion. Let me try to enumerate why I believe that auto-save should be the default ;-):

• On the esp8266 the default when setting the wifi config is to save it to flash.
• On our laptop, when you associate with an access point it also gets stored and comes up again. You don't say "save this network" explicitly anywhere.
• If you don't auto-save you really have at least two configs, the saved one and the current one, so you really need more functions to set the current, set/save the saved, get the current, and get the saved one.
• The esp8266 actually has a notion of storing multiple network configs, like your laptop, and picking the appropriate one after a scan, but I have not seen code that uses that
• When you have persistent connections the notion of saving becomes a bit odd IMHO: you're saying you want a current connection that is re-established across certain events (such as loosing signal) but erased when certain other events occur (power cycle)? So it's really semi-persistent with some definition for what type of event causes loss of persistence? Boot? Reset? Power-cycle? Which do and which don't?

:-D

---

Posted at 2015-10-03 by @gfwilliams

My reasoning is it's a lot easier and more platform independent if you save the information into JsVars - but if you do that then you can't just trigger a save() when you call connect as you'll blow away existing saved code.

Is there some other area of flash in the ESP8266 where the data could be saved? If so then you could store WiFi details in that - it's just that the solution wouldn't be portable to any other device.