Espruino Internals

[gfwilliams]

Posted at 2015-04-09 by @gfwilliams

I recently got asked some pretty deep questions about Espruino by e-mail - I'm copying here as they're probably useful to anyone hacking on Espruino...

The structure of JsVar is: https://github.com/espruino/Espruino/blob/4e9f34831ff9f200cc5e261c30a2af9f145b980c/src/jsvar.h#L122

The structure of JsVarData is : https://github.com/espruino/Espruino/blob/4e9f34831ff9f200cc5e261c30a2af9f145b980c/src/jsvar.h#L110

For this structure of JsVarData, I do not understand the effect of “native“ and “ref”.

'Native' is information for functions that are implemented as native code rather than JavaScript (so basically compiled C code). 'Ref' contains references to other children/siblings in the linked list/tree structure.

And why does it use union structure. What is the advantage of the union structure?

See http://en.wikipedia.org/wiki/Union_type#C.2FC.2B.2B

Basically unions overlap in memory, so all the elements of JsVarData share the same bytes. There is some documentation under the definition that describes how bytes are used for different types of variable:

https://github.com/espruino/Espruino/blob/master/src/jsvar.h#L144

Also:

http://www.espruino.com/Internals

What puzzled me most is the structure of JsVarDateRef: https://github.com/espruino/Espruino/blob/4e9f34831ff9f200cc5e261c30a2af9f145b980c/src/jsvar.h#L98

What do the “nextSibling”, ”prevSibling”, ”refs”, ”firstChild” and “lastChild” mean? Could you give me an example for understanding every part of the JsVar structure wil be assigned?

“nextSibling”, ”prevSibling” are when the JsVar is used in a linked list
refs is for reference counting - it's the number of references to the JsVar - if that is 0 and there are no locks then the variable can be freed.
”firstChild” and “lastChild” point to the beginning and end of a linked list of children.

The best way to get an idea of how they're all used is to type trace() in Espruino. This will dump out the datastructure in a human readable form (to see how, look at jsvTrace).

And what is the whole structure between variable and another variable? A Binary tree or a list? Please give me an example.

It's a linked list. But a JsVar can have children, in which case it is a linked list of children too.

If I upload the code digitalWrite([A5,A6],0x11) to the board in the terminal, I do not know how the every part of JsVar Structure will be assigned for the array [A5,A6].

[A5,A6] goes to JsVar(Array, JsVar(Name_int, 0, JsVar(Pin,A5)), JsVar(Name_int, 1, JsVar(Pin, A6)))

(Note: the use of JsVar() is just pseudo-code to denote a tree structure)

But arrays can be 'packed' - if you used a simple number like [4,5] you'd get: JsVar(Array, JsVar(Name_int_int, 0, 4), JsVar(Name_int_int, 1, 5))

And how does the array [A5,A6] add to the JsExecIfo->root? It is a binary tree or a list, or some struct else?

It actually doesn't go into JsExecIfo->root as it isn't a global variable, but it's stored as a linked list while executing the function digitalWrite

[espruino-discuss3]

Posted at 2015-04-09 by @gfwilliams

And then...

Byte	Name	STRING	STR_EXT	NAME_STR	NAME_INT	INT	DOUBLE	OBJ/FUNC/ARRAY	ARRAYBUFFER	FLAT_STR	NATIVE_FUNC
0 - 3	varData	data	data	data	data	data	data	nativePtr	size	size	nativePtr
4 - 5	next	-	data	next	next	-	data		format	-	argTypes
6 - 7	prev	-	data	prev	prev	-	data		format	-	-
8 - 9	refs	refs	data	refs	refs	refs	refs	refs	refs	refs	refs
10-11	first	-	data	child	child	-	-	first	stringPtr	-	-
12-13	last	nextPtr	nextPtr	nextPtr	-	-	r?	last	-	nextPtr	-
14-15	Flags	Flags	Flags	Flags	Flags	Flags	Flags	Flags	Flags	Flags	Flags

I saw it already last week, but I can not tell the difference among NAME, INT, NAME_INT, NAME_INT_INT? also STRING and STR_EXT.

You'd have to look further into the code - look at the comments here: https://github.com/espruino/Espruino/blob/master/src/jsutils.h#L254

And in the table, what is the difference between “varData” and “data”, “nextPtr” and “next”?

The 2nd column 'Name' is just the name of the field - it doesn't really mean anything other than being descriptive (and possibly the name of the variable)

And what does the “child” means, it is firstChild?

Some variable types have just a single child. See jsvHasSingleChild

digitalWrite([A5,A6],0x11), I see the function about this is : https://github.com/espruino/Espruino/blob/4e9f34831ff9f200cc5e261c30a2af9f145b980c/src/jswrap_io.c#L233

In the function, jsvLock(pinName) uses jsvGetAddressOf(pinName), and jsvGetAddressOf(pinName) just return &jsVars[ref-1]. Does it mean all variable are stored in jsVar[JSVAR_CACHE_SIZE], including local variable and global variable, like Serial1?

Yes. jswrapper may 'make' the global variable if it doesn't already exist though.

If it is, what is the function of the jsvSkipName(pinNamePtr). I see the definition of the jsvSkipName(), It is : https://github.com/espruino/Espruino/blob/4e9f34831ff9f200cc5e261c30a2af9f145b980c/src/jsvar.c#L1542

I don’t understand that “if a is a name ” . What does “a name” mean?

see 'lvalue' on wikipedia - a Name in Espruino is one of those - a variable that is a pointer to another. For instance in a = 1, 1 is not a name (it cannot be written) but a is. However, after assignment a will point to the value 1.

What is the effect of the function indeed ?

It skips the name. In the case above, 'jsvSkipName(a)' would skip over 'a' and return 1.

And what is the relationship of JsExecIfo->root and jsVar[JSVAR_CACHE_SIZE]

JsExecIfo->root is basically the value of the 'global' variable. jsVar[JSVAR_CACHE_SIZE] holds all variables, including 'global'.

---

Posted at 2015-04-09 by alexanderbrevig

Extremely useful! Sticky or link to here from github README maybe?
This would've saved me some hours a couple of months ago.

Good job with the writeup!

---

Posted at 2015-04-09 by @gfwilliams

Good idea - I'll put a link on http://www.espruino.com/Internals

---

Posted at 2015-08-05 by RandyHarmon

Sorry if I've missed this detail in existing communication. I'm interested in how variable names are stored (with its data?) and matched from executing code. I'm assuming if Espruino uses a linked list for variables, that a variable name from executing source has to be found in the JsVar structure?

I'm particularly curious about the string-length overheads (on mem & CPU both) during execution of code having variable names, and what optimizations may already be in use or available (e.g. pre-munging var names with Esprima, or ???).

Another angle on optimization (if relevant) is code-path frequency, in that (assuming a given variable name is evaluated faster if shorter), we could rewrite most-commonly-ACCESSED vars with the shortest names (it's hard to think Esprima could help much with that)

Another question has to do with namespacing of vars. It seems to me that an object global.JUNK = { foo: 'bar' } implies a variable 'foo' in JUNK's namespace (yes?) which puts an interesting twist on potentially rewriting 'foo'. Seems like JUNK has a linked list of its properties, but again with the variable-name question.

Finally, if I have obj1 = { foo: "bar" }; obj2 = obj1, then these two variables both point to the same object. Probably the answer about var names will demonstrate how this is implemented too, but if not, that extra info would be interesting too.

Thanks in advance.

R

---

Posted at 2015-08-06 by @gfwilliams

Have you checked out http://www.espruino.com/Performance and http://www.espruino.com/Internals ?

If you something like A.foo() it does a linear search in A, but then if A is some kind of built-in class (or has a prototype that is one) it'll do a binary search through the built-in functions.

Variable names of 4 chars or left are stored in a single JsVar, so are very efficient (so minified code works great). Also, if the value they represent is small (for instance a 16 bit int) it is stored in the exact same JsVar. If not (for instance it's an object) it points to another JsVar that represents the object.

The initial search is pretty quick too, as Espruino does something like:

while (var) {
  if (var.first4Bytes==first4BytesOfName) 
    var.checkProperly(name);
  var = var.next;
}

To take advantage of this, modules are already minified.

Another question has to do with namespacing of vars.

Because you've got linked lists, if you've got a bunch of functions/variables you're better off adding your own tree structure by putting them in objects as you suggest. Generally you'd do that anyway though...

So for instance my car's ECU uses Espruino, and it has a bunch of pins that control things. I put those in an object called PINS, so now Espruino isn't searching through them for everything, but only when I access PINS.tempSensor or whatever it is.

Finally, if I have obj1 = { foo: "bar" }; obj2 = obj1, then these two variables both point to the same object.

It's best to poke around with the trace() command:

>obj1 = { foo: "bar" }; obj2 = obj1
={ 
  "foo": "bar"
 }
>trace()
#1[r2,l1] Object { 
  // internal stuff here you don't care about
  #18[r1,l2] Name String [1 blocks] "obj1"    #17[r2,l1] Object { 
      #22[r1,l2] Name String [1 blocks] "foo"        #23[r1,l1] String [1 blocks] "bar"  
    } 
  #24[r1,l2] Name String [1 blocks] "obj2"    #17[r2,l1] Object { 
      #22[r1,l2] Name String [1 blocks] "foo"        #23[r1,l1] String [1 blocks] "bar"  
    } 
}

So you actually have just 5 JsVars (the number with the # in front is a unique ID):

• 2 for obj1 and obj2
• 1 for the object
• 1 for foo
• 1 for bar

But if foo was an int...

>obj1 = { foo: 42 }; obj2 = obj1   
={ "foo": 42 }
>trace()                        
#1[r2,l1] Object { 
  // stuff
  #18[r1,l2] Name String [1 blocks] "obj1"    #5[r2,l1] Object { 
      #36[r1,l2] Name String [1 blocks] "foo"= int 42
    } 
  #24[r1,l2] Name String [1 blocks] "obj2"    #5[r2,l1] Object { 
      #36[r1,l2] Name String [1 blocks] "foo"= int 42
    } 
}

Suddenly its only 4 JsVars, because both foo and 42 get jammed into the same block.

In the same way, if you had a long name that wouldn't fit in one var, another one gets added:

>obj1 = { fooBarBaz: "bar" }; obj2 = obj1  
={ 
  "fooBarBaz": "bar"
 }
>trace(obj1)
#41[r2,l1] Object { 
  #23[r1,l2] Name String [2 blocks] "fooBarBaz"    #46[r1,l1] String [1 blocks] "bar"  
}

Now the object itself takes 4 blocks, so the whole thing (obj1 and obj2) takes 6.