some questions about the internal (2)

[espruino-discuss3]

Posted at 2015-06-24 by Long

Hi,Gordon!
Here are some questions I want to ask about the internal.

1.About the FlatString.
(1)The blocks of FlatString is

size_t jsvGetFlatStringBlocks(JsVar *v) {

  assert(jsvIsFlatString(v));

  return ((size_t)v->varData.integer+sizeof(JsVar)-1) / sizeof(JsVar);

}

why is it not (size_t)v->varData.integer / sizeof(JsVar) ?

(2) From the formula, it implys that 'FlatString' uses the whole JsVar to contain the data. So it has no LastChild . But In the function of jsvAppendStringVar():

void jsvAppendStringVar(JsVar *var, const JsVar *str, size_t stridx, size_t maxLength) {
       JsVar *block = jsvLockAgain(var);
       assert(jsvIsString(var));
      // Find the block at end of the string...
      while (jsvGetLastChild(block)) {
             JsVarRef next = jsvGetLastChild(block);
            jsvUnLock(block);
            block = jsvLock(next);
      }
       // find how full the block is
      size_t blockChars = jsvGetCharactersInVar(block);
      .......

"assert(jsvIsString(var));" means var can not be a StringEXT, but the while loop uses the 'LastChild' to get the final block. So it can not be a 'FlatString'. So which type of JsVar does this code deal with?

(3) And in which condition can we use 'FlatString' ,not ' StringEXT '?

(4) And why in the function of jsvGetNativeFunctionPtr():

void *jsvGetNativeFunctionPtr(const JsVar *function) {
  /* see descriptions in jsvar.h. If we have a child called JSPARSE_FUNCTION_CODE_NAME
then we execute code straight from that */
  JsVar *flatString = jsvFindChildFromString((JsVar*)function, JSPARSE_FUNCTION_CODE_NAME, 0);
  if (flatString) {
    flatString = jsvSkipNameAndUnLock(flatString);
    **void *v = (void*)((size_t)function->varData.native.ptr + (char*)jsvGetFlatStringPointer(flatString));**
    jsvUnLock(flatString);
    return v;
  } else
    return (void *)function->varData.native.ptr;
}

why is it not void *v = (void*)((size_t)function->varData.native.ptr?

2.About tha Array.
In the function of jsvGetArrayItem():

JsVar *jsvGetArrayItem(const JsVar *arr, JsVarInt index) {

  JsVarRef childref = jsvGetLastChild(arr);

  JsVarInt lastArrayIndex = 0;

  // Look at last non-string element!

  while (childref) {

    JsVar *child = jsvLock(childref); 
    if (jsvIsInt(child)) {
      lastArrayIndex = child->varData.integer;
      // it was the last element... sorted!
      if (lastArrayIndex == index) {
        return jsvSkipNameAndUnLock(child);
      }
      jsvUnLock(child);
      break;
    }
    // if not an int, keep going
    childref = jsvGetPrevSibling(child);
    jsvUnLock(child);
  }
.......

The while loop means 'Array' have 'Name_String' childName. how can it have ?
And why does the jsvArrayPop() just pop the 'Name_Int' childName. What about the 'Name_String' childName?

3.Is the effect of 'LOCK' (like jsvUnLock() and jsvLock()) just to check whether the JsVar can be freed or not? Or does it have something else effect?

Hope you can give me some tips about these questions. Sorry for that I am troubling with the text edit for formatting.

[espruino-discuss3]

Posted at 2015-06-24 by @gfwilliams

Just fixed your formatting - it's not that hard once you get the hang of it. Just add the 3 ticks before and after code, and make sure there is a newline before and after them.

1. FlatString blocks - why is it not (size_t)v->varData.integer / sizeof(JsVar)

If sizeof(JsVar)=16 and you want to store 15 characters, you need 1 block. But 15/16 in integer math is 0. Same for 1 character.

So what you do is add 15 then divide by 16, so for character, (1+15)/16 = 1

(2) From the formula, it implys that 'FlatString' uses the whole JsVar to contain the data. So it has no LastChild

Flat Strings use 1 JsVar at the start (which includes the type, reference count, an integer describing the length, and a LastChild pointer). Subsequent JsVars store the string data.

That means that every Flat String does have a LastChild, and that you can actually append data to a Flat string (although then it doesn't stay flat!). It allows all the existing String manipulation functions to treat Flat Strings as if they were normal strings.

3. And in which condition can we use 'FlatString' ,not ' StringEXT '?

I don't understand the question I'm afraid.

(4) And why in the function of jsvGetNativeFunctionPtr(): why is it not void v = (void)((size_t)function->varData.native­.ptr?

It's because in something like var a = E.nativeCall(1, "void ()", atob("w4D...my...base64...encoded...data...AAAg")) the pointer needs to be relative to the Flat String that the native function references.

At the bottom of the function you can see that it is what you suggest if no Flat String is supplied.

The while loop means 'Array' have 'Name_String' childName. how can it have ?

Arrays can have NAME_STRING, NAME_INT, etc as child. We check that it's the correct child type first using jsvIsInt.

Why does the jsvArrayPop() just pop the 'Name_Int' childName. What about the 'Name_String' childName?

Because if it didn't, the following would fail:

var a = [5,6,7];
a["foo"] = "Oh No!";
a.pop(); // we expect 7 here, not "Oh No!"

Is the effect of 'LOCK' (like jsvUnLock() and jsvLock()) just to check whether the JsVar can be freed or not? Or does it have something else effect?

Its stores whether any bit of code has a pointer to the variable or not. If a variable is referenced but not locked then it needs to be kept, but it can be moved around because nothing has a pointer to it.

Nothing uses that just yet, but at some point in the future there may be a 'defragmentation' step, if we start to run into problems with allocation for Flat Strings.

Espruino was also designed to allow it to execute code from external memory - in which case the lock count could be used to see if the variable was needed in RAM any more or not. Again, I haven't had to implement that yet, but it could mean that Espruino could run on (for instance) a cheap microcontroller with 8kB RAM, while running from SPI memory with 1MB of RAM.

Out of interest, why are you diving so deep inside Espruino? Do you have some project in mind that you plan to use it for?

---

Posted at 2015-06-24 by Long

oh, Thank you for your patience ! I am just for fun, and maybe use it for make a library for my colleague's project --mixly_arduino.(see here https://github.com/xbed/mixly_arduino ).

And in which condition can we use 'FlatString' ,not ' StringEXT '?

There are JSV_FLAT_STRING and JSV_STRING_EXT. JSV_STRING_EXT is extra character data for string. Why not use FlatString to replace JSV_STRING_EXT? Maybe I don't realize something else?

If a variable is referenced but not locked then it needs to be kept, but it can be moved around because nothing has a pointer to it.

So the 'refs' can also have the same effect of the 'LOCK'. Does it?

---

Posted at 2015-06-24 by @gfwilliams

Why not use FlatString to replace JSV_STRING_EXT?

Ahh, right. It comes down to memory allocation. Suppose we did:

a="Lots and lots of data in here"
a+="!";

If we used flat strings, you'd probably have to allocate a whole new flat string and copy the result in. Not only does that take a while, but it might not be possible to allocate a big continuous chunk of memory due to fragmentation.

For example in C, if you did:

// lets say we have 128kB RAM
void *ptrs[128];
for (i=0;i<128;i++) ptrs[i] = malloc(1024); 
// all RAM allocated
// Now let's free *every other* item
for (i=0;i<128;i+=2) free(ptrs[i]); 
// And now lets try and allocate one (slightly larger) memory block
malloc(1500); 
// this will fail. Even though we have 64kB free, it's fragmented and there isn't one continuous block of 1500 bytes

People tend to ignore it, but in an interpreted language where you're doing this kind of thing all the time it's dead easy to get fragmented memory. Using StringExt is slightly less efficient but it means you never hit this problem.

So the 'refs' can also have the same effect of the 'LOCK'

Yes, they're almost identical.

• 'ref' is the number of references from other JsVars
• 'lock' is the number of references from native code

The only difference right now is that when Garbage collecting, a variable can be freed even if refs>0, but it can never be freed if locks>0