inlineC float byVal & byRef working example

[espruino-discuss2]

Posted at 2023-12-22 by d3nd3-o0

It took me a while to get working, so thought I'd share what worked. Can be useful for off-loading some heavy duty calculations to c, to increase throughput.

var c = E.compiledC(`
// int fadd_byval(int,int)
// void fadd_byref(int,int,int)
typedef union {
    float f;
    int i;
} floatint;
int fadd_byval(int arg1,int arg2)
{

  floatint farg1,farg2,farg3;
  farg1.i=arg1;farg2.i=arg2;

  farg3.f = farg1.f + farg2.f;

  return farg3.i;
}
void fadd_byref(floatint* arg1, floatint* arg2, floatint* arg3)
{
  floatint farg1,farg2;
  farg1.i=arg1->i;
  farg2.i=arg2->i;

  floatint out;
  out.f = farg1.f + farg2.f;

  arg3->i = out.i;
}

`);

{
let cFloatArray = (val,len)=>{
  val = typeof val !== 'undefined' ? val : 0.0;
  len = typeof len !== 'undefined' ? len : 1;

  let fs = E.toFlatString((new Float32Array(len)).buffer);
  if ( !fs ) throw new Error("Required to be a flatstring");
  let f = new Float32Array(E.toArrayBuffer(fs));

  let i = new Int32Array(f.buffer);
  let r = E.getAddressOf(f.buffer,true);
  if ( r % 4 != 0 ) throw new Error("Required to be a aligned");

  f.fill(val);
  return {
    f: f,
    i: i,
    ref: r,
    store: s=>{
      i[0]=s;
    },
    get: ()=>f[0]
  };
};

//pass by val.
let f1 = cFloatArray(0.9);
let f2 = cFloatArray(1.2);
let fSum = cFloatArray();
fSum.store( c.fadd_byval(f1.i[0],f2.i[0]) );
print(`pass by val answer : ${fSum.get()}`);


//pass by reference.
f1 = cFloatArray(0.65);
f2 = cFloatArray(3.4);
let fOut = cFloatArray();
c.fadd_byref(f1.ref,f2.ref,fOut.ref);
print(`pass by ref answer : ${fOut.get()}`);

1. Use string literal `` in E.compiledC().
2. Keep it declared in var = global scope.
3. Flatstring + 4 byte Alignment (Thanks fanoush)

Edit: Getting the ref version to work was extremely time-consuming because of some strange crash of reading the variables directly + performing floating math on them. Yet copying the inputs off of the stack seems to solve it. Also it is necessary to write to the pointers in int form ,not float.
Edit: Fixed the crashes by forcing flat string. (cos alignment)

[espruino-discuss2]

Posted at 2023-12-22 by d3nd3-o0

This code is a simpler layout of the above by reference. It could be there is an error in the wrapper stuff in cFloat object.

This mb works? Not sure yet.

var c = E.compiledC(`
// void fadd_byref(int,int,int)

typedef union {
    float f;
    int i;
} floatint;

void fadd_byref(floatint* arg1, floatint* arg2, floatint* arg3)
{
  floatint farg1,farg2;
  farg1.i=arg1->i;
  farg2.i=arg2->i;

  floatint out;
  out.f = farg1.f + farg2.f;

  arg3->i = out.i;
}

`);

let f = new Float32Array(1);
let r = E.getAddressOf(f,true);
print(`a : ${r}`);
f[0] = 0.3;

let f1 = new Float32Array(1);
let r1 = E.getAddressOf(f1,true);
print(`a : ${r1}`);
f1[0] = 0.63;

let f2 = new Float32Array(1);
let r2 = E.getAddressOf(f2,true);
print(`a : ${r2}`);

c.fadd_byref(r,r1,r2);
print(f[0]);
print(f1[0]);
print(f2[0]);

Edit: Yes this works, there is something wrong with the cFloat wrapper function I created. Yet I dont know what it is. So good news for the float ref working I guess.

Make sure to use this:

let fs = E.toFlatString(new Float32Array(1).buffer);
if ( !fs ) throw new Error("Required to be a flatstring");
let f = new Float32Array(E.toArrayBuffer(fs));

---

Posted at 2023-12-22 by d3nd3-o0

So when I wrap stuff in functions, it crashes, when the code is not inside a function call, it works. This must be a bug...?

Nevermind, so when I save this as a file on the watch its self. It doesn't have an issue running it. Why is this? I always get confused with the prompt (save to ram) and its execution differences to file-based running.

---

Posted at 2023-12-23 by @fanoush

did not have time to test your code but in general there is garbage collector and memory defragmentation done from time to time so the address of variables in memory may change, also the E.getAddressOf(xxxx,true) may return zero if the variable data is not a flat block of memory (both arrays and strings can be fragmented).
SO you should test the address for zero and better allocate flat string via E.toFlatString (or E.toString in older versions), there is also E.toArrayBuffer to convert such flat string to array.
Passing by reference makes better sense when the array is larger (and flat) otherwise passing by value is simpler/faster.
You can use trace(variable) to see what it is made of. If it is not backed by flat string I am guessing that it can freely move in memory when garbage collection comes.
You can use peek8/16/32 to see if the address really points to expected data.

---

Posted at 2023-12-23 by d3nd3-o0

Thanks for that heads up. I have tested that the return value from E.getAddressOf is not 0, and I still have to upload to file for it to work. Mb its related to that old post I made about the interpreter uploading line by line idk, I'm trying to think how it can go wrong here, its essentially a call to E.NativeCall(offsetInBlob, funcTemplate, binaryBlob). Thinking...

Btw: Whats really annoying/strange is that if you add some random lines of code, like eg. adding print("a") into the function, it magically starts working. So such tiny irrelevant edits contribute to the crash (when running via ram)...

---

Posted at 2023-12-23 by @fanoush

Try to use only something made from flat string for E.getAddressOf. it can crash because of bad address (=stale pointer corrupting different data) or maybe it can be unaligned address, flat strings are aligned to 32 bits, other data may not be.

---

Posted at 2023-12-23 by d3nd3-o0

Good news, I got it working by forcing to Flat string!!! You solved it! Initial post updated with fixed code.

---

Posted at 2023-12-23 by d3nd3-o0

It could be that getAddressOf(var,true) does not properly do what its described it does? Because it does not return 0 when an apparent non flatstring based flat ArrayBuffer is given to it.

Mb its wrong to assume that all Int32Array etc are flat, cos if you give small length, its not?

["flatAddress","bool","(boolean) If `true` and a Flat String or Flat ArrayBuffer is supplied, return the address of the data inside it - otherwise 0. If `false` (the default) return the address of the JsVar itself."]

Unless I am missing some information about flat array buffers, perhaps it does not refer to the underlying buffer...

That is why I didn't think about flatstring being the culprit for so longer, was so happy checking the return value of E.getAddressOf(). Perhaps it only applies to strings?

E.toString() notes ->
You can still check if the returned value is a Flat string using E.getAddressOf(str, true)!=0, or can use E.toFlatString instead.

The reason it was important for this to work with RAM Upload was because that is how I test most of my development, I didn't want to make LOADS of writes to the flash just when i am debugging my code.

---

Posted at 2023-12-23 by @fanoush

It could be that getAddressOf(var,true) does not properly do what its described it does?

Maybe. It returns address if it makes sense = data is stored in one block, even normal short string can have the data in one block
https://github.com/espruino/Espruino/blob/5f6ad65e8c8bd51a2c1be17fe68ee162ce4e8c88/src/jsvar.c#L1745

This is also the case for your array

>var f=new Float32Array(1)
=new Float32Array(1)
>E.getAddressOf(f.buffer,true)
=536895915
>peek32(536895915)
=0
>f[0]=0.5
=0.5
>peek32(536895915).toString(16)
="3f000000"
>f[0]=0.25
=0.25
>peek32(536895915).toString(16)
="3e800000"
>trace(f)
#832[r1,l1] Float32Array (offs 0, len 1)  #828[r1,l0] ArrayBuffer (offs 0, len 4)    #844[r1,l0] String [1 blocks] "\0\0\x80>"
=undefined

It can be seen the float array is backed by short string that takes up single block so it is makes sense to get the address of it.

However you can see that in my case the address is not aligned (ends with 5) so reading or writing float value from/to FPU register with such pointer will crash on unaligned memory access.

see also https://stackoverflow.com/questions/63834136/how-to-avoid-unaligned-access-exceptions-with-float-on-cortex-m4

---

Posted at 2023-12-23 by d3nd3-o0

That is interesting information. So my theory about it working from storage isn't so complete. I further compared upload as "test.js" and used load("test.js") to call it. It doesnt' work there, the example where it works is when I load the file by renaming it to test.boot.js.

I checked the alignment of the address, and it seems related to what you describe. Somehow code loaded at boot.js stage is more aligned? crazy.

---

Unrelated...

>let f = new Float32Array(1);
=new Float32Array(1)
>f[0] = 0.5
=0.5
>E.getAddressOf(f.buffer,true)
=195582
>peek32(E.getAddressOf(f.buffer,true)).toString(16)
="4"

This is similar to your code above, but in the Emulator. I get weird output always "4".

---

Also this might sound stupid, but how can it store the float in 1 block(16bits). I print the float with :

>trace(f)
#4452[r1,l1] Float32Array (offs 0, len 1)  #5210[r1,l0] ArrayBuffer (offs 0, len 4)    #2358[r1,l0] String [1 blocks] "\0>\x1CF"
=undefined
>peek32(r).toString(16)
="461c3e00"
>f[0]
=9999.5

Its using 24 bits, 0x46, 0x1c,0x3e. Yet 1 block? How is that possible?
process.memory().blocksize prints 14

---

E.getSizeOf() for the flat string was 4. ( One block for the Float32Array view, One block for the ArrayBuffer view, 2 block for the backing flatstring data )

for the normal string backed example, it was 3. ( One block for the Flaot32Array view, One block for the ArrayBuffer view, 1 block for the backing string data ).

Is my understanding correct of the above?

---

Posted at 2023-12-24 by d3nd3-o0

let r = E.getAddressOf(f.buffer,true);
print(`addr aligned 4: ${r % 4}`);

I can now confirm that alignment is the thing that affect all cases. I was just lucky before.

so

 Somehow code loaded at boot.js stage is more aligned? crazy.
``` is false and just based on luck.

Still haven't figured out the size confusion yet though.