Deriving typed array ids from RTTI

[dcodeIO]

This is a little idea to get rid of the requirement to specify an additional entry file.

Instead of exporting the IDs, it is also possible to obtain these from RTTI, which is located in static memory at exports.__rtti_base. RTTI is somewhat limited, but as-bind's currently supported types fit it well.

// ╒═══════════════════ Typeinfo interpretation ═══════════════════╕
//    3                   2                   1
//  1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0  bits
// ├─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┴─┤ ◄─ __rtti_base
// │                             count                             │
// ╞═══════════════════════════════════════════════════════════════╡ ┐
// │                      Typeinfo#flags [id=0]                    │ id < count
// ├ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┤
// │                      Typeinfo#base  [id=0]                    │
// ├───────────────────────────────────────────────────────────────┤
// │                              ...                              │

All the ArrayBufferViews have a special flag TypeinfoFlags.ARRAYBUFFERVIEW. Their integer or floating point type can be derived from TypeinfoFlags.VALUE_ALIGN_X in combination with TypeinfoFlags.VALUE_SIGNED and TypeinfoFlags.VALUE_FLOAT.

However, doing so imposes the limitation that if no Int16Array for example is used within the module, it won't be present in RTTI. Instantiating one should not be useful anyway in this case as there is nothing accepting an Int16Array on the Wasm side.

This would also work with Array<T>. These use TypeinfoFlags.ARRAYBUFFERVIEW | TypeinfoFlags.ARRAY and have the same flags otherwise. However, special care must be taken if T is managed, which is indicated by TypeinfoFlags.VALUE_MANAGED.

The other supported types have fixed IDs btw and are always present, so one doesn't need an export for these:

• ArrayBuffer: 0
• String: 1
• ArrayBufferView: 2 (this can be used to find typed arrays, which use this ID as a base)

[dcodeIO]

Attempted to make a PR for it, but it seems the change has unforeseen consequences on other parts of as-bind that I do not yet understand. In case it's useful, here's a little snippet of the RTTI part :)

[torch2424]

Oops! I just saw this! My apologies @dcodeIO ! 🙃

So thanks for explaining this, and how it would work! So the thing about the entry file, is I eventually had plans for allowing users to pass closures, and things like creating their own classes and attempting to pass those back and forth. Thus we'd probably still need the entry file for those use cases? 🤔

And I'll take a look at #21 tommorrow, as it is currently 1am for me haha! 😂 And I'll keep this in a pinned tab so I don't forget haha! 😄

[dcodeIO]

Yeah, I guess we'll need some sort of code generation on the Wasm side eventually, though I imagine that this will then involve a transform of sorts to identify the stuff that must be exposed, essentially reducing what a user has to specify to --transform as-bind, implicitly adding a (generated) entry file. Not sure if the ability to add an entry (not just a library) file is in place, but if it isn't, that should be easy to add.

[torch2424]

@dcodeIO So in my head (heaven't actually tried it yet), I was hoping to have some helpers in the entry file, to help facilitate these use cases. Since, currently I think the idof with a custom class should work. So as long as I had a helper that exposed the class Id or something, I though that would work? But again, haven't tried it, and it's a half-formed idea 😂

That being said, maybe it is time to start working on code generation now. Since it's the eventual way to go anyways? 🤔

[dcodeIO]

When it comes to classes stuff becomes a bit more complex, yeah. The recommended way as of today isn't exactly ideal, which is to export the classes one needs to instantiate / work with externally from the entry file. There's a little undocumented (and potentially very useful) feature atm that exports the class id as the name of the class for instance, so in

export class MyClass {
  constructor() {}
  foo(): void {}
}

one gets the exports

• MyClass = the id
• MyClass#constructor
• MyClass#foo

but one downside is that one doesn't have more granular control about what is exported. For example, one might not need MyClass#foo externally, or the constructor. One solution that might fall short in some cases is to only make those members module exports that are explicitly declared public. But overall a convention like this has the potential to give as-bind everything it needs to make instances of classes.

[torch2424]

Ah,

So creating classes externally was something I was thinking of doing later, had no clue that was already semi-supported, thank you! 😄 👍

I was thinking of doing something like this:

Let's just say (for wasmboy for an example, not something I want to do), I want to return a "Save State" object from the module. For example,

AssemblyScript:

class SaveState {
  gameboyMemory: Uint8Array;

  constructor(gameboyMemory): void {
    this.gameboyMemory = gameboyMemory;
  }
}

export function getSaveState(): SaveState {
  let gameboyMemory = new Uint8Array(100);
  gameboyMemory.fill(24);
  let saveState = new SaveSatate();
  return saveState;
}

Then in my JavaScript I could in my as-bind instantiated module:

Javascript

const mySaveState = myWasmModule.exports.getSaveState();
console.log(mySaveState) // { gameboyMemory: [24, 24, ...]}

And if I do the inverse, and pass an object in, I could allocate it, fill the memory correctly.

[dcodeIO]

Regarding the example, the equivalent of what one could do currently with just the loader is:

export class SaveState { // note the export
  gameboyMemory: Uint8Array;

  constructor(gameboyMemory): void {
    this.gameboyMemory = gameboyMemory;
  }
}

export function getSaveState(): SaveState {
  let gameboyMemory = new Uint8Array(100);
  gameboyMemory.fill(24);
  let saveState = new SaveSatate();
  return saveState;
}

const mySaveStatePtr = myWasmModule.exports.getSaveState();
const mySaveState = myWasmModule.exports.SaveState.wrap(mySaveStatePtr);
const gameboyMemoryPtr = mySaveState.gameboyMemory;
const gameboyMemory = myWasmModule.exports.__getArrayView(gameboyMemoryPtr);
console.log(gameboyMemory);
myWasmModule.exports.__release(gameboyMemoryPtr);
myWasmModule.exports.__release(mySaveStatePtr);

Not sure how helpful this is, but yeah, somewhat supported already, just not really convenient :). Mostly because the properties of classes made by the loader still have pointer fields, and don't translate these to objects implicitly but require either .wrap for classes or the runtime helpers when dealing with strings or arrays.

[torch2424]

Oh! @dcodeIO thank you so much! 😄 When I get the time, I'll totally go and implement this! 😄 👍

Yeah the only thing that I'd really have to do is document that the class needs to be exported, or else it won't work.

That being said, going back to the original issue. If this won't be a problem without an entry file. I'll probably just release the next major / minor without the entry file. And I think we'd be good to go? 😄

[aminya]

👍 for this feature. From this example, I can see that it adds support for classes!
#20 (comment)