Store ABI type information as a custom section in .masp packages

[marijamijailovic]

Feature description

Right now the package manifest only has flattened types. For example, a function that takes two Points(custom struct Point) and return Point just shows params: [Felt, Felt, Felt, Felt], results: [Felt, Felt]. You can't tell from the manifest that those felts are part of an Point.
This info exists during compilation in the compiler frontend (compiler/frontend/wasm/src/). Parameter names, struct fields, and type names are all there in the canonical ABI sections — they just get thrown away during flattening.

The idea is to get this info before flattening and put it in a custom section in sections: Vec<Section>.

Example of what the section would contain:

• Type definitions:

  point = record { x: felt, y: felt }
  account_id_data = record { prefix: felt, suffix: felt }

• Function signatures:

  point-add(a: point, b: point) -> point
  account-to-word(id: account_id_data) -> word

Why is this feature needed?

With this miden call can accept an formatted struct Point as input, and show formatted output, instead of making users type 4 felts, and see 2 felts as output.

[bitwalker]

The project assembly PR 0xMiden/miden-vm#2877 will start emitting packages that contain the various debug info sections with type information used for debugging. That may be worth experimenting with, since I think this use case has some overlap a bit with the desire to display values in source language terms during debugging.

More generally though, the signatures in the manifest are not intended to be representative of the source language signature, but instead the signature that callers must agree on at the call boundary. For Rust-compiled functions, the actual source-level function is not directly called, instead it is called through bindings that speak the Wasm Canonical ABI. The signature of that function is represented in the WIT manifest encoded in the package (we aren't yet emitting this section, but plan to very soon). It is the WIT signature for a method that tells you what to expect at the call boundary in terms of high-level types.

There are several things going on here:

1. You have some tool, e.g. miden call which presumably needs to take arguments provided on the command line, validate them against some schema, and then encode them in terms of the Wasm Canonical ABI in order to actually call a method on a contract with those arguments.
2. You need to know the Canonical ABI signature to be able to do this, which must be extracted from WIT - either in textual form, or by extracting the textual form from an assembled package.
3. Lastly, you need to be able to correctly encode values provided via the command line to felts on the operand stack so you can invoke the callee correctly.
4. Conversely, you need to be able decode the results of the call in order to display them to the user.

I think the easiest way to facilitate this might be to compile a Rust program on-the-fly using the Miden compiler, which binds against the desired callee, and handles all of the details of calling and printing values in Rust terms for you (we will be adding println-like functionality soon that could be used for this).

That's a lot of extra work under the covers (i.e. compiling a package which you then execute with the client), but I think ultimately ends up being more robust and flexible of a solution.

/cc @greenhat @djolertrk

[marijamijailovic]

Thanks for the detailed breakdown! I'm still going through PR #2877. The on-the-fly Rust compilation approach is interesting. I'll explore what that would look like on our end and follow up once I have a clearer picture.