Embedding a Spin Runtime

[ignatz]

First and foremost I wanted to say "Hi 👋 ", and what an awesome project 🙏

...back to topic: I was looking around for possible alternatives for the Deno v8 runtime in my project: TrailBase. I was already play around with wasmtime and possible concurrency models, build/dev flows for user-code, ... and then stumbled upon spin. I'm so excited to see that you folks have pretty much built what I had in mind - and certainly much better.

I understand that your main integrations are probably your CLI (and maybe a separate shared runtime for your cloud product). Yet, I was wondering how you folks would feel about someone else integrating your runtime or parts thereof? I wouldn't wanna create extra work for you or churn on either side. Maybe that's something you've already though about?

Either way ❤️ spin

[lann]

Spin is definitely designed to allow embedding; there is even another open source example: https://github.com/spinframework/containerd-shim-spin (mostly implemented here)

^ That probably represents the highest level of abstraction that you might want to embed; it is possible to integrate at other levels with varying degrees of effort.

[lann]

It is possible with WASIp2's wasi-io but in practice it's quite difficult to implement correctly if you want to be able to compose async operations between different interfaces. We also deferred in part because we were a little too optimistic about the timeline for p3 async... 😬

[ignatz]

Thanks, that's the conversation I was hoping for. I'm not the sharpest knife in the drawer, but I've looked at your code and would like to understand the current state and the motivation behind: technical and non-technical.

It is possible with WASIp2's wasi-io but in practice it's quite difficult to implement correctly if you want to be able to compose async operations between different interfaces

Right. I hoped spin had solved it and wanted to tap into it.

I love that all the WASIs are explicit and simple, but if implementing them async was a pain (maybe I'm drawing the wrong conclusions), have you considered implementing a more general async fabric (IPC system) and multiplex all the PG, MySql, Redis, ... over it?

and that's why I asked this. I'm happy to just be a sounding board to reflect on past decisions and trade-offs. Arguably, even with WASIp3 there may be value-add from a generic IPC system: API vs ABI stability, middleware: tracing, auth, ... 🤷‍♀️

Let me swap my engineering hat for my product one...My main goal is to offer the best WASM runtime to my users.

I am evaluating spin, because:

• I was hoping I'd get a working solution for async out of the box.
• I understand that there's value by standardizing on a consistent environment on a technical level, for users targeting the runtime and us growing the community together.
• To make sure we're all on the same page. Me adopting spin is risk for you and for me. I don't want to change runtimes every other Tuesday. It's the most painful surgery you can do to your users.

Don't get me wrong, not trying to sound dramatic or important, realistically I have 5 users 🙃 , hopefully more in the future. I just want to provide the best possible runtime and I'm happy to solve hard problems - that's where the most fun and the most value for users is to be had 😅 🤷‍♀️

[lann]

WRT async there are (at least) two places you might care about async:

• Host: Wasmtime implements async with a userspace fibers implementation that lets the host treat synchronous guest code as if it was async. If a guest calls into an async host-implemented import and that import "yields" (returns Pending), execution will jump back to the host entrypoint to get back to the async executor. Even busy guest execution will periodically yield to the host thanks to "epoch interruption" that gets injected into the guest. Lots of detail on this topic here.

• Guest: WASIp2 implements posix-poll-like non-blocking IO in the wasi-io interface. The Spin Rust SDK uses this interface to implement its async executor. This is where things get hairy: any code that wants to do composable async (e.g. wait on concurrent futures) with Spin needs to integrate tightly with that executor; see for example this (currently very experimental) grpc client. There has been a more recent effort to standardize the Rust WASIp2 ecosystem with an async executor in the wstd crate but I believe migrating to that would be a breaking change for the SDK so we'll probably just wait and make the jump to WASIp3 instead. Another major disadvantage of the WASIp2 approach is that there is no standard way for different components to interoperate async-ly; you could try to define your own ABI for this but making it work cross-language would be "non-trivial" and one of the challenges behind a lot of the big ideas in the WASIp3 async design.

In that context, (where) do you see an IPC system fitting in?

[ignatz]

Good stuff 👍

You'll typically care about both. In the Spin case, you have an HTTP or MQTT request come in, you do a bunch of concurrent database calls and then compute the response, i.e. you need:

• The concurrency in your WASM guest (your (2))
• And the host-implemented database handlers yielding back (1), potentially yielding all the way back to handle another HTTP/MQTT request in another "instance".
• And having "epoch interruption"/preemption is just the icing on the cake when running many untrusted components in a shared environment

Very excited you folks have been looking at grpc. Was the intention to then multiplex async postgres, mysql, redis over it and achieve async? Or it this more to directly receive work from gRPC similar to HTTP and MQTT? (on a tangent, doesn't grpc use HTTP as the wire transport? If HTTP already supports bidirectional streaming, you may be able to re-use the integration 🤷‍♀️ )

There has been a more recent effort to standardize the Rust WASIp2 ecosystem with an async executor in the wstd crate but I believe migrating to that would be a breaking change for the SDK so we'll probably just wait and make the jump to WASIp3 instead.

Very interesting, I'll take a look. That's why I brought up "mio", wishfully hoping that it may alleviate some of the generality and correctness challenges round bringing async into WASM

Another major disadvantage of the WASIp2 approach is that there is no standard way for different components to interoperate async-ly; you could try to define your own ABI for this but making it work cross-language would be "non-trivial"...

Talking myself through it, what you're saying is that spin's executor, wstd, mio are all Rust-specific solutions, and for other guest environments one would have to solve this problem all over again?

This is were my previous statement of optimistic ignorance comes in:

Anyway, this is all Rust guess specific, it might be easier for other languages which bring in their own event-loop... will see

I haven't looked into it but I would have hoped that some languages already solve this, e.g. if I write async JS or Dart executed on an event-loop, the compiled-in event-loop should ideally yield back when drained. I may be wrong and for other languages, like go, that's certainly not the case.

When I looked at the timelines of WASIp3, I felt it was already lagging (I'm all in favor of taking the time to release something solid). But that's only the spec. Then both the runtime and the guest-compilers will have to adopt it. I figured it would probably be years before one could support a broad set of guest languages on WASIp3.

That's when I started looking for WASIp2-based options.

With Spin explicitly supporting Rust, JS, Python and Go (and many more such as C#), do you foresee a more timely timeline?

[lann]

Was the intention to then multiplex async postgres, mysql, redis over it and achieve async? Or it this more to directly receive work from gRPC similar to HTTP and MQTT?

The linked repo implements just a guest gRPC client on top of WASIp2's wasi-http interface. It should be possible to implement a server similarly.

doesn't grpc use HTTP as the wire transport?

The (best-supported) gRPC transport does indeed use HTTP/2 but it depends on some HTTP features which aren't otherwise commonly used - like bidirectional streaming and trailers - which has lead to some subtle spec and implementation problems (ref WebAssembly/wasi-http#128).

what you're saying is that spin's executor, wdtd, mio are all Rust-specific solutions, and for other guest environments one would have to solve this problem all over again?

Yes, correct. Also: unlike WASIp2's polling model, WASIp3's async features have been designed to integrate more tightly with many different languages' concurrency models.

Aside: there is an active PR adding WASIp2 guest support to mio: tokio-rs/mio#1836

Then both the runtime and the guest-compilers will have to adopt it.

This is in full swing.

With Spin explicitly supporting Rust, JS, Python and Go (and many more such as C#), do you foresee a more timely timeline?

I would expect Rust, JS, and maybe Python to have the first WASIp3 implementations; those are the languages with toolchains maintained by the same people/organizations who are implementing Component Model async today. Other languages will depend on resourcing that I don't think anyone can predict yet and indeed I would expect the long tail of language support to take years.

The good news is that WASIp3 guests will still be able to take advantage of async execution even with existing WASIp2 implementations, e.g. a p3 guest should be able to call into two different p2 component instances concurrently.