WASI P3 HTTP middleware

[itowlson]

As part of WASI P3 work, we want to explore implementing HTTP middleware using component composition. The purpose is to allow application developers to use off-the-shelf or custom components in a pipeline to add or customise behaviour without needing to bake it into the core application component.

For example, the following site requires a CORS check and authentication, and enriches the request with geolocation information. None of these are core application concerns, and they could ideally be pulled off the shelf rather than custom coded.

Manifest

Spin cannot naturally do this in the current dependency model, because dependencies model only imports of the application component. (It can do one level of middleware unnaturally, by making the main component the middleware and having it depend on the application, but 1. that currently allows only one level and 2. that's unnatural, ew.)

So to support middleware we have to decide:

1. Do we try to wedge it into the current dependencies section?
2. If not, where do we want to put it?

The trouble with using the dependencies section is that it's a map, whereas the middleware pipeline is a sequence. All middleware components use the same interface, and the order of plugging them together is significant. I'm sure some creative soul has an idea for how to overcome this mismatch, but it defeated me. Ideas welcome!

If we can't use the dependencies section, where do we put it? The two options appear to be the trigger or the component:

# Trigger?
[[trigger.http]]
route = "/..."
component = "cat-videos"
pipeline = ["cors", "auth", "geo"]

# Or alternative trigger?
[[trigger.http]]
route = "/..."
component = ["cors", "auth", "geo", "cat-videos"]

# Or component?
[component.cat-videos]
source = "catvids.wasm"
pipeline = ["cors", "auth", "geo"]

I'm attracted to putting it on the trigger, because:

1. It's HTTP specific - it can be part of the HTTP trigger schema rather than the common component schema
2. It frames middleware as part of the serving pipeline rather than as inherent to the component

(okay mostly 1 I admit)

However, all our composition and Wasm loading infrastructure is component centric. During prototyping (https://github.com/itowlson/spin/tree/http-middleware) I found it really hard to see how the relevant subsystems could reach into trigger-specific config this way - for example how would spin registry push know that this field on the HTTP trigger required binaries to be resolved and packaged?

So for prototyping I've been putting it on the component. But I'd value discussion and feedback on this.

Interface

Middleware needs to be able to pass a request on to the next entry in the chain. In principle this could be done by composing each component's wasi:http/handler import onto the next one's export, then the middleware code calling handle on the incoming request to pass it on. But many middleware items will want to do their own HTTP stuff, e.g. authenticating using OIDC or whatever.

WebAssembly/wasi-http#167 proposes adding an origin interface to the WASI-HTTP proxy world. This would be an additional interface identical to handler, but with the semantics of "pass it on along the chain" rather than "fling it across the network." My prototype called this next and had it in a Spin-specific package, but the concept is the same. If origin lands soon enough then we can use that, otherwise we use our own and adopt the spec one when it arrives.

Unlike dependencies, Spin (or the HTTP trigger, or whatever) needs special knowledge of the middleware origin/next interface, because it's not amenable to plug composition (and we don't want devs to have to specify the mapping on every pipeline entry!).

Permissions

Currently, dependencies get the choice of "main component permissions or nothing," governed by the dependencies_inherit_configuration flag. We should not expect fine-grained permissions in the middleware timeframe, so middleware will likely inherit the same permissions behaviour, governed by the same flag. This fits nicely with "middleware is an attribute of the component" but not so nicely with it being on the trigger (because if the component needs to allow-list the OIDC provider then it is not middleware-agnostic after all).

This is a bit of a brain dump and I am sure other brains have thought about this a lot more than mine so please weigh in.

[lann]

spin/crates/manifest/src/schema/v2.rs

Line 133 in e4a1600

/// Reserved for future use.

🙂

I think my theory was something like:

[[trigger.http]]
component = "cat-video"
components.middleware = ["cors", "auth", "geo"]

[itowlson]

The worry with that is still that the meaning of the middleware key and the stuff it's meant to do is specific to the HTTP trigger. But good catch as it does provide a home for "hey loader you're gonna need these things for Wasming."

I will have an investigate of this - thanks!

[lann]

Another option we have is to implement middleware more like service chaining with the "linking" implemented in the host rather than composition. That has its own tradeoffs but seems better than the watered-down functionality of dependencies.

This would also give us a solution to the "origin" problem, where we could use another magic hostname like e.g. next.middleware.spin.alt or what have you.

[radu-matei]

Another option we have is to implement middleware more like service chaining with the "linking" implemented in the host

@lann does that mean you would have to explicitly call the middleware in your code handler over HTTP?

[radu-matei]

The worry with that is still that the meaning of the middleware key and the stuff it's meant to do is specific to the HTTP trigger.

@itowlson -- I am not convinced I follow the above given the very first sentence of this issue:

As part of WASI P3 work, we want to explore implementing HTTP middleware using component composition.

Yes, I imagine we could have "middleware" for non-HTTP components as well in the future.
But I worry that a) I don't see the feature requests for that (not saying it doesn't exist, just that overwhelmingly, we've been talking HTTP specific middleware), and b) in my opinion it would be a loss delaying HTTP middleware longer given a).

What do you think?

[itowlson]

@radu-matei The concern is not non-HTTP middleware. Such a thing is totally out of scope unless someone asks for it.

The concern is that it how the composition subsystem is to understand the trigger-specific semantics of the middleware sub-key. I.e. I think this solves the "how does spin registry push know that these are components that need to be included as binaries in the OCI artifact" but I haven't yet convinced myself it solves the "the thing that understands the trigger-specific key has access to the right things at the right time" problem. It's definitely worth exploring though: the HTTP trigger would presumably have specific code, I just need to assure myself that it can be made to line up with the composition stuff without putting specific knowledge of the middleware sub-key into agnostic code.

[itowlson]

A specific concern is precomposition in OCI artifacts. By default, Spin performs all composition at registry push time (for compatibility with hosts that don't do composition). But how does spin registry push know how to interpret the components.middleware table (or any other trigger-config equivalent), when that table and its semantics are specific to the HTTP trigger?

(Lann's suggestion of service-chaining-a-like would bypass this although possibly at the expense of borking hosts that don't support service chaining. Not sure, we can certainly look into this too.)

(Also, to be clear, as an interim fix we could give spin registry push special knowledge of HTTP trigger keys: it would kinda be kicking the can down the road, but it's quite possible we'd kick it far enough that we would never need to think about it ever again.)

[itowlson]

Okay, I started looking at doing it in the HTTP trigger and there's another little nasty: we have an assumption that we can look up InstancePres by component ID. Per-trigger middleware means we can no longer rely on that assumption: two triggers might point to the same component ID, but have different middleware resulting in different compositions. I'll investigate how easy it is to modify the current assumption: I think the current mapping is RouteMatch -> component ID -> InstancePre in which case we might be able to get away with switching to something richer in the middle.