You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Primary reason is to extend capabilities of proxies with business logic. Ideally running on different proxies and providers. Run in Nginx, istio, fastly, and works exactly the same on the platforms. Started as part of istio to allow code at runtime with security, but that’s not the only reason. Primary reason is abstraction across different providers.
55
+
56
+
Have support for logging, metrics, network filters, HTTP and gPRC callouts, can do auth check, have key-value store (non-durable, only used to coordinate e.g. telemetry, not meant as DB), background services and timers, FFI.
57
+
58
+
Current adoption pretty good, supported in Envoy and Istio Proxy. Alibaba added support to MOSN. Created embeddable Go host library. Have SDKs for AS, C++, Go, Rust. ABI spec is kind of lagging behind, so they are all using last year’s version which isn’t reflected in today’s presentation.
59
+
60
+
Compared to serverless, we are executing in-process as part of request/response flow, trying to minimize memory overhead, running multiple requests in same VM. Handle as much as possible in the optimized host e.g. TLS processing
61
+
62
+
Led to a design that’s using reactor, each entry point is an event. Minimize number of calls, e.g. include number and size of headers so you don’t need to do extra call. Transforming pre-opened connections.
63
+
64
+
[walk through Example slide]
65
+
66
+
Changes since last update: trying to align with WASI. Prev memory allocations were happening on host side, e.g. get host headers would call plugin and would then copy to own memory. Context_id is passed as arg, prev this was implied and not part of ABI. Added support to have multiple instances of timers, queues, etc.
67
+
68
+
Wasi-context. We run multiple plugins as part of the same wasm instance if there are multiple plugins compiled as part of the same wasm module. Don’t expect this to land in WASI in this form.
69
+
70
+
Wasi-common. Retrieve data, get/set property. I put them in common because they are shared across http proxy and bytestream, etc. Perhaps should move towards something more generic.
71
+
72
+
Wasi-bytestream. TCP, UDP stream, QUIC.
73
+
74
+
Wasi-http or wasi-http-proxy.
75
+
76
+
wasi-http-client .
77
+
78
+
**Sam Clegg:** So if WASI developed async, you could use that.
79
+
80
+
**Dan Gohman:** The Wasm stack switching proposal is the place where that async work is happening.
81
+
82
+
**Sam Clegg:** I wonder if this changes—instead of having callout ID, you would use the stack.
83
+
84
+
Wasi-grpc. Similar set of APIs. bidirectional open http2 request. Wondering whether this should be separate component, or whether should be layered on top of http. Left as dedicated abi because of small details.
85
+
86
+
**Dan Gohman:** are the on functions calling _____
87
+
88
+
Now for straightforward components: wasi-log. Print_f or std_out. Level so we don’t waste time for hosts that dont’ care about certain levels.
89
+
90
+
**Dan Gohman:** Minimizing calls because APIs are expensive?
91
+
92
+
**Piotr Sikora:** Again, this is running in process. Because of that, we’re looking at 300-400 nanosecond. Maybe premature optimization.
93
+
94
+
Wasi-kvstore. Not sure whether this is something that should be standardized in WASI. Can create new or open existing KV store. Make sure it’s atomic.
95
+
96
+
wasi-queue . similar to prev. We just use it to shift stats from each of the workers to aggregated.
97
+
98
+
Wasi-timer. Used to periodically call into the plugins so it can perform some background task.
99
+
100
+
Wasi-metrics. Maintained by the host.
101
+
102
+
Wasi-ffi. Support for foreign functions.
103
+
104
+
Next steps: alignment with WASI. Write witx, need to migrate to handles, need to see whether can leverage wasi-io.
105
+
106
+
New features: load balancing, tighter integration with cache control, ability to follow redirects because right now ABI makes it ugly, support for starttls because some start with plain text and then start tls in middle of connection, possibly UDP datagrams.
107
+
108
+
Any questions, comments?
109
+
110
+
**Sam Clegg:** What VM options do you provide?
111
+
112
+
**Piotr Sikora:** We’re running on V8, but envoy supports both Wasmtime and WAVM, and PR in progress for WAMR.
113
+
114
+
**Sam Clegg:** When you have the on callbacks, those are exports from the module, and the others are imports. Is this the first time we’re talking about exports of a wasm module.
115
+
116
+
**Dan Gohman:** Yes, currently tooling gaps
117
+
118
+
**Sam Clegg:** Once we have funcrefs, would it be better to use those than IDs.
119
+
120
+
**Luke Wagner:** One idea in other contexts, is distinction between interface and profile. It sounds like we’re talking about profiles to describe proxy filter.
121
+
122
+
**Piotr Sikora:** Definitely
123
+
124
+
**Luke Wagner:** There might be perf improvement if they aren’t callbacks
125
+
126
+
**Sam Clegg:** Funcrefs would require some sort of bind
127
+
128
+
**Luke Wagner:** IT wants to statically know the functions, so that works better when they are static exports.
129
+
130
+
**Dan Gohman:** At some level core wasm doesn’t like mutually recursive instances. This seems to have APIs that would want that, eg log interface. Have any ideas to resolve?
131
+
132
+
**Piotr Sikora:** One is host and one is module.
133
+
134
+
**Dan Gohman:** Thinking in terms of virtualization.
135
+
136
+
**Sam Clegg:** You would need mutual instantiation in order to satisfy.
137
+
138
+
**Piotr Sikora:** Those would be statically linked at compile time. Right now we solve this through SDK, so delivered as part of the same WASI module.
139
+
140
+
**Dan Gohman:** Goal is to make all of these virtualizable. How would we do that in the case of wasm module implementation. To be sure, this is a loaded question.
141
+
142
+
**Piotr Sikora:** Log doesn’t depend on anything.
143
+
144
+
**Dan Gohman:** You do have the program depending on the log module, the log module
145
+
146
+
**Luke Wagner:** You may need to split this into two modules. Containment relationship
147
+
148
+
**Sam Clegg:** Presumably only one log module, so there’s a multiplexing here, is that right? Could have 1000 plugins and there’s only 1 log instance.
149
+
150
+
**Luke Wagner:** So you’d have 1 of the inner things, and N of the outer things. Might be an interesting question of whether this could just be a function call.
151
+
152
+
**Dan Gohman:** Maybe the log function could return current log level.
153
+
154
+
**Sam Clegg:** Highlights some sort of design space we need to think about
155
+
156
+
**Andrew Brown:** essence of the problem here. IIn order to work, every module needs to register somewhere and there’s no good place.
157
+
158
+
**Dan Gohman:** Good question. Does everyone get their own log instance. Can you run multiple plugins in same store.
159
+
160
+
**Piotr Sikora:** Yes. share an instance
161
+
162
+
**Andrew Brown:** The instance we’re talking about is a host instance for loggin.
163
+
164
+
**Dan Gohman:** Sort of both. Should de-emphasize the instance
165
+
166
+
**Sam Clegg:** Something that changed over the last year or so. We now think more about virtualizability. Never made a formal announcement, but that’s the way that’s it’s going.
167
+
168
+
**Piotr Sikora:** Now I understand the issue. So I guess the question is do we see these components as something that we want to include.
169
+
170
+
**Dan Gohman:** Certainly a lot of functionality that’s common to a lot of things. I think question is with the callback interface, will that be generalized enough, or is this more of a specialized use case? People doing HTTP that are using no callbacks. Do we need separate APIs for that?
171
+
172
+
**Piotr Sikora:** Good question, prefer same APIs.
173
+
174
+
**Dan Gohman:** The callback model requires fundamentally different model. E.g. C++ doesn’t have a way of getting called ___
175
+
176
+
**Piotr Sikora:** No main function
177
+
178
+
**Dan Gohman:** That’s a pretty fundamental difference between models. That’s why we’re working on difference between command and reactor. This is an open question.
179
+
180
+
**Sam Clegg:** I guess you’re saying that anything that has these callback patterns is by definition a reactor.
181
+
182
+
**Andrew Brown:** Kind of changes requirements of WASI APIs, because if you’re one of these reactors that can look at HTTP headers, you have to export that function, which is a WASI requirement that we didn’t have before.
183
+
184
+
**Piotr Sikora:** But that’s the whole point of WASI reactors
185
+
186
+
**Dan Gohman:** What I would like to see then, could someone mock up end-to-end command style program that has callbacks
187
+
188
+
**Piotr Sikora:** Those are by definition mutually exclusive
189
+
190
+
**Dan Gohman:** That gets to my question. Do we need to have separate APIs. If we’re going to have one, then I’d like to see how it works.
191
+
192
+
**Andrew Brown:** It would be nice to just have one.
193
+
194
+
**Piotr Sikora:** Blocking
195
+
196
+
**Dan Gohman:** In regard to blocking, WASI is deferring to Wasm’s async model. Not practical in short term, so I think there’s space for doing something now with a custom async model, but that’s the current space that we’re in. But feels like it has bearing on whether it bifurcates the API.
197
+
198
+
**Andrew Brown:** Bringing this up brings up a great question. On other note, does this support client?
199
+
200
+
**Piotr Sikora:** Any idea when async may be widely available.
201
+
202
+
**Luke Wagner:** Like idea someone else brought up, maybe doesn’t have to wait.
203
+
204
+
**Lin Clark**: Next steps?
205
+
206
+
**Andrew Brown**: Seems like a convo about callbacks
207
+
208
+
**Luke Wagner:** Should have more to say about async after next Tuesday’s CG meeting
209
+
210
+
**Lin Clark:** Ok, I’ll follow up with those involved in the discussion to see whether this is offline chat or should be covered in next meeting.
0 commit comments