Skip to content

Commit b221fca

Browse files
dicejalexcrichton
dicej
and
alexcrichton
authored
update component-model-async plumbing (#11123)
* [DO NOT MERGE] update `component-model-async` plumbing This pulls in the latest Component Model async ABI code from the `wasip3-prototyping` repo, including various API refactors and spec updates. This includes all the changes to the `wasmtime` crate from `wasip3-prototyping` _except_ that the `concurrent` submodule and child submodules contain only non-functional stubs. For that reason, and the fact that e.g. `Func::call_async` is now implemented in terms of `Func::call_concurrent`, most of the component model tests are failing. This commit is not meant to be merged as-is; a follow-up commit (to be PR'd separately) will contain the real `concurrent` implementation, at which point the tests will pass again. I'm splitting these into separate PRs to make review easier. Signed-off-by: Joel Dice <[email protected]> * Undo wit-bindgen changes No longer necessary after other refactors * Move back to crates.io-based wit-bindgen * Undo upgrade of http-body-util (deferred for future PR) * Add back in arbitrary use of async Looks like it may have been lost by accident * Make imports more conventional for Wasmtime * Some minor changes * Privatize a component field * Cut down a bit on #[cfg] * Undo a no-longer-necessary `pub` * add doc comments for `{Future,Stream,ErrorContext}Any` Signed-off-by: Joel Dice <[email protected]> * rename `concurrent` stub module to `concurrent_disabled` ...and avoid panicking in the stubs. Signed-off-by: Joel Dice <[email protected]> * fix test regression Signed-off-by: Joel Dice <[email protected]> * revert `call_async` and `post_return_impl` changes These will need to wait until the `component-model-async` feature is fully implemented. Signed-off-by: Joel Dice <[email protected]> * remove unused struct Signed-off-by: Joel Dice <[email protected]> * add `Options::callback` field This isn't used yet, but will be used when the real `component-model-async` implementation is merged. Signed-off-by: Joel Dice <[email protected]> * Remove no-longer-needed feature * Trim reexports from Wasmtime Some of these are no longer needed or can be avoided with small changes. Some deps are likely needed in the next commit but they'll be best added there. * Update test expectations * More trimming of Cargo.toml * Defer `*Buffer` traits to next PR Not needed for this PR I believe. * Use conventional Wasmtime imports + remove dummy_waker * Reduce duplication in `*_disabled` * Remove some unncessary bounds * Remove some `for<'a>` bounds where unnecessary * Remove another bound * Defer more functions to the next PR `drop_fibers` is different in the next PR, so defer it to then. * Remove some reexports no longer necessary Bindings generation changed awhile back so these aren't needed, defer the implementations to the next PR. * Remove unnecessary drop This was already moved to `run_manual_drop_routines` * Defer a `pub(crate)` to a future PR * Expand comments in traphandlers * Defer some types to the next PR * Update linker documentation * Add `Send`/`Sync` bounds to `ComponentType` This commit is extracted from from review of #11123 and #11127. While not literally present in those PRs it's my own personal conclusion that it's best to just go ahead and add these bounds at the "base" of the component trait hierarchy. The current implementation in #11123 adds bounds in many locations and this would remove the need to add bounds everywhere and instead have everything inherited through the `Lift` and `Lower` traits. This raises the question of: why? The main conclusion that I've reached leading to this change is that Wasmtime currently will store `R`, a return value, on the stack during the lowering process back into linear memory. This might involve allocation, however, meaning that wasm can be invoked and a context switch could happen. For Wasmtime's `unsafe impl` of `Send` and `Sync` on fibers to be sound it requires that this stack-local variable is also `Send` and `Sync` as it's an entirely user-provided type. Thus I've concluded that for results it's always required for these to be both `Send` and `Sync` (or at the very least, `Send`). Given that I've gone ahead and updated to require both `Send` and `Sync` for both params and results. This is not expected to actually have any impact in practice since all primitives are already `Send`/`Sync` (minus `Rc` impls all removed here) and all `bindgen!`-generated types are compositions of `Send`/`Sync` primitives meaning that they're also `Send` and `Sync`. * Remove some now-unnecessary bounds * Fix build after #11160 * Remove some now-unnecessary duplicate bounds * Uncomment test that now works * Undo accidental doc wrap * Clarify comment on `Value` types Don't leave `TODO` in public-facing documentation ideally * Actually resolve the conflict (forgot to commit) * Avoid returning boxed futures in APIs * Defer making constructors more public to a future PR * Make a method name more conventional * Refactor `linear_lift_into_from_memory` * Drop the `max_count` parameter in favor of slicing the `WasmList` itself. Avoids situations such as what happens if `max_count` is larger than the length of the list. * Don't have the default implementation collect to a vector and then push all that onto a different vector. Instead push each item individually through `extend`. * De-indent a block of code added * Remove unsafety from `prepare_call` Mostly move the parameters themselves to the closure to avoid raw pointers/drop/etc. This will have the consequence of in the future `call_async` is going to now require `Params: 'static` but that seems more-or-less inevitable at this point. * Go back to returning box, alas. * Apply same treatment to lift function Make it a closure and reduce some levels of indirection of the various functions in play. * Refactor `lower_params` to require less context. Relax the bounds on the closure specified since it's immediately called and then additionally take out parameters/captures that the closure can carry itself. * Don't pass extraneous `Instance` parameter This can now be inferred from `Func`. * Clean up some SAFETY comments * Generalize the signature of `lift_results` * Move `lift_results` function to `Func` Also rename the lift/lower helpers to `with_{lift,lower}_context` * Remove parameter from `with_lift_context` Like `with_lower_context` this is fine to capture in the closure passed in. * Simplify the dynamic lifting logic Don't call `with_lift_context` in two locations, only call it once with a dynamic parameter. * Refactor away the `Func::lift_results_sync` helper * Use `with_lift_context` in `call_raw` * Simplify a call to `Func::call_unchecked_raw` * Ungate `with_lift_context` to fix non-cm-async build * Fix compile (bad cherry-pick conflict resolution) * Use `with_lower_context` in `call_raw` Trying to unify the async/concurrent paths as much as possible. * Move params out of `call_raw` Let closures capture the params, no need to thread it through as an unnecessary argument. * Clean up unsafety in `Func::call_raw` * Accurately mark `call_raw` itself as `unsafe`, then document why callers should be safe. * Don't have one large `unsafe` block in `call_raw`, instead split it up with separate safety comments. * Move a one-off type definition closer to its use * Avoid intermediate allocations in dynamic calls * Simplify a future-return site * Deduplicate checking parameter count * Simplify a future invocation with `?` * Simplify a variable declaration * Simplify some function signatures * Remove outdated safety comment * Refactor to not require `Params: 'static` on `call_async` * Remove no-longer-necessary SAFETY comment * Fix typos * Add a fast-path with no `Box` for sync host functions Speeds up host calls by ~20% and puts them back on parity with the beforehand numbers Wasmtime has. * Synchronize signatures of async/concurrent dynamic calls Use slices for both instead of vecs for one and slices for the other. Required some slight rejiggering. Apparently one can solve a closure problem with another closure, then one surely has no more closure problems. * Fix non-cm-async build --------- Signed-off-by: Joel Dice <[email protected]> Co-authored-by: Alex Crichton <[email protected]>
1 parent 95942cb commit b221fca

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

52 files changed

+4166
-1104
lines changed

‎Cargo.lock‎

Lines changed: 1 addition & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎Cargo.toml‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -92,7 +92,7 @@ rustix = { workspace = true, features = ["mm", "process"] }
9292

9393
[dev-dependencies]
9494
# depend again on wasmtime to activate its default features for tests
95-
wasmtime = { workspace = true, features = ['default', 'winch', 'pulley', 'all-arch', 'call-hook', 'memory-protection-keys'] }
95+
wasmtime = { workspace = true, features = ['default', 'winch', 'pulley', 'all-arch', 'call-hook', 'memory-protection-keys', 'component-model-async'] }
9696
env_logger = { workspace = true }
9797
log = { workspace = true }
9898
filecheck = { workspace = true }
@@ -129,6 +129,7 @@ wasmtime-test-macros = { path = "crates/test-macros" }
129129
pulley-interpreter = { workspace = true, features = ["disas"] }
130130
wasm-encoder = { workspace = true }
131131
cranelift-native = { workspace = true }
132+
futures = { workspace = true }
132133

133134
[target.'cfg(windows)'.dev-dependencies]
134135
windows-sys = { workspace = true, features = ["Win32_System_Memory"] }
@@ -323,6 +324,7 @@ io-extras = "0.18.1"
323324
rustix = "1.0.3"
324325
# wit-bindgen:
325326
wit-bindgen = { version = "0.43.0", default-features = false }
327+
wit-bindgen-rt = { version = "0.43.0", default-features = false }
326328
wit-bindgen-rust-macro = { version = "0.43.0", default-features = false }
327329

328330
# wasm-tools family:
@@ -336,6 +338,7 @@ wasm-mutate = "0.235.0"
336338
wit-parser = "0.235.0"
337339
wit-component = "0.235.0"
338340
wasm-wave = "0.235.0"
341+
wasm-compose = "0.235.0"
339342

340343
# Non-Bytecode Alliance maintained dependencies:
341344
# --------------------------
@@ -379,7 +382,7 @@ tempfile = "3.1.0"
379382
filecheck = "0.5.0"
380383
libc = { version = "0.2.112", default-features = true }
381384
file-per-thread-logger = "0.2.0"
382-
tokio = { version = "1.30.0", features = [ "rt", "time" ] }
385+
tokio = { version = "1.43.0", features = [ "rt", "time" ] }
383386
hyper = "1.0.1"
384387
http = "1.0.0"
385388
http-body = "1.0.0"

‎crates/c-api/src/component/val.rs‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -307,6 +307,9 @@ impl From<&Val> for wasmtime_component_val_t {
307307
Val::Result(x) => wasmtime_component_val_t::Result(x.into()),
308308
Val::Flags(x) => wasmtime_component_val_t::Flags(x.as_slice().into()),
309309
Val::Resource(_resource_any) => todo!(),
310+
Val::Future(_) => todo!(),
311+
Val::Stream(_) => todo!(),
312+
Val::ErrorContext(_) => todo!(),
310313
}
311314
}
312315
}

‎crates/component-macro/tests/expanded/char_concurrent.rs‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -342,10 +342,11 @@ pub mod exports {
342342
(char,),
343343
>::new_unchecked(self.return_char)
344344
};
345-
wasmtime::component::__internal::FutureExt::map(
346-
callee.call_concurrent(store.as_context_mut(), ()),
347-
|v| v.map(|(v,)| v),
348-
)
345+
let future = callee.call_concurrent(store.as_context_mut(), ());
346+
async move {
347+
let (ret0,) = future.await?;
348+
Ok(ret0)
349+
}
349350
}
350351
}
351352
}

‎crates/component-macro/tests/expanded/flags_concurrent.rs‎

Lines changed: 42 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -771,10 +771,12 @@ pub mod exports {
771771
(Flag1,),
772772
>::new_unchecked(self.roundtrip_flag1)
773773
};
774-
wasmtime::component::__internal::FutureExt::map(
775-
callee.call_concurrent(store.as_context_mut(), (arg0,)),
776-
|v| v.map(|(v,)| v),
777-
)
774+
let future = callee
775+
.call_concurrent(store.as_context_mut(), (arg0,));
776+
async move {
777+
let (ret0,) = future.await?;
778+
Ok(ret0)
779+
}
778780
}
779781
pub fn call_roundtrip_flag2<S: wasmtime::AsContextMut>(
780782
&self,
@@ -792,10 +794,12 @@ pub mod exports {
792794
(Flag2,),
793795
>::new_unchecked(self.roundtrip_flag2)
794796
};
795-
wasmtime::component::__internal::FutureExt::map(
796-
callee.call_concurrent(store.as_context_mut(), (arg0,)),
797-
|v| v.map(|(v,)| v),
798-
)
797+
let future = callee
798+
.call_concurrent(store.as_context_mut(), (arg0,));
799+
async move {
800+
let (ret0,) = future.await?;
801+
Ok(ret0)
802+
}
799803
}
800804
pub fn call_roundtrip_flag4<S: wasmtime::AsContextMut>(
801805
&self,
@@ -813,10 +817,12 @@ pub mod exports {
813817
(Flag4,),
814818
>::new_unchecked(self.roundtrip_flag4)
815819
};
816-
wasmtime::component::__internal::FutureExt::map(
817-
callee.call_concurrent(store.as_context_mut(), (arg0,)),
818-
|v| v.map(|(v,)| v),
819-
)
820+
let future = callee
821+
.call_concurrent(store.as_context_mut(), (arg0,));
822+
async move {
823+
let (ret0,) = future.await?;
824+
Ok(ret0)
825+
}
820826
}
821827
pub fn call_roundtrip_flag8<S: wasmtime::AsContextMut>(
822828
&self,
@@ -834,10 +840,12 @@ pub mod exports {
834840
(Flag8,),
835841
>::new_unchecked(self.roundtrip_flag8)
836842
};
837-
wasmtime::component::__internal::FutureExt::map(
838-
callee.call_concurrent(store.as_context_mut(), (arg0,)),
839-
|v| v.map(|(v,)| v),
840-
)
843+
let future = callee
844+
.call_concurrent(store.as_context_mut(), (arg0,));
845+
async move {
846+
let (ret0,) = future.await?;
847+
Ok(ret0)
848+
}
841849
}
842850
pub fn call_roundtrip_flag16<S: wasmtime::AsContextMut>(
843851
&self,
@@ -855,10 +863,12 @@ pub mod exports {
855863
(Flag16,),
856864
>::new_unchecked(self.roundtrip_flag16)
857865
};
858-
wasmtime::component::__internal::FutureExt::map(
859-
callee.call_concurrent(store.as_context_mut(), (arg0,)),
860-
|v| v.map(|(v,)| v),
861-
)
866+
let future = callee
867+
.call_concurrent(store.as_context_mut(), (arg0,));
868+
async move {
869+
let (ret0,) = future.await?;
870+
Ok(ret0)
871+
}
862872
}
863873
pub fn call_roundtrip_flag32<S: wasmtime::AsContextMut>(
864874
&self,
@@ -876,10 +886,12 @@ pub mod exports {
876886
(Flag32,),
877887
>::new_unchecked(self.roundtrip_flag32)
878888
};
879-
wasmtime::component::__internal::FutureExt::map(
880-
callee.call_concurrent(store.as_context_mut(), (arg0,)),
881-
|v| v.map(|(v,)| v),
882-
)
889+
let future = callee
890+
.call_concurrent(store.as_context_mut(), (arg0,));
891+
async move {
892+
let (ret0,) = future.await?;
893+
Ok(ret0)
894+
}
883895
}
884896
pub fn call_roundtrip_flag64<S: wasmtime::AsContextMut>(
885897
&self,
@@ -897,10 +909,12 @@ pub mod exports {
897909
(Flag64,),
898910
>::new_unchecked(self.roundtrip_flag64)
899911
};
900-
wasmtime::component::__internal::FutureExt::map(
901-
callee.call_concurrent(store.as_context_mut(), (arg0,)),
902-
|v| v.map(|(v,)| v),
903-
)
912+
let future = callee
913+
.call_concurrent(store.as_context_mut(), (arg0,));
914+
async move {
915+
let (ret0,) = future.await?;
916+
Ok(ret0)
917+
}
904918
}
905919
}
906920
}

‎crates/component-macro/tests/expanded/floats_concurrent.rs‎

Lines changed: 10 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -409,10 +409,11 @@ pub mod exports {
409409
(f32,),
410410
>::new_unchecked(self.f32_result)
411411
};
412-
wasmtime::component::__internal::FutureExt::map(
413-
callee.call_concurrent(store.as_context_mut(), ()),
414-
|v| v.map(|(v,)| v),
415-
)
412+
let future = callee.call_concurrent(store.as_context_mut(), ());
413+
async move {
414+
let (ret0,) = future.await?;
415+
Ok(ret0)
416+
}
416417
}
417418
pub fn call_f64_result<S: wasmtime::AsContextMut>(
418419
&self,
@@ -429,10 +430,11 @@ pub mod exports {
429430
(f64,),
430431
>::new_unchecked(self.f64_result)
431432
};
432-
wasmtime::component::__internal::FutureExt::map(
433-
callee.call_concurrent(store.as_context_mut(), ()),
434-
|v| v.map(|(v,)| v),
435-
)
433+
let future = callee.call_concurrent(store.as_context_mut(), ());
434+
async move {
435+
let (ret0,) = future.await?;
436+
Ok(ret0)
437+
}
436438
}
437439
}
438440
}

‎crates/component-macro/tests/expanded/integers_concurrent.rs‎

Lines changed: 45 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -921,10 +921,11 @@ pub mod exports {
921921
(u8,),
922922
>::new_unchecked(self.r1)
923923
};
924-
wasmtime::component::__internal::FutureExt::map(
925-
callee.call_concurrent(store.as_context_mut(), ()),
926-
|v| v.map(|(v,)| v),
927-
)
924+
let future = callee.call_concurrent(store.as_context_mut(), ());
925+
async move {
926+
let (ret0,) = future.await?;
927+
Ok(ret0)
928+
}
928929
}
929930
pub fn call_r2<S: wasmtime::AsContextMut>(
930931
&self,
@@ -941,10 +942,11 @@ pub mod exports {
941942
(i8,),
942943
>::new_unchecked(self.r2)
943944
};
944-
wasmtime::component::__internal::FutureExt::map(
945-
callee.call_concurrent(store.as_context_mut(), ()),
946-
|v| v.map(|(v,)| v),
947-
)
945+
let future = callee.call_concurrent(store.as_context_mut(), ());
946+
async move {
947+
let (ret0,) = future.await?;
948+
Ok(ret0)
949+
}
948950
}
949951
pub fn call_r3<S: wasmtime::AsContextMut>(
950952
&self,
@@ -961,10 +963,11 @@ pub mod exports {
961963
(u16,),
962964
>::new_unchecked(self.r3)
963965
};
964-
wasmtime::component::__internal::FutureExt::map(
965-
callee.call_concurrent(store.as_context_mut(), ()),
966-
|v| v.map(|(v,)| v),
967-
)
966+
let future = callee.call_concurrent(store.as_context_mut(), ());
967+
async move {
968+
let (ret0,) = future.await?;
969+
Ok(ret0)
970+
}
968971
}
969972
pub fn call_r4<S: wasmtime::AsContextMut>(
970973
&self,
@@ -981,10 +984,11 @@ pub mod exports {
981984
(i16,),
982985
>::new_unchecked(self.r4)
983986
};
984-
wasmtime::component::__internal::FutureExt::map(
985-
callee.call_concurrent(store.as_context_mut(), ()),
986-
|v| v.map(|(v,)| v),
987-
)
987+
let future = callee.call_concurrent(store.as_context_mut(), ());
988+
async move {
989+
let (ret0,) = future.await?;
990+
Ok(ret0)
991+
}
988992
}
989993
pub fn call_r5<S: wasmtime::AsContextMut>(
990994
&self,
@@ -1001,10 +1005,11 @@ pub mod exports {
10011005
(u32,),
10021006
>::new_unchecked(self.r5)
10031007
};
1004-
wasmtime::component::__internal::FutureExt::map(
1005-
callee.call_concurrent(store.as_context_mut(), ()),
1006-
|v| v.map(|(v,)| v),
1007-
)
1008+
let future = callee.call_concurrent(store.as_context_mut(), ());
1009+
async move {
1010+
let (ret0,) = future.await?;
1011+
Ok(ret0)
1012+
}
10081013
}
10091014
pub fn call_r6<S: wasmtime::AsContextMut>(
10101015
&self,
@@ -1021,10 +1026,11 @@ pub mod exports {
10211026
(i32,),
10221027
>::new_unchecked(self.r6)
10231028
};
1024-
wasmtime::component::__internal::FutureExt::map(
1025-
callee.call_concurrent(store.as_context_mut(), ()),
1026-
|v| v.map(|(v,)| v),
1027-
)
1029+
let future = callee.call_concurrent(store.as_context_mut(), ());
1030+
async move {
1031+
let (ret0,) = future.await?;
1032+
Ok(ret0)
1033+
}
10281034
}
10291035
pub fn call_r7<S: wasmtime::AsContextMut>(
10301036
&self,
@@ -1041,10 +1047,11 @@ pub mod exports {
10411047
(u64,),
10421048
>::new_unchecked(self.r7)
10431049
};
1044-
wasmtime::component::__internal::FutureExt::map(
1045-
callee.call_concurrent(store.as_context_mut(), ()),
1046-
|v| v.map(|(v,)| v),
1047-
)
1050+
let future = callee.call_concurrent(store.as_context_mut(), ());
1051+
async move {
1052+
let (ret0,) = future.await?;
1053+
Ok(ret0)
1054+
}
10481055
}
10491056
pub fn call_r8<S: wasmtime::AsContextMut>(
10501057
&self,
@@ -1061,10 +1068,11 @@ pub mod exports {
10611068
(i64,),
10621069
>::new_unchecked(self.r8)
10631070
};
1064-
wasmtime::component::__internal::FutureExt::map(
1065-
callee.call_concurrent(store.as_context_mut(), ()),
1066-
|v| v.map(|(v,)| v),
1067-
)
1071+
let future = callee.call_concurrent(store.as_context_mut(), ());
1072+
async move {
1073+
let (ret0,) = future.await?;
1074+
Ok(ret0)
1075+
}
10681076
}
10691077
pub fn call_pair_ret<S: wasmtime::AsContextMut>(
10701078
&self,
@@ -1081,10 +1089,11 @@ pub mod exports {
10811089
((i64, u8),),
10821090
>::new_unchecked(self.pair_ret)
10831091
};
1084-
wasmtime::component::__internal::FutureExt::map(
1085-
callee.call_concurrent(store.as_context_mut(), ()),
1086-
|v| v.map(|(v,)| v),
1087-
)
1092+
let future = callee.call_concurrent(store.as_context_mut(), ());
1093+
async move {
1094+
let (ret0,) = future.await?;
1095+
Ok(ret0)
1096+
}
10881097
}
10891098
}
10901099
}

0 commit comments

Comments
 (0)