Pre-proposal for a WG: Multi-modal dev services #48724
Replies: 4 comments 6 replies
-
|
Great write up @holly-cummins ! Regarding WebAssembly (WASM), yes, it’s definitely feasible, though the complexity and trade‑offs depend on the service in question. Take running PostgreSQL, for example: porting it to WASI is ongoing and involves some effort . For completeness, it is possible to run containers via WASM using projects like container2wasm. Happy to help and support this great initiative. |
Beta Was this translation helpful? Give feedback.
-
|
We are in the exact scenario described. We do not have a container runtime available. We occasionally use MQ Series. For quarkus tests, our current workaround to start an in-memory artemis and replace the mq connection factory with an artemis connection factory. It would be really nice to have the possibility to start a non-containerized mq series. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks to @andreaTP we now have a prototype running PostgreSQL v17 (with limitations..) via WASM/Chicory: Would love to see this as a "fallback mode" for the PosgreSQL devservices based on containers.
@holly-cummins your analysis is spot-on - there's multiple modes we'll probably want to support eventually and it might make sense to design the mode-choice and related configuration properly so to have such services configured and behave in a consistent way, but at the same time I think it would be fine to allow rapid feedback for just |
Beta Was this translation helpful? Give feedback.
-
|
Hey @holly-cummins, thanks for the great write-up! I really love the idea and the direction this is heading. I do have a slight concern regarding the "shell script" fallback, though. Shell scripts can be notoriously tricky to get right, especially when trying to tailor and maintain them for every possible OS and architecture combination. The maintenance burden there could get pretty heavy. Other than that, I completely agree with the rest of the proposal! It actually got me wondering: would it be possible to orchestrate WASM modules directly from a Compose file? (Would love your thoughts on this, @ozangunalp!). Also, if we go the WASM route, how would we handle dynamic port allocation and network discoverability? We are already struggling a bit with network discoverability as it is, so I'm curious about how we'd go about retrieving those ports and wiring things up once they are assigned to a module at runtime (and how when running container IT it will connect to a WASM app). |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
I don't think this is ready for a working group, not least because #49 is still going, but I want to get a few thoughts down in a place where we can add to them.
The problem
For users in restricted environments, such as banks, most dev services just don't work. A container runtime isn't allowed, which means any dev service that uses containers can't run. The only dev services that can work are ones which don't rely on containers, such as H2 and Amazon Lambda. This is a real shame, because dev services are fantastic.
We offer some workarounds to missing container runtimes, such as using Testcontainers cloud instead of a local container runtime, but no generalised solution.
The inverse problem that also turns out to be a problem
We also occasionally get requests to allow dev services that don't run in containers to run in containers. For example, sometimes it's desirable to have the AWS Lambda dev service run in a container so that it can access the docker shared network
Ideas for solutions
At the moment, every dev service has to be either "container" or "not-container," each of which has drawbacks. What if a dev service could provide multiple implementations, and the Quarkus runtime would select the most suitable one for the environment? For example, an extension's processor could return a prioritised list, with attached conditions for each option (
isContainerRuntimeAvailable(), etc).Here are are some possible implementations:
It's worth noting that we also have a related problem, "should this dev service start a new container or re-use a compose container or or re-use an external service?" We've solved this on an ad-hoc basis per service by doing discovery and chaining, but that implementation could fit neatly into a the "prioritised list of suppliers + metadata" solution.
cc @cescoffier @ozangunalp
Beta Was this translation helpful? Give feedback.
All reactions