alchemiscale user survey 2025

[dotsdl]

As 2025 comes to a close, we'd like to solicit feedback from our users to help us determine where to prioritize effort in 2026. If you have a moment, please answer the following questions as responses to this thread!

---

If you've used alchemiscale:

1. What is something you love about how alchemiscale works for you today?
2. What is one of your biggest frustrations when using alchemiscale as of today?
3. Can you share with us any ideas you have for how the project might address the frustrations you have?

If you've never used alchemiscale:

1. What barriers, if any, keep you from trying alchemiscale out?
2. Can you describe one major pain point in your work that, if alchemiscale were shown to effectively address it, would be compelling enough for you to try it?
3. Have you reviewed the alchemiscale docs? If so, what questions do you have that weren't addressed by them?

[IAlibay]

I have used / am using alchemiscale.

What is something you love about how alchemiscale works for you today?

Being able to launch tens of thousands of jobs in one go without having to worry about management is just amazing.

What is one of your biggest frustrations when using alchemiscale as of today?

I'll mention two instead of one:

1. We often find myself unable to use alchemiscale when validating openfe on experimental stack updates. Here I mean, for example, new versions of OpenMM or OpenFE that have been released (maybe as an RC) but aren't "stable" because we've not had a chance to validate them.
2. It's hard to get hardware provenance information. Quite often we want to check why a job is behaving in a certain way, or even just measure performance across a host of different GPUs. Alchemiscale should be ideally suited for this, but there doesn't seem to be any easy way to fetch hardware execution information for each DAG.

Ideas for ways these frustrations can be addressed.

1. Could it be possible to have an "experimental" stack deployment, which is limited to a "dev" scope that undergoes less pre-deployment validation? The idea would be that a select few individuals would then use this dev scope purely for validating upcoming stack changes. This validation would then, hopefully, speed up root stack updates.
2. @mikemhenry proposed moving logging utilities not too long ago, we could move it into gufe and have it attach data to DAG results when calling execute_DAG? It would have to be an optional thing (since GPU probing is contentious).

[jthorton]

I have used / am using alchemiscale.

What is something you love about how alchemiscale works for you today?

Submitting and running calculations at a massive scale is fantastic, allowing me to accomplish much more than I could with local execution of OpenFE only. This has had a huge impact on my work, so thank you so much for everything you have done so far!

What is one of your biggest frustrations when using alchemiscale as of today?

1. We commonly use alchemiscale to run benchmarks for new versions of OpenFE or force field candidates, in some cases, it would be useful to analyse the simulation trajectory to understand major discrepancies in performance however these are not available in alchemiscale.
2. Continuations and extensions: these are vital so that we can run longer simulations like those involved with charge changing transforms or more expensive protocols like SepTop or ABFE, which have much larger wall times, vastly increasing the amount of calculations we can run through alchemiscale.
3. Software provenance information attached to the result to probe the exact versions used in the calculation.

[lilyminium]

Chalk me down as a happy user of alchemiscale.

What is something you love about how alchemiscale works for you today?

I love being able to submit and run calculations at scale, and having it "just work". The communication and documentation has been fantastic around releases, so that even as someone only tangentially keeping an eye on the Slack channel I knew when and how to update my images.

What is one of your biggest frustrations when using alchemiscale as of today?
Can you share with us any ideas you have for how the project might address the frustrations you have?

1. To add onto Josh, being able to retrieve the simulations would be incredibly useful.

2. Personally, I am not a heavy enough user to really run into any major frustrations. One issue I did see was some calculations stuck in the "Running" phase for much longer than seemed reasonable, and I had to manually cancel and restart those, where they finished as expected. Maybe some kind of error cycling would be useful.

[lwhitm1]

I have used / am an Alchemiscale user.

What is something you love about how alchemiscale works for you today?

The scale at which I can run calculations is amazing and the interface is intuitive and very easy to use.

What is one of your biggest frustrations when using alchemiscale as of today?
Can you share with us any ideas you have for how the project might address the frustrations you have?

I agree with Josh and Lily that being able to retrieve/analyze simulation trajectories would be great. On vav18, I’ve been able to pull files and run short simulations of the systems without explicit topology files to ensure jobs wouldn’t blow up, but we can’t really visualize them without the topology file or a way to reconstruct it.

It would also be great to have alchemiscale-fah support running the new OpenFE protocols (I.e. SepTop and protein mutation rbfe)

[jandom]

If you've used alchemiscale:

That's the one for me :D

What is something you love about how alchemiscale works for you today?

It is remarkably general and well-structured. There are a couple of concepts that you need to get your head around, but once you do it becomes much easier to navigate. Extension/customization is also possible because of the good structure (eg my saladcloud work) but that's more of an expert-feature.

What is one of your biggest frustrations when using alchemiscale as of today?

There is no single big frustration but there certainly are a few papercuts

• resuming (but that's getting sorted)
• while errors are 1st class citizens, the observability of tasks is quite low – what's the % competition of a task, which tasks are restarting a lot, which tasks have been actioned/claimed, which tasks are solvent, which are complex jobs. get_network_status is the main tool here but it really hides a lot of the complexity.
• a GUI is missing to be able to just monitor the networks (nevermind prep/analysis)
• hard to understand how much S3 space is consumed by each task (this only becomes a problem with interruptible workloads, either the Domanski-flavor or the DAG/Dotson-flavor)

Can you share with us any ideas you have for how the project might address the frustrations you have?

• most of these are getting addressed already to be fair, my broad message would be "product over tool" focus – not just for this project but everything in the OMSF ecosystem

[JenkeScheen]

I am using alchemiscale at scale - we tend to turn ~1000 compounds per month through alchemiscale on a good month.

What is something you love about how alchemiscale works for you today?

Echoing the others, I love the way that alchemiscale abstracts away many of the complexities that come with managing/babysitting FECs. There's a clear advantage over having to interact with a HPC directly. For us at ASAP specifically, the fact that alchemiscale is able to orchestrate/distribute across multiple clusters at the same time is excellent.

What is one of your biggest frustrations when using alchemiscale as of today?

I agree with the others but I'll try not to just repeat them. Some other notes:

1. I interact with lots of peers in the field that work for biotech/pharma and they're generally impressed with alchemiscale but it seems the barrier for entry is high. Folks seem to think that hooking up to their own cluster is not feasible/not worth the effort. If the barrier could be lowered somehow it could pull more people into this circle.
2. this one's a bit niche: if I run n replicates for a task, I'd like to be able to retrieve the prediction for each replicate rather than only the mean between all replicates. This would be useful for debugging but also for prospective QC (e.g. discard edges that have high variance).

Can you share with us any ideas you have for how the project might address the frustrations you have?

1. some sort of high-level document that details the installation process would possibly help folks understand that it's not impossible. Looking at the installation docs currently it's a bit black-magic.
2. this is pure dev work, possibly in the openfe layer of things.

[IAlibay]

prediction for each replicate

Not to debug too much in a wishlist, but if you're using the HREX method, have you tried using get_individual_estimates?

[dotsdl]

Opened this issue to address (2): #474