Feedback from real-world use of BayBE in closed-loop lab optimization

[SissiFeng]

Hi awesome BayBE developers:

We’ve evaluated BayBE for lab-related Bayesian optimization workflows that require dynamic adaptation modes. BayBE provides a solid foundation, yet we encountered several technical limitations—that make it difficult to integrate into evolving or system-wide workflows.

1. Input interface constraints
   current input works well for batch design of experiments, but poses challenges when parameter spaces, constraints, or workflow definitions need to be updated programmatically or on-the-fly within a running pipeline.

2. Limited exposure of internals
   lacks granular control over acquisition functions, kernel types, or initialization strategies. So when we try to tune the optimizer for noisy processes or domain-informed settings must extend or modify internal logic directly, it diminishes usability.

3. Minimal lifecycle management & callback support
   recommend-measure loops do not provide explicit hooks like on_new_point, on_iteration_end, or checkpoint callbacks. As developers embedding BayBE in larger orchestration systems, we found this restricts smooth integration with experiment controllers and iterative data pipelines.

4. Integration overhead
   lacks detailed schemas or type specifications. For teams building UI dashboards or microservice wrappers, responsibly handling I/O streams and modelling data structures becomes cumbersome without introspection tools or typed APIs—leading to integration overhead.

5. Compatibility & dependency conflicts with new Python versions
   Currently, BayBE does not support Python 3.13, and several dependencies used within the package are pinned to older versions that conflict with widely used tools in the Python data ecosystem.

We believe extending BayBE with flexible IO APIs, plugin-based optimizer configuration, lifecycle callback hooks, and typed schema support would significantly enhance adoption in real-world, integrated environments.

If there’s interest, we’re open to contributing to these enhancements.

[Scienfitz]

Hi @SissiFeng

this is much appreciated. What you describe will require some digestion from oour side, so perhaps expect multiple answers. Here some immediate things that come to mind after first read:

1)
Flexibily evolving parameters, constarints and objectives are a bit challenging under the current methodology. While there might be bits that we can allow changing more flexibly I don't see much improvement possible here quickly. Do you have concrete ideas / requests? An adaptive searchspace for instance would be tremendous contribution.

3)
Did you look at custom hooks? With this you can initiate callback not just on a predefined set of events, but in fact on any internally called method. Just be careful with the registration persistence.

4)
Having this is not unrealistic as we use a sophisticated machinery based on attrs and cattrs. Once these packages provide making a schema, your request is solved (maybe attrs master @AdrianSosic knows a timeline or issue where you can express your interest in this)

5)
3.13 support was released in the latest version 0.13.1 just days ago. It always takes a while due to internal dependencies, not much we can do about it.

Is there any reason why you have to use 3.13 and not e.g. 3.10?

I would also be interested in what exact packages you refer to in regards to the pin to older versions (we might be able to get rid but are simply not aware at the moment)

[AdrianSosic]

Hi @SissiFeng,

Thanks a lot for taking the time to compile this list – feedback like this is incredibly helpful! @Scienfitz has already pointed to some of existing features and ongoing work, but I'd also like to share my thoughts and specifically ask you for some more details on the individual points to make them actionable for us.

1. As already mentioned, having full support for adaptive search spaces with a clean interface would be awesome, but this requires a lot of additional machinery. What I would like to point out, though: while expanding search spaces is a complex story, we already have quite some functionality for reducing existing search spaces, with more features to come. In fact, there is already an "Excluding Configurations" user guide that showcases for example the toggle_discrete_candidates method. Also, there is a refactoring of the search space classes planned that will bring easy-to-use functionality like a filter for convenient reductions of the search spaces. What is still missing though is some clever algorithm to automatically trigger these mechanisms depending on the taken experimentation trajectory. Could you please comment on what functionality you'd ideally wish for?

2. I'm not entirely sure what you mean here, could you please clarify? Maybe some info from my side: While we certainly don't have convenience methods for all sorts of customizations (and there is also certainly a lot of documentation still missing), most parts of BayBE are already designed in a way that you can fully customize them. For example, you can easily pass your own kernels to the existing GP model, or even use your own surrogate models or recommenders and they will automatically blend into the recommendation process. In fact, two nice (but not yet documented) features are our SurrogateProtocol and RecommenderProtocol interfaces, which allow you to cook up your own model/recommender logic and insert them into our framework without even having to interact with any of BayBE's internals. Let me know what concrete feature you are missing and we can see how to enable it for you if it's not yet there 👍🏼

3. Yes, I can see that callbacks can be an essential part for integration into other systems. The custom hooks approach mentioned by @Scienfitz is our first generic step in that direction, which – at least in theory – should already enable all possible interactions a user might need. However, I am very open to extend the callback functionality by adding some additional hook registration mechanism for certain specific points in the process. The only question is: which are the points that justify having additional explicit hooks attached? Would be happy to hear your thought here, for example, where specifically should on_new_point be called and with which context? Perhaps you can outline some of your use cases for us.

4. Typed APIs and schema generation: very much agree! Right now, there is no easy way for us to enable this but there are ongoing discussions (since quite a while) on the cattrs side. Not sure if we'll get a handle on it very soon, though. I know that, for example, pydantic would offer something like that, but we moved away from it for other reasons. One thing that might still help you: while there is no automatic way to generate the schema, we have at least summarized all the necessary rules in our serialization user guide. So, in principle, using that in combination with the class constructor signatures, you can derive the schema manually. If you have concrete suggestions how to improve the status quo, let me know!

5. Let us know if the problem is now resolved with our latest 0.13.1 release. We've spent some effort not only to enable Python 3.13 but also significantly reduced the dependency footprint in general (with potentially more reduction to come). But I'd be curious to hear which specific pins are causing trouble for you. We can certainly work on it to solve the problem!

Looking forward to hear your thoughts 🙃

[AdrianSosic]

Hi @SissiFeng: would be great to hear some of your thoughts about the discussed points ☺️

[AdrianSosic]

Hi @SissiFeng, did you have some time to go through our answers? We'd highly appreciate if you could share some more insights so that we can create some concrete action items before closing this issue 🙃

[Scienfitz]

outdated