Considering multi-run plans (RunEngine only)

[thopkins32]

Having a multi-run plan setup could be a nice way to handle the optimization loop provided by blop.plans.optimize and blop.plans.optimize_step.

We can track the optimization metadata in the start document, such as the experiment name, experiment identifier, last optimizer suggestion id, objectives, parameters, evaluation function name, constraints, etc. We can track suggestions and outcomes as event data. And we can have callbacks for visualization that operate on only the outer run.

This allows for the blop.plans.default_acquire (or any custom acquisition plan) to continue opening and closing its own BlueskyRun so that the EvaluationFunction can continue to read back the data properly.

Furthermore, this would allow you to save a single "optimization session" in the event-model architecture. You can look back on this session in particular, even though multiple sessions may comprise the full dataset for your optimizer.

Each session would be distinguished as using:

session_uid = RE(optimize(...))

Some things that we would likely have to do to make this work:

• Make suggestions and outcomes into bluesky.protocols.Readable signals
• Add open_run and close_run to the optimize plan
• Add bps.rd to the optimize_step plan after the evaluation function returns outcomes
• Callbacks for visualization
• (Optional) Utility to string together optimization sessions into the full optimizer state

That being said, I don't think this is a solution for storing optimizer state as a whole. Integration with bluesky-adaptive or bluesky-queueserver will most likely not be using this multi-plan setup. They will likely queue up the acquisition plan (blop.plans.default_acquire) directly, with the optimizer not having anything to do with Bluesky. Externally tracking this optimizer state is not difficult since Ax provides the ability to store experiments with a StorageConfig. Nevertheless, having some metadata about the optimization session seems important to store in the Bluesky documents for linking purposes.

[thopkins32]

Took a stab at this: https://github.com/NSLS-II/blop/tree/multi-run-plan. It's a bit more complex than I initially realized because we will have to turn each parameter and outcome into a Readable signal with structure.

Multiple suggestions per iteration means that these are arrays. So each event of the outer run would look something like:

optimizer stream:
  - iteration: 3
  - acquisition_uid: "abc123"
  - x1: [0.5, 0.7]        # suggested values for each point
  - x2: [1.2, 0.9]
  - himmelblau: [4.2, 3.1] # outcomes

It makes sense to keep optimize_step the way it is and do this in optimize.