Visualiser integration

[mastadle]

As far as I can tell, the ionpulse sequence visualiser is not yet integrated.
Currently, the visualiser expects two endpoints where the hardware description and the current sequence json can be retrieved. Additionally, it subscribes to socket io notifications.

Ideally, whenever the hardware_processing worker gets a new task from its queue, I update the sequence json endpoint and emit a notification with the updated sequence data. I assume I should create a new Controller in the api_service (e.g. HardwareController) that exposes this functionality.

I would be thankful for any pointers that help me to implement this properly.

[aferk]

I would suggest we do not touch the hardware worker for this:

The Visualiser has 2 "roles":

1. Help visualize what is going on currently (connection to the hardware worker)
   Since we can not really offer a live view ("currently going on") in ICON, I would opt to integrate this into the experiment results. (which is the closest to a live view, but per experiment). Here the visualizer can display the latest JSON from the HDF file, also for past experiments (ideally we would integrate this with a slider or a datapoint).
2. Help with debugging (export a sequence)
   This could be integrated in a similar fashion to the above, where currently we require the emulator for debugging. Ultimately, we could also implement a dry-run mode, where the "harware worker" does nothing on hardware, but just returns the JSON which can then be displayed by ICON.

[mastadle]

1. I agree that integration into the experiment results makes sense and is certainly the direction we should go. For this to happen, the visualizer must first use echarts instead of plotly and observable-hq/plot. This will take a little while. Alternatively, I can port the job interface from ICON to the visualizer, but that seems backwards to me.
2. Indeed, that's the first use-case I had in mind. I want to create an easy-to-use software package for evaluating the control system, which would need exactly the dry-run mode you just mentioned. I see two ways of implementing this:
   2.1. The Hardware Processing Worker does not just tell the Hardware Controller to run the JSON sequence and then creates a PostProcesssingTask but also calls the "Visualizer Notifier", which pushes the update to the Visualizer via socketio..
   2.2. Add another socketio notification to "write_experiment_data_by_job_id" that does not include the job id. Then the visualizer just listens to that one.

This leaves the hardware description. I would create a new "controlsystem_controller" endpoint in the api folder and generate the hardware description there and return it to the visualizer.

[mosmuell]

The json sequence is already pushed via socket.io:

icon/src/icon/server/data_access/repositories/experiment_data_repository.py

Line 497 in 1625237

emit_queue.put(

So knowing the job id, you could already get this.

I'm not sure why you would add another controller? The JSON is stored in the HDF file and as such regarded as experiment data, so I would put anything CRUD based into the ExperimentDataController.

[mosmuell]

PS: ah I see. The hardware description is something separate.

Maybe one could argue that the experiment metadata and the hardware description are similar and combine them in one controller?
I chose the controllers sort of arbitrarily to fit the use cases I had. Feel free to update or restructure them as necessary.