Pods for bluesky(-adaptive)

This is a docker-compose yaml and ContainerFile files that will stand up a pod that attempt to mimic the full beamline / remote compute model (as we want to run at NSLS-II). The intent is to provide a realistic environment for local development.

Run the pod

cd compose/acq-pod
podman-compose --in-pod true up

To get a bluesky terminal in this pod run

bash launch_bluesky.sh

From inside bsui a set of simulated devices are available, as are databases and tiled servers. Data can be accessed via databroker, a tiled profile, or a remote tiled server.

RE(scan([det], motor, -10,10,10)) # will produce a liveplot and data
db[1] # or db[uid] can be used to access the data
from tiled.client import from_profile, from_uri
c = from_profile("MAD")
c[1] # or c[uid]
c = from_uri("http://proxy:11973/tiled")
c[1] # or c[uid]

Setting the environment variable BLUESKY_PROFILE_DIR to an ipython profile will allow you to use a custom profile in the launch_bluesky.sh script or the Queue Server container. In both cases, the RunEngine (RE), databroker, and Kafka subscriptions must be initialized in the startup profile.

To get a default QT gui for the queue server run

bash launch_bluesky.sh bluesky queue-monitor

On a Mac, XQuartz is required to display the output of the Best Effort Callback or QT UI. You will need to add your en0 IP address to Xauth. One route to this is described here.

$ export DISPLAY_MAC=`ifconfig en0 | grep "inet " | cut -d " " -f2`:0
$ xauth list
pc34.home/unix:0  MIT-MAGIC-COOKIE-1  491476ce33cxxx86d4bfbcea45
pc34.home:0  MIT-MAGIC-COOKIE-1  491476ce33cxxx86d4bfbcea45
$ export DISPLAY=$DISPLAY_MAC
$ xauth
Using authority file /Users/pat/.Xauthority
xauth> add 192.168.64.1:0 . 491476ce33cxxx86d4bfbcea45
xauth> exit
Writing authority file /Users/pat/.Xauthority
$ xauth list
pc34.home/unix:0  MIT-MAGIC-COOKIE-1  491476ce33cxxx86d4bfbcea45
pc34.home:0  MIT-MAGIC-COOKIE-1  491476ce33cxxx86d4bfbcea45
192.168.64.1:0  MIT-MAGIC-COOKIE-1  491476ce33cxxx86d4bfbcea45
pc34:docker-tips pat$ docker run -e DISPLAY=$DISPLAY_MAC -v ~/.Xauthority:/root/.Xauthority -it jess/gimp

There is a jupyterlab instance, a tiled instance, and a Queueserver http API instance running the pod which are proxied via nginx. If the pod is running http://localhost:11973 will provide links to each.

Notes

If using a version of podman-compose between 1.1 and 1.2 there is a bug that has been fixed, but not yet released. For a work around, rename the Containerfile to Dockerfile and use podman-compose.

Terms

• image : The binary blob that can be run as a container
• container : A running image. You can have many containers running the same image simultaneously. As part of starting the container you can pass in environmental variables and mount directories from the host into the container (read-only or read/write)
• pod : A collection of running containers that share a conceptual local network. When the pod is created you can control which ports are visible to the host machine. When using podman-compose the other containers can be accessed via DNS with their names.

Contents

Get podman

Podman and buildah are packaged on many Linux distributions. Refer to the official installation guide for specific instructions. These instructions cover how to install podman. Also install buildah in exactly the same fashion.

You will also need podman compose

Enable "rootless" usage

Unlike Docker, podman and buildah can be used without elevated privileges (i.e. without root or a docker group). Podman only needs access to a range of uids and gids to run processes in the container as a range of different "users". Enable that like so:

sudo usermod --add-subuids 200000-201000 --add-subgids 200000-201000 $USER
podman system migrate

For additional details and troubleshooting, see the rootless tutorial.

Configure for display over SSH

If the machine where you will be running podman is one you are connected to via SSH, then you will need to configure the SSH daemon to accept connections routed through podman---specifically, connections to its IP address rather than localhost.

Add this line to /etc/ssh/sshd_config.

X11UseLocalhost no

If podman is running on the machine you are sitting in front of, or if you would like to run in "headless" mode, no action is required.

Repository Contents

Other examples

• Wright Group bluesky-in-a-box
• sst-pods [based on an early version of this repo]