running_locally

Running Numtracker service locally

For local development or testing there are two main approaches, either building and running locally or using docker/podman and running it in a container.

Using containers

Running in a container is the easier option if you only need to develop applications against the API and don't need to work on the numtracker codebase itself.

The image is published to ghcr.io and the latest version is available using the latest tag. Alternatively, tagged versions are also available.

The simplest way to run the service is

podman run -p8000:8000 ghcr.io/diamondlightsource/numtracker

This will make the service available on http://localhost:8000 with no authentication required. When the container is shutdown (via podman kill <CONTAINER_NAME>), all configuration data is lost and will need to be re-entered the next time the container is run.

Note

Podman will by default assign a random name to the container each time it is run. This can make it difficult to shutdown. You can assign a name when running using --rm --name <container_name> which will allow it to be shutdown using podman kill <container_name>. The --rm flag means the container will be deleted when it is killed allowing the name to be re-used.

Persisting state

Numtracker writes to its file system for two reasons which can be configured separately.

The main sqlite database file is where the path templates and DB scan numbers are held. It can be configured via the --db flag to the serve command. To persist this configuration between restarts, set up a volume in the container and set the db location to be within it, eg

mkdir numtracker_data
podman run \
    -p8000:8000 \
    -v${PWD}/numtracker_data:/data \
    ghcr.io/diamondlightsource/numtracker serve --db /data/numtracker.db

This will create the database in the numtracker_data directory and re-use it for subsequent restarts. You can also query the DB locally (via sqlite3 cli or similar) if needed for debugging.

The filesystem is also used to interact with the number file system used by GDA. This can be configured in the same way, potentially re-using the same directory. Within the mounted directory, there should be a directory named after the beamline that should be writing number files, eg a directory could look something like

numtracker_data
├── b21
│   └── 983.b21
├── i11
│   └── 43921.i11
├── i11-1
│   └── 98432.i11-1
├── i22
│   └── 1343.i22
└── numtracker.db

These subdirectories need to be present when the service starts. Any configured beamline that does not have a matching directory will have its scan number stored only in the DB. The file extension used for the number file is the value configured for trackerFileExtension in the beamline's configuration.

To enable the filesystem tracker files as well as persisting the DB, run the service as

podman run \
    -p8000:8000 \
    -v${PWD}/numtracker_data:/data \
    ghcr.io/diamondlightsource/numtracker serve \
        --db /data/numtracker.db \
        --root-directory /data

Building and running locally

Numtracker is written in rust and makes use of the cargo build system. You will need the rust toolchain and cargo available - see rustup.rs for details. You will need a recent version of rust, 1.83 at time of writing. Ensure that $CARGO_HOME is set to a directory with plenty of space available (ie, not a network home directory) as the cache directory can get very large.

Once cargo is available, getting running is straightforward

1. Clone the project

   git clone [email protected]:DiamondLightSource/numtracker.git

2. Run the application

   cargo run serve

This will run numtracker with its default settings. The DB will be written to numtracker.db in the root of the project directory. There will be no tracker files.

To configure the filesystem tracker integration, pass the directory containing the beamline directories (described above) using the --root-directory flag.

cargo run serve --root-directory numtracker_data