Skip to content

Comments

feat: Add example Python linear registration script and control over non-display dimension based annotation clipping#883

Open
seankmartin wants to merge 66 commits intogoogle:masterfrom
MetaCell:feat/example-linear-registration
Open

feat: Add example Python linear registration script and control over non-display dimension based annotation clipping#883
seankmartin wants to merge 66 commits intogoogle:masterfrom
MetaCell:feat/example-linear-registration

Conversation

@seankmartin
Copy link
Contributor

@seankmartin seankmartin commented Feb 16, 2026

Summary

  1. Adds a new Python script to the Python examples. This script allows for a number of transformation estimations to help estimate image registration based on points placed in the viewer by the script user.
  2. Change to the Python ViewerState to expose the displayDimensions of the Neuroglancer viewer state.
  3. Introduces a method for annotations to change the clip weighting of non-display dimensions, the weight for this is stored in the state.

Setup

From the repository main directory:

npm run build-python
python -m pip install .
python -m pip install scipy

Linear registration workflow

The workflow is described in more detail in the docstring in the example file. But in short the most likely use path is as follows:

  1. Open neuroglancer as normal and setup layers representing the reference/fixed data and the moving data that you want to align to the fixed data.
  2. Copy the url and launch the example script here via python -i example_linear_registration.py --url https://neuroglancer.demo.appspot.com/...
  3. Follow the instructions in the new viewer to put fixed data in the left panel, and moving data in the right panel. Then start placing registration points. The affine transform will automatically update when the viewer state updates and the annotation points are different, but debounced.
  4. When finished, dump the state for later reference.

In images that might look like this. This image is after placing the fixed and reference layers in the right layer groups:
image

And then after placing a number of points:
image

Here is a video of using the tool on the default included demo data, the points placed here could be a bit better, this is moreso to give an idea of the different steps of estimation and actions on pressing the custom functions bound to keys for the workflow:
https://github.com/user-attachments/assets/41f27c77-94d9-452d-9317-8fda9c2c3827

New annotation layer state - clipDimensionsWeight

One problem arose during this script setup. As part of the script we created a second coordinate space. The annotations representing the registration points would contain information for both coordinate spaces. This allowed to conveniently know which points were linked, allow moving to the same spot in the fixed image and moving image quickly, allowing adjusting the points easily etc. There was one issue with this though, which is that the non-display dimension clipping meant that moving in the left panel caused the annotations in the right panel to disappear, and vice versa. Here shows the issue, and the state change proposed to allow changing the default weight on dimensions that are non-display dimensions. Setting that weight to zero means that no clipping happens on the specified dimension when it is a non display dimension.

2026-02-05.12-39-31.mp4

There are a few points to note about this though:

  1. We did consider having this instead be on the user defined shader instead, since clipping happens at the shader level. The awkward part about that though is that because for optimization the shader clips early, it means that the shader clipping of annotations runs before the user main runs. As such we'd lose some important optimization, especially if the user shader is heavy. It could instead be a new kind of shader directive like the #uicontrol directive, but unsure if we want to introduce that kind of pattern. Either way, it could be something like #clipdimweight[0]=0.0 #clipdimweight[2]=0.5.
  2. In the current state, the mapping is done by name. But right now there is not a link setup so that if the user renames a dimension in the viewer then the corresponding name will update in any annotation layer's clipDimensionsWeight. This is definitely a bit undesirable, but considering that I would imagine this state feature is likely only used by advanced users that might be ok. Alternatively an array could be used for the weights instead of an object. Regardless, thought it would be good to agree on the state representation / overall strategy here before worrying too much about those details.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant