-
Notifications
You must be signed in to change notification settings - Fork 3
Tutorial: Scripting Ot2Rec in Jupyter Notebooks
We will be processing EMPIAR-10364 in this example. You will also need Ot2Rec, IMOD, and MotionCor2 installed. You'll need to run this on a GPU enabled Linux system. For reference, our system is a virtual Linux machine with 12 Intel Xeon Gold 5218 2.30 GHz CPUs and 1 Tesla V100 GP.
Note: This Jupyter notebook is available to download here.
Download the data from EMPIAR-10364.
import Ot2Rec as o2r
import magicgui
import yamlThe next code cell will bring up a MagicGUI dialogue box where you can fill in your input.
o2r.main.new_proj()Master metadata file created.
Alternatively, if you prefer to set the user inputs in Jupyter directly, you can create the input arguments directly and "deconstruct" the o2r.main.new_proj function.
# Set user args, look at o2r.magicgui for names of fields etc
new_proj_args = magicgui.widgets.FunctionGui(o2r.magicgui.get_args_new_proj)
new_proj_args.project_name.value = "TS"
new_proj_args.source_folder.value = "../data"
new_proj_args.no_mdoc
print(new_proj_args)
# Create main yaml config file
o2r.params.new_master_yaml(new_proj_args)
# Create new empty metadata object and turn it into a yaml
meta = o2r.metadata.Metadata(
project_name="TS",
job_type="master"
)
meta.create_master_metadata()
with open("TS_master_md.yaml", "w") as f:
yaml.dump(meta.metadata, f, indent=4)<FunctionGui magicgui.widgets._function_gui.<class 'magicgui.widgets._function_gui.FunctionGui'>(project_name='TS', source_folder=PosixPath('/ceph/groups/els/EMPIAR-10364/example/../data'), folder_prefix='', file_prefix='', ext='mrc', stack_field=0, index_field=1, tiltangle_field=2, no_mdoc=True)>
A yaml file containing the setup details for the project has now been created.
Ensure motioncor2 is available on your system for this step. Again, you can choose to enter your user inputs in magicgui, or set them directly. Both ways are included below.
# Use MagicGUI to capture inputs
o2r.motioncorr.create_yaml()Master config read successfully.
Master metadata read successfully.
Previous MotionCor2 metadata not found.
MotionCor2 metadata updated.
MotionCor2 metadata file created.
# *OR* set user args manually
mc2_args = magicgui.widgets.FunctionGui(o2r.magicgui.get_args_mc2)
mc2_args.project_name.value = "TS"
mc2_args.pixel_size.value = 2.243
o2r.motioncorr.create_yaml(mc2_args) # see tests for more examples of setting the user inputs programaticallyMaster config read successfully.
Master metadata read successfully.
Previous MotionCor2 metadata not found.
MotionCor2 metadata updated.
MotionCor2 metadata file created.
Now we will run motioncor2. Since this is an example, I have edited the TS_mc2.yaml process list to only process one tilt-series, but you can leave the process list as is if you'd like.
The following code cell is a "deconstructed" version of what happens when you call o2r.mc.run in the terminal.
mc2_params = o2r.params.read_yaml(
project_name="TS",
filename="TS_mc2.yaml",
)
proj_md = o2r.metadata.read_md_yaml(
project_name="TS",
job_type="motioncorr",
filename="TS_master_md.yaml",
)
mc2_obj = o2r.motioncorr.Motioncorr(
project_name="TS",
mc2_params=mc2_params,
md_in=proj_md,
logger=o2r.logger.Logger("o2r_motioncor2.log")
)
mc2_obj.run_mc2()
mc2_obj.export_metadata()Ot2Rec-MotionCor2 started.
Processing TS 1...: 100%|████████████████████████████████████████████| 1/1 [02:34<00:00, 154.23s/it]
Ot2Rec-MotionCor2 jobs finished.
Ensure IMOD is available on your system. User inputs can be captured from the MagicGUI or programatically as before.
# MagicGUI method
o2r.align.create_yaml()# Programatically
align_args = magicgui.widgets.FunctionGui(o2r.magicgui.get_args_align)
align_args.project_name.value = "TS"
align_args.rot_angle.value = 175.51
align_args.stack_bin_factor.value = 8 # to speed things up
o2r.align.create_yaml(align_args)# Run
align_params = o2r.params.read_yaml(
project_name="TS",
filename="TS_align.yaml",
)
mc2_md = o2r.metadata.read_md_yaml(
project_name="TS",
job_type="align",
filename="TS_mc2_mdout.yaml"
)
align_obj = o2r.align.Align(
project_name="TS",
md_in=mc2_md,
params_in=align_params,
logger_in=o2r.logger.Logger("o2r_imod_align.log"),
)
align_obj.create_stack_folders()
align_obj.create_rawtlt()
align_obj.create_stack()
align_obj.align_stack()Ot2Rec-align (IMOD) started: newstack.
Creating stack for TS 1...: 100%|█████████████████████████████████████| 1/1 [00:06<00:00, 6.17s/it]
All Ot2Rec-align (IMOD): newstack jobs successfully finished.
Ot2Rec-align (IMOD) started: batchruntomo.
Aligning TS 1...: 100%|██████████████████████████████████████████████| 1/1 [03:37<00:00, 217.02s/it]
All Ot2Rec-align (IMOD) jobs successfully finished.
# Create yaml
o2r.recon.create_yaml()IMOD alignment metadata read successfully.
Previous IMOD reconstruction metadata not found.
IMOD reconstruction metadata updated.
IMOD alignment metadata file created.
# Programatically
recon_args = magicgui.widgets.FunctionGui(o2r.magicgui.get_args_recon)
recon_args.project_name.value = "TS"
o2r.recon.create_yaml(recon_args)IMOD alignment metadata read successfully.
Previous IMOD reconstruction metadata not found.
IMOD reconstruction metadata updated.
IMOD alignment metadata file created.
# Run
recon_params = o2r.params.read_yaml(
project_name="TS",
filename="TS_recon.yaml",
)
align_md = o2r.metadata.read_md_yaml(
project_name="TS",
job_type="reconstruct",
filename="TS_align_mdout.yaml"
)
recon_obj = o2r.recon.Recon(
project_name="TS",
md_in=align_md,
params_in=recon_params,
logger_in=o2r.logger.Logger("o2r.imod_recon.log"),
)
recon_obj.recon_stack()Ot2Rec-reconstruction (IMOD) started.
Reconstructing TS 1...: 100%|████████████████████████████████████████| 1/1 [03:49<00:00, 229.67s/it]
All Ot2Rec-recon (IMOD) jobs successfully finished.
You can generate the Ot2Rec reports by running o2r.report.run TS in the project directory within a conda env with ot2rec_report installed.