Merge pull request #86 from Lucaslab-Berkeley/development_v1.1

Backend updates for v1.1 release

Author: mgiammar

.gitignore

@@ -119,6 +119,10 @@ ENV/
 # Temporary files
 *.tmp
 
+# Log files
+*.log
+*.out
+
 
 small_image_test_data/
 medium_image_test_data/

benchmark/benchmark_match_template.py

@@ -12,8 +12,7 @@
 import subprocess
 import time
 from pathlib import Path
-from pprint import pprint
-from typing import Any, dict
+from typing import Any
 
 import click
 import numpy as np
@@ -35,6 +34,16 @@ def download_comparison_data() -> None:
         ["zenodo_get", f"--output-dir={DOWNLOAD_DIR}", ZENODO_URL], check=True
     )
 
+    # Change the paths pointing to the tests/tmp directory to benchmark/tmp directory
+    # in the downloaded YAML file
+    with open(YAML_PATH) as f:
+        yaml_text = f.read()
+
+    yaml_text = yaml_text.replace("tests/tmp", "benchmark/tmp")
+
+    with open(YAML_PATH, "w") as f:
+        f.write(yaml_text)
+
 
 def setup_match_template_manager() -> MatchTemplateManager:
     """Instantiate the manager object and prepare for template matching."""
@@ -78,7 +87,9 @@ def benchmark_match_template_single_run(
     ##################################################
     # This is using the timing model where the time -- T -- to compute N
     # cross-correlations is dependent on some device rate -- r -- and a constant setup
-    # cost in terms of time (this is what we measure):
+    # cost in terms of time. This setup time (or core-deadtime) is part of distributing
+    # data to each device, compiling helper functions, and other overhead. What we can
+    # measure is the total time:
     #
     # T_N = N/r + k
     #
@@ -229,7 +240,7 @@ def main(orientation_batch_size: int, num_runs: int, output_file: str):
     print(f"  Output file: {output_file}")
 
     result = run_benchmark(orientation_batch_size, num_runs)
-    pprint(result)
+    # pprint(result)
     save_benchmark_results(result, output_file)
 
 

docs/benchmarks.md

@@ -0,0 +1,45 @@
+# Benchmarks of the Leopard-EM package
+
+Identifying freely oriented macromolecules using Two-Dimensional Template Matching (2DTM) is a computationally intensive task since we must compute millions of cross-correlograms on large cryo-EM micrographs.
+Efficiency of the `match_template` program is therefore a key consideration going into the Leopard-EM package.
+
+We include some benchmarking results across different GPU hardware to provide an estimate of `match_template` performance.
+These results can help guide users in planning out 2DTM analyses of their data using Leopard-EM and serve as a reference for expected performance.
+
+## Benchmarking Setup
+
+Leopard-EM includes a benchmarking script at `benchmark/benchmark_match_template.py` (if downloaded from source) which you can use to determine performance _on your own hardware_.
+This script runs the `match_template` program using the following parameters:
+
+- Micrograph size: 4096 x 4096 pixels (Falcon 4i) or 5760 x 4092 pixels (K3)
+- Template size: 512 x 512 x 512 pixels
+- Number of defocus planes: 11
+- Variable orientation batch size configurable using `--orientation-batch-size`
+
+Note that we empirically observe that template size has negligible effect on performance.
+Total search times are extrapolated from throughput to a full orientation search of ~1.58 million orientations with 13 defocus planes (~20.5 million total cross-correlations).
+
+## Version 1.1 benchmarks
+
+### Falcon 4i images (4096 x 4096 pixels)
+
+GPU name                 | VRAM  | Image size | Throughput (cross-corr/sec) | 2DTM search time (hours) |
+------------------------ | ----- | ---------- | --------------------------- | -------------------------- |
+GeForce 2080 Ti          | 11 GB | 4096×4096  | 343.0                       | 16.70                      |
+RTX 6000 Ada / L40s      | 48 GB | 4096×4096  | 744.5                       | 7.69                       |
+RTX 6000 Blackwell Max-Q | 96 GB | 4096×4096  | 1394.7                      | 4.10                       |
+A100                     | 40 GB | 4096x4096  | 923.4                       | 6.19                       |
+H100                     | 80 GB | 4096×4096  | 1650.8                      | 3.47                       |
+
+### K3 images (5760 x 4092 pixels)
+
+GPU name                 | VRAM  | Image size | Throughput (cross-corr/sec) | 2DTM search time (hours) |
+------------------------ | ----- | ---------- | --------------------------- | -------------------------- |
+GeForce 2080 Ti          | 11 GB | 5760×4092  | 217.1                       | 26.40                      |
+RTX 6000 Ada / L40s      | 48 GB | 5760×4092  | 431.7                       | 13.30                      |
+RTX 6000 Blackwell Max-Q | 96 GB | 5760×4092  | 799.7                       | 7.15                       |
+A100                     | 40 GB | 5760×4092  | 530.2                       | 10.79                      |
+H100                     | 80 GB | 5760×4092  | 897.9                       | 6.37                       |
+
+!!! note "K3 image benchmarks"
+    Note that we have not optimized Leopard-EM v1.1 for K3 images in particular. Future versions should include optimizations for non-square images which will improve performance on K3 data.

docs/index.md

@@ -7,11 +7,25 @@ description: Overview of the Leopard-EM package for 2DTM in Python
 
 Welcome to the **L**ocation & ori**E**ntati**O**n of **PAR**ticles found using two-**D**imensional t**E**mplate **M**atching (Leopard-EM) online documentation!
 Leopard-EM is a Python implementation of Two-Dimensional Template Matching (2DTM) which itself is a data processing method in cryo-EM for locating and orienting particles using a reference structure.
-This package currently reflects the functionality described in Lucas, *et al.* (2021)[^1] with additional programs to maximize the usefulness of 2DTM as well as other user-friendly features for integrating into broader data science workflows.
+<!-- This package currently reflects the functionality described in Lucas, *et al.* (2021)[^1] with additional programs to maximize the usefulness of 2DTM as well as other user-friendly features for integrating into broader data science workflows. -->
 
 !!! note "Citing this work"
 
-    If you use Leopard-EM in your research, please cite (coming soon!).
+    If you use Leopard-EM in your research, please cite the [Leopard-EM preprint](https://doi.org/10.1101/2025.08.26.672452):
+
+    ```
+    @article {Giammar2025.08.26.672452,
+        author = {Giammar, Matthew David and Dickerson, Joshua Luke and Hall, Laina Nicole and Lucas, Bronwyn Ayla},
+        title = {Leopard-EM: An extensible 2DTM package to accelerate in situ structural biology},
+        elocation-id = {2025.08.26.672452},
+        year = {2025},
+        doi = {10.1101/2025.08.26.672452},
+        publisher = {Cold Spring Harbor Laboratory},
+        URL = {https://www.biorxiv.org/content/early/2025/08/29/2025.08.26.672452},
+        eprint = {https://www.biorxiv.org/content/early/2025/08/29/2025.08.26.672452.full.pdf},
+        journal = {bioRxiv}
+    }
+    ```
 
 ## Installation
 
@@ -202,6 +216,6 @@ See the [Installation -- For Developers](#for-developers) section for instructio
 
 The code in this repository is licensed under the **BSD 3-Clause License**. See the [LICENSE](LICENSE) file for full details.
 
-## References
+<!-- ## References
 
-[^1]: Lucas BA, Himes BA, Xue L, Grant T, Mahamid J, Grigorieff N. Locating macromolecular assemblies in cells by 2D template matching with cisTEM. Elife. 2021 Jun 11;10:e68946. doi: 10.7554/eLife.68946. PMID: 34114559; PMCID: PMC8219381.
+[^1]: Lucas BA, Himes BA, Xue L, Grant T, Mahamid J, Grigorieff N. Locating macromolecular assemblies in cells by 2D template matching with cisTEM. Elife. 2021 Jun 11;10:e68946. doi: 10.7554/eLife.68946. PMID: 34114559; PMCID: PMC8219381. -->

docs/tutorials/batch_processing.md

@@ -431,11 +431,23 @@ if [ -z "$CONFIG_FILE" ]; then
 fi
 
 # Run the match_template script with the selected config file
+# NOTE: You may need to wrap the `python run_match_template.py $CONFIG_FILE`
+#       call within an `srun` command depending on your cluster configuration
+#       to properly expose the GPU devices to the process.
 echo "Running match_template with config file: $CONFIG_FILE"
 python run_match_template.py $CONFIG_FILE
 
 ```
 
+??? Caution "Making GPU devices discoverable within SLURM via srun"
+
+    Depending on your cluster configuration, you may need to wrap the `python run_match_template.py $CONFIG_FILE` command within an srun command to properly expose the GPU devices to the command.
+    For example, this might look like:
+
+    ```bash
+    srun --nodes=1 --ntasks=1 --cpus-per-task=8 --gres=gpu:L40:1 python run_match_template.py $CONFIG_FILE
+    ```
+
 ## Conclusion
 
 In this tutorial, we walked through how to set up batch processing of micrographs with Leopard-EM using a SLURM array job.