Merge branch 'develop' into pr/1087

Author: AdvancedImagingUTSW

.gitignore

@@ -296,3 +296,5 @@ test.n5
 test_dir
 test.xml
 *_log.txt
+/docs/source/05_reference/_autosummary
+/docs/source/05_reference/_autosummary

docs/source/01_getting_started/01_quick_start/01_quick_start.rst

@@ -0,0 +1,60 @@
+.. _Quick_Start_Guide:
+
+=================================
+Quick Start Guide
+=================================
+
+This quick start guide covers how to install **navigate**, launch it in synthetic hardware mode to confirm it is working, and save an image to disk.
+
+Installation
+------------
+
+1. *Install Miniconda.* Download and install Miniconda from the `official website <https://docs.conda.io/en/latest/miniconda.html>`_.
+
+2. *Create and Activate a Conda Environment.* Launch a Miniconda Prompt (or a Terminal on MacOS) and enter the following.
+
+   .. code-block:: console
+
+      (base) conda create -n navigate python=3.9.7
+      (base) conda activate navigate
+
+3. *Install* **navigate**.
+
+   .. code-block:: console
+
+      (navigate) pip install navigate-micro
+
+4. *Launch* **navigate** *in synthetic hardware mode.* This will allow you to test its functionality without actual hardware.
+
+   .. code-block:: console
+
+      (navigate) navigate -sh
+
+Saving a Z-Stack to Disk
+------------------------
+
+To save an image to disk, follow these steps:
+
+* Launch **navigate** in synthetic hardware mode as described above.
+* Next to the :guilabel:`Acquire` button on the upper left, make sure that the acquisition mode in the dropdown is set to "Continuous Scan".
+* Press :guilabel:`Acquire` (:kbd:`ctrl + enter`) and confirm that a synthetic noise image is displayed in the :guilabel:`Camera View` window.
+
+.. Note::
+    At least one channel must be selected (checkbox marked) in the :guilabel:`Channel Settings` window, and all of the parameters (e.g., :guilabel:`Power`) in that row must be populated.
+
+* Select the :guilabel:`Channels` tab. In the :guilabel:`Stack Acquisition Settings` window in this tab, press the :guilabel:`Set Start Pos/Foc` button. This specifies the starting ``Z`` and ``F`` (e.g., Focus) positions for the stack acquisition.
+* Select the :guilabel:`Stage Control` tab (:kbd:`ctrl + 3`), move the ``Z`` stage to the desired position (e.g., ``100`` μm), go back to the :guilabel:`Channels` tab (:kbd:`ctrl + 1`), and press the :guilabel:`Set End Pos/Foc` button. This specifies the ending ``Z`` and ``F`` positions for the stack acquisition.
+* In the :guilabel:`Stack Acquisition Settings` frame, you can now adjust the step size, which determines the number of slices in a z-stack.
+* In the :guilabel:`Timepoint Settings` window, select :guilabel:`Save Data` by marking the checkbox. If the number of timepoints is set to ``1``, only a single stack will be acquired.
+* Next to the :guilabel:`Acquire` button on the upper left, change the acquisition mode in the dropdown to "Z-Stack", and press :guilabel:`Acquire` (:kbd:`ctrl + enter`).
+* A :guilabel:`File Saving Dialog` popup window will appear.
+    * With the exception of :guilabel:`Notes`, all fields must be populated. Any spaces in the fields will be replaced with an underscore.
+    * :guilabel:`Notes` is saved with the metadata, and can be useful for describing the experiment.
+    * :guilabel:`Solvent` is useful for tissue clearing experiments.
+    * :guilabel:`File Type` can be set to :guilabel:`.TIFF`, :guilabel:`OME-TIFF`, :guilabel:`H5`, or :guilabel:`N5`. The latter two options are pyramidal file formats that are best used for large datasets and are immediately compatible with `BigDataViewer <https://imagej.net/plugins/bdv/>`_ and `BigStitcher <https://imagej.net/plugins/bigstitcher/index>`_.
+    * Press :guilabel:`Acquire` to begin the acquisition.
+    * Once complete, the data can be opened using standard image processing software such as `Fiji <https://imagej.net/software/fiji/>`_.
+
+.. image:: images/save_dialog.png
+    :align: center
+    :alt: File Saving Dialog

docs/source/user_guide/images/save_dialog.png renamed to docs/source/01_getting_started/01_quick_start/images/save_dialog.png

@@ -0,0 +1,149 @@
+=====================
+Software Installation
+=====================
+
+.. _software_installation:
+
+Computer Specifications
+=======================
+
+Below are the recommended specifications for **navigate**.
+
+Operating System Compatibility
+------------------------------
+
+.. important::
+   **navigate** is developed for use on Windows-based systems. This is due to the compatibility of device drivers for various microscope hardware components, such as cameras, stages, and data acquisition cards, which are predominantly designed for the Windows environment. The software is only partially tested on MacOS and Linux systems. Users considering the use of **navigate** software on Linux should proceed with caution and be prepared for potential compatibility issues. For optimal performance and compatibility, it is strongly recommended to run **navigate** on a Windows machine.
+
+.. _computer_considerations:
+
+Computer Considerations
+-----------------------
+
+**navigate** will run on a mid-range laptop with at least 8 GB of RAM and a processor with two cores. Most of its operations are undemanding. Saving data at a reasonable rate, however, will require an SSD. The hardware configuration for an example microscope control machine is shown below.
+
+.. important::
+   Scientific cameras are capable of rapidly generating large amounts of high-resolution data. As such, the read/write speed of the data storage device is critical for smooth operation of the software. For example, for a standard Hamamatsu camera with a 2048 x 2048 sensor, operating at 16-bit depth and 20 frames per second, the data save rate is approximately ~167 MB/s. While such capabilities are well within the capabilities of modern SSDs, they are beyond the capabilities of most HDDs. Therefore, it is recommended to use a fast SSD for data saving operations.
+
+.. collapse:: Example Computer Configuration
+
+    - *Base Platform*
+        - **Product Name**: `Colfax SX6300 Workstation <https://www.colfax-intl.com/workstations/sx6300>`_
+        - **Colfax Part #**: CX-116263
+
+    - *Primary and Secondary CPU*
+        - **CPU Model**: Intel Xeon Silver 4215R
+        - **Configuration**: 8 Cores / 16 Threads
+        - **Frequency**: 3.2 GHz
+        - **Cache**: 11 MB
+        - **TDP**: 130W
+        - **Memory Support**: 2400 MHz
+
+    - *Memory*
+        - **Type**: Registered ECC DDR4
+        - **Speed**: 3200 MHz
+        - **Configuration**: 16 GB per socket, 8 sockets per CPU
+        - **Total RAM**: >64 GB (recommended)
+
+    - *Operating System Drive*:
+        - **Type**: M.2 NVMe SSD
+        - **Model**: Micron 7450 Max
+        - **Capacity**: 800 GB
+        - **Endurance**: 3 DWPD
+
+    - *Primary Data Drive*:
+        - **Type**: NVMe SSD
+        - **Model**: Samsung PM9A3
+        - **Capacity**: 7.68 TB
+        - **Interface**: U.2 Gen4
+
+    - *Secondary Data Drive*:
+        - **Type**: SATA HDD
+        - **Model**: Seagate Exos X20
+        - **Capacity**: 20 TB
+        - **Speed**: 7200 RPM
+        - **Cache**: 256 MB
+        - **Interface**: SATA 6.0 Gb/s
+
+    - *Video Card*
+        - **Model**: PNY nVidia T1000
+        - **Memory**: 4 GB
+        - **Interface**: PCI Express
+
+    - *Network Interface*
+        - **Model**: Intel X710-T2L RJ45 Copper
+        - **Type**: Dual Port 10GbE
+        - **Interface**: PCI-E x 8
+
+    .. note::
+       The specifications listed are based on an example system configuration and can be adjusted based on specific needs and availability.
+
+---------------------
+
+Quick install
+=============
+
+**Setup your Python Environment**
+
+Head over to the `miniconda website <https://docs.conda.io/en/latest/miniconda.html>`_ and install the appropriate version based on your operating system.
+
+.. tip::
+    It is also handy to have the `conda cheatsheet <https://docs.conda.io/projects/conda/en/4.6.0/_downloads/52a95608c49671267e40c689e0bc00ca/conda-cheatsheet.pdf>`_ open when first using miniconda to get accustomed to the commands available.
+
+* Windows: Use the Windows taskbar search to find ``Anaconda Prompt (Miniconda3)``. Given how frequently you will use this, we recommend pinning it to your taskbar. * Linux/Mac: Open a Terminal.
+
+**Create a Python environment called navigate that uses Python version 3.9.7**
+
+.. code-block:: console
+
+    (base) MyComputer ~ $ conda create -n navigate python=3.9.7
+
+**Activate the navigate environment**
+
+.. code-block:: console
+
+    (base) MyComputer ~ $ conda activate navigate
+
+The active environment is shown in parentheses on the far-left. Originally, we were in the miniconda ``(base)`` environment. After activating the navigate environment, it should now show ``(navigate)``.
+
+**Install navigate via pip**
+
+To install the latest stable release of **navigate**, run the following command:
+
+.. code-block:: console
+
+    (navigate) MyComputer ~ $ pip install navigate-micro
+
+To install the bleeding edge version of **navigate**, run the following command:
+
+.. code-block:: console
+
+    (navigate) MyComputer ~ $ pip install git+https://github.com/TheDeanLab/navigate.git
+
+**Run navigate software**
+
+.. code-block:: console
+
+    (navigate) MyComputer ~ $ navigate
+
+.. note::
+    If you are running the software on a computer that is not connected to microscope hardware, you can add the flag ``-sh`` (``--synthetic-hardware``) to launch the program:
+
+    .. code-block:: console
+
+        navigate -sh
+
+Launching **navigate**
+======================
+
+Open an ``Anaconda Prompt (Miniconda3)`` and enter the following.
+
+.. code-block:: console
+
+    (base) conda activate navigate
+    (navigate) navigate
+
+.. tip::
+    If you are running Windows, you can create a desktop shortcut to **navigate** by right-clicking the Desktop, navigating to New and then Shortcut and entering ``%windir%\system32\cmd.exe "/c" C:\path\to\miniconda\Scripts\activate.bat navigate && navigate`` into the location text box.
+
+    This provides a convenient executable shortcut to launch the software, which is advantageous for users who are not comfortable with the command line.

docs/source/01_getting_started/02_software_installation/02_software_installation.rst

@@ -2,21 +2,13 @@
 I Want To...
 ============
 
-This section contains how-to documents organized into three levels of difficulty:
-beginner, intermediate, and advanced.
+This section contains how-to documents organized into three levels of difficulty: beginner, intermediate, and advanced.
 
-- The beginner how-to is intended for users who have little to no experience with
-  computer programming and simply wish to acquire an image.
+- The beginner how-to is intended for users who have little to no experience with computer programming and simply wish to acquire an image.
+- The intermediate how-to is intended for users who want to learn how to use the graphical user interface to create their own smart acquisition workflows.
+- The advanced how-to is intended for users who have experience with computer programming and wish to extend the functionality of **navigate** by adding new devices or writing their own acquisition features.
 
-- The intermediate how-to is intended for users who want to learn how to use the
-  graphical user interface to create their own smart acquisition workflows.
-
-- The advanced how-to is intended for users who have experience with computer
-  programming and wish to extend the functionality of **navigate** by adding new
-  devices or writing their own acquisition features.
-
-Through this tiered approach, we hope to provide a gentle introduction to the software
-while also reaching the maximum number of users.
+Through this tiered approach, we hope to provide a gentle introduction to the software while also reaching the maximum number of users.
 
 .. toctree::
    :maxdepth: 1

docs/source/i_want_to.rst renamed to docs/source/01_getting_started/03_i_want_to/03_i_want_to.rst

@@ -0,0 +1,101 @@
+======================================
+ Add a New Hardware Device (Advanced)
+======================================
+
+**navigate** includes several standard hardware device types. These include:
+
+- Cameras
+- Data Acquisition Cards
+- Filter Wheels
+- Galvo Scanners
+- Lasers
+- Deformable Mirrors
+- Remote Focusing Systems
+- Shutters
+- Stages
+- Zoom Devices
+
+To add a new piece of hardware to one of these device types requires knowledge about the software's device abstraction layer. Here’s a detailed guide to help you integrate a new ``CustomStage`` device into **navigate**. The same principles work for other device types.
+
+.. note::
+    A strong knowledge of Python and object-oriented programming is required to integrate new hardware devices into **navigate**.
+
+----------------
+
+What is the Device Abstraction Layer?
+-------------------------------------
+
+To ensure compatibility and extendability, **navigate** utilizes a device abstraction layer, which allows the same commands to be used across different hardware devices. For example, all stages in **navigate** are programmed to include the `stop()` command, which can be used to stop the stage's movement. When someone hits the :guilabel:`Stop Stage` button on the GUI, this action is relayed from the ``Controller`` to the ``Model`` and ultimately the ``CustomStage``, which communicates with the hardware in a device-specific format to stop the stage's movement.
+
+--------------
+
+Device Integration Approaches
+-----------------------------
+
+There are two primary approaches to integrating new hardware into **navigate**:
+
+- **Plugin**: If you want to continue to work with an up-to-date version of **navigate**, consider integrating your new hardware device as a plugin. This allows you to pull updates from the main repository without losing your custom hardware integration. It also allows you to integrate non-standard device types. Learn more about the plugin architecture :ref:`here <plugin>`, and how to write a custom plugin :ref:`here <advanced>`.
+- **Fork**: Alternatively, you can fork the **navigate** repository on GitHub and modify it directly. This is useful for custom, in-house developments. In select circumstances, you can contribute your changes back to the main repository through a pull request. Please contact the **navigate** development team for guidance on this approach.
+
+--------------
+
+Device Class Creation
+---------------------
+- New hardware devices must have a corresponding device class in navigate. To ensure consistency and reduce redundancy, each device must inherit the appropriate abstract base class. For instance, a ``CustomStage`` device would inherit from ``StageBase``.
+- Classes should follow CamelCase naming conventions and reflect the device they control (e.g., ``NewportStage`` for a stage from the manufacturer Newport).
+- Place the new device class within the appropriate device directory, `src/navigate/model/devices/`.
+- Place related API or hardware documentation within the appropriate manufacturer directory, typically under `src/navigate/model/devices/APIs/`.
+
+--------------
+
+Establish Device Communication
+------------------------------
+
+- Each device requires a unique method to initialize a connection, which may involve APIs, serial communication, or other protocols. This method should be separate from the device class and is typically located at the beginning of the device file.
+- For example, a function named `build_custom_stage_connection()` would handle the connection setup for ``CustomStage`` class.
+- By separating the connection setup from the device class, you can easily interact with the hardware device outside of the larger **navigate** ecosystem, which can be useful for debugging and testing (e.g., within a Jupyter notebook).
+
+--------------
+
+Device Class Constructor
+------------------------
+
+- The constructor for the device class (`__init__`) should accept parameters for the `microscope_name`, `device_connection`, `configuration_file`, and an optional `device_ID` (useful when multiple instances of the same device are used).
+- The constructor should load and enforce device settings from the `configuration_file`. For a new stage, this could be defining the axes mapping between **navigate** and the device, `{x:'X', y:'Y', z:'Z'}`.
+- Ensure the device class uses the connection established by your `build_custom_stage_connection` method.
+
+--------------
+
+Device Class Methods
+--------------------
+
+- Implement any necessary device-specific methods within your device class.
+- Essential methods are inherited from the base class (e.g., ``StageBase`` for the ``CustomStage``), but you can override them or add new methods as needed for specialized functionality.
+
+--------------
+
+Startup and Configuration
+-------------------------
+
+- Utilize or modify methods within `src/navigate/model/device_startup_functions` to configure and start your device upon system initialization.
+- These functions should handle configuration parsing and the device communication setup.
+- Implement a retry mechanism, such as `auto_redial`, to handle communication issues robustly, attempting multiple times before failing.
+
+--------------
+
+Integration with Microscope Object Configurations
+-------------------------------------------------
+
+- Each microscope configuration in **navigate** that uses the new device should receive a reference to the established communication object.
+- This setup is defined in the *configuration.yaml* and handled within the `device_startup_functions`, ensuring each configuration has access to the necessary hardware.
+
+--------------
+
+Testing and Validation
+----------------------
+- Thoroughly test the new hardware integration to ensure it functions correctly within navigate, across all intended use cases and configurations.
+- The naming convention for test files is: `test_` + module name.
+- Device test files are located in `test\model\devices\`
+- Device testing utilizes the `pytest` package.
+
+By following these steps, you can effectively integrate new hardware into the **navigate** platform, enhancing its functionality and ensuring it meets specific experimental needs.