Merge pull request #78 from daniel-unyi-42/install_update

Installation updates and fixes

Author: EliHei2

Dockerfile

@@ -1,27 +1,37 @@
+# Base image with CUDA 12.1 runtime and cuDNN
 FROM nvidia/cuda:12.1.1-cudnn8-runtime-ubuntu22.04
 
+# Install essential system tools and Python
 RUN apt-get update -y && apt-get install -y --no-install-recommends \
     git \
     wget \
-    tmux \
-    vim \
-    htop \
-    zip \
+    curl \
     unzip \
+    htop \
+    vim \
     build-essential \
     python3 \
-    python3-pip && \
-    apt-get clean && \
+    python3-pip \
+    python3-venv \
+    && apt-get clean && \
     rm -rf /var/lib/apt/lists/*
 
-RUN python3 -m pip install --no-cache-dir --upgrade pip && \
-    pip install --no-cache-dir debugpy
+# Create a symbolic link to enable `python` instead of `python3`
+RUN ln -s /usr/bin/python3 /usr/bin/python
 
+# Install global python tools
+RUN python -m pip install --no-cache-dir --upgrade pip && \
+    pip install --no-cache-dir debugpy jupyterlab
+
+# Set working directory
 WORKDIR /workspace
 
+# Clone the repository and install segger with CUDA 12 support
 RUN git clone https://github.com/EliHei2/segger_dev.git /workspace/segger_dev && \
-    pip install -e "/workspace/segger_dev[cuda12,rapids12,cupy12,faiss]"
-
-EXPOSE 5678
+    pip install -e "/workspace/segger_dev[cuda12]"
 
+# Set environment variables
 ENV PYTHONPATH=/workspace/segger_dev/src:$PYTHONPATH
+
+# expose ports for debugpy and jupyterlab
+EXPOSE 5678 8888

README.md

@@ -2,7 +2,7 @@
 
 [![pre-commit.ci status](https://results.pre-commit.ci/badge/github/EliHei2/segger_dev/main.svg)](https://results.pre-commit.ci/latest/github/EliHei2/segger_dev/main)
 
-**Important note (Dec 2024)**: As segger is currently undergoing constant development we highly recommending installing segger directly via github.
+**Important note (Dec 2024)**: As segger is currently undergoing constant development, we highly recommend installing it directly via GitHub.
 
 **segger** is a cutting-edge tool for **cell segmentation** in **single-molecule spatial omics** datasets. By leveraging **graph neural networks (GNNs)** and heterogeneous graphs, segger offers unmatched accuracy and scalability.
 
@@ -51,75 +51,33 @@ segger tackles these with a **graph-based approach**, achieving superior segment
 
 ## Installation
 
-**Important note (Dec 2024)**: As segger is currently undergoing constant development we highly recommending installing segger directly via github.
-
-### Important: PyTorch Geometric Dependencies
-
-Segger **relies heavily** on PyTorch Geometric for its graph-based operations. One **must** install its dependencies (such as `torch-sparse` and `torch-scatter`) based on their system’s specifications, especially the **CUDA** and **PyTorch** versions.
-
-Please follow the official [PyTorch Geometric Installation Guide](https://pytorch-geometric.readthedocs.io/en/latest/install/installation.html) to install the correct versions of `torch-sparse`, `torch-scatter`, and other relevant libraries.
-
-Below is a quick guide for installing PyTorch Geometric dependencies for **torch 2.4.0**:
-
-#### For CUDA 11.x:
-
-```bash
-pip install pyg_lib torch_scatter torch_sparse torch_cluster torch_spline_conv -f https://data.pyg.org/whl/torch-2.4.0+cu121.html
-```
-
-#### For CUDA 12.x:
-
-```bash
-pip install pyg_lib torch_scatter torch_sparse torch_cluster torch_spline_conv -f https://data.pyg.org/whl/torch-2.4.0+cu118.html
-```
-
-Afterwards choose the installation method that best suits your needs.
+**Important note (Dec 2024)**: As segger is currently undergoing constant development, we highly recommend installing it directly via GitHub.
 
 ### GitHub Installation
 
 For a straightforward local installation from GitHub, clone the repository and install the package using `pip`:
 
-#### Clone the repo
-
 ```bash
 git clone https://github.com/EliHei2/segger_dev.git
-git cd segger_dev
+cd segger_dev
+pip install -e ".[cuda12]"
 ```
 
-
-#### Pip Installation (RAPIDS and CUDA 11)
-
-For installations requiring RAPIDS and CUDA 11 support, run:
-
-```bash
-pip install -e ".[rapids11]"
-```
-
-#### Pip Installation (RAPIDS and CUDA 12)
-
-For installations requiring RAPIDS and CUDA 12 support, run:
-
-```bash
-pip install -e ".[rapids12]"
-```
+segger requires **CUDA 11** or **CUDA 12** for GPU acceleration.
+You can find more detailed information in our **[Installation Guide](https://elihei2.github.io/segger_dev/installation/)**.
+To avoid dependency conflicts, we recommend installing segger in a virtual environment or using a containerized environment.
 
 ### Docker Installation
 
-Segger provides an easy-to-use Docker container for those who prefer a containerized environment. To pull the latest Docker image:
+We provide an easy-to-use Docker container for those who prefer a containerized environment. To pull and run the Docker image:
 
 ```bash
-docker pull danielunyi42/segger_dev:latest
+docker pull danielunyi42/segger_dev:cuda121
+docker run --gpus all -it danielunyi42/segger_dev:cuda121
 ```
 
-The Docker image comes with all dependencies packaged, including RAPIDS. It currently supports only CUDA 12.2, and we will soon release a version that supports CUDA 11.8.
-
-### Singularity Installation
-
-For users who prefer Singularity, you can pull the Docker image as follows:
-
-```bash
-singularity pull docker://danielunyi42/segger_dev:latest
-```
+The official Docker image comes with all dependencies pre-installed, including the CUDA toolkit, PyTorch, and CuPy.
+The current images support **CUDA 11.8** and **CUDA 12.1**, which can be specified in the image tag.
 
 ---
 

docs/installation.md

@@ -1,71 +1,141 @@
-## Segger Installation Guide
+## segger Installation Guide
 
-Select the appropriate installation method based on your requirements.
+segger provides multiple installation options to suit your requirements. You can install it using:
+
+- **Virtual environments** (recommended for most users)
+- **Containerized environments** (Docker or Singularity)
+- **Editable mode from GitHub** (for developers or users who want to modify the source code)
+
+!!! tip "Recommendation"
+    To avoid dependency conflicts, we recommend installing segger in a virtual environment or a container environment.
+
+segger requires **CUDA 11** or **CUDA 12** for GPU acceleration.
+
+### :snake: Installation in Virtual Environment
+
+#### Using `venv`
 
-=== ":rocket: Micromamba Installation"
 ```bash
-micromamba create -n segger-rapids --channel-priority 1 \
-  -c rapidsai -c conda-forge -c nvidia -c pytorch -c pyg \
-  rapids=24.10 python=3.* 'cuda-version>=12.0,<=12.1' jupyterlab \
-  'pytorch=*=*cuda*' 'pyg=*=*cu121' pyg-lib pytorch-sparse
-micromamba install -n segger-rapids --channel-priority 1 --file mamba_environment.yml
-micromamba run -n segger-rapids pip install --no-deps ./
+# Step 1: Create and activate the virtual environment.
+python3.10 -m venv segger-venv
+source segger-venv/bin/activate
+
+# Step 2: Install segger with CUDA support.
+pip install --upgrade pip
+pip install .[cuda12]
+
+# Step 3: Verify the installation.
+python --version
+pip show segger
+
+# step 4 [Optional]: If your system doesn't have a universally installed CUDA toolkit, you can link CuPy to PyTorch's CUDA runtime library.
+export LD_LIBRARY_PATH=$(pwd)/segger-venv/lib/python3.10/site-packages/nvidia/cuda_nvrtc/lib:$LD_LIBRARY_PATH
 ```
 
-=== ":snake: Conda Installation"
+#### Using `conda`
+
 ```bash
+# Step 1: Create and activate the conda environment.
 conda create -n segger-env python=3.10
 conda activate segger-env
-conda install pytorch torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia
-conda install pyg -c pyg
-pip install .
+
+# Step 2: Install segger with CUDA support.
+pip install --upgrade pip
+pip install .[cuda12]
+
+# Step 3: Verify the installation.
+python --version
+pip show segger
+
+# Step 4 [Optional]: If your system doesn't have a universally installed CUDA toolkit, you can link CuPy to PyTorch's CUDA runtime library.
+export LD_LIBRARY_PATH=$(conda info --base)/envs/segger-env/lib/python3.10/site-packages/nvidia/cuda_nvrtc/lib:$LD_LIBRARY_PATH
 ```
 
-=== ":whale: Docker Installation"
+#### How to Choose Between `[cuda11]` and `[cuda12]`
+1. **Check Your NVIDIA Driver Version**: Run `nvidia-smi`. Use `[cuda11]` for driver version ≥ 450.80.02 or `[cuda12]` for version ≥ 525.60.13.
+2. **Check for a CUDA Toolkit**: Run `nvcc --version`. If it outputs a CUDA version (11.x or 12.x), choose the corresponding `[cuda11]` or `[cuda12]`.
+3. **Default to PyTorch CUDA Runtime**: If CUDA toolkit is not installed, segger can use PyTorch's bundled CUDA runtime. You can link CuPy as shown in Step 4 of the venv/conda installation.
+
+
+### :whale: Installation in Container Environment
+
+####  Using `docker`
+
 ```bash
+# Step 1: Pull the official Docker image.
 docker pull danielunyi42/segger_dev:cuda121
+
+# Step 2: Run the Docker container with GPU support.
+docker run --gpus all -it danielunyi42/segger_dev:cuda121
 ```
 
-The Docker image comes with all required packages pre-installed, including PyTorch, RAPIDS, and PyTorch Geometric.
-The current images support CUDA 11.8 and CUDA 12.1, which can be specified in the image tag.
+The official Docker image comes with all dependencies pre-installed, including the CUDA toolkit, PyTorch, and CuPy.
+The current images support **CUDA 11.8** and **CUDA 12.1**, which can be specified in the image tag.
 
-For users who prefer Singularity:
+#### Using `singularity`
 
 ```bash
+# Step 1: Pull the official Docker image.
 singularity pull docker://danielunyi42/segger_dev:cuda121
+
+# Step 2: Run the Singularity container with GPU support.
+singularity exec --nv segger_dev_cuda121.sif
 ```
 
-=== ":octocat: Github Installation"
+The Singularity image is derived from the official Docker image and includes all pre-installed dependencies.
+
+#### :file_folder: Directory Mapping for Input and Output Data
+
+Directory mapping allows:
+
+- **Access to input data** (spatial transcriptomics datasets) from your local machine inside the container.
+- **Saving output data** (segmentation results and logs) generated by segger to your local machine.
+
+Setting up directory mapping is really easy:
+
+- **For Docker**:
 ```bash
-git clone https://github.com/EliHei2/segger_dev.git
-cd segger_dev
-pip install -e "."
+docker run --gpus all -it -v /path/to/local/data:/workspace/data danielunyi42/segger_dev:cuda121
 ```
 
-=== ":rocket: Pip Installation of RAPIDS with CUDA 11"
+- **For Singularity**:
 ```bash
-pip install "segger[rapids11]"
+singularity exec --nv -B /path/to/local/data:/workspace/data segger_dev_cuda121.sif
 ```
+  - Place your input datasets in `/path/to/local/data` on your host machine.
+  - Inside the container, access these datasets from `/workspace/data`.
+  - Save results to `/workspace/data`, which will be available in `/path/to/local/data` on the host machine.
+
+### :smirk_cat: Editable GitHub installation
+
+For developers or users who want to modify the source code:
 
-=== ":rocket: Pip Installation of RAPIDS with CUDA 12"
 ```bash
-pip install "segger[rapids12]"
+git clone https://github.com/EliHei2/segger_dev.git
+cd segger_dev
+pip install -e ".[cuda12]"
 ```
 
 !!! warning "Common Installation Issues"
-    - **Python Version**: Ensure you are using Python >= 3.10. Check your version with:
+    - **Python Version**: Ensure you are using Python >= 3.10. Check your Python version by running:
       ```bash
       python --version
       ```
-      If necessary, upgrade to the correct version.
+      If your version is lower than 3.10, please upgrade Python.
 
-    - **CUDA Compatibility (GPU)**: For GPU installations, ensure the correct CUDA drivers are installed. Verify your setup with:
+    - **CUDA Compatibility (GPU)**: For GPU installations, verify that your system has the correct NVIDIA drivers installed. Run:
       ```bash
       nvidia-smi
       ```
-      Ensure your CUDA version is compatible with the package.
+      Ensure that the displayed CUDA version is compatible with your selected `[cuda11]` or `[cuda12]` extra.
+
+        - Minimum driver version for **CUDA 11.x**: `450.80.02`
+        - Minimum driver version for **CUDA 12.x**: `525.60.13`
 
-    - **Permissions**: If you encounter permission errors, use the `--user` flag to install without admin rights:
+    - **Permissions**: If you encounter permission errors during installation, use the --user flag to install the package without requiring administrative privileges:
       ```bash
-      pip install --user .
+      pip install --user .[cuda12]
       ```
+      Alternatively, consider using a virtual environment (venv or conda) to isolate the installation.
+
+    - **Environment Configuration**: Ensure that all required dependencies are installed in your environment.

docs/notebooks/segger_tutorial.ipynb

@@ -506,17 +506,11 @@
     "    min_transcripts=5,\n",
     "    cell_id_col='segger_cell_id',\n",
     "    use_cc=False,\n",
-    "    knn_method='cuda',\n",
+    "    knn_method='kd_tree',\n",
     "    verbose=True,\n",
     ")\n"
    ]
   },
-  {
-   "cell_type": "markdown",
-   "id": "3aa5002c",
-   "metadata": {},
-   "source": []
-  },
   {
    "cell_type": "markdown",
    "id": "0a823035",