Enhance model loading and configuration handling in MonaiBundleInferenceOperator

- Introduced helper functions to streamline model loading from directory-based bundles and ensure proper configuration parsing.
- Improved error handling for missing model files and configuration metadata.
- Refactored the get_bundle_config function to utilize the new helper for better code organization and readability.
- Updated the NiftiDataLoader to handle image dimensionalities more effectively using SimpleITK metadata.

Signed-off-by: Victor Chang <[email protected]>

Author: mocsharp

monai/deploy/operators/monai_bundle_inference_operator.py

@@ -82,20 +82,34 @@ def convert_and_save(self, nii_path):
         image = image_reader.Execute()
         image_np = SimpleITK.GetArrayFromImage(image)
 
-        # Handle different dimensionalities properly
-        if image_np.ndim == 3:
-            # Standard 3D volume: transpose from (z, y, x) to (x, y, z)
-            image_np = np.transpose(image_np, [2, 1, 0])
-        elif image_np.ndim == 4:
-            # 4D volume with channels: (c, z, y, x) to (c, x, y, z)
-            image_np = np.transpose(image_np, [0, 3, 2, 1])
-        elif image_np.ndim == 2:
-            # 2D slice: transpose from (y, x) to (x, y)
-            image_np = np.transpose(image_np, [1, 0])
+        # Get image metadata to properly distinguish between different image types
+        spatial_dims = image.GetDimension()  # Actual spatial dimensions (2D, 3D, etc.)
+        num_components = image.GetNumberOfComponentsPerPixel()  # Components/channels per pixel
+        
+        self._logger.debug(f"Image spatial dimensions: {spatial_dims}, components per pixel: {num_components}, array shape: {image_np.shape}")
+
+        # Handle different dimensionalities properly using SimpleITK metadata
+        if spatial_dims == 2:
+            if num_components == 1:
+                # 2D grayscale: transpose from (y, x) to (x, y)
+                image_np = np.transpose(image_np, [1, 0])
+            else:
+                # 2D with multiple components/channels: transpose from (y, x, c) to (x, y, c)
+                # SimpleITK stores multi-component 2D images as (y, x, c)
+                image_np = np.transpose(image_np, [1, 0, 2])
+        elif spatial_dims == 3:
+            if num_components == 1:
+                # 3D grayscale volume: transpose from (z, y, x) to (x, y, z)
+                image_np = np.transpose(image_np, [2, 1, 0])
+            else:
+                # 3D volume with multiple components: transpose from (z, y, x, c) to (x, y, z, c)
+                # SimpleITK stores multi-component 3D images as (z, y, x, c)
+                image_np = np.transpose(image_np, [2, 1, 0, 3])
         else:
-            # For other dimensions, log a warning and return as-is
+            # For other spatial dimensions, log a warning and return as-is
             self._logger.warning(
-                f"Unexpected {image_np.ndim}D NIfTI file shape {image_np.shape} from {nii_path}, returning without transpose"
+                f"Unexpected {spatial_dims}D spatial image with {num_components} components per pixel, "
+                f"array shape {image_np.shape} from {nii_path}, returning without transpose"
             )
 
         return image_np

monai/deploy/operators/nii_data_loader_operator.py

@@ -52,6 +52,12 @@ uv run pg gen MONAI/spleen_ct_segmentation --output my_app
 uv run pg run my_app --input /path/to/test/data --output ./results
 ```
 
+> [!NOTE]  
+> Input can be NIfTI files (.nii, .nii.gz) or DICOM series directories Refer to [config.yaml](tools/pipeline-generator/pipeline_generator/config/config.yaml) for details.
+
+> [!NOTE]
+> For DICOM input types, refer to the model documentation for DICOM series processing limitations.
+
 ### List Available Models
 
 List all models from configured endpoints:
@@ -107,6 +113,7 @@ Options:
 - `--format`: Input/output data format (optional): auto, dicom, or nifti (default: auto)
   - For tested models, format is automatically detected from configuration
   - For untested models, attempts detection from model metadata
+  - **DICOM Limitation**: Refer to the model documentation for multi-series support.
 - `--force, -f`: Overwrite existing output directory
 
 Generate with custom application class name:
@@ -196,7 +203,7 @@ output/
 ├── README.md             # Documentation
 ├── operators/            # Custom operators (if needed)
 │   └── nifti_operators.py
-└── model/                 # Downloaded MONAI Bundle
+└── model/                 # Downloaded model files
     ├── configs/
     ├── models/
     └── docs/

tools/pipeline-generator/README.md

@@ -13,9 +13,9 @@ By bridging the gap between model packaging and application deployment, this pro
 
 ## **Background**
 
-The **MONAI Bundle** is a standardized format designed to package deep learning models for medical imaging. It includes the model weights, metadata, transforms, and documentation needed to make the model self-contained and portable.
+The **[MONAI Bundle](https://docs.monai.io/en/latest/mb_specification.html#archive-format)** is a standardized format designed to package deep learning models for medical imaging. It includes the model weights, metadata, transforms, and documentation needed to make the model self-contained and portable.
 
-The **Holoscan SDK** is NVIDIA’s real-time streaming application SDK for AI workloads in healthcare and edge devices. The **MONAI Deploy App SDK** is designed for building composable inference applications, often used in radiology and imaging pipelines.
+The **Holoscan SDK** is NVIDIA’s real-time streaming application SDK for AI workloads in healthcare and edge devices. The **MONAI Deploy App SDK** builds on top of the Holoscan SDK, adding medical imaging domain-specific operators and functionalities to enable the creation of composable inference applications, particularly for radiology and imaging pipelines.
 
 As of MONAI Core, bundles can also be exported in a **Hugging Face-compatible format**, which allows sharing through the Hugging Face Model Hub. Supporting this format increases reach and adoption.
 
@@ -28,9 +28,8 @@ As of MONAI Core, bundles can also be exported in a **Hugging Face-compatible fo
 
 ## **Assumptions/Limitations**
 
-- The tool does not convert input formats given that each model may expect a different type of input
 - The tool does not convert output formats given that each model may output a different type of result
-- The tool supports only torchscript (ts) models
+- The generated application does not convert input formats given that each model may expect a different type of input
 
 ## **Scope**
 
@@ -55,7 +54,7 @@ This project includes:
   - PyTorch state dict (.pt): Loaded with model definition code/config
   - Hugging Face-compatible: Recognized and unpacked with reference to Hugging Face conventions
 - **AI Inference Operator Integration**
-  - Python and C++ support for TorchScript/ONNX-based inference
+  - Python support for TorchScript/ONNX-based inference
   - Auto-configures model inputs/outputs based on network_data_format
   - Embeds optional postprocessing like argmax, thresholding, etc.
 - **Preprocessing/Postprocessing Pipeline**
@@ -70,7 +69,6 @@ This project includes:
 - **Tooling**
   - Command-line tool to:
     - Validate MONAI Bundles
-    - Convert .pt → .ts/.onnx
     - Generate MONAI Deploy and Holoscan-ready configs
     - Extract and display metadata (task, inputs, author, etc.)
 

tools/pipeline-generator/docs/design.md

@@ -168,21 +168,21 @@ Run directly using Python as shown above.
 
 ### Container Deployment
 
-Package the application as a container using Holoscan CLI:
+Package the application as a container using MONAI Deploy CLI:
 
 ```bash
 # Package for x64 workstations
-holoscan package app -c app.yaml --platform linux/amd64 -t {{ model_short_name|lower }}:latest
+monai-deploy package app -c app.yaml --platform linux/amd64 -t {{ model_short_name|lower }}:latest
 
 # Package for IGX Orin devkits
-holoscan package app -c app.yaml --platform linux/arm64 -t {{ model_short_name|lower }}:latest
+monai-deploy package app -c app.yaml --platform linux/arm64 -t {{ model_short_name|lower }}:latest
 ```
 
 Run the containerized application:
 
 ```bash
-# Using Holoscan CLI
-holoscan run -i /path/to/input -o /path/to/output {{ model_short_name|lower }}:latest
+# Using MONAI Deploy CLI
+monai-deploy run -i /path/to/input -o /path/to/output {{ model_short_name|lower }}:latest
 
 # Or using Docker directly
 docker run -v /path/to/input:/input -v /path/to/output:/output {{ model_short_name|lower }}:latest

tools/pipeline-generator/pipeline_generator/templates/README.md.j2

@@ -56,14 +56,14 @@ def __init__(
         self._case_sensitive = bool(case_sensitive)
 
         # State tracking
-        self._files: list[Path] = []
+        self._files: List[Path] = []
         self._current_index = 0
 
         super().__init__(fragment, *args, **kwargs)
 
     def _find_files(self) -> List[Path]:
         """Find all files matching the specified extensions."""
-        files = []
+        files: List[Path] = []
 
         # Normalize extensions for comparison
         if not self._case_sensitive:

tools/pipeline-generator/pipeline_generator/templates/operators/generic_directory_scanner_operator.py

@@ -4,7 +4,6 @@
 # MONAI Deploy App SDK and dependencies
 monai-deploy-app-sdk>=3.0.0
 
-
 # Required by MONAI Deploy SDK (always needed)
 pydicom>=2.3.0 # Required by MONAI Deploy SDK even for NIfTI apps
 highdicom>=0.18.2     # Required for DICOM segmentation support