-
Notifications
You must be signed in to change notification settings - Fork 64
Add README to the remote inference example app #553
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 1 commit
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,137 @@ | ||||||
| # AI Remote Inference App - Spleen Segmentation | ||||||
|
|
||||||
| This example application demonstrates how to perform medical image segmentation using **Triton Inference Server** for remote inference calls. The app processes DICOM CT series to segment spleen anatomy using a deep learning model hosted on a remote Triton server. | ||||||
|
|
||||||
| ## Overview | ||||||
|
|
||||||
| This application showcases: | ||||||
| - **Remote inference using Triton Inference Server**: The app connects to a Triton server to perform model inference remotely rather than loading models locally, by sending and receiving input/output tensors corresponding to the model dimensions including channels | ||||||
| - **Triton client integration**: The built-in `TritonRemoteModel` class in the [triton_model.py](https://github.com/Project-MONAI/monai-deploy-app-sdk/blob/137ac32d647843579f52060c8f72f9d9e8b51c38/monai/deploy/core/models/triton_model.py) module contains and acts as a Triton inference client, communicating with an already loaded model network on the server. It supports the same API as the in-process model class (e.g., a loaded TorchScript model network), so that the application inference operator does not need to change when switching between in-process and remote inference | ||||||
MMelQin marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||||||
| - **Model metadata parsing**: Uses Triton's model folder structure which contains the `config.pbtxt` configuration file to extract model specifications including name, input/output dimensions, and other metadata. The parent folder of the Triton model folder needs to be used as the model path for the application | ||||||
MMelQin marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||||||
|
|
||||||
| ## Architecture | ||||||
|
|
||||||
| The application follows a pipeline architecture: | ||||||
|
|
||||||
| 1. **DICOM Data Loading**: Loads DICOM study from input directory | ||||||
| 2. **Series Selection**: Selects appropriate CT series based on configurable rules | ||||||
| 3. **Volume Conversion**: Converts DICOM series to 3D volume | ||||||
| 4. **Remote Inference**: Performs spleen segmentation via Triton Inference Server | ||||||
| 5. **Output Generation**: Creates DICOM segmentation and STL mesh outputs | ||||||
|
|
||||||
| ## Key Components | ||||||
|
|
||||||
| ### Triton Integration | ||||||
|
|
||||||
| The `SpleenSegOperator` leverages the `MonaiSegInferenceOperator` which: | ||||||
| - Uses the loaded model network which in turn acts as a **Triton inference client** and connects to a remote Triton Inference Server that actually serves the named model | ||||||
| - Handles preprocessing and postprocessing transforms | ||||||
| - No explicit remote inference logic is required in these two operators | ||||||
|
|
||||||
| ### Model Configuration Requirements | ||||||
|
|
||||||
| The application requires a Triton model folder which contains a **Triton model configuration file** (`config.pbtxt`) to be present on the application side, and the parent path to the model folder will be used as the model path for the application. This example application has the following model folder structure: | ||||||
|
|
||||||
| ``` | ||||||
| models_client_side/spleen_ct/config.pbtxt | ||||||
| ``` | ||||||
|
|
||||||
| The path to `models_client_side` is the model path for the application while `spleen_ct` is the folder of the named model with the folder name matching the model name. The model name in the `config.pbtxt` file is therefore intentionally omitted. | ||||||
|
||||||
| The path to `models_client_side` is the model path for the application while `spleen_ct` is the folder of the named model with the folder name matching the model name. The model name in the `config.pbtxt` file is therefore intentionally omitted. | |
| The path to `models_client_side` serves as the model path for the application, while `spleen_ct` is the named model folder with its name matching the model name. Therefore, the model name is intentionally omitted from the `config.pbtxt` file. |
Uh oh!
There was an error while loading. Please reload this page.