MRI Helper

System Overview

The MRI Helper is an experimental asynchronous medical imaging application that processes NIfTI files for visualization and segmentation using deep learning models. The SegResNet model used for segmentation is trained by the MONAI community. The primary use cases include deployment within clinics or hospitals, either on-premise locally or on AWS within private subnets to ensure sensitive data privacy protection.

Docker Compose Development Architecture

%%{init: {'theme':'base', 'themeVariables': { 'fontSize': '18px', 'fontFamily': 'Arial, sans-serif', 'primaryColor': '#ffffff', 'primaryTextColor': '#000000', 'primaryBorderColor': '#000000', 'lineColor': '#000000', 'secondaryColor': '#ffffff', 'tertiaryColor': '#ffffff'}}}%%
graph TD
    %% Client Layer
    subgraph Client["<b>Client Layer</b>"]
        User["<b>User/Doctor</b>"]
        Browser["<b>Web Browser</b>"]
    end
    
    %% Application Layer
    subgraph App["<b>Application Layer</b>"]
        Frontend["<b>React Frontend</b><br/>Port 8000<br/>VTK.js 3D Viewer<br/>File Upload UI"]
    end
    
    %% API Layer
    subgraph API["<b>API Layer</b>"]
        FastAPI["<b>FastAPI Backend</b><br/>Port 8001<br/>MRI Master API<br/>File Upload Handler<br/>SSE Events"]
    end
    
    %% Processing Layer
    subgraph Processing["<b>Processing Layer</b>"]
        Celery["<b>Celery Worker</b><br/>ONNX Models<br/>SegResNet<br/>NIfTI Processing<br/>VTK Conversion"]
    end
    
    %% Infrastructure Layer
    subgraph Infrastructure["<b>Infrastructure Layer</b>"]
        RabbitMQ["<b>RabbitMQ</b><br/>Message Queue<br/>Port 5672/15672"]
        Redis["<b>Redis</b><br/>Cache & Pub/Sub<br/>Port 6379<br/>Task Status"]
        MinIO["<b>MinIO</b><br/>File Storage<br/>Port 9000/9001<br/>NIfTI & VTI Files"]
    end
    
    %% Data Flow - File Upload
    User -->|"<b>1. Upload NIfTI</b>"| Browser
    Browser -->|"<b>2. POST /workflow/start</b>"| Frontend
    Frontend -->|"<b>3. Upload File</b>"| FastAPI
    FastAPI -->|"<b>4. Save to MinIO</b>"| MinIO
    FastAPI -->|"<b>5. Trigger Task</b>"| Celery
    FastAPI -->|"<b>6. Store Task ID</b>"| Redis
    
    %% Data Flow - Processing
    Celery -->|"<b>7. Get Task</b>"| RabbitMQ
    Celery -->|"<b>8. Load NIfTI</b>"| MinIO
    Celery -->|"<b>9. Process with ONNX</b>"| Celery
    Celery -->|"<b>10. Save VTI</b>"| MinIO
    Celery -->|"<b>11. Update Status</b>"| Redis
    
    %% Data Flow - Real-time Updates
    Redis -->|"<b>12. SSE Events</b>"| FastAPI
    FastAPI -->|"<b>13. Progress Updates</b>"| Frontend
    Frontend -->|"<b>14. 3D Visualization</b>"| User
    
    %% Styling with larger fonts and spacing
    classDef clientClass fill:#e1f5fe,stroke:#01579b,stroke-width:3px,font-size:16px,font-weight:bold
    classDef appClass fill:#f3e5f5,stroke:#4a148c,stroke-width:3px,font-size:16px,font-weight:bold
    classDef apiClass fill:#e8f5e8,stroke:#1b5e20,stroke-width:3px,font-size:16px,font-weight:bold
    classDef processingClass fill:#fff3e0,stroke:#e65100,stroke-width:3px,font-size:16px,font-weight:bold
    classDef infraClass fill:#e0f2f1,stroke:#004d40,stroke-width:3px,font-size:16px,font-weight:bold
    
    class User,Browser clientClass
    class Frontend appClass
    class FastAPI apiClass
    class Celery processingClass
    class RabbitMQ,Redis,MinIO infraClass

Loading

File Processing Workflow

1. File Upload Process

1. User uploads NIfTI file (.nii.gz) through the React frontend interface
2. Frontend sends POST request to the /workflow/start endpoint
3. FastAPI validates file type and size constraints
4. File is saved to MinIO storage with a unique UUID identifier
5. Celery task is triggered for background processing
6. Task ID is stored in Redis for progress tracking

2. Processing Pipeline

1. Celery worker retrieves the task from RabbitMQ queue
2. Loads NIfTI file from MinIO storage system
3. Preprocesses the volume (normalization, cropping operations)
4. Executes ONNX model (SegResNet for brain tumor segmentation)
5. Converts results to VTK format (.vti files)
6. Saves processed files back to MinIO storage
7. Updates task status in Redis for real-time monitoring

3. Real-time Updates

1. Redis pub/sub system publishes status updates
2. FastAPI SSE endpoint streams events to the frontend
3. Frontend receives progress updates in real-time
4. VTK.js renders 3D visualization when processing is complete
5. User can download segmentation results as NIfTI files

Component Details

Frontend Layer

• Technology Stack: React 19 + TypeScript + Vite
• 3D Visualization: VTK.js for medical imaging rendering
• UI Components: Radix UI + Tailwind CSS
• Key Features: File upload interface, real-time progress tracking, interactive 3D volume rendering

API Layer

• Framework: FastAPI (Python)
• Port: 8001
• Core Features:
  • NIfTI file upload handling (.nii.gz format)
  • Server-Sent Events (SSE) for real-time progress updates
  • CORS middleware for cross-origin request handling
  • GZip compression for optimized data transfer
  • Health check endpoints for monitoring

Task Processing Layer

• Framework: Celery with RabbitMQ message broker
• AI Models: ONNX SegResNet models (128x128x128 and 208x208x144)
• Processing Tasks:
  • process_volume: Converts NIfTI files to NumPy arrays
  • segment_volume: Performs brain tumor segmentation using AI
  • prepare_3d_data: Converts processed data to VTK format for visualization

Storage Layer

• Storage Solution: MinIO (S3-compatible object storage)
• Supported File Types: NIfTI, VTK (.vti), NumPy (.npy)
• Key Features: Automatic file cleanup, secure file serving, persistent storage

Message Queue & Cache

• RabbitMQ: Task queue management and worker coordination
• Redis:
  • Task result backend storage
  • Pub/Sub system for real-time status updates
  • Session and cache storage

Deployment Architecture

Development Environment (Docker Compose)

• All services containerized for consistent development
• Local MinIO storage for file management
• Simplified setup for rapid development iteration

Data Flow Summary

1. File Upload: User uploads NIfTI file through React frontend interface
2. Processing Initiation: FastAPI saves file and triggers Celery workflow
3. AI Segmentation: Celery workers process files using ONNX models
4. Format Conversion: Results converted to VTK format for visualization
5. 3D Rendering: Frontend renders interactive 3D volumes using VTK.js
6. Result Download: Users can download segmentation results as NIfTI files

Key Features

• Real-time Progress Tracking: Server-Sent Events for live status updates
• Scalable Processing: Celery workers can be scaled horizontally for performance
• Medical Imaging Standards: Full support for NIfTI medical imaging format
• Interactive 3D Visualization: Advanced 3D rendering of MRI volumes
• AI-Powered Segmentation: Automated brain tumor segmentation using SegResNet
• Secure Deployment: Designed for on-premise and private cloud deployment