Advanced Lung Cancer Detection Using Optimized Data-Efficient Image Transformers

This repository contains code, experiments, and results for a research project focused on lung cancer detection and analysis using deep learning techniques. The project is being conducted under the guidance of Dr. Pawan Kumar Singh.

Project Overview

Lung cancer is one of the leading causes of cancer-related deaths worldwide. Early detection and accurate diagnosis are crucial for improving patient outcomes. In this project, we leverage state-of-the-art vision transformer models, specifically attention-based architectures, to analyse medical imaging data for lung cancer research.

Supervisor

• Dr. Pawan Kumar Singh
  Google Scholar Profile

Methods

We utilise the facebook/deit-base-distilled-patch16-224 attention-based model for image analysis. Vision Transformers (ViT) have shown promising results in various computer vision tasks, and we apply this architecture to our lung cancer dataset to evaluate its effectiveness.

Key Techniques Used

• Image Preprocessing Pipeline:

  • CLAHE (Contrast Limited Adaptive Histogram Equalization)
  • Bicubic Interpolation (Resizing to 224×224)
  • Gaussian Blurring
  • Otsu's Thresholding
  • Morphological Transformations (Erosion + Dilation)
  • Grayscale Normalization using AutoImageProcessor from HuggingFace

• Model:

  • facebook/deit-base-distilled-patch16-224

• Training Enhancements:

  • Stratified 5-Fold Cross-Validation
  • Learning Rate Scheduling (CosineAnnealingLR)
  • Early Stopping with Patience Tracking
  • Reduce LROnPlateau
  • Label Smoothing Regularization
  • DataLoader with multiprocessing (num_workers, pin_memory)

• Optimizers Used:

  • AdamW

• Metrics:

  • Accuracy
  • Classification Report
  • ROC Curve and AUC (per class)
  • Confusion Matrix

Data Efficient Image Transformer(DeiT) Results

IQ-OTH/NCCD Dataset

Classification report obtained using the DeiT-based model:

Confusion Matrix obtained using the DeiT-based model:

Highest current accuracy achieved using DeiT: 99.54%

LIDC-IDRI Dataset

Classification report obtained using the DeiT-based model:

Confusion Matrix obtained using the DeiT-based model:

Highest current accuracy achieved using DeiT: 95.64%

Acknowledgements

• This research is conducted under the supervision of Dr. Pawan Kumar Singh.
• Special thanks to the contributors and maintainers of the Hugging Face Transformers library for providing the pre-trained models and tools that facilitated this research.

Team Members

• Dr. Pawan Kumar Singh (Supervisor)
• Yash Raj Singh - Jadavpur University (Researcher)
• Ishant Sarthak Singh - Jadavpur University (Researcher)
• Aditya Anand - Jadavpur University (Researcher)

License

This project is for research and educational purposes only.