SCR-Progression: Retinal Layer Annotation Prediction in OCT B-Scan Images

Project Overview

This repository contains deep learning models and comprehensive analysis tools for retinal layer prediction in Optical Coherence Tomography (OCT) B-scan images. The primary focus is on predicting the Inner Limiting Membrane (ILM) and Bruch's Membrane (BM) layers, with ongoing work to improve model performance through architecture innovation and extensive evaluation metrics.

Repository Structure

SCR-Progression/
├── scr/                         # Source code directory
│   ├── models/                  # Model implementations
│   │   ├── EncDec_Hybird_pytorch.py            # Encoder-Decoder CNN (implemented)
│   │   ├── UNet_hybrid.py                      # Attention-based Hybrid U-Net (next)
│   │   ├── Base-UNet.py                        # Base U-Net implementation
│   │   ├── Swin_pytorch.py                     # Swin Transformer model
│   │   └── train_segformer.py                  # SegFormer transformer model
│   ├── inference/               # Model inference and analysis tools
│   │   ├── PredvsGT_Pref_metrics_heatmaps.py   # Comprehensive metrics analysis
│   │   ├── bscan_quality_assessment.py         # B-scan quality evaluation
│   │   ├── EncDec_inference_pred.py            # Encoder-decoder inference
│   │   └── BScan_Analysis_Guide.md             # Analysis documentation
│   ├── encoder-decoder_inference.py            # Main inference script
│   └── noise_reduction.py                      # Image preprocessing
├── tools/                      # Data conversion and utilities
│   ├── e2e_to_hdf5_converter.py                # E2E to HDF5 converter
│   ├── mat2hdf5.py                             # MATLAB to HDF5 converter
│   └── modelrun.slurm                          # SLURM job scripts
├── notebooks/                  # Jupyter notebooks for analysis
│   ├── data_processing.ipynb                   # Main data processing pipeline
│   ├── cnn_data_ops.ipynb                      # CNN data operations
│   ├── pred_heatmaps_data_ops.ipynb            # Prediction heatmap analysis
│   └── mask_generate.ipynb                     # Mask generation utilities
├── docs/                       # Documentation and sample images
│   ├── bscan_boundary_predictions.png          # Model prediction examples
│   └── depth_heatmap_comparison.png            # Heatmap analysis samples
├── configs/                    # Configuration files
├── inference_results/          # Model outputs and analysis results
├── logs/                       # Training logs and checkpoints
└── README.md                   # Project documentation

Models Implemented

We are implementing 2 main deep learning architectures for retinal layer annotation prediction:

1. Encoder-Decoder CNN

• Architecture: Convolutional Encoder-Decoder with skip connections
• Purpose: Baseline CNN model for layer boundary prediction
• Files:
  • scr/models/EncDec_Hybird_pytorch.py - Main implementation
  • scr/inference/EncDec_inference_pred.py - Inference pipeline

2. Attention-based Hybrid U-Net

• Architecture: U-Net with attention mechanisms and hybrid feature extraction
• Purpose: Attention based model for improved layer detection

Additional Models (Research Exploration)

• SegFormer Model: Vision Transformer optimized for segmentation (scr/models/train_segformer.py)
• Swin Transformer: Hierarchical vision transformer (scr/models/Swin_pytorch.py)

Model Results and Visualizations

B-scan Annotation Predictions

Our Encoder-Decoder CNN model demonstrates high accuracy in predicting retinal layer boundaries. Below are examples of model performance on OCT B-scan images:

Layer Boundary Detection

The model accurately identifies and segments the Inner Limiting Membrane (ILM) and Bruch's Membrane (BM) boundaries:

Ground Truth vs Model Predictions

• Left: Ground truth annotations overlaid on original B-scan image
• Right: Model predictions overlaid on original B-scan image
• Red lines: Inner Limiting Membrane (ILM)
• Blue lines: Bruch's Membrane (BM)

Model successfully captures complex retinal layer morphology

Depth Heatmap Analysis

The analysis pipeline generates detailed heatmaps showing layer depth variations across the retinal surface:

ISI_2_B Volume Analysis

• Ground Truth: Mean depth 62.9±11.7 pixels
• Predicted: Mean depth 60.9±10.1 pixels
• Error: Close agreement with <2 pixel mean difference

Heatmap comparison showing excellent spatial correlation between ground truth and predictions

Datasets

The project works with 2 different datasets of processed OCT B-scan images:

1. Duke Control Dataset

• Source: Duke AMD dataset
• Format: MATLAB (.mat) files
• Content: Annotated B-scan dataset of AMD and Control patients
• Reference: Duke AMD Dataset
• Conversion: Use tools/mat2hdf5.py to convert .mat files to HDF5 format

2. Internal Nemours Dataset

• Source: Nemours Children's Hospital
• Format: E2E files (Heidelberg OCT format)
• Content: Annotated B-scan dataset of SCR and Control patients
• Access: Restricted
• Conversion: Use tools/e2e_to_hdf5_converter.py to convert .e2e files to HDF5 format

Getting Started

Prerequisites

# Core dependencies
conda install numpy pandas opencv-python matplotlib h5py
conda install -c conda-forge tensorflow torch torchvision
conda install -c conda-forge transformers scikit-learn scipy

# For E2E file processing
pip install eyepy

# For experiment tracking
pip install wandb

---

This repository is part of ongoing research in medical image analysis and computer vision for ophthalmology department at Nemours Children's Hospital.