🔬 MedLens — Multimodal AI Clinical Assistant for Prostate Cancer

End-to-end multimodal AI pipeline for prostate cancer MRI analysis. Processes real clinical DICOM data, localizes lesions with PI-RADS scoring, classifies Gleason grade groups, and generates structured PDF clinical reports via LLM — built on the PROSTATEx dataset (346 patients, 5.93 GB).

This project is the engineering implementation behind the Scopus-indexed paper: LASRMIS: LLM-Assisted Structured Reporting from Medical Image Segmentation (2026).

---

Pipeline

DICOM Input (T2W + ADC)
  → Volume Loading       (PyDICOM, sorted by ImagePositionPatient Z)
  → NIfTI Mask Overlay   (PROSTATEx_masks, best-slice projection)
  → PI-RADS Scoring      (per-finding clinical significance + Gleason grade)
  → MRI Visualization    (T2W grayscale + ADC heatmap + lesion overlays)
  → LLM Report Draft     (Groq LLaMA 3.1, structured clinical language)
  → PDF Export           (ReportLab, patient summary + findings + recommendation)

Folder Structure

MedLens/
├── app.py            ← Gradio UI entry point
├── imaging.py        ← DICOM loading, NIfTI masks, visualization
├── report.py         ← PDF clinical report generation
├── requirements.txt
├── data/             ← patient_index.pkl, patient_pirads.pkl, lesion_index.pkl
│   └── .gitkeep      (indexes generated by data prep — see Setup below)
└── README.md

Dataset

PROSTATEx (The Cancer Imaging Archive)

• 346 patients, 5.93 GB
• Modalities: T2-Weighted (T2W) + Apparent Diffusion Coefficient (ADC) maps
• Annotations: PI-RADS scores, lesion masks (NIfTI), Gleason grade groups
• 70 cancer-positive, 130 benign cases with clinical ground truth

Dataset access: cancerimagingarchive.net/collection/prostatex Masks: github.com/rcuocolo/PROSTATEx_masks

Setup

1. Install dependencies

git clone https://github.com/Vi-bha/MedLens
cd MedLens
pip install -r requirements.txt

2. Download PROSTATEx dataset

from tcia_utils import nbia
import pandas as pd

df_series = nbia.getSeries(collection='PROSTATEx')
df_series = pd.DataFrame(df_series)
df_filtered = df_series[
    (df_series['Modality'] == 'MR') &
    (df_series['SeriesDescription'].str.contains('t2|adc', case=False, na=False))
]
nbia.downloadSeries(series_data=df_filtered, input_type="df", path="./prostatex_data")

3. Clone lesion masks

git clone https://github.com/rcuocolo/PROSTATEx_masks.git ./prostatex_masks

4. Build patient indexes

Run the data preparation notebook (notebooks/data_prep.ipynb) to generate data/patient_index.pkl, data/patient_pirads.pkl, and data/lesion_index.pkl.

5. Run

export GROQ_API_KEY=your_key_here   # free at console.groq.com
python app.py

Open http://localhost:7860 in your browser.

Sample Output

Patient	PI-RADS	Status
ProstateX-0000	4	🔴 Cancer Detected
ProstateX-0001	2	🟢 Benign
ProstateX-0005	5	🔴 Cancer Detected
ProstateX-0009	2	🟢 Benign

Each analysis produces:

• T2W MRI with red/yellow lesion mask overlays
• ADC heatmap with co-registered masks
• Structured clinical findings per lesion (Gleason grade, clinical significance)
• Automated PDF report with recommendation

Tech Stack

Component	Technology
DICOM Loading	PyDICOM (sorted by Z-position)
Mask Processing	NiBabel (NIfTI), SciPy (zoom/resize)
Visualization	Matplotlib (T2W grayscale + ADC heatmap)
LLM Reports	Groq LLaMA 3.1 8B Instant
PDF Export	ReportLab
UI	Gradio 4.x

Related Publication

This project implements the engineering pipeline behind:

LASRMIS: LLM-Assisted Structured Reporting from Medical Image Segmentation Vibhavari Tummewar et al. — Scopus-indexed, 2026

Evaluated 3 prompting strategies (minimal, structured, full-context) on 32 cases (96 reports). Full-context prompting achieved 86.2% clinical completeness vs 37.5% (minimal). Zero hallucinations in structured and full-context conditions. Validated 10-element rubric (κ = 0.864, Almost Perfect agreement).

Related Projects

Project	Description	Demo
ResearchMind	Autonomous AI scientist querying PubMed 35M+ papers	🤗 Live
PaperLens	RAG pipeline for research paper Q&A	🤗 Live

Author

Vibhavari Tummewar — M.Tech Advanced Computing, MANIT Bhopal Scopus-indexed researcher in LLM-assisted medical AI systems.