🌟 Open-AISP

Open-AISP is a toy-level, open-source AI-ISP (Artificial Intelligence Image Signal Processor) pipeline framework for beginners. The workflow follows a staged ISP bring-up: realistic RAW degradation, neural RAW reconstruction, traditional static HDR fusion from RAW/DNG data, then AI tone mapping and post enhancement.

📚 Documentation

Language	Entry
🌐 English	English Documentation
🇨🇳 中文	中文文档

🗓️ Timeline

Date	Update
2026-04-27	Add bilingual Mkdocs documentation for Open-AISP
2026-04-25	Init repo with raw-sim and JDD modules

---

🧩 Core Modules

📸 1. Raw Simulation (raw-sim)

This module focuses on generating highly realistic degraded raw data that mimics physical sensor characteristics, which is essential for training robust AI-ISP models.

• 🔄 Unprocess Pipeline: Reverses high-quality RGB images (DIV2K & Flickr2K) back to the raw domain, inlcuding 4x4 quadbayer and 2x2 binning color filter format.
• 🎛️ Gaussian-Poisson Mixed Noise Model: Accurately simulates the physical noise introduced during the photoelectric conversion process, including shot noise and read noise with calibrated noise model.
• Optical PSF Degradation: Introduces Point Spread Function (PSF) degradation to mimic the optical blurring and chromatic aberration of real-world camera lenses.

Input SRGB (DIV2K)	Output Raw (ISO6400)

🧠 2. Joint Denoising and Demosaicing (MF-JDD)

The JDD module handles the early-stage core reconstruction tasks in the ISP pipeline.

• 📊 Brust Image denoise: Utilizes deep learning architectures to learn the complex mapping from multi-frame raw inputs to a high-quality single-frame linear RGB image.
• ⚙️ Hardware Noise Estimation: Estimate the noise map based on analog/digital gain and noise parameter calibration, and guide the JDD model to perform denoising.
• [Todo] Burst Image alignment: efficient multi-frame image alignment and warp.

Opencv Demosaic	JDD Output	Ground truth

🌅 3. Multi-frame HDR Synthesis (HDR)

The current HDR module is a traditional static fusion baseline. It uses MIT5K DNG images, decodes them to linear RGB, dynamically estimates over/normal/under exposure brackets from luminance distribution, and fuses them with photometric weights plus Gaussian/Laplacian pyramids.

• Static multi-exposure fusion: Estimate scene-dependent brackets from one MIT5K DNG image and fuse them with Gaussian/Laplacian pyramids.
• No AI-HDR yet: DeepHDR-style moving bursts and neural HDR merging are reserved for the next HDR iteration.

Under exposure	Normal exposure	Over exposure	Fused HDR (TM)

🎨 4. [TODO] Learning-based Tonemapping (AITM)

Data-driven AI Local Tonemapping Module

🪄 5. [TODO] Diffusion-based Image Post Enhancement (DiffIPE)

Image Post-processing and Enhancement Based on Large-Scale Pre-training and Adversarial Distillation of Single-Step Diffusion Models

---

📅 Project Roadmap

• ✅ Raw Image Simulation (raw-sim)
  • Basic unprocess pipeline (sRGB to Raw)
  • Standard Bayer / QuadBayer / Binning sensor formats
  • Calibrated Gaussian-Poisson mixed noise modeling
  • Lens optical degradation (PSF)
• ✅ Joint Denoise & Demosaic (JDD)
  • Neural network architecture setup
  • Multi-frame fusion implementation
  • Hardware noise estimation integration
  • Burst image alignment registration (WIP)
• ⏳ Multi-frame HDR Synthesis (MF-HDR)
  • Traditional static multi-exposure simulation and fusion from MIT5K DNG images
  • AI-HDR fusion for moving multi-exposure bursts
• ⏳ Deep Local Tonemapping (LTM)
  • Specialized linear RGB to sRGB dataset preparation
  • End-to-end neural tonemapping model implementation
• ⏳ Diffusion Post-Enhancement (IPE)
  • Latent diffusion baseline setup for dark-light / noise extreme scenarios
  • Inference efficiency optimization (Distillation / Single-Step)

---

🚀 Quick Checks

Run commands from the repository root.

Generate one RAW simulation sample:

python raw-sim/scripts/generate_raw.py \
  --input raw-sim/examples/input/sample.png \
  --output raw-sim/simu_pairs \
  --camera-json raw-sim/configs/cameras/example_camera_10bit_RGGB_binning.json \
  --patch-size 128 \
  --num-patches 1

Run quick JDD inference with the bundled checkpoint:

python JDD/scripts/infer.py \
  --checkpoint JDD/assets/latest.pth \
  --input raw-sim/examples/input/sample.png \
  --output JDD/runs/quick_infer \
  --camera-module-json JDD/configs/camera_module_10bit_binning_precali_noise_rggb_ag1to64.json \
  --mode full \
  --max-images 1

Download a small MIT5K DNG subset:

python HDR-fusion/scripts/download_mit5k.py --ids 3

Run static HDR fusion:

PYTHONPATH=HDR-fusion python -m hdr_fusion.cli \
  --input HDR-fusion/datasets/MIT5K/dng/a0003.dng \
  --output HDR-fusion/outputs/a0003_fused.hdr \
  --metrics HDR-fusion/outputs/a0003_metrics.json

🔗 References & Related Projects

• openISP
• fast-openISP
• ISPLab