RF-DETR Change Detection for HiRISE

This repository contains a Python pipeline that detects and quantifies mass-wasting events in time-series HiRISE imagery of Mars. Object detection alone is brittle on planetary surfaces. Boxes flicker between epochs, confidences are uncalibrated, and a single missed match can be mistaken for fresh debris. We side-step these failure modes by fusing RF-DETR detections with a pixel-level SSIM change heatmap into a single calibrated probability per detection. That probability is then fed into a Monte-Carlo volume sampler that propagates detector, classifier, and geometric uncertainty in one shot.

Key Features:

1. Pseudo-labelling: imporved RF-DETR performance via a self-supervised training loop.

2. Calibrated detector: raw RF-DETR confidences are IoU-matched to ground truth via isotonic regression.

3. Heatmap evidence: per-detection SSIM features are extracted from the heatmap inside each bounding box.

4. Fused probability: folds detector and change-evidence into a single number ($p_{event} = p_{conf} \cdot p_{new}$ ).

5. MC volume estimation: integrates bernoulli gating, box-coordinate jitter, and lognormal shape error.

6. GPU enabled: whenever possible calculations are vectorized and offloaded to the GPU for maximum efficiency.

🧠 Method Details

Figure 1: co-registered t₁-t₂ image pairs and their difference heatmaps. Bounding boxes are matched across timesteps using a 1-to-1 Hungarian assignment, where yellow labels denote persistent debris and red labels denote new debris. The strong spatial correlation between the red bounding boxes and the intense signals in the difference heatmaps indicates adequate co-registration and tracking performance.

Detection confidence alone only answers whether an object is present, leaving persistent and newly arrived debris indistinguishable. The SSIM heatmap isolates the "is it newly arrived" status independently by capturing localized structural changes. Fusing both signals collapses the joint probability into a single Bernoulli parameter that downstream samplers can consume directly. This fused score degrades gracefully: a high-confidence new detection on a flat heatmap (likely matcher failure) represents a persistent object and is pushed toward zero. Conversely, a low-confidence new detection on a hot heatmap (detector indicates the candidate is likely a false positive rather than actual debris) is capped by the detector’s calibrated probability, ensuring that a change signal alone does not mistakenly elevate background noise.

Key Algorithms

                                  4 -----------+
                                  | Pxl-Spread | ---------------------+
                                  +------------+                      |
                                                                      v
1 --------+   2 --------------+   3 -----------+   5 --------+   6 -------+
| RF-DETR | > | Infer & Match | > | Classifier | > | p_event | > | Volume |
+---------+   +---------------+   +------------+   +---------+   +--------+
                              |                         ^
                              |                         |
                              Calibarted (p_conf) ------+

1. Self-supervised RF-DETR: is a real-time transformer architecture for object detection built on a DINOv2 vision transformer backbone. We incorporate pseudo-labels into the training objective to help the detector improve its decision boundaries. Since raw sigmoid outputs tend to be over-confident, we also perform calibration using an isotonic regression fit. This turns the rank-only scores into real probabilities ($p_{conf}$).

2. Bipartite $t_1 \to t_2$ matching: we identify new objects by matching them to existing objects in the previous frame (fig. 1). This is done using a one-to-one Hungarian assignment based on a combined centroid distance + area cost matrix. Pairs above dist_limit are masked with a sentinel so scipy's solver never assigns them. Detections matched within the cutoff are tagged $\texttt{P}$ (persistent), the rest $\texttt{N}$ (new).

3. SSIM + Logistic regression: for every consecutive pair $(t_1, t_2)$ we compute a per-pixel dissimilarity heatmap, optionally blended with a Sobel gradient-magnitude difference and masked for dead pixels. For each detection, four features are sampled from the heatmap: Gaussian-weighted interior mean ($\mu_{in}$), interior max ($\mu_{max}$), annulus contrast, and fraction of pixels above the 90th percentile, with neighbouring boxes masked out to prevent contamination (fig. 3). These features are fed to a logistic regression fitted on $\texttt{N}/\texttt{P}$ labels using GroupShuffleSplit on patch_id to prevent within-patch leakage, then wrapped in CalibratedClassifierCV with isotonic calibration to yield a reliable $p_{new}$ for every detection.

4. MC-dropout pixel-spread: a dropout-enabled detector runs $K$ stochastic forward passes per patch. Surviving detections are clustered greedily across passes by intersection-over-min, and the per-cluster pixel-spread is bootstrapped as the empirical localisation-noise distribution.

5. Probability fusion. Detector and change-evidence probabilities are assumed conditionally independent given the detection footprint. The fused score ($p_{event} = p_{conf}\cdot p_{new}$) is interpretable as $P(\text{detection is real}) \cdot P(\text{detection is new} \mid \text{real})$ and ready to drive the MC sampler.

6. Volume sampling. Each detection is modelled as an ellipsoid inscribed in a box of extent $W \times H \times \min(W, H)$. The $\min(W, H)$ choice for the out-of-plane axis is based on settled-debris physics: a tumbled block rests on its flattest face. Three uncertainty sources are realised jointly:

   $$V_i = b_i \cdot \varepsilon_i \cdot \frac{\pi}{6} \cdot w_i \cdot h_i \cdot \min(w_i, h_i)$$

   where $b_i \sim \text{Bernoulli}(p_{\text{event}, i})$, $w_i$ and $h_i$ are coordinates perturbed by a Gaussian with $\sigma$ drawn from the pixel-spread pool, and $\varepsilon_i \sim \text{LogNormal}(0, \sigma_{logV}^2)$ absorbs shape-model and depth-assumption error symmetrically on the log scale.

📂 Repository structure

configs/
│   ├─train.yaml ----------------- # Training stage config
│   ├─pseudo.yaml ---------------- # Pseudo-label generation config
│   └─change.yaml ---------------- # Change-detection + volume estimation config
|
marsdet/
│   ├─config.py ------------------ # Pydantic configs + YAML loaders
│   ├─data/ ---------------------- # Dset IO, LMDB builder, loaders, radiometric norm
│   ├─models/ -------------------- # RF-DETR wrapper, confidence calib, event classifier
│   ├─change/ -------------------- # Heatmap, SSIM features, Hungarian match, volume MC
│   └─viz/ ----------------------- # Overlays, plots, debug grids
|
scripts/
│   ├─train.py ------------------- # Stage 1 — fine-tune RF-DETR
│   ├─generate_pseudo_labels.py -- # Stage 2 — score-and-filter pseudo-labelling
│   └─detect_changes.py ---------- # Stage 3 — full change-detection pipeline
|
notebooks/
│   └─analysis.ipynb ------------- # Post-hoc diagnostics on _detections.parquet
|
pyproject.toml
README.md

🚀 Getting Started

Prerequisites

The pipeline depends on PyTorch, RF-DETR, and LMDB. A conda environment with CUDA is recommended.

# 1. Create environment
(base) x@y:~$ conda create -n marsdet python=3.10
(base) x@y:~$ conda activate marsdet

# 2. Install PyTorch (adjust for your CUDA version)
(marsdet) x@y:~$ pip install torch torchvision --index-url https://download.pytorch.org/whl/cu118

# 3. Install the package (editable)
(marsdet) x@y:~$ pip install -e .

🛠️ Usage

The pipeline is split into three independent stages, each driven by a YAML file under configs/ and parsed by the Pydantic models in marsdet/config.py.

1. Training — scripts/train.py

data_dir/
│   ├─annotations/ 
|   |   ├─manual/ ------------------ # Manual labels
|   |   └─pseudo/ ------------------ # Pseudo labels
|   |
│   ├─visualizations/  ------------- # Debug overlays
│   └─images/ ---------------------- # Training .pngs
|
detr_dir/
    ├─datasets/ -------------------- # YOLO-formatted dataset
    └─outputs/ --------------------- # Saved model checkpoints

The training stage fine-tunes RF-DETR-Base on a YOLO-formatted dataset assembled from manual annotations under data_dir/annotations/manual/ and optional pseudo-labels under data_dir/annotations/pseudo/. Pseudo-labels mix into the training fold only (never validation) to avoid evaluation contamination. Augmentation uses the AUG_AERIAL preset, multi-scale training with expanded scales, and early stopping on EMA validation loss. The detector outputs a normalised xyxy box and a sigmoid confidence per query, later turned into a calibrated probability by the change-detection stage.

(marsdet) x@y:~$ python scripts/train.py

Note

Config is read from configs/train.yaml. Adjust model_name, epochs, batch_size, grad_accum_steps, and include_pseudo. The input_size must be divisible by 56 to match the RF-DETR backbone stride.

2. Pseudo-labelling — scripts/generate_pseudo_labels.py

Figure 2: Pseudo-label quality scoring. Left, a low-score ($\sim0.2$) image with few unconfident detections. Center, medium-score ($\sim0.4$) image with a couple moderately confident detections. Right, a high-score ($\sim0.7$) image with many confident detections and a sharp retention ratio between the low and high confidence regimes. The composite score is the cube-root of the three normalised terms.

The pseudo-labelling stage runs the trained detector on un-annotated products and ranks each image by a composite quality score. Three signals combine multiplicatively:

$$\text{score} = (c_{norm} \cdot v_{norm} \cdot r)^{1/3}$$

where $c_{norm}$ is the high-confidence mean normalised against the upper anchor, $v_{norm} = n_{high}/(n_{high} + \text{target}\underline{ }\text{count})$ is a soft count saturation term, and $r = n_{high}/n_{low}$ is the retention ratio between the strict and loose confidence floors. The cube-root keeps the score on a $(0, 1]$ scale and prevents any one term from dominating. The top-K images per location (controlled by pseudo_topk) are written as YOLO .txt files. Optional side-by-side debug overlays land under <data_dir>/visualizations/<pid>/ with the score embedded in the filename so triage by eye is quick.

(marsdet) x@y:~$ python scripts/generate_pseudo_labels.py

Note

Config is read from configs/pseudo.yaml. Set model_name to a checkpoint stem under <detr_dir>/outputs, tune rr_conf_range and target_count to your scene density, and set pid: null to process every product directory.

3. Change-detection — scripts/detect_changes.py

Figure 3: Visualization of the feature extractor applied to a synthetic heatmap. (Top-Left) Feature localization showing bounding boxes (solid), mu_max footprints (dashed), and contrast annuli (dotted circles). (Top-Right) Comparison of the extracted feature values per detection. (Bottom) Breakdown of the neighbor signal exclusion logic using detection #4 as an example.

The change-detection stage runs end-to-end and writes three artefacts to output_dir. The five stages are:

• (1) Pixel-spread sampling. A dropout-enabled RF-DETR runs num_passes stochastic forwards per patch. Detections are clustered greedily by intersection-over-min across passes; clusters seen in $\geq 2$ passes survive. The per-cluster L2 norm of the four coordinate stds becomes one sample of the empirical localisation-noise distribution. Stage runs first so the MC model can be freed before the deterministic one loads.

• (2) Detector calibration. Raw RF-DETR confidences on a labelled split are matched against ground-truth boxes using the same fused-distance Hungarian as the main loop. An isotonic regression on (conf, is_TP) pairs becomes the calibrator pickled to _rf_detr_iso.pkl.

• (3) Main loop. Builds the LMDB store of temporally-aligned and radiometrically normalized patches if absent, then iterates the loader. For every $(t_1, t_2)$ pair: compute the batched SSIM heatmap, predict boxes on both frames, match $t_2 \to t_1$ via Hungarian, assign $\texttt{N}/\texttt{P}$ status, harvest the four SSIM features per $t_2$ detection, and emit a row per detection with calibrated confidence ($p_{conf}$), features, and persistence status. Debug side-by-side snapshots optional. Output lands in _detections.parquet.

• (4) Event classifier. A calibrated logistic regression fits on rows with conf > cls_conf_threshold to avoid label noise from low-quality detections, then scores the full table so every row carries a $p_{new}$. The fused probability $p_{event} = p_{conf} \cdot p_{new}$ is attached as a column.

• (5) Volume MC. Only rows with status == 'N' feed the MC, persistent detections do not contribute new mass. A vectorised $(N, n_{mc})$ grid realises the Bernoulli gate, Gaussian coordinate jitter, and lognormal shape error in one pass. Per-interval summaries ($mean$, $std$, $p05$, $p50$, $p95$) are written to _yearly_volumes.parquet.

(marsdet) x@y:~$ python scripts/detect_changes.py

Note

Config is read from configs/change.yaml. The calib_dir should point at a labelled YOLO split independent of the training data. The gsd knob (metres per pixel) controls the unit of the output volumes. The LMDB store under data_dir/_lmdb is cached. Delete it manually if you change patch tiling, radiometric normalisation, or the reference Mars Year.

Caution

The companion repository hierarchical-coregistration produces the pixel-aligned multi-year patches that this pipeline consumes. Sub-pixel residuals from that stage propagate directly into the SSIM heatmap and the pixel-spread distribution, so co-registration quality is a first-order input to volume uncertainty.

📝 License

Distributed under the Apache 2.0 License. See LICENSE for more information.

✉️ Contact

Oleksii Martynchuk — [email protected]

🤝 Acknowledgements

This project was made possible thanks to the support and resources provided by:

• Technische Universität Berlin (TU Berlin)
• German Aerospace Center (DLR) Berlin
• HiRISE (High Resolution Imaging Science Experiment) team at the University of Arizona
• HEIBRIDS School for Data Science

Additional thanks to the open-source community and the maintainers of RF-DETR, scikit-learn, and PyTorch.