TrackAid-DT: An Egocentric Vision Dataset and Benchmarks for Lane Detection in Visually Impaired Athlete Guidance

Authors: Gagan Narang, Alessandro Galdelli, Oleksandr Kuznetsov, Primo Zingaretti, Adriano Mancini

Journal: Computer Vision and Image Understanding

---

Overview

TrackAid-DT introduces:

• A pixel-annotated egocentric dataset for athlete guidance.
• A benchmark of modern semantic segmentation architectures.
• A complete training, evaluation, and inference framework for lane perception in assistive sports applications.

The project focuses on autonomous lane guidance for visually impaired athletes through egocentric vision. TrackAid-DT provides a dataset collected from chest-mounted cameras during real running sessions and evaluates multiple deep learning architectures for detecting the walkable running lane under realistic athletic conditions.

The dataset contains pixel-level lane annotations acquired under diverse environmental and motion conditions, including daytime, nighttime, low-light environments, sprint sessions, and challenging viewpoints. Together with the dataset, this repository provides reproducible pipelines for training, evaluating, and benchmarking semantic segmentation models for assistive sports applications.

---

Dataset

TrackAid-DT consists of 747 pixel-annotated images collected from chest-mounted cameras during real running sessions on athletics tracks. The original annotations are provided in COCO Segmentation format and can be converted to binary segmentation masks for training with this repository. All other datasets used in the study were collected from publicly available resources. We share the TrackAid-DT dataset; for details about the additional public data, please refer to the full study: TrackAid-DT.

For the training and evaluation code, images and binary masks are expected to follow the folder structure defined in config.py. The following augmentation was applied to create 3 versions of each source image:

• 50% probability of horizontal flip
• Random rotation of between -10 and +10 degrees
• Random exposure adjustment of between -15 and +15 percent
• Random Gaussian blur of between 0 and 1.5 pixels
• Salt and pepper noise was applied to 1.01 percent of pixels

---

Data Collection

TrackAid-DT was collected on standard outdoor athletics tracks using a chest-mounted monocular camera to capture an egocentric view of the running lane. Data acquisition involved multiple athletes performing running sessions under diverse environmental and motion conditions to reflect realistic deployment scenarios.

The dataset includes recordings acquired during daytime, nighttime, and low-light conditions, as well as straight and curved track segments. Additional variability was introduced through different running speeds, athlete positions within the lane, and viewpoint changes naturally occurring during athletic activity. To further improve diversity and robustness, supplementary recordings were collected using smartphone cameras under different orientations and acquisition settings.

From the recorded videos, frames were uniformly sampled and manually annotated with pixel-level masks representing the walkable running lane. The final dataset comprises 747 annotated images, providing a challenging benchmark for semantic segmentation models operating in real-world athletic environments.

Dataset Characteristics

• Pixel-level lane annotations
• Egocentric chest-mounted viewpoint
• Multiple athletes and recording sessions
• Daytime and nighttime acquisitions
• Low-light scenarios
• Sprint and jogging conditions
• Straight and curved track segments

Each image is paired with a binary segmentation mask representing the walkable running lane.

Models

The benchmark includes the following segmentation architectures. All networks are implemented using TensorFlow and Keras:

Model	Description
UNet	Baseline encoder-decoder architecture with skip connections
UNet++	Enhanced variant with nested skip connections
UNetL	Lightweight version of UNet
UNet++L	Lightweight version of UNet++
TransUNet	Hybrid CNN and Transformer architecture
LightMUNet	Uses Mamba State Space Models instead of Transformers

🔍 The TransUNet model implementation was adapted from the repository TransUNet-tf.

---

All models are trained and evaluated under a unified experimental protocol to enable a fair comparison across different architectural families.

---

Installation

This project requires Python 3.10.

Clone the repository and install the required dependencies:

git clone https://github.com/vrai-group/TrackAid-DT.git

cd TrackAid-DT

pip install -r requirements.txt

The requirements.txt file contains all dependencies required to reproduce the experiments presented in the paper, including libraries for:

• Deep learning model training
• Dataset loading and preprocessing
• Data augmentation
• Model evaluation and benchmarking
• Segmentation inference
• Visualization and result generation

---

Repository Structure

TrackAid-DT/
├── main.py              # Main entry point for training, evaluation, and inference
├── train.py             # Training, evaluation, inference, and cross-validation sessions
├── predict.py           # Prediction, evaluation metrics, and mask-saving utilities
├── test.py              # Optional batch inference utility over saved best models
├── config.py            # Dataset paths, input settings, and training parameters
├── dataclass.py         # Dataset and parameter data structures
├── models/              # Segmentation architectures
├── utils/               # Dataset loading, saving, logging, denoising, and helper utilities
├── images/              # Figures used in this README
└── requirements.txt     # Python dependencies

---

Usage

The standard entry point of the repository is:

python main.py

Training, evaluation, and inference are selected by enabling the corresponding function call inside main.py. In the current workflow, train.py and predict.py provide the underlying classes and utilities, while main.py orchestrates the experiment.

Dataset paths, model-saving folders, input size, number of channels, and hyperparameters are configured in:

config.py

Training and Evaluation

To train the selected segmentation models and evaluate them after training, enable start_model_wise() inside train_eval_session() in main.py:

TrainingAndEvaluationSession(
    logger,
    dataset,
    models,
    params,
    False,
    denoised,
    extra_filter
).start_model_wise()

Then run:

python main.py

This workflow trains the models listed in models, evaluates each trained model on the configured test split, saves predicted masks, and stores evaluation metrics such as IoU, F1-score, precision, recall, and accuracy.

To train only one model, modify the models list in main.py. For example:

models = [UNetPP]

To train all benchmark models, use:

models = [UNetL, UNet, UNetPPL, UNetPP, TransUNet, PretrainedTransUNet, LightMUNet]

Test / Evaluation of a Saved Model

To test an already trained model from main.py, disable the training call and enable the evaluation/inference call with the path of the saved model:

TrainingAndEvaluationSession(
    logger,
    dataset,
    [UNetPP],
    params,
    True,
    denoised,
    extra_filter
).start_inference(
    model_path="./saves_check_Unet/lane/UNetPP/model.hdf5",
    model_dir_name="UNetPP"
)

Then run:

python main.py

The model_path argument must point to the saved model file, usually model.hdf5. The model_dir_name argument is used to name the output folder containing the predicted masks and inference summary.

Single-Image Prediction Test

For a quick single-image sanity check, disable train_eval_session(logger) in main.py and enable:

test_prediction()

Then run:

python main.py

By default, this loads the model path configured in EvaluationSession.test_prediction() and saves the input image, ground-truth mask, and predicted mask in the prediction output folder.

Inference Output

Predicted masks are saved under the prediction path configured in config.py:

Paths.PREDICTIONS

For each model, the framework creates a dedicated output directory and writes the predicted binary masks together with an inference summary.

Optional Batch Inference Utility

The repository may also include test.py, which can run batch inference over all models stored inside a best_models/ directory. However, the primary documented workflow uses main.py as the single entry point.

---

Results

Quantitative Results

Stability is reported as the standard deviation of IoU across the 8 cross-validation folds; lower values indicate more stable performance.

Training from Scratch (8-Fold Cross-Validation)

Model	IoU (μ ± σ)	F1 (μ ± σ)	Recall (μ ± σ)	Precision (μ ± σ)	Accuracy (μ ± σ)	Stability
U-Net	0.854 ± 0.102	0.918 ± 0.059	0.921 ± 0.072	0.917 ± 0.063	0.973 ± 0.027	0.102
U-Net-L	0.738 ± 0.178	0.837 ± 0.121	0.813 ± 0.143	0.869 ± 0.113	0.953 ± 0.039	0.178
UNet++	0.859 ± 0.113	0.920 ± 0.068	0.906 ± 0.101	0.940 ± 0.050	0.972 ± 0.033	0.113
UNet++-L	0.763 ± 0.165	0.855 ± 0.109	0.836 ± 0.132	0.882 ± 0.102	0.957 ± 0.038	0.165
TransUNet	0.865 ± 0.120	0.923 ± 0.073	0.940 ± 0.061	0.910 ± 0.099	0.978 ± 0.019	0.120
Pretrained TransUNet*	0.929 ± 0.064	0.962 ± 0.036	0.958 ± 0.037	0.965 ± 0.036	0.988 ± 0.011	0.064
LightM-UNet	0.697 ± 0.182	0.808 ± 0.128	0.777 ± 0.156	0.851 ± 0.113	0.944 ± 0.046	0.182

KITTI Pretraining (8-Fold Cross-Validation)

Model	IoU (μ ± σ)	F1 (μ ± σ)	Recall (μ ± σ)	Precision (μ ± σ)	Accuracy (μ ± σ)	Stability
U-Net	0.921 ± 0.064	0.958 ± 0.037	0.960 ± 0.035	0.956 ± 0.041	0.987 ± 0.011	0.064
U-Net-L	0.884 ± 0.089	0.936 ± 0.052	0.936 ± 0.044	0.937 ± 0.065	0.981 ± 0.016	0.089
UNet++	0.917 ± 0.067	0.956 ± 0.038	0.957 ± 0.033	0.954 ± 0.044	0.986 ± 0.011	0.067
UNet++-L	0.815 ± 0.121	0.893 ± 0.076	0.881 ± 0.081	0.908 ± 0.081	0.968 ± 0.024	0.121
TransUNet	0.796 ± 0.136	0.880 ± 0.086	0.877 ± 0.123	0.895 ± 0.089	0.962 ± 0.036	0.136
Pretrained TransUNet*	0.899 ± 0.073	0.945 ± 0.042	0.943 ± 0.049	0.948 ± 0.044	0.983 ± 0.014	0.073
LightM-UNet	0.578 ± 0.244	0.702 ± 0.198	0.647 ± 0.216	0.793 ± 0.204	0.918 ± 0.066	0.244

* Pretrained TransUNet denotes the model initialized with ImageNet-pretrained weights.

Results Summary

TrackAid-DT was evaluated using 8-fold cross-validation across multiple semantic segmentation architectures, including convolutional, transformer-based, and state-space models. Among the models evaluated without task-specific KITTI pretraining, Pretrained TransUNet achieved the best overall performance, reaching an IoU of 0.929 ± 0.064 and an F1-score of 0.962 ± 0.036. Task-specific pretraining on the KITTI dataset significantly improved the performance of convolutional architectures. In this setting, U-Net achieved the highest performance with an IoU of 0.921 ± 0.064 and an F1-score of 0.958 ± 0.037, closely followed by UNet++. These results demonstrate that TrackAid-DT provides a challenging benchmark for egocentric lane perception while highlighting the effectiveness of modern segmentation architectures for assistive athlete guidance under diverse real-world conditions.

Qualitative Results

Example predictions demonstrate accurate lane segmentation across a wide range of running conditions, supporting reliable perception for athlete guidance applications.

---

Generalization Test

To evaluate robustness beyond the cross-validation protocol, all trained models were additionally assessed on an independent external test set that was not used during training or validation. This evaluation provides insight into the ability of the models to generalize to unseen running conditions, athletes, viewpoints, and environmental scenarios. The results indicate that the top-performing architectures maintain strong segmentation performance under previously unseen conditions, demonstrating the potential of TrackAid-DT for real-world deployment in assistive athlete guidance applications.

Deployment and Outdoor Validation

Beyond offline benchmarking, the proposed framework was deployed and evaluated on multiple embedded hardware platforms to assess its suitability for real-time operation in wearable assistive systems. The trained models were converted and optimized for edge inference, enabling efficient execution on resource-constrained devices while maintaining reliable segmentation performance. To validate the practical feasibility of the system, a complete prototype integrating egocentric vision, onboard processing, and vibrotactile feedback was tested in real outdoor running scenarios. During these trials, athletes navigated standard athletics tracks while receiving real-time haptic guidance based on the estimated lane position. The system demonstrated stable lane perception under varying illumination conditions, curved track geometries, and different running speeds, providing timely feedback that supported lane maintenance throughout the running sessions. These experiments highlight the potential of TrackAid-DT not only as a benchmark dataset for semantic segmentation research, but also as a foundation for developing real-world assistive technologies that promote greater autonomy and independence for visually impaired athletes.

To read more about these tests, we recommend the following resources, and read the full paper.

Contact:

Dr. Alessandro Galdelli: [email protected] Dr. Gagan Narang: [email protected]

Read our full study, and cite if you find it useful!

Full paper: TrackAid-DT

If you use this repository or the TrackAid-DT dataset in your research, please cite:

@article{TrackAidDT,
  title={TrackAid-DT: An Egocentric Vision Dataset and Benchmarks for Lane Detection in Visually Impaired Athlete Guidance},
  journal={Computer Vision and Image Understanding},
  volume={269},
  pages={104788},
  year={2026},
  author={Gagan Narang and Alessandro Galdelli and Oleksandr Kuznetsov and Primo Zingaretti and Adriano Mancini},
  doi={10.1016/j.cviu.2026.104788}
}