ImageSentiment

A deep learning computer vision pipeline for classifying multi-dimensional human emotion utilizing Object Detection mapping.

Overview

This repository contains a unified Python architecture for processing the Emotion6 sentiment classification dataset.

It implements a two-stage approach to classifying eight distinct emotion probabilities:

1. Object Detection: Extracts bounding boxes using YOLOv5 around relevant context windows (i.e. humans, faces).
2. Transfer Learning: Passes the YOLO-cropped regions through a fine-tuned VGG16 backbone architecture containing customized Global Average Pooling and Dropout headers.

Architecture Improvements

The pipeline has been thoroughly refactored from disorganized Jupyter Notebook files into a standard, modular Python package structure:

• src/dataset.py: Condenses data loading into a live tf.data.Dataset pipeline. This dynamic generator prevents sudden RAM crashes by processing YOLO crops, standardizing sizes, and normalizing arrays line-by-line rather than pre-loading a massive monolithic Numpy Array explicitly.
• src/model.py: Defines the VGG16 model with Keras Functional API. Implements essential Regularization via internal Data Augmentation nodes (RandomFlip, RandomRotation, RandomZoom) to force generalization on simpler datasets.
• src/train.py: Orchestrates deep learning workflows. Contains safeguards such as EarlyStopping execution via callback to halt model degrading, and automatic configuration of .h5 model checkpoints.

Getting Started

1. Requirements

• Python 3.7+
• TensorFlow 2.x
• OpenCV
• Pandas

2. Prepare Data

Ensure your /data/ folder contains the appropriate raw images/ directory extracted from your source archive. If you contain raw labels (e.g., ground_truth.txt), you can optionally convert them via Pandas.

3. Usage

You can execute the entire pipeline via the interactive Python orchestrator.

1. Launch run_pipeline.ipynb
2. Specify the absolute or relative pathing to your generated yolo5_output.csv and label.csv files.
3. Run the top-level block to invoke train_emotion_model seamlessly.

from src.train import train_emotion_model

model, history = train_emotion_model(
    images_dir="data/images",
    labels_csv="data/label.csv",
    yolo_csv="data/yolo5_output.csv",
    output_dir="checkpoints",
    batch_size=32,
    epochs=20,
    fine_tune=False
)

By default, this will freeze the underlying VGG16 convolutions to train the final sentiment header. Set fine_tune=True explicitly in your configuration to adjust deeper structural weights on subsequent epochs.