AI Powered Attention Monitor

Real-time attention monitoring system that uses multiple AI models to detect and analyze student behavior in classroom settings.

Table of Contents

1. Quick Start
2. Development Setup
3. Features
4. Models
5. API Endpoints
6. Project Structure
7. Running Tests
8. Linting & Formatting

1. Quick Start

Important

The backend must be running before you start the frontend.

# Terminal 1 — backend
uv sync && uv run server.py

# Terminal 2 — frontend
cd frontend && pnpm run build && pnpm run start

Then open http://localhost:3000 — sign up with a STU-XXXXXX or TCH-XXXXXX ID.

2. Development Setup

2.1 Install uv (First Time Only)

uv is a modern Python package manager used instead of pip.

macOS / Linux:

curl -LsSf https://astral.sh/uv/install.sh | sh

Windows (PowerShell):

irm https://astral.sh/uv/install.ps1 | iex

2.2 Install Dependencies

uv sync

2.3 Configure Environment

cp .env.example .env
# Edit .env with your settings

Database options:

• SQLite (default): No configuration needed, creates app.db automatically.
• PostgreSQL: Set DATABASE_TYPE=postgresql and DATABASE_URL in .env.

2.4 Start the Backend

uv run server.py

Backend starts on http://localhost:5000. On first run it creates app/app.db and all tables automatically.

2.5 Start the Frontend

cd frontend
pnpm install      # only needed once
pnpm run build
pnpm run start

Frontend starts on http://localhost:3000.

Use pnpm run dev instead of build + start for hot-reload during development.

3. Features

Feature	Description	Status
Gaze Estimation	Tracks eye direction using Gaze360 LSTM model to detect when students look away	✅ Implemented
Phone Detection	Detects mobile phone usage via YOLOv11 object detection	✅ Implemented
Facial Landmarks	68-point facial landmark detection using STAR Loss model	✅ Implemented
Activity Recognition	Classifies 10 classroom activities using SE-ResNet (95% accuracy)	✅ Implemented
Session Reports	Auto-generates PDF reports with stats, event breakdown, and event log	✅ Implemented
Email Reports	Sends session PDF reports via email (SMTP)	✅ Implemented
Multi-Student Tracking	Tracks multiple students with persistent IDs using YOLO + ByteTrack	🧪 Prototype
Real-Time Alerts	Sends alerts via Telegram, Discord, and Email on distraction events	✅ Implemented

4. Models

Model	File	Purpose
Gaze360 (L2CS)	models/gaze360_model.pth.tar	Gaze direction estimation
STAR Loss	models/star_loss.pth	68-point facial landmarks
YOLOv11s	models/yolo11s.pt	Phone detection + person tracking
ResEmoteNet	models/activity_model.pth	10-class activity recognition

5. API Endpoints

Method	Endpoint	Description
GET	/api/health	Server health + loaded models
POST	/api/predict?model=l2cs	Gaze estimation
POST	/api/predict?model=star	Facial landmarks
POST	/api/predict?model=yolo	Phone detection
POST	/api/predict?model=activity	Activity recognition
POST	/api/session/end	End session + generate PDF report
GET	/api/session/<id>/report	Download session PDF

6. Project Structure

app/
├── api/              # Flask routes + API gateway
├── events/           # Observer pattern (Subject/Observer)
├── models/           # SQLAlchemy DB models (Session, Event)
├── services/
│   ├── inference.py  # Model loading + prediction
│   ├── monitor.py    # Attention analysis + alerting
│   ├── tracker.py    # Multi-student tracking (ByteTrack)
│   ├── report.py     # PDF report generation + email
│   ├── store.py      # Database event logging
│   └── notification.py
│       └── notifications/connectors.py  # Email, Discord, Telegram
├── utils/            # Constants, logger, decorators
├── model.py          # PyTorch model architectures
└── config.py         # App configuration
bin/                  # Test scripts
models/               # Trained model weights
notebook/             # Training notebooks (Colab)
reports/              # Generated PDF reports

7. Running Tests

uv run python bin/test_api.py       # API gateway test
uv run python bin/test_report.py    # PDF report generation
uv run python bin/test_tracker.py   # Multi-student tracking (webcam)
uv run python bin/test_activity.py  # Activity recognition (webcam)

8. Linting & Formatting

uv run ruff check .    # Lint
uv run ruff format .   # Format (Black-compatible)