ExtractIQ Engine

AI-Powered Structured Information Extraction Engine for Noisy Customer Support Data

🌐 Live Demo • 📄 Documentation • ⚡ API Docs • 🎥 Demo Video

🚀 A production-inspired AI extraction platform that transforms noisy customer support tickets into schema-compliant JSON using LLMs, strict Pydantic validation, and an intelligent Model-Driven Repair Loop.

⭐ Built for OneInbox AI Engineer Internship Hackathon 2026

⚡ Production-Inspired Architecture

🧠 Zero Regex Repair Strategy

📊 Project Highlights

✅ Production-ready deployment
🤖 AI-Powered Structured Extraction
🔄 Model-Driven Repair Loop
📄 Strict Pydantic Schema Validation
🛡️ PII Redaction
📦 Batch Extraction
📈 Analytics Dashboard
📜 Extraction History
❤️ Health Monitoring
🌐 Render + Vercel Deployment
🗄️ Supabase Database
⚡ Featherless AI + Google Gemma 4 31B

Extract structured, schema-valid JSON from messy customer support tickets using LLMs with a Model-Driven Repair Loop.

Built for the OneInbox AI Engineer Internship Hackathon 2026

📈 Project Metrics

Feature	Status
AI Extraction	✅
Model-Driven Repair Loop	✅
Strict Schema Validation	✅
Batch Extraction	✅
Analytics Dashboard	✅
Health Monitoring	✅
Version Endpoint	✅
Structured Logging	✅
Docker Deployment	✅
CI/CD Ready	✅
Request Correlation IDs	✅
API Documentation	✅

Short Description

ExtractIQ Engine is a production-inspired AI extraction system that transforms noisy customer support tickets into clean, structured JSON.

Instead of relying on fragile regex patterns, it uses LLMs, strict Pydantic validation, and a Model-Driven Repair Loop to guarantee highly reliable outputs.

💡 Why ExtractIQ?

Traditional information extraction systems rely on fragile regular expressions or unconstrained LLM outputs, making them unreliable when processing noisy customer support conversations.

ExtractIQ Engine addresses this challenge through an AI-first extraction pipeline powered by strict schema validation and a Model-Driven Repair Loop. Instead of silently accepting invalid outputs or masking failures with handcrafted regex rules, validation errors are fed back into the LLM, enabling targeted self-correction until a schema-compliant response is produced or the retry limit is reached.

The result is a production-inspired extraction workflow that prioritizes structural integrity, transparency, and maintainability.

Problem Statement

Customer support tickets are messy — missing punctuation, broken sentences, mixed languages, typing mistakes, missing fields, and unstructured conversations. Traditional regex-based extraction breaks easily, and even modern LLMs frequently produce invalid JSON or violate predefined schemas.

Solution

ExtractIQ Engine introduces a Model-Driven Repair Loop:

Extract — LLM extracts structured data from ticket
Validate — Pydantic enforces strict schema compliance
Detect — Validation errors are captured with exact field details
Repair — Errors are fed back to the LLM for targeted correction
Repeat — Loop continues until valid or retry limit reached

⭐ Key Features

🤖 AI-Powered Extraction

Extracts structured information from noisy customer support tickets using state-of-the-art LLMs (Featherless AI, Groq).

✅ Strict Schema Validation

Every response must satisfy nested Pydantic models with enum enforcement, type checking, and required field validation.

🔄 Model-Driven Repair Loop

Automatically repairs invalid outputs by feeding validation errors back to the LLM. No regex fallbacks, no force-fitting.

🛡️ PII Redaction

Emails, phone numbers, and zip codes are automatically redacted before any data reaches third-party LLM providers.

📊 Analytics Dashboard

Real-time Streamlit dashboard showing success rates, repair rates, latency, category distribution, and historical trends.

📂 Batch Extraction

Upload and process multiple tickets simultaneously with individual error handling.

📜 Extraction History

Complete history of all extractions with pagination, search, and detailed repair attempt logging.

❤️ Health Monitoring

Comprehensive health endpoint with per-dependency status checks (API, database, LLM provider, disk).

📈 Metrics API

Live extraction statistics including success rates, average processing time, retry distribution, and failure breakdown.

🧠 MVP Architecture

Request IDs with end-to-end correlation
Structured JSON logging
Version endpoint
Error tracking with correlation IDs
Global exception handlers
CORS configuration

🚀 Key Innovations

🔄 Model-Driven Repair Loop

Validation errors generated by Pydantic are automatically used to guide the LLM toward producing corrected, schema-compliant responses.

🧠 AI-First Extraction

Instead of handcrafted parsing rules, the engine leverages modern LLMs for intelligent structured extraction.

🛡 Privacy-Aware Processing

Sensitive information is redacted before being transmitted to external AI providers.

📊 Production Observability

Every request is tracked using structured logging, Request IDs, Correlation IDs, metrics, and health endpoints.

🏗 Architecture

The architecture follows a modular layered design that separates preprocessing, extraction, validation, persistence, and analytics into independent components.

This improves maintainability, simplifies debugging, and allows future scalability without changing the core extraction pipeline.

The system follows a modular pipeline architecture:

┌─────────────┐
│  Raw Ticket  │
└──────┬──────┘
       │
       ▼
┌─────────────┐
│ Preprocess   │  ← PII redaction, normalization, language detection
└──────┬──────┘
       │
       ▼
┌─────────────┐
│ LLM Extract  │  ← instructor + Groq/Featherless
└──────┬──────┘
       │
       ▼
┌─────────────┐     ┌──────────────┐
│  Validate    │────→│  Repair Loop │  ← Feed errors back to LLM
└──────┬──────┘     └──────────────┘
       │
       ▼
┌─────────────┐
│ Validated    │  ← Schema-compliant JSON
│    JSON      │
└──────┬──────┘
       │
       ▼
┌─────────────┐
│  Database    │  ← SQLite (SQLAlchemy)
└──────┬──────┘
       │
       ▼
┌─────────────┐
│  Analytics   │  ← Streamlit / React Dashboard
└─────────────┘

Component Diagram

┌─────────────────────────────────────────────────────┐
│                    FastAPI Server                     │
│  ┌──────────┐  ┌──────────┐  ┌────────────────────┐ │
│  │  Routes   │→│  Service  │→│  Extraction Engine  │ │
│  └──────────┘  └──────────┘  └────────────────────┘ │
│                      │               │               │
│                      ▼               ▼               │
│               ┌──────────┐  ┌────────────────────┐ │
│               │ Database │  │  LLM Provider       │ │
│               │   (DB)   │  │  (Groq/Featherless) │ │
│               └──────────┘  └────────────────────┘ │
└─────────────────────────────────────────────────────┘

🔄 Workflow Pipeline

User Uploads Ticket
       │
       ▼
Text Cleaning & PII Redaction
       │
       ▼
LLM Extraction (instructor + Groq/Featherless)
       │
       ▼
JSON Validation (Pydantic TicketExtraction)
       │
       ├── Valid ──→ Database Storage ──→ Response
       │
       └── Invalid ──→ Repair Loop
                            │
                            ├── (≤3 attempts) Re-prompt with errors
                            │
                            └── Max retries exceeded → Return failure

Processing Stages

Ticket Ingestion
Text Cleaning
PII Redaction
LLM Extraction
Schema Validation
Intelligent Repair Loop
Database Storage
Analytics Generation
API Response

⚙ Tech Stack

Backend

Technology	Purpose
FastAPI	REST API framework
Python 3.12	Runtime
Pydantic v2	Schema validation & settings
SQLAlchemy 2.0	ORM
SQLite	Database
instructor	Structured LLM output
Uvicorn	ASGI server

AI / LLM

Technology	Purpose
Featherless AI	Primary LLM provider (default: google/gemma-4-31B-it)
Groq	Optional alternate provider (llama-3.3-70b-versatile)

Frontend

Technology	Purpose
React 19	UI framework
Vite	Build tool
TailwindCSS	Styling
Recharts	Charts
Framer Motion	Animations
TypeScript	Type safety

Dashboard

Technology	Purpose
Streamlit	Analytics dashboard
Plotly	Interactive charts
Pandas	Data processing

DevOps

Technology	Purpose
Docker	Containerization
GitHub Actions	CI/CD
pytest	Testing
Black	Code formatting
isort	Import sorting
Flake8	Linting
mypy	Type checking
pre-commit	Git hooks

🎯 Design Principles

Modular Architecture
AI-First Engineering
Production-Inspired Design
Strict Validation
Privacy by Default
Scalable Components
Observable APIs

📁 Project Structure

ExtractIQ-Engine/
│
├── backend/
│   ├── app/
│   │   ├── api/              # API routes, models, middleware, error models
│   │   │   ├── routes.py     # FastAPI route handlers
│   │   │   ├── models.py     # Request/response Pydantic models
│   │   │   ├── service.py    # Business logic orchestrator
│   │   │   ├── middleware.py # Request ID & correlation middleware
│   │   │   └── error_models.py  # Standard error response model
│   │   ├── database/
│   │   │   ├── database.py   # SQLAlchemy engine & session
│   │   │   ├── models.py     # ORM models
│   │   │   └── repository.py # Data access layer
│   │   ├── evaluation/
│   │   │   ├── collector.py  # Evaluation record collector
│   │   │   ├── metrics.py    # Evaluation metrics calculations
│   │   │   ├── models.py     # Evaluation record schema
│   │   │   └── repository.py # JSONL-backed evaluation persistence
│   │   ├── services/
│   │   │   ├── health_service.py  # Dependency health checks
│   │   │   ├── metrics_service.py # Metrics aggregation
│   │   │   └── version_service.py # Version info
│   │   ├── config.py         # Configuration (backward-compatible)
│   │   ├── settings.py       # Pydantic Settings (centralized config)
│   │   ├── schema.py         # TicketExtraction Pydantic schema
│   │   ├── extraction.py     # Extraction & repair loop
│   │   ├── preprocessing.py  # Text cleaning & PII redaction
│   │   ├── confidence.py     # Confidence scoring
│   │   ├── metadata.py       # Extraction metadata builder
│   │   ├── logging.py        # Structured JSON logging
│   │   └── repair_logging.py # Repair attempt logging
│   ├── tests/
│   │   ├── conftest.py       # Shared fixtures & test configuration
│   │   ├── test_api.py       # API endpoint tests
│   │   ├── test_schema.py    # Schema validation tests
│   │   ├── test_repair_loop.py  # Repair loop tests
│   │   ├── test_database.py  # Database operation tests
│   │   ├── test_utilities.py # Utility module tests
│   │   ├── test_error_handling.py  # Error handling tests
│   │   ├── test_integration_mocked.py  # Mocked integration tests
│   │   └── test_evaluation.py  # Evaluation module tests
│   ├── scripts/              # Utility scripts
│   ├── data/                 # Data files (CSV, JSONL, SQLite)
│   ├── reports/              # Evaluation reports
│   ├── Dockerfile            # Multi-stage Docker build
│   ├── requirements.txt      # Production dependencies
│   ├── requirements-dev.txt  # Development dependencies
│   ├── pyproject.toml        # Tool configuration
│   └── pytest.ini            # Pytest configuration
│
├── frontend/
│   ├── src/
│   │   ├── components/       # React components
│   │   ├── pages/            # Page components
│   │   ├── lib/              # API client & utilities
│   │   └── assets/           # Static assets
│   ├── Dockerfile
│   ├── package.json
│   ├── vite.config.ts
│   └── tailwind.config.js
│
├── dashboard/
│   ├── app.py                # Streamlit app
│   ├── analytics.py          # Analytics components
│   ├── charts.py             # Chart components
│   └── config.py             # Dashboard config
│
├── docs/
│   ├── architecture.png      # Architecture diagram
│   ├── architecture.md       # Architecture documentation
│   ├── api.md                # API documentation
│   ├── api_examples.md       # API examples
│   ├── banner.png            # Project banner
│   ├── changelog.md          # Release history
│   ├── code_of_conduct.md    # Code of conduct
│   ├── contributing.md       # Contributing guide
│   ├── deployment.md         # Deployment guide
│   ├── logo.png              # Project logo
│   ├── model-card.md         # Model card
│   ├── observability.md      # Observability guide
│   ├── roadmap.md            # Roadmap
│   ├── screenshots/          # Application screenshots
│   ├── security.md           # Security guide
│   ├── workflow.md           # Workflow documentation
│   ├── adr/                  # Architecture decision records
│   └── reports/              # Documentation reports
│       ├── audit_report.md
│       ├── benchmark.md
│       ├── evaluation.md
│       ├── failure_analysis.md
│       ├── metric_verification_report.md
│       ├── performance.md
│       └── project_metrics.md
│
├── reports/                  # Generated outputs
│
├── .github/
│   └── workflows/
│       └── ci.yml            # GitHub Actions CI pipeline
│
├── docker-compose.yml        # Multi-service orchestration
├── .pre-commit-config.yaml   # Pre-commit hooks
├── .editorconfig             # Editor configuration
├── .gitignore                # Git ignore rules
├── .dockerignore             # Docker ignore rules
├── README.md                 # This file
└── LICENSE                   # MIT License

🏆 Production Engineering

✔ Global Exception Handling

✔ Request IDs

✔ Correlation IDs

✔ Structured Logging

✔ Environment Configuration

✔ Health Monitoring

✔ Version Endpoint

✔ Metrics API

✔ Batch Processing

✔ Docker Support

✔ REST API Documentation

🚀 Quick Start

Prerequisites

Python 3.12+
Node.js 20+
An API key from Groq or Featherless AI

1. Clone the Repository

git clone https://github.com/RaihanBasha7/ExtractIQ-Engine.git
cd ExtractIQ-Engine

2. Backend Setup

cd backend
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

pip install -r requirements.txt

3. Configure Environment

cp .env.example .env

Edit .env and add your API key:

LLM_PROVIDER=featherless
FEATHERLESS_API_KEY=fl_your_key_here
FEATHERLESS_MODEL=deepseek-ai/DeepSeek-V4-Pro
FEATHERLESS_BASE_URL=https://api.featherless.ai/v1
MAX_REPAIR_RETRIES=3

To switch to Groq as the provider, change LLM_PROVIDER=groq and set GROQ_API_KEY and GROQ_MODEL instead.

4. Run the Backend

uvicorn app.main:app --reload --host 0.0.0.0 --port 8000

5. Run the Frontend

cd frontend
npm install
npm run dev

6. Access the Application

Service	URL
Frontend	http://localhost:5173
API Docs	http://localhost:8000/docs
Dashboard	http://localhost:8501

🔌 API Reference

Note on example responses: The JSON response bodies below show the expected structure and field types. Numeric values (response times, latency, confidence) are illustrative and will vary based on provider, network conditions, and input complexity. See docs/reports/benchmark.md for actual measured values.

Health Check

GET /health

Response:

{
  "status": "healthy",
  "timestamp": "2026-07-20T10:30:00Z",
  "response_time_ms": 12.34,
  "checks": {
    "api": { "status": "ok", "response_time_ms": 0.1 },
    "database": { "status": "ok", "response_time_ms": 1.2 },
    "llm_provider": { "status": "ok", "response_time_ms": 5.0, "provider": "featherless", "model": "deepseek-ai/DeepSeek-V4-Pro" },
    "disk": { "status": "ok", "response_time_ms": 0.3 }
  }
}

Version Info

GET /version

Response:

{
  "service": "OneInbox API",
  "version": "0.1.0",
  "api_version": "v1",
  "provider": "featherless",
  "model": "google/gemma-4-31B-it",
  "environment": "development",
  "python_version": "3.12.4",
  "timestamp": "2026-07-20T10:30:00Z"
}

Single Extraction

POST /v1/extract
Content-Type: application/json

{
  "ticket_id": "TKT-001",
  "raw_text": "From: John Doe <[email protected]>\nSubject: Need help\n\nI can't log into my account. It says 'invalid password'. Help!"
}

Response:

{
  "ticket_id": "TKT-001",
  "success": true,
  "data": {
    "ticket_id": "TKT-001",
    "customer": { "name": "John Doe", "account_id": null },
    "issue": {
      "category": "account",
      "subcategory": "login failure",
      "product_or_service": null,
      "urgency": "high"
    },
    "sentiment": "frustrated",
    "entities": { "order_ids": [], "dates_mentioned": [], "amounts_mentioned": [] },
    "requested_action": "help with login",
    "resolution_status": "unresolved"
  },
  "confidence": 0.85,
  "metadata": {
    "repair_attempts": 0,
    "latency_ms": 450,
    "provider": "featherless",
    "model": "deepseek-ai/DeepSeek-V4-Pro",
    "validation": "passed",
    "timestamp": "2026-07-20T10:30:00Z"
  },
  "retry_count": 0,
  "latency_seconds": 0.45,
  "language": "en",
  "request_id": "REQ-000001",
  "cleaned_text": "From: John Doe <[REDACTED_EMAIL]>\nSubject: Need help\n\nI can't log into my account...",
  "repair_attempts": []
}

Batch Extraction

POST /v1/extract/batch
Content-Type: application/json

{
  "tickets": [
    { "ticket_id": "TKT-001", "raw_text": "Need help with my order" },
    { "ticket_id": "TKT-002", "raw_text": "Account locked, please help" }
  ]
}

Extraction History

GET /v1/history?limit=10&offset=0

Metrics

GET /v1/metrics

Combined System Status

GET /v1/system

🧪 Testing

The project uses pytest for testing.

Running Tests

cd backend

# Run all tests
python -m pytest

# With coverage report
python -m pytest --cov=app --cov-report=term

# With HTML coverage report
python -m pytest --cov=app --cov-report=html

# Run specific test file
python -m pytest tests/test_api.py

# Run tests by marker
python -m pytest -m "unit"
python -m pytest -m "integration"

# Verbose output
python -m pytest -v

Test Structure

Test File	Coverage
`test_api.py`	API endpoints (health, version, metrics, history, extract, batch)
`test_schema.py`	Schema validation, enums, nested objects, optional fields
`test_repair_loop.py`	Repair loop execution, retry counting, error classification
`test_database.py`	CRUD operations, metrics aggregation, history retrieval
`test_utilities.py`	Text cleaning, PII stripping, request ID, confidence, metadata
`test_error_handling.py`	Error responses, exception handlers, validation errors
`test_integration_mocked.py`	End-to-end pipeline with mocked LLM
`test_evaluation.py`	Evaluation records, metrics, repository

Code Quality

# Format code
black --line-length=120 app/ tests/

# Sort imports
isort --profile=black app/ tests/

# Lint
flake8 app/ tests/

# Type check
mypy app/

🐳 Docker Deployment

Build and Run with Docker

# Build the backend image
docker build -t extractiq-engine -f backend/Dockerfile .

# Run the container
docker run -p 8000:8000 \
  -e LLM_PROVIDER=featherless \
  -e FEATHERLESS_API_KEY=your_key \
  -e FEATHERLESS_MODEL=deepseek-ai/DeepSeek-V4-Pro \
  -e FEATHERLESS_BASE_URL=https://api.featherless.ai/v1 \
  extractiq-engine

Full Stack with Docker Compose

# Start all services
docker-compose up -d

# View logs
docker-compose logs -f

# Stop services
docker-compose down

This starts:

Backend on port 8000
Frontend on port 5173
Health checks ensure proper startup order

📊 Deployment Overview

Layer	Platform
Frontend	Vercel
Backend	Render
Database	SQLite (PostgreSQL Ready)
API	FastAPI
AI Provider	Featherless AI

🛣️ Roadmap

Short Term

✅ Completed

[✅] PostgreSQL support (via SQLAlchemy)
[✅] Authentication & API keys
[✅] Rate limiting middleware
[✅] Request validation improvements

Medium Term

Multi-provider LLM routing (LiteLLM)
Redis queue + Celery workers
Webhook notifications
Human review dashboard
Export integration (CSV, JSON, PDF)

Long Term

⚠️ Known Limitations

SQLite — Suitable for development/small-scale. For production, migrate to PostgreSQL.
Process-local request IDs — Current REQ-XXXXXX generator resets on restart. Use UUIDv7 for distributed deployments.
Rate limits — Free-tier LLM providers have daily token limits. Consider upgrading or adding provider failover.
Single-worker — The repair loop runs synchronously. For high throughput, implement Celery workers.
Language detection — Uses langdetect which may not detect all languages accurately.
No authentication — API endpoints are currently open. Add authentication before production deployment.

🏅 Project Highlights

Built for OneInbox AI Engineer Internship Hackathon 2026
Production-inspired AI extraction architecture
Strict Pydantic schema enforcement
Model-Driven Repair Loop
Interactive analytics dashboard
RESTful API with OpenAPI documentation
Dockerized deployment
Modular FastAPI backend

👨‍💻 Author

Shaik Raihan Basha

B.Tech CSE | AI/ML Engineer

GitHub: @RaihanBasha7
LinkedIn: Shaik Raihan Basha

🤝 Contributing

Please read CONTRIBUTING.md for details on our code of conduct and the process for submitting pull requests.

📄 License

See LICENSE for full details.

⭐ If you found this project useful, consider giving it a star!

Made with ❤️ by Shaik Raihan Basha

Name		Name	Last commit message	Last commit date
Latest commit History 18 Commits
.github		.github
backend		backend
dashboard		dashboard
data		data
demo		demo
docs		docs
frontend		frontend
.dockerignore		.dockerignore
.editorconfig		.editorconfig
.gitignore		.gitignore
.pre-commit-config.yaml		.pre-commit-config.yaml
CODE_OF_CONDUCT.md		CODE_OF_CONDUCT.md
CONTRIBUTING.md		CONTRIBUTING.md
LICENSE		LICENSE
README.md		README.md
SECURITY.md		SECURITY.md
TREE.md		TREE.md
docker-compose.yml		docker-compose.yml

Folders and files

Latest commit

History

Repository files navigation

ExtractIQ Engine

AI-Powered Structured Information Extraction Engine for Noisy Customer Support Data

📊 Project Highlights

📈 Project Metrics

📋 Table of Contents

Short Description

💡 Why ExtractIQ?

Problem Statement

Solution

⭐ Key Features

🤖 AI-Powered Extraction

✅ Strict Schema Validation

🔄 Model-Driven Repair Loop

🛡️ PII Redaction

📊 Analytics Dashboard

📂 Batch Extraction

📜 Extraction History

❤️ Health Monitoring

📈 Metrics API

🧠 MVP Architecture

🚀 Key Innovations

🔄 Model-Driven Repair Loop

🧠 AI-First Extraction

🛡 Privacy-Aware Processing

📊 Production Observability

🏗 Architecture

Component Diagram

🔄 Workflow Pipeline

Processing Stages

⚙ Tech Stack

Backend

AI / LLM

Frontend

Dashboard

DevOps

🎯 Design Principles

📁 Project Structure

🏆 Production Engineering

🚀 Quick Start

Prerequisites

1. Clone the Repository

2. Backend Setup

3. Configure Environment

4. Run the Backend

5. Run the Frontend

6. Access the Application

🔌 API Reference

Health Check

Version Info

Single Extraction

Batch Extraction

Extraction History

Metrics

Combined System Status

🧪 Testing

Running Tests

Test Structure

Code Quality

🐳 Docker Deployment

Build and Run with Docker

Full Stack with Docker Compose

📊 Deployment Overview

🛣️ Roadmap

Short Term

✅ Completed

Medium Term

Long Term

⚠️ Known Limitations

🏅 Project Highlights

👨‍💻 Author

🤝 Contributing

📄 License

⭐ If you found this project useful, consider giving it a star!

About

Topics

Resources

Packages