Skip to content

Thundercloud12/nl_to_sql

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

64 Commits
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

πŸš€ RELIX - Natural Language to SQL

Transform data questions into AI-driven SQL queries through conversational intelligence.


πŸ“‹ What It Does

Upload data files (Excel, CSV) β†’ Ask questions in English β†’ Get SQL results + AI insights. No SQL knowledge required.

Key Features:

  • 🎯 Natural language query interface
  • πŸ“ Auto schema detection & metadata generation
  • πŸ”„ Multi-turn conversations with context awareness
  • ❓ Smart clarification for ambiguous queries
  • πŸ’¬ LLM-powered result summarization
  • πŸ” Multi-tenant with Clerk auth

πŸ—οΈ Architecture at a Glance

User Query β†’ FastAPI Backend β†’ LangGraph Workflow β†’ DuckDB SQL Execution β†’ Gemini Insights β†’ Response

Data Flow

1. File Upload β†’ Convert to Parquet β†’ Generate Schema (DuckDB + LLM)
2. User Query β†’ LangGraph State Machine β†’ Intelligent Routing
3. Planner LLM β†’ Generate Execution Plan (tables, filters, aggregations)
4. Execute SQL β†’ Generate Insights β†’ Return to User

🌟 LangGraph Workflow (Core Intelligence)

8 Nodes in Intelligent Sequence:

Node Purpose Decision
Input Validate query β†’ Planner
Planner ⭐ LLM generates SQL plan Router decides next step
Router Conditional branching 5 possible routes
Clarify Ask user questions β†’ END
Schema Fetch table details β†’ Planner (loop)
Preprocess Data transformations β†’ SQL Executor
SQL Executor Execute on DuckDB β†’ Output
Output Format results + insights β†’ END

Router Logic

if plan.needs_clarification:
    β†’ user_clarification (ask user)
elif plan.metadata_requests:
    β†’ schema_info (fetch + replan)
elif plan.preprocessing_operations:
    β†’ preprocessing (clean data)
elif plan.execution_mode == "sql":
    β†’ sql_executor (execute)
else:
    β†’ output (direct response)

Example Plan (Planner Output)

{
  "tables": ["sales"],
  "filters": ["date >= '2025-10-01'"],
  "operations": ["AVG(amount)"],
  "group_by": ["category"],
  "preprocessing_operations": [
    {"type": "fill_nulls", "column": "amount", "method": "mean"}
  ],
  "execution_mode": "sql"
}

πŸ“ Project Structure

backend/                    # FastAPI + LangGraph
β”œβ”€β”€ main.py               # Endpoints, session management
β”œβ”€β”€ data_ingestion/
β”‚   └── graph_builder.py  # Parquet conversion, schema generation
β”œβ”€β”€ llm/
β”‚   β”œβ”€β”€ plan_generator.py # ⭐ LangGraph workflow (8 nodes)
β”‚   β”œβ”€β”€ interpretor.py    # SQL execution
β”‚   └── llm_tracker.py    # LLM analytics
└── utils/                # Rate limiting, DB, cloud storage

my-app/                    # Next.js Frontend
β”œβ”€β”€ app/
β”‚   β”œβ”€β”€ page.tsx          # Landing page
β”‚   β”œβ”€β”€ chat/             # Query interface
β”‚   β”œβ”€β”€ upload-file/      # File upload
β”‚   └── api/              # Backend routes
└── components/           # UI components

πŸ’» Tech Stack

Backend: FastAPI, LangGraph 1.0, Gemini API, DuckDB, PostgreSQL, Supabase
Frontend: Next.js 16, React 19, TypeScript, Tailwind CSS, Clerk auth


πŸš€ Quick Start

Backend

cd backend
python -m venv venv && source venv/bin/activate
pip install -r requirements.txt

# .env
GOOGLE_API_KEY=your_key
DATABASE_URL=postgresql://...
Supabase _URL=Supabase ://...

uvicorn main:app --reload --port 8000

Frontend

cd my-app
npm install

# .env.local
NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY=your_key
NEXT_PUBLIC_API_URL=http://localhost:8000

npm run dev  # http://localhost:3000

πŸ“Š Query Execution Flow

User asks: "Average revenue by region in Q4?"

  1. Planner Node (LLM) β†’ Understands intent

    • Tables: [sales]
    • Filters: [date BETWEEN '2025-10-01' AND '2025-12-31']
    • Operations: [AVG(revenue)]
    • Group: [region]
  2. Router β†’ execution_mode = "sql" β†’ Routes to SQL Executo

  3. SQL Executor

    SELECT region, AVG(revenue)
    FROM sales
    WHERE date BETWEEN '2025-10-01' AND '2025-12-31'
    GROUP BY region
  4. Output Node β†’ Gemini LLM converts results to insights

    • "North leads with $2,450 avg (40% higher than West)..."
  5. Return β†’ Results + Insights + Save to session history


πŸ”„ Multi-Turn with Clarification

If query is ambiguous:

  1. Planner detects ambiguity β†’ needs_clarification = True
  2. Router β†’ Clarification Node
  3. Clarification Node β†’ Sends question to user
  4. User responds β†’ Context appended β†’ Planner replans

Example:

System: "Define 'recent' - last week or month?"
User:   "Last 30 days"
β†’ Replans with clarification context

πŸ” Security & Multi-Tenancy

  • Clerk authentication
  • All queries filtered by user_id + data_source_id
  • Per-datasource session isolation
  • Secure file storage (Supabase )

⚑ Performance

  • Parquet files - 10x faster than CSV
  • DuckDB - In-process SQL (no latency)
  • Row limiting - Schema on 10K rows max
  • Rate limiting - ~50 LLM calls/min
  • Connection reuse - Avoid repeated loads

πŸ“Š Data Ingestion Pipeline

1. Upload File (Excel/CSV)
   ↓
2. Convert to Parquet (memory-efficient)
   ↓
3. Build Schema via DuckDB
   - Detect types
   - Normalize columns
   - Sample rows
   ↓
4. Generate Metadata (LLM)
   - Table summaries
   - Relationship detection
   ↓
5. Store
   - Parquet β†’ Supabase 
   - Metadata β†’ PostgreSQL

πŸ› οΈ API Endpoints

Files: POST /upload_and_process
Queries: POST /query, POST /continue_conversation, POST /clarify
Sessions: POST /save_session


πŸ“ˆ Error Handling

  • SQL validation before execution
  • Type coercion (auto-convert strings to numbers)
  • NULL handling via preprocessing
  • Missing column detection β†’ requests schema
  • Failed queries β†’ retry with corrections
  • Ambiguous queries β†’ ask user

🎯 Future Roadmap

  • Multi-step joins with relationship inference
  • Chart/visualization generation
  • Scheduled queries & alerts
  • Advanced caching
  • Federated queries (multiple datasources)
  • Explainability reports

πŸ“„ Monitoring

LLM Tracking (llm_tracker.py):

  • Calls per query
  • Token usage
  • Latency
  • Cost estimation

🀝 Contributing

  1. Create feature branch: git checkout -b feature/your-feature
  2. Make changes
  3. Submit PR

Built with ❀️ using LangGraph, DuckDB, and Gemini API

About

Resources

Stars

2 stars

Watchers

0 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors