AI Knowledge Assistant

A fully local Retrieval-Augmented Generation (RAG) system built using FastAPI, Streamlit, Ollama, ChromaDB, and Sentence Transformers.

This project allows users to chat with their own documents, synthetic company knowledge bases, PDFs, reports, books, and notes using a local Large Language Model (LLM) without relying on cloud APIs.

The assistant supports both:

• Document-grounded question answering (RAG)
• General knowledge conversations using the LLM

---

Features

Document Intelligence

• Supports PDF, Markdown, DOCX, and CSV documents
• Extracts and chunks document text automatically
• Stores embeddings locally using ChromaDB
• Retrieves relevant chunks during questioning
• Provides source-aware grounded answers

Hybrid AI Assistant

The system intelligently handles:

Document Questions

Examples:

• What is the leave policy?
• Explain normalization in DBMS
• Summarize the uploaded report

These use retrieval from indexed documents.

General Knowledge Questions

Examples:

• Who is Sachin Tendulkar?
• What is Artificial Intelligence?
• Explain black holes

These bypass retrieval and use the local LLM directly.

---

Tech Stack

Component	Technology
Backend API	FastAPI
Frontend UI	Streamlit
LLM Runtime	Ollama
Vector Database	ChromaDB
Embedding Model	paraphrase-MiniLM-L3-v2
Embedding Framework	sentence-transformers
Language	Python
Testing	Pytest

---

System Architecture

Documents / Synthetic Data
        ↓
Document Ingestion
(core/ingest.py)
        ↓
Chunking
(core/chunker.py)
        ↓
Embedding Generation
(core/embedder.py)
        ↓
Vector Storage (ChromaDB)
(data/vector_store/)
        ↓
Retriever
(core/retriever.py)
        ↓
RAG Pipeline
(core/rag.py)
        ↓
Ollama Local LLM
        ↓
FastAPI Backend
(api/)
        ↓
Streamlit UI
(ui/app.py)

---

Project Structure

ai-knowledge-assistant/
│
├── core/
│   ├── __init__.py
│   ├── ingest.py
│   ├── chunker.py
│   ├── embedder.py
│   ├── retriever.py
│   ├── rag.py
│   ├── redactor.py
│   └── config.py
│
├── api/
│   ├── __init__.py
│   ├── main.py
│   ├── routes/
│   │   ├── ask.py
│   │   └── reindex.py
│   ├── middleware/
│   │   └── auth.py
│   └── schemas.py
│
├── ui/
│   ├── app.py
│   ├── components/
│   │   ├── chat.py
│   │   ├── citations.py
│   │   ├── export.py
│   │   └── upload.py
│   └── .streamlit/
│       └── config.toml
│
├── data/
│   ├── raw/
│   ├── processed/
│   │   └── chunks.json
│   └── vector_store/
│
├── tools/
│   ├── synth_data.py
│   ├── evaluate.py
│   ├── logger.py
│   ├── export.py
│   └── reindex.py
│
├── tests/
│   ├── test_ingest.py
│   └── test_retriever.py
│
├── .env
├── requirements.txt
├── README.md
└── Makefile

---

How the RAG Pipeline Works

Step 1 — Document Ingestion

Documents are loaded from:

data/raw/

Supported formats:

• PDF
• DOCX
• Markdown
• CSV
• TXT

---

Step 2 — Chunking

Large documents are split into overlapping chunks.

Purpose:

• improves semantic retrieval
• reduces token overload
• increases retrieval precision

---

Step 3 — Embedding Generation

Each chunk is converted into vector embeddings using:

paraphrase-MiniLM-L3-v2

This model is lightweight, fast, and optimized for semantic similarity tasks.

---

Step 4 — Vector Storage

Embeddings are stored locally inside:

data/vector_store/

using ChromaDB.

---

Step 5 — Retrieval

When the user asks a question:

1. the query is embedded
2. similarity search is performed
3. top-k relevant chunks are retrieved

---

Step 6 — Answer Generation

Retrieved chunks are passed to the Ollama LLM as context.

The LLM generates a grounded answer with citations.

---

Prerequisites

Software Requirements

• Python 3.10+
• Ollama installed
• Git
• 8 GB RAM minimum
• 16 GB RAM recommended for larger models

---

Ollama Setup

Install Ollama:

https://ollama.com

Pull a model:

ollama pull llama3

or:

ollama pull mistral

Start Ollama:

ollama serve

---

Installation

1. Clone Repository

git clone https://github.com/your-username/ai-knowledge-assistant.git

cd ai-knowledge-assistant

---

2. Create Virtual Environment

Windows

python -m venv .venv

.venv\Scripts\activate

Linux / Mac

python3 -m venv .venv

source .venv/bin/activate

---

3. Install Dependencies

pip install -r requirements.txt

---

Environment Configuration

Create a .env file:

OLLAMA_MODEL=llama3

EMBEDDING_MODEL=paraphrase-MiniLM-L3-v2

TOP_K=5

CHUNK_SIZE=500

CHUNK_OVERLAP=50

API_AUTH_TOKEN=change-me

---

Adding Documents

Place documents inside:

data/raw/

Example:

data/raw/dbms.pdf
data/raw/company_policy.pdf
data/raw/report.docx

---

Generating Synthetic Company Data

Generate sample company documents:

python tools/synth_data.py

This creates:

• leave policies
• onboarding workflows
• SOPs
• FAQs
• incident response documents

These are useful for testing enterprise RAG behavior.

---

Building the Vector Index

Run:

python tools/reindex.py

This performs:

1. ingestion
2. chunking
3. embedding generation
4. vector database creation

---

Running the Backend

Start FastAPI:

uvicorn api.main:app --host 127.0.0.1 --port 8000 --reload

FastAPI docs:

http://localhost:8000/docs

---

Running the Streamlit UI

Start Streamlit:

streamlit run ui/app.py

UI opens at:

http://localhost:8501

---

Example Queries

General Knowledge

Who is Sachin Tendulkar?
What is Artificial Intelligence?
Explain machine learning

Document Questions

What is the leave policy?
Explain normalization in DBMS
Summarize the uploaded report

---

Retrieval Verification

To manually test retrieval:

python -c "from core.retriever import retrieve_chunks; r = retrieve_chunks('What is normalization?', top_k=3); print(r)"

If retrieval works correctly, you should see:

source: dbms.pdf

inside the output.

---

Export Features

The assistant supports exporting answers to:

• Word (.docx)
• Excel (.xlsx)

via Streamlit UI controls.

---

Running Tests

Run unit tests:

pytest

or:

make test

---

Evaluation

Run evaluation pipeline:

python tools/evaluate.py

Measures:

• Recall@K
• citation grounding
• retrieval quality

---

Security Features

• Local-only execution
• No cloud APIs required
• PII redaction supported
• Token-based API authentication
• .env excluded from Git

---

Important Notes

Reindex Required

Whenever new documents are added:

data/raw/

you MUST rebuild the vector index:

python tools/reindex.py

Otherwise the assistant cannot retrieve information from new documents.

---

Supported Use Cases

Academic

• DBMS books
• notes
• research papers
• university reports

Enterprise

• SOPs
• HR policies
• onboarding workflows
• incident response docs
• internal FAQs

Personal Knowledge Base

• notes
• PDFs
• documentation
• manuals

---

Future Improvements

• Hybrid search (BM25 + embeddings)
• Reranking
• Streaming responses
• Multi-user authentication
• Conversation memory
• GPU acceleration
• Advanced metadata filtering

---

Author

Developed by Venkatesh Integrated M.Tech AI VIT Bhopal University