Skip to content

RAG_README.md

Latest commit

 

History

History
230 lines (186 loc) · 9.47 KB

RAG_README.md

File metadata and controls

230 lines (186 loc) · 9.47 KB
Raw

RAG-Powered Conversational Agent with LangGraph4j

Overview

This project implements a Conversational AI Agent with Retrieval Augmented Generation (RAG) using LangGraph4j and LangChain4j in a Spring Boot application. The agent intelligently decides when to use RAG based on the user's query and maintains conversation context across a session.

Features

  • 🧠 Conversation Memory: Remembers context within a session for natural multi-turn conversations
  • 📚 RAG-Powered Search: Uses vector similarity search to find relevant company documents
  • 🎯 Focused Responses: Only answers greetings/chitchat OR company knowledge questions
  • ❌ Honest "I Don't Know": Explicitly says when information isn't available

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                        User Query                               │
│                   (with Session ID)                             │
└─────────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│                 Conversation Memory                             │
│     (Stores chat history per session for context)              │
└─────────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────┐
│                    LangGraph4j Agent                            │
│  ┌─────────────────────────────────────────────────────────┐   │
│  │  Decides based on query:                                 │   │
│  │  - Greeting/Chitchat → Direct response (no tools)        │   │
│  │  - Company question → searchKnowledgeBase (RAG)          │   │
│  │  - General knowledge → Politely decline                  │   │
│  └─────────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────┘
                              │
                              ▼ (if RAG needed)
┌─────────────────────────────────────────────────────────────────┐
│                       RAG Pipeline                              │
│  ┌──────────────┐   ┌──────────────┐   ┌────────────────────┐  │
│  │   Embedding  │   │   Vector     │   │     Retrieval      │  │
│  │    Model     │──▶│    Store     │──▶│    & Response      │  │
│  │(AllMiniLm)   │   │ (InMemory)   │   │                    │  │
│  └──────────────┘   └──────────────┘   └────────────────────┘  │
└─────────────────────────────────────────────────────────────────┘

Agent Behavior

Query Type Agent Action Example
Greetings Responds directly, warmly "Hello!", "How are you?"
Company Questions Uses RAG to search knowledge base "What is the leave policy?"
General Knowledge Politely declines "What is the Battle of Panipat?"
RAG No Results Says "I don't know" "What is the pizza policy?"

Components (All Free!)

Component Technology Description
Embedding Model AllMiniLmL6V2 (ONNX) Free, local embedding model that runs in-process
Vector Store InMemoryEmbeddingStore Pure Java implementation
LLM Groq (Llama 3.1) Using Groq's free tier
Agent Framework LangGraph4j + LangChain4j Java implementation of LangGraph
Memory ConversationMemory Session-based chat history

API Endpoints

Agent Endpoints

Method Endpoint Description
POST /api/agent/execute Execute agent with session memory
GET /api/agent/chat?message= Simple chat without tools or memory
GET /api/agent/tools List available tools
GET /api/agent/health Health check with RAG status

Session Management

Method Endpoint Description
POST /api/agent/session/new Create a new session
DELETE /api/agent/session/{id} Clear a session's history

RAG-Specific Endpoints

Method Endpoint Description
GET /api/agent/rag/search?query= Direct RAG search
GET /api/agent/rag/status RAG service status
POST /api/agent/rag/reload Reload documents

Example Usage

1. Start a Conversation (with Session)

# First message - creates a new session
curl -X POST http://localhost:8080/api/agent/execute \
  -H "Content-Type: application/json" \
  -d '{"message": "Hello! My name is John."}'

Response:

{
  "success": true,
  "response": "Hello John! Welcome! How can I help you today?",
  "sessionId": "abc-123-xyz",
  "messageCount": 2,
  "toolsUsed": [],
  "type": "agent_with_tools"
}

2. Continue the Conversation (same session)

# Use the same sessionId to continue
curl -X POST http://localhost:8080/api/agent/execute \
  -H "Content-Type: application/json" \
  -d '{"message": "What is the leave policy?", "sessionId": "abc-123-xyz"}'

Response:

{
  "success": true,
  "response": "Hi John! Based on our company policies, employees are entitled to 20 days of annual leave per year...",
  "sessionId": "abc-123-xyz",
  "messageCount": 4,
  "toolsUsed": ["searchKnowledgeBase"],
  "type": "agent_with_tools"
}

3. Ask a General Knowledge Question

curl -X POST http://localhost:8080/api/agent/execute \
  -H "Content-Type: application/json" \
  -d '{"message": "What is the Battle of Panipat?"}'

Response:

{
  "success": true,
  "response": "I'm a company assistant and can only help with company-related questions like policies, products, and FAQs. For general knowledge questions, please use a search engine.",
  "toolsUsed": [],
  "type": "agent_with_tools"
}

4. Ask About Something Not in Knowledge Base

curl -X POST http://localhost:8080/api/agent/execute \
  -H "Content-Type: application/json" \
  -d '{"message": "What is the pizza ordering policy?"}'

Response:

{
  "success": true,
  "response": "I'm sorry, I don't have information about pizza ordering in our knowledge base.",
  "toolsUsed": ["searchKnowledgeBase"],
  "type": "agent_with_tools"
}

Sample Questions to Try

Will Use RAG (searchKnowledgeBase tool):

  • "What is the leave policy?"
  • "How many days can I work from home?"
  • "How do I reset my password?"
  • "What are the pricing tiers for EMS?"
  • "Tell me more about that" (after asking about a policy)

Direct Response (no tools):

  • "Hello!"
  • "Hi, my name is Alice"
  • "How are you today?"
  • "Thank you!"
  • "Goodbye"

Will Politely Decline (no tools):

  • "What is the Battle of Panipat?"
  • "Explain quantum physics"
  • "What's the weather today?"
  • "What is 2 + 2?"

Documents Location

Place your documents in:

src/main/resources/documents/

Current sample documents:

  • company_policies.txt - Leave policies, WFH rules, expense guidelines
  • product_info.txt - EMS product features and specifications
  • faq.txt - Frequently asked questions

How It Works

  1. Session Management: Each conversation has a unique sessionId that tracks message history.

  2. System Prompt: The agent has a carefully crafted system prompt that defines:

    • How to handle greetings
    • When to use RAG
    • When to decline (general knowledge)
    • How to say "I don't know"

  3. Query Processing: When a user sends a query:

    • The conversation history is loaded
    • The LLM decides: greet, use RAG, or decline
    • If RAG is used, results are summarized naturally
    • Response is added to conversation history

  4. Memory Trimming: Sessions are limited to 20 messages to prevent memory issues.

Adding Documents

  1. Add .txt, .pdf, or .md files to src/main/resources/documents/
  2. Restart the application, or call POST /api/agent/rag/reload