Skip to content

Chatbot crashes when FAISS index is missing instead of handling gracefully #312

@j4b3-21

Description

@j4b3-21

Jenkins and plugins versions report

Environment

Jenkins: 2.504.3
OS: Mac OS X - 26.4
Java: 21.0.10 - Homebrew (OpenJDK 64-Bit Server VM)

resources-ai-chatbot:999999-SNAPSHOT (private-2bd2f235-bhuvanmacharya)

What Operating System are you using (both controller, and any agents involved in the problem)?

Operating System (Controller):
macOS (Apple Silicon / ARM64)

Operating System (Agents):
No agents involved (local development only)

Reproduction steps

  1. Fresh Setup
git clone https://github.com/jenkinsci/resources-ai-chatbot-plugin.git
cd resources-ai-chatbot-plugin
  1. Build frontend
    make build-frontend

  2. Start API Server
    make api

  3. Setup Backend
    mvn hpi:run

  4. Start Jenkins
    http://localhost:8080/jenkins

  5. Open Chatbot UI

Navigate to Jenkins UI and open chatbot and send any message

  1. Result
    • API returns 500 Internal Server Error
    • No fallback to LLM prior knowledge

Expected Results

The chatbot should respond to user queries even when the FAISS index or embeddings are missing.

If the index is unavailable, the system should gracefully handle the situation by either:
• Falling back to the LLM’s prior knowledge, or
• Returning a clear and actionable error message

The backend should not crash or return a 500 Internal Server Error.

Actual Results

When sending a message to the chatbot, the backend attempts to load the FAISS index file (plugins_index.idx). Since the file does not exist, the request fails with a 500 Internal Server Error.

The backend logs show:
RuntimeError: could not open .../data/embeddings/plugins_index.idx
No such file or directory

As a result:
• The chatbot does not respond
• The request fails with HTTP 500
• No fallback or user-friendly error is provided

Anything else?

•	This issue occurs on a fresh setup without running the data collection / embedding pipeline
•	The backend starts successfully and the embedding model initializes correctly
•	The failure only occurs when the system attempts to load the FAISS index during retrieval
•	This makes the chatbot unusable until embeddings are manually generated
•	A graceful fallback or clearer error message would improve developer onboarding and reliability

Are you interested in contributing a fix?

Yes, I’d be interested in contributing a fix for this issue.

I plan to add graceful handling when the FAISS index is missing, so the chatbot either falls back to LLM responses or returns a clear error instead of crashing.

Guidance on the preferred behavior (fallback vs. explicit error) would be helpful.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels
    No fields configured for Enhancement.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions