Local API Setup — Windows (Victus PC with RTX 3050)

This guide explains how to run GopalBot's AI backend on your own Windows machine so that zero data leaves your hardware and there is no corporate involvement (no Meta, no Groq, no OpenAI).

Architecture

Discord Server
      │
      ▼
Railway Bot  (always online, handles Discord events)
      │
      ▼
Your Victus PC — local_api_server.py  (port 8000)
      │
      ▼
Ollama — Mistral 7B  (port 11434, GPU-accelerated on RTX 3050)

Part A — Install Ollama

Download the Windows installer from https://ollama.ai
Run the installer and follow the prompts.
Ollama will start automatically as a background service on port 11434.

Verify: Open PowerShell and run:
curl http://localhost:11434
You should see: Ollama is running

Part B — Pull the Mistral 7B Model

Open PowerShell and run:

ollama pull mistral

This downloads the Mistral 7B model (~4 GB). It is fast and runs natively on the RTX 3050 with instant responses.

Verify:

curl http://localhost:11434/api/generate `
  -Method POST `
  -ContentType "application/json" `
  -Body '{"model":"mistral","prompt":"hello","stream":false}'

You should receive a JSON response containing a "response" field.

Part C — Install Python Dependencies (Local Server)

pip install -r requirements_local.txt

Part D — Start the Local API Server

python local_api_server.py

You should see:

============================================================
GopalBot Local API Server
============================================================
Listening on  : http://0.0.0.0:8000
Ollama target : http://localhost:11434
Model         : mistral
============================================================

Verify:
curl http://localhost:8000/health
Expected: {"model": "mistral", "status": "ok"}

Part E — Test the Full Stack

curl http://localhost:8000/api/generate `
  -Method POST `
  -ContentType "application/json" `
  -Body '{"prompt":"who are you?"}'

You should get an instant Mistral response in JSON.

Railway Bot Configuration

The Railway-hosted bot.py reads the LOCAL_API_URL environment variable to know where to find your local API. You have two options:

Option 1 — Same network (LAN)

If Railway and your PC are on the same network (unlikely), set:

LOCAL_API_URL=http://<your-pc-lan-ip>:8000

Option 2 — ngrok (Recommended for remote access)

Download ngrok from https://ngrok.com
Run:
```
ngrok http 8000
```
Copy the https://xxxx.ngrok-free.app URL.

In Railway, set the environment variable:

LOCAL_API_URL=https://xxxx.ngrok-free.app

Redeploy the Railway service.

Now the Railway bot can reach your local Mistral instance from anywhere.

What Happens When Your PC Is Off?

When local_api_server.py is not running (or your PC is off), the Railway bot detects the connection failure and automatically replies:

"My creator's PC is offline right now 🔌 I'm powered by Mistral running on their Victus with RTX 3050 — turn it on to chat with me properly!"

No crashes, no errors — just a friendly message.

Keeping the Server Running in the Background

To keep the server running without a visible terminal window, use:

Start-Process pythonw -ArgumentList "local_api_server.py" -WindowStyle Hidden

Or create a simple batch file start_gopalbot.bat:

@echo off
start "" /B pythonw local_api_server.py
echo GopalBot local API started.

Quick Reference

Step	Command
Start Ollama	Starts automatically as a Windows service
Pull model (first time)	`ollama pull mistral`
Start local API	`python local_api_server.py`
Health check	`curl http://localhost:8000/health`
Test generation	See Part E above
Expose via ngrok	`ngrok http 8000`

Privacy Guarantee

✅ All AI inference runs on your RTX 3050 — nothing is sent to external servers.
✅ Mistral AI is an independent French company — no Meta involvement.
✅ Ollama is fully open-source.
✅ GopalBot code is owned by tomato9553-bit.
✅ Discord messages stay between your server and Discord's servers only.

FilesExpand file tree

LOCAL_API_SETUP.md

Latest commit

History