SwayUp.ai - Real-Time AI Sales Call Assistant

SwayUp.ai is an MVP assistant for live sales conversations. It listens to call chunks, transcribes speech, detects buying intent/sentiment, and returns in-the-moment coaching suggestions plus call-level performance insights for the sales rep.

Why this project is compelling

• Real-time AI pipeline: audio/text -> ASR (Automatic Speech Recognition) -> intent/sentiment -> coaching tip
• Practical architecture: FastAPI backend + Streamlit dashboard + local LLM (Ollama)
• Resilient behavior: fallback logic when LLM or Sheets is unavailable
• Demo-ready scripts: smoke test + synthetic stream + one-command runner

What it does

• Captures short audio chunks from microphone or accepts direct text chunks
• Transcribes audio with faster-whisper
• Classifies intent/sentiment with LangChain + Ollama (tinyllama)
• Produces actionable coaching text for objection handling
• Builds session-level performance insights and benchmarks the call against configurable sales standards
• Attempts to log each chunk to Google Sheets
• Displays a live dashboard in Streamlit

Architecture

1. Input: microphone (/call/ingest) or synthetic/text (/call/ingest_text)
2. ASR: Whisper (faster-whisper) converts speech to text transcript
3. AI analysis: intent, sentiment, entities, and next action
4. Coaching: generates a short prompt the rep can say next
5. Performance analysis: aggregates the whole call into benchmarked insights and a coaching score
6. Storage/UI: optional Google Sheets logging + Streamlit live view

Fallback logic

1. If the Ollama LLM is slow/down:

   • The System uses backup rule-based logic to still guess intent/sentiment and give a coaching tip.

2. If Google Sheets logging fails:

   • The API still returns a normal response to the user
   • It add a warning note in coach_tip instead of stopping the flow.

Tech stack

• Backend: FastAPI, Uvicorn, Pydantic
• ASR: faster-whisper
• LLM orchestration: LangChain + langchain-ollama
• LLM runtime: Ollama (tinyllama)
• Frontend: Streamlit
• Logging sink: Google Sheets (gspread, google-auth)

Prerequisites

• Python 3.10+
• Ollama installed
• tinyllama model pulled

Install Ollama:

curl -fsSL https://ollama.com/install.sh | sh

Pull model:

ollama pull tinyllama

Linux container note (if microphone capture fails):

sudo apt-get update
sudo apt-get install -y libportaudio2 portaudio19-dev libasound2-dev

Setup

Linux / macOS:

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Windows (PowerShell):

python -m venv .venv
.\.venv\Scripts\activate
pip install -r requirements.txt

Environment configuration

1. Copy .env.example to .env
2. Configure values:
   • GOOGLE_SHEETS_SPREADSHEET_ID (required for Sheets logging)
   • GOOGLE_SHEETS_WORKSHEET_NAME (default: Logs)
   • MIC_DEVICE (optional input device index)
   • WHISPER_MODEL (small is default)
   • OLLAMA_MODEL (tinyllama default)
   • SALES_BENCHMARK_PROFILE (general, enterprise, or smb)
   • SALES_STANDARDS_JSON (optional path or raw JSON string for custom benchmark thresholds)
3. Place Google service account JSON at secrets/google-service-account.json

Important: if Google Sheets is not configured, the API still runs and includes a non-fatal note in coach_tip.

Quickstart (recommended)

Fastest live demo launcher (opens API docs + Streamlit automatically):

chmod +x scripts/live_demo.sh
./scripts/live_demo.sh

Standard all-in-one orchestrator:

Run the one-command orchestrator:

chmod +x scripts/overnight_agent.sh
./scripts/overnight_agent.sh

This script:

1. Ensures the venv/dependencies exist
2. Starts Ollama if needed
3. Starts FastAPI server
4. Runs smoke test
5. Starts mic streaming, or falls back to synthetic stream when no mic is found

Logs are written to:

• logs/api.log
• logs/ollama.log
• logs/smoke.log
• logs/stream.log

Manual run (step-by-step)

Terminal 1: start Ollama

ollama serve

Terminal 2: start API

source .venv/bin/activate
uvicorn app.main:app --host 127.0.0.1 --port 8000 --reload

Terminal 3: run smoke test

source .venv/bin/activate
python scripts/smoke_test_api.py

Terminal 4 (option A): microphone stream

source .venv/bin/activate
python -m scripts.run_mic_to_api

Terminal 4 (option B): synthetic stream

source .venv/bin/activate
python -m scripts.run_synthetic_to_api

Optional frontend:

source .venv/bin/activate
streamlit run frontend/streamlit_app.py --server.port 8501 --server.address 127.0.0.1

API endpoints

• GET /health
• POST /call/start
• POST /call/stop
• GET /call/latest
• GET /call/insights
• POST /call/ingest (audio samples)
• POST /call/ingest_text (raw transcript text)

Swagger docs: http://127.0.0.1:8000/docs

Verified run status (March 26, 2026)

Validated in Ubuntu dev container with Python 3.12:

• API health check passed
• smoke test passed end-to-end
• direct ingest_text call produced correct pricing objection classification
• Streamlit app launched successfully on port 8501

Observed behavior:

• First ingest_text inference can be slow (about 42s in this environment) and may return regex fallback coaching if LLM is slow/unavailable.
• In containers, no microphone may be exposed; synthetic streaming is the reliable fallback demo path.

Results snapshot

Sample call flow (real run):

curl -X POST http://127.0.0.1:8000/call/start
curl -X POST http://127.0.0.1:8000/call/ingest_text \
   -H 'Content-Type: application/json' \
   -d '{"transcript":"Your pricing is too high for our budget right now"}'
curl -X GET http://127.0.0.1:8000/call/latest
curl -X POST http://127.0.0.1:8000/call/stop

Representative ingest_text response:

{
   "call_id": "0dfcf9cf-06ca-4fd3-9482-c282a3cb29ba",
   "chunk_index": 2,
   "transcript": "Your pricing is too high for our budget right now",
   "intent": "pricing_objection",
   "sentiment": "negative",
   "entities": [],
   "action": "Ask one clarifying question, then map value to the buyer's stated concern.",
   "coach_tip": "Fallback intent used because the LLM was unavailable/slow or returned invalid output."
}

The latest call payload also includes insights, which exposes a benchmark score, per-metric checks, strengths, and the highest-priority coaching areas.

Observed latency in this environment:

• first ingest_text call: ~42s
• subsequent calls: typically faster once model is warm

Live demo script (2-3 minutes)

1. Start stack with ./scripts/live_demo.sh
2. Open API docs at /docs and show endpoints
3. Run synthetic streamer and show live intent/sentiment updates
4. Open Streamlit dashboard and highlight real-time coaching prompts
5. Explain fallback resilience (works even if Sheets/LLM has issues)

Project layout

• app/: FastAPI app, ASR, intent analysis, coaching logic, Sheets integration
• scripts/: smoke test, microphone/synthetic streamers, overnight runner, live demo launcher
• frontend/: Streamlit live dashboard
• logs/: runtime logs
• secrets/: local service credentials (not for git)

Scripts

• smoke test: a lightweight integration test that quickly confirms if the app works end-to-end (validates critical paths like API up, call start, ingest, and response.)
• microphone streamers: Scripts that capture live mic audio and send it to the backend in small chunks.
• synthetic streamers: Scripts that send prewritten sample text/audio instead of real mic input.
• coaching logic: The rule/AI layer that decides what sales advice to return. ( decision policy that maps 'intent + sentiment' into action recommendations and coach tips)
• Sheets integratrion: Sending analyzed results to Google Sheets for tracking/reporting using Google Sheets API

License

This repository is proprietary and all rights are reserved. See LICENSE for full terms.

No use, copying, modification, distribution, or contributions are allowed without prior written permission.

Future improvements (V2 & Above)

• Add async queue for non-blocking inference at higher throughput
• Add richer entity extraction and CRM field mapping
• Add automated tests for endpoint contract + fallback paths
• Add deployment profile (Docker Compose) for one-command local demo