Learnable, configurable, and pluggable Omni Personal Assistant for everyone

AlphaAvatar is a self-hostable Omni Personal Assistant framework designed to evolve into an intelligent personal butler — a continuous, personalized, and proactive assistant that can remember, understand, plan, and act on behalf of the user.

It is built around a plugin-based real-time Agent architecture, combining:

• 🧠 Memory for long-term user, assistant, and tool interaction history
• 🧬 Persona for user understanding, identity continuity, and personalization
• 💡 Reflection for self-improvement and long-term behavioral adaptation
• 📅 Planning for task decomposition, reminders, and future-oriented actions
• ⚙️ Behavior for response style, workflow policy, and proactive assistance
• 🧰 Tools through MCP, RAG, DeepResearch, and external integrations
• 😊 Virtual Character for real-time voice/avatar interaction

✨ Fully self-hostable and privacy-first — AlphaAvatar can run locally or on your own infrastructure, giving you control over your data, memory, tools, and behavior.

💡 AlphaAvatar is not just a chatbot. It is a foundation for building stateful, proactive, multimodal, and self-evolving personal AI assistants.

• [2026/06] Released AlphaAvatar version 0.6.0: Added the Status plugin, sampled visual input support, and status-aware DeepResearch / RAG / MCP tool feedback.

• [2026/05] Released AlphaAvatar version 0.5.4:

  • Added LanceDB-backed MCP tool retrieval, enabling AlphaAvatar to semantically search relevant MCP tools from Agent queries.
  • Refactored system prompt and runtime prompt composition, improved Persona runtime state tracking, added temporary-user to real-user identity merging, and improved RAG runtime behavior.
  • Released AlphaAvatar version 0.5.5: Fixed the inference runner registration lifecycle for production start mode, ensuring plugins runners are registered after config parsing and before LiveKit creates the inference executor.

• [2026/04] Released AlphaAvatar version 0.5.3:

  • Added localized Markdown backup for the Memory plugin.
  • Added LanceDB as the default local VDB option when Qdrant credentials are not provided.

• [2026/03] Released AlphaAvatar version 0.5.0:

  • Added the MCP plugin, enabling retrieval and concurrent invocation of MCP tools.
  • Released AlphaAvatar version 0.5.1: Added WhatsApp channel support via Baileys.
  • Released AlphaAvatar version 0.5.2: Added the AlphaAvatar Voice plugin with Voice.ai TTS support.

• [2026/02] Released AlphaAvatar version 0.4.0:

  • Added RAG support through RAG-Anything.
  • Optimized the Memory and DeepResearch modules.
  • Released AlphaAvatar version 0.4.1: Fixed Persona plugin bugs and added a new MCP plugin.

• [2026/01] Released AlphaAvatar version 0.3.0:

  • Added DeepResearch support through the Tavily API.
  • Released AlphaAvatar version 0.3.1: Added tool-call memory extraction during user–assistant interactions.

Install stable AlphaAvatar version from PyPI:

uv venv .my-env --python 3.11
source .my-env/bin/activate
pip install alpha-avatar-agents

Install latest AlphaAvatar version from GitHub:

git clone --recurse-submodules https://github.com/AlphaAvatar/AlphaAvatar.git
cd AlphaAvatar

uv venv .venv --python 3.11
source .venv/bin/activate
uv sync --all-packages

Start your agent in dev mode to connect it to LiveKit and make it available from anywhere on the internet.

🧩 Step 1. Configure Environment Variables

cd AlphaAvatar

# Copy template
cp .env.template .env.dev

Edit .env.dev and set required environment variables.

📦 Step 2. Download Required Files

alphaavatar download-files

✅ Step 3. Run the Agent

ENV_FILE=.env.dev alphaavatar dev examples/agent_configs/voice/pipeline_openai_tools.yaml
# or
ENV_FILE=.env.dev alphaavatar dev examples/agent_configs/mm/pipeline_openai_tools.yaml

To see more supported modes, please refer to the LiveKit doc.

To see more examples, please refer to the Examples README

AlphaAvatar is built around a real-time multimodal Agent runtime. It separates streaming interaction, model reasoning, runtime context, plugin state, tool execution, and channel adapters.

                         AlphaAvatar Runtime
                         ───────────────────

        ┌──────────────────────────────────────────────┐
        │                 LiveKit Room                 │
        │        audio / text / video / data streams   │
        └──────────────────────┬───────────────────────┘
                               │
                               ▼
        ┌──────────────────────────────────────────────┐
        │                 AgentSession                 │
        │      real-time turn detection / streaming IO │
        └──────────────────────┬───────────────────────┘
                               │
                               ▼
        ┌──────────────────────────────────────────────┐
        │                 AvatarEngine                 │
        │  Realtime Model Pipeline                     │
        │  STT / VAD / LLM / TTS / video-ready input   │
        └──────────────────────┬───────────────────────┘
                               │
                               ▼
        ┌──────────────────────────────────────────────┐
        │          Prompt & Runtime Context Layer       │
        │  system prompt / runtime prompt / time / mode │
        │  memory / persona / plan / reflection / rules │
        └──────────────────────┬───────────────────────┘
                               │
        ┌──────────────────────┼───────────────────────┐
        │                      │                       │
        ▼                      ▼                       ▼
┌──────────────────┐  ┌──────────────────┐   ┌──────────────────┐
│  Memory Plugin   │  │  Persona Plugin  │   │   Tools Plugins  │
│ conversation     │  │ user profile     │   │ MCP / RAG        │
│ tool memory      │  │ speaker / face   │   │ DeepResearch     │
│ avatar memory    │  │ runtime state    │   │ external APIs    │
└──────────────────┘  └──────────────────┘   └──────────────────┘

┌──────────────────┐  ┌──────────────────┐   ┌──────────────────┐
│ Reflection Plugin│  │ Planning Plugin  │   │ Behavior Plugin │
│ self-analysis    │  │ tasks / plans    │   │ response policy │
│ adaptation       │  │ reminders        │   │ workflow rules  │
└──────────────────┘  └──────────────────┘   └──────────────────┘
                               │
                               ▼
        ┌──────────────────────────────────────────────┐
        │              UserPath Workspace              │
        │  local data / cache / logs / memory / RAG    │
        │  persona state / artifacts / user files      │
        └──────────────────────┬───────────────────────┘
                               │
        ┌──────────────────────┴───────────────────────┐
        │                                              │
        ▼                                              ▼
┌──────────────────────┐                    ┌──────────────────────┐
│ Native Web / App     │                    │ Channel Adapters     │
│ voice / text / video │                    │ WhatsApp / future    │
│ WebRTC / UI          │                    │ messaging platforms  │
└──────────────────────┘                    └──────────────────────┘

AlphaAvatar supports multiple Access Channels, allowing different types of users — from end users to developers — to interact with the system.

AlphaAvatar now provides a browser-based realtime demo interface built on LiveKit.

👉 Try the Web Demo: https://www.alphaavatar.ai/demo

The Web Demo supports:

• 🎙️ Real-time voice interaction
• 💬 Text chat with the Avatar
• 📷 Camera preview and video-ready interaction
• 🔊 Agent audio playback
• 😊 Virtual character / avatar stage
• 🧠 Full plugin support, including Memory, Persona, RAG, MCP, and DeepResearch
• 🌍 Browser timezone metadata, enabling AlphaAvatar to understand local login time

The Web Demo is the recommended way to try AlphaAvatar with a full realtime multimodal experience.

Interact with AlphaAvatar directly inside messaging platforms.

Capabilities:

• 💬 Text-based conversation
• 🎤 Voice message interaction
• 🧰 Tool invocation via chat interface

📦 Channel introduction: README

▶️ Start WhatsApp Channel

Make sure AlphaAvatar Agent is already running (see Quick Start above).

ENV_FILE=.env.dev sh examples/channels/start_whatsapp.sh

💡 The WhatsApp channel runs as an independent bridge process and connects to the Agent runtime.

A dedicated AlphaAvatar mobile application providing:

• 🎙️ Real-time voice communication
• 😊 Live2D / Virtual character visualization
• 🧠 Persistent memory & persona

Developers can immediately access AlphaAvatar via the LiveKit Playground.

👉 https://agents-playground.livekit.io/

After starting your AlphaAvatar server:

1. Connect to your LiveKit instance
2. Configure the Agent name in the Playground (must match avatar_name, default: Assistant) to enable Explicit Dispatch.
3. Connect to the agent room
4. Start testing real-time interaction

Supported capabilities:

• 🎙️ Voice interaction
• 🧠 Memory extraction
• 🔍 RAG retrieval
• 🧰 MCP tool invocation
• 😊 Virtual character display

💡 AlphaAvatar is currently developer-first, with a Web Demo available for realtime interaction.

More user-facing web and mobile experiences are under active development.