Skip to content

verawu/memES

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2,126 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

MemES (Mem0 Elasticsearch Storage) — a production-grade memory system for AI agents, derived from mem0. Extends vector retrieval with multi-signal fusion ranking, self-managing memory quality, and full-lifecycle storage on Elasticsearch.

📖 Architecture Guide →


What is MemES?

MemES is an intelligent memory layer for AI assistants and agents, built on the foundation of mem0. It retains mem0's core API and provider ecosystem while adding three architectural layers that transform memory from simple storage into a complete memory management system:

  • Multi-Signal Fusion Retrieval — Beyond vector-only search, fusing 5 signals (semantic, BM25 keyword, entity relationships, time decay, access hotness) for precise relevance ranking
  • Self-Managing Memory Quality — LLM-gated candidate promotion prevents low-confidence memories from polluting primary storage; hotness scoring surfaces frequently-accessed memories and archives cold ones automatically
  • Production-Grade Storage — Elasticsearch ILM (hot → warm → cold → delete) for tiered data management, plus a pluggable HistoryStore backend (SQLite for dev, ES for production)

Key Benefits

Area mem0 (upstream) MemES
Retrieval Single vector semantic search 5-way signal fusion (semantic + BM25 + entity + recency + hotness)
Memory Quality Store-and-use, no filtering LLM confidence gating, evidence-based promotion, cold archival
Storage Pluggable vector stores ES-native ILM lifecycle management, pluggable history backend
History SQLite only SQLite (dev) + Elasticsearch (prod) backends via abstract interface

Architecture Overview

MemES organizes memory management into three layers. Each is covered in detail in the Architecture Guide.

1. Five-Way Fusion Retrieval

Replaces single-vector retrieval with a multi-signal pipeline that covers semantic relevance, exact keyword matching, entity relationships, temporal recency, and access frequency.

Five-Way Fusion Retrieval Pipeline

Signals:

# Signal Source Normalization
1 Semantic Cosine similarity (vector search) Native [0, 1]
2 BM25 Keyword BM25 via Elasticsearch Sigmoid, query-length-adaptive
3 Entity Boost Entity store linked memories Cosine × decay, capped 0.5
4 Time Decay 3 functions × 4 presets (task/preference/knowledge/default) [0, 1], weight 0.5
5 Hotness Access frequency × access recency Sigmoid(freq) × exp(recency), weight 0.5

Fusion: final = min((S₁ + S₂ + E₃ + R₄ + H₅) / max_possible, 1.0) — preserves score granularity (unlike RRF which discards confidence).

2. Hotness Scoring + Candidate Promotion

Two mechanisms that keep your memory store clean and relevant:

  • Hotness Scoring — Tracks access frequency and recency for every memory. Hot memories rank higher; cold memories (age ≥ 30d, hotness < 0.1) are auto-archived.
  • Candidate Promotion — New memories start as "candidates." An LLM confidence gate scores each extraction (0.5–1.0). Below-threshold entries stay hidden from search until confirmed by repeated evidence (≥2 mentions or confidence ≥ 0.9).

Hotness Scoring and Candidate Promotion

3. ILM Tiered Storage + Pluggable History

Built on Elasticsearch's Index Lifecycle Management for automatic data tiering:

Phase Timing Action
HOT Immediate SSD writes, replica ×1
WARM 30 days Forcemerge to 1 segment
COLD 90 days Frozen, read-only
DELETE 365 days Auto cleanup

History storage uses a pluggable backend architecture (HistoryStoreBase abstract interface):

  • SQLiteManager — Embedded, zero-config, ideal for development
  • ElasticsearchHistoryStore — Production distributed, two data streams for history + messages

Switch backends with zero code changes.

ILM Tiering and History Backend


Multi-Tenant Authentication & Authorization

MemES adds a two-layer authentication system for production deployment, built on Elasticsearch security primitives.

Server-Level API Key Auth

The FastAPI server (server/main.py) supports optional single-key authentication via the X-API-Key header:

ADMIN_API_KEY=your-secret-key    # set to enable; empty = open access

When enabled, all endpoints (except /docs and /openapi.json) require a valid X-API-Key header. Uses secrets.compare_digest for timing-attack resistant comparison.

Collection-Based Multi-Tenant Middleware

For multi-tenant deployments, MemES provides an Elasticsearch-backed collection isolation system (mem0/middleware/). Each collection (previously called workspace) gets:

  • Dedicated ES indicesmem0_{collection_id} for memory, mem0_{collection_id}_entities for graph entities
  • Scoped ES roles & API keys — a role bound to the collection's index pattern, with a corresponding ES API key
  • Isolated Memory instances — LRU-cached, collection-scoped Memory objects with dedicated history DB

Authentication flow:

Layer Header Purpose
Collection access X-Workspace-Key ES API key scoped to a single collection
Admin X-Admin-Key Management operations (create/delete collections, rotate keys)

Management API (/v1/collections):

Endpoint Method Description
/v1/collections/ POST Create collection + generate initial API key
/v1/collections/ GET List collections (filter by owner_id)
/v1/collections/{id} GET/DELETE Get or delete a collection
/v1/collections/{id}/keys POST Create additional API keys
/v1/collections/{id}/keys/rotate POST Rotate all API keys
/v1/collections/keys/{key_id} DELETE Revoke a specific API key

Integration points:

  • ASGI MiddlewareCollectionMiddleware injects collection context into request.state for downstream handlers
  • FastAPI Dependenciesget_collection_context / get_collection_memory for per-route injection, including auto-error on missing keys
  • Standalone usageCollectionAuth.authenticate(api_key) validates any ES API key and returns the resolved CollectionContext

Configuration via environment variables:

WORKSPACE_ES_HOST=localhost
WORKSPACE_ES_PORT=9200
WORKSPACE_ES_API_KEY=...           # or WORKSPACE_ES_USER + WORKSPACE_ES_PASSWORD
WORKSPACE_ADMIN_KEY=...            # admin key for collection management
WORKSPACE_ES_CLOUD_ID=...          # Elastic Cloud ID (optional)
WORKSPACE_USE_SSL=true             # default false

Quickstart

Installation

pip install mem0ai

For NLP-enhanced hybrid search (BM25 + entity extraction):

pip install mem0ai[nlp]
python -m spacy download en_core_web_sm

TypeScript SDK:

npm install mem0ai

Basic Usage

from openai import OpenAI
from mem0 import Memory

client = OpenAI()
memory = Memory()

# Store a memory
memory.add("User prefers dark mode and keyboard shortcuts", user_id="alice")

# Search with 5-way signal fusion
results = memory.search("What does Alice prefer?", user_id="alice")
print(results)

See the Architecture Guide for detailed configuration of fusion weights, ILM policies, and promotion thresholds.


Provider Ecosystem

MemES retains full compatibility with mem0's provider ecosystem:

  • LLMs (24): OpenAI, Anthropic, Gemini, Groq, Ollama, Together, AWS Bedrock, Azure OpenAI, DeepSeek, vLLM, LiteLLM, xAI, and more
  • Vector Stores (30): Qdrant, Pinecone, Chroma, Weaviate, Milvus, MongoDB, Redis, Elasticsearch, pgvector, Supabase, Faiss, and more
  • Embeddings (15): OpenAI, Azure OpenAI, Gemini, HuggingFace, FastEmbed, Together, AWS Bedrock, Ollama, Vertex AI
  • Graph Stores (4): Neo4j, Memgraph, Kuzu, Apache AGE
  • Rerankers (5): Cohere, HuggingFace, LLM-based, Sentence Transformer, Zero Entropy

Documentation & Support

Citation

@article{mem0,
  title={Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory},
  author={Chhikara, Prateek and Khant, Dev and Aryan, Saket and Singh, Taranjeet and Yadav, Deshraj},
  journal={arXiv preprint arXiv:2504.19413},
  year={2025}
}

License

Apache 2.0 — see LICENSE.

About

A production-grade memory system for AI agents, derived from mem0. Extends vector retrieval with multi-signal fusion ranking, self-managing memory quality, and full-lifecycle storage on Elasticsearch.

Resources

Contributing

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages