Reading papers is still a solo grind: PDF in one window, notes somewhere else, Google tabs everywhere, ChatGPT on the side, Anki later, Zotero maybe, and your own half-broken memory trying to hold the thread.

ResearchMate turns one paper into a living workspace. You open the paper, and suddenly you are not alone: a small lab appears around it.

Instead of one chatbot, ResearchMate gives you a lab bench of grounded helpers: one explains, one challenges, one builds your wiki, one creates practice, one tracks what you forget, and one connects the paper to your longer research direction.

Every paragraph can become a Feynman explanation, a flashcard, a quiz, a wiki node, a debate, a related-paper search, or a research idea.

The language model (Gemma 4 on Cerebras) is the fast, grounded voice.

Cognee is the lab notebook that makes it a partner instead of a document Q&A box, a hybrid graph+vector memory that every agent reads before it acts and writes after you work.

---

How To Use

1. Open ResearchMate.
2. Create a project or reopen an existing one.
3. Add the papers ResearchMate is allowed to use.
4. Build the project graph.
5. Open a concept from the graph, then start studying.
6. Read in the PDF workspace and select text when you want grounded help.
7. Use Wiki for quick drill-downs.
8. Use Chat for questions, /quiz <topic>, /flashcards <topic>, and paper recommendations.
9. Use Pair Buddy for Socratic/Feynman-style explanation practice.
10. Open Citation Graph to inspect and verify paper-to-paper relationships (best with two or more related uploaded papers).
11. Commit when you want ResearchMate to write observations into memory and update the project. Reopen the project afterward and continue studying to see it pick up where you left off.

---

Feature Gallery

Project Start Window

• Project-first workflow: create a named project, reopen an existing one, and add or remove papers over time.
• Material gating: ResearchMate does not study from empty space; papers are visibly the source of truth.
• Control room: project status, memory health, citation graph artifacts, settings, and activity updates are surfaced from the start window.
• Editable behavior: project preferences, learner profile, and internal agent skills can be adjusted through ResearchMate Settings.

Paper-Centered Workspace

• Read uploaded papers inside the app.
• Select text to ground Wiki and Chat.
• Add margin notes beside the paper.
• Send detected figures or regions into study tools.

Knowledge Graph As Table Of Contents

• Auto-generates a concept map from uploaded material.
• Uses honest states: ongoing or completed, not fake mastery scores.
• Uses Cognee memory to reshape future study.
• Commit runs the idea-observer, writes memory, and lets the Graph Curator add explored concepts.

Citation Graph

• Shows uploaded project papers as graph nodes.
• Connects papers with verified paper-to-paper citation/reference edges.
• Provides context for related-paper recommendations.

Wiki

• Turns selected terms into grounded explanation cards.
• Supports active recall prompts.
• Can attach grounded visuals through the visual service.

Chat

• Answers from project material and selected context.
• Supports /quiz <topic> and /flashcards <topic> as inline tools.
• Mic dictation is wired for transcription-only input: record audio, insert the transcript into the draft, then edit/send manually.
• Routes paper recommendation requests through project, citation, and memory context.
• Optional Net Support adds explicit web citations when configured.

Pair Buddy

• Runs Socratic pair-reading conversations.
• Feynman mode lets the student teach a concept back.
• Style feedback is stored as learner preference memory.
• Voice Buddy is visible as a future shell; dictation infrastructure exists, but full voice-out and hands-free STT -> LM -> TTS loop are deferred.

Sandboxed Visuals

• Generates visuals lazily.
• Declines when there is no meaningful grounding.
• Runs generated HTML in a sandboxed iframe.
• Repairs runtime errors through /sandbox/repair.

Local Memory Layers

• Cognee memory lives locally under ~/.studybuddy/cognee/.
• Embeddings run locally through fastembed / sentence-transformers.
• Memory failures are surfaced as degraded/unavailable instead of silently becoming plain RAG.
• research-memory-reviewer is loaded as Cognee skill content for project memory review.

Signoz Observability Layers

---

Project Motivation

Reading research is a long-horizon, accumulating activity but the tools for it have no memory and no shape.

• Chatbots forget. A pure-RAG tutor (or a raw LLM) knows only what the current document says. Reload the same paper next week and it re-explains what you've mastered, re-asks what you've answered, and never notices you've been stuck on the same idea for three sessions. There is no you in the loop.
• Material arrives over time, but tools are one-shot. Real study grows a corpus incrementally - add a chapter today, a problem set next week, connect them later. Session-based tools treat every upload as a disposable, isolated event with no through-line.
• LLMs hallucinate, and "it rendered" isn't "it's right." A model will happily fabricate a citation, a formula, or a visualization that runs cleanly and teaches something false.
• "Understanding" gets faked with numbers. A monotone score bolted onto a graph measures activity, not whether an idea actually holds up against the source.

Solution

ResearchMate is a grounded, memory-load-bearing research partner.

Your material groups into resumable projects; a knowledge graph gives it shape; every answer is sourced from your content; and a local hybrid graph+vector memory of how you read, struggle, and explore underlies every agent - so the lab rearranges itself around this learner and this research direction, not just the document open right now.

flowchart LR
  U["Upload your material<br/>(grouped into Projects)"] --> G["Auto-extracted<br/>Knowledge Graph"]
  G --> S["Workspace: Wiki · Chat<br/>Pair Buddy · Citation Graph"]
  S --> C["Commit"]
  C --> M[("Cognee memory<br/>day 10 &gt; day 1")]
  M -.->|"read before every action<br/>(Brain · Tutor · Evaluator · Curator)"| G

Loading

• Grounded-only. Agents organises and rephrases your material and cites every claim ([Source: file, chunk N]); it never invents facts from its weights (except explicitly-cited web results in Net mode).
• Memory changes behavior, not just prompts. The Brain restructures the curriculum on your history, the Tutor adapts per-concept, the Evaluator judges your ideas against your trajectory, and the Graph Curator spawns nodes for what you explored off-path (detailed below).
• Ideas judged against sources, students never scored. Two honest node states (ongoing / completed) and a cited-prose idea-observer replace fabricated mastery numbers.
• Hallucination-resistant by construction. Verbatim, self-checking anchors and a plan→QC→code→QC visual loop make fabricated content unusable, not merely discouraged.

---

Memory: the layers that make it a study partner

The single most important thing about ResearchMate is that it doesn't start from zero every time. That isn't one system, though; it's a small set of purpose-built memory layers working together, gated by a shared promotion rule so nothing becomes durable memory just because a model said it. Cognee is the newest and most novel of those layers (durable cross-project personalization), but it isn't the only thing standing between this app and a stateless RAG wrapper: project memory persists a project's own accumulated claims independently of Cognee, and canonical evidence persists independently of both.

Full memory architecture: layers, the write pipeline, the read boundary, and failure visibility (verified against code)

The layers

Layer	Backend	Scope	What it actually holds
Source evidence	SQLite + FTS5 (EvidenceStore, canonical) + ChromaDB paper_evidence (dense index)	Per-project	What uploaded papers and bounded regions actually say, never mutated by an agent, only by re-ingestion or deletion
Project memory	ChromaDB project_memory collection (ProjectMemoryStore, app/rag/project_memory.py)	Per-project, durable across the project's life	This project's own accumulated, evidence-grounded observations: claims, methods, open questions, notes
Student profile	Cognee research_profile dataset	Cross-project, durable	Recurring learner traits (preferences, recurring strengths/weaknesses); the one layer that survives a project ending
Memory-integrity ledger	Local JSONL under ~/.studybuddy/memory_integrity/	Operational, not agent-readable	Append-only promotion-decision audit trail, crash-safe pending-review state, a temporal-recall fallback

How a raw interaction becomes durable memory

Nothing an agent says becomes memory just by being said. On Commit, run_commit_adaptation() (app/services/commit_adaptation.py) reviews unreviewed student interactions plus already-approved staged profile candidates in one Cerebras call, and emits typed MemoryCandidate objects (app/services/memory_candidates.py), each tagged with a destination ("project" or "student") and an attribution (explicit_student, idea_observer_interaction, or idea_observer_profile_proposal).

Every candidate is then checked by MemoryPromotionGate (app/services/memory_promotion.py):

• Project-destined candidates are rejected outright if they're an assistant-only claim with no backing interaction_ids, if their evidence_ids aren't valid canonical (ev_-prefixed) EvidenceStore ids, or if the statement is empty. Otherwise they're promoted straight into project memory.
• Student-destined candidates promote immediately only for an explicit student signal (the student said it about themselves). Everything else needs the same claim (matched by a normalized recurrence_key) to reappear across at least 2 different projects with confidence ≥ 0.75 before it's trusted as a durable cross-project trait: one confident-sounding session is never enough on its own.
• Every decision, promoted or rejected, and why, is written to an append-only ledger at ~/.studybuddy/memory_integrity/promotion_decisions.jsonl.

The Cognee write itself (flush_project() → cognee.improve(dataset="research_profile", session_ids=[project_id])) is the expensive step and is crash-safe: a pending_review_store status machine (ready_to_flush → cognee_flush_started → cognee_flush_succeeded / cognee_flush_uncertain) means a process crash mid-flush gets quarantined on restart instead of being silently retried or duplicated.

How memory comes back

query_prior_knowledge(topic, project_id, mode) (app/services/student_memory.py) is the single boundary every agent goes through, and it routes by mode:

• mode="project" reads ProjectMemoryStore (Chroma) directly, this path never touches Cognee.
• mode="profile" / "concept" / "temporal" read Cognee's research_profile dataset via cognee.recall(query_type=SearchType.GRAPH_COMPLETION | SearchType.TEMPORAL, only_context=True, ...), falling back to a local temporal JSONL ledger if Cognee's own temporal recall errors or returns nothing. A DatabaseNotCreatedError / "No data found" is treated as a healthy empty state (fresh install, no Commit yet), not a failure.

Four consumers, four structural effects: this is the actual mechanism behind "day 10 feels different from day 1".

Agent	Reads	Structural effect
Brain Agent	prior knowledge before building the tree	tags each section new / review / scaffold; collapses mastered ground, inserts scaffold nodes for revealed gaps, orders known→new
Evaluator (idea-observer)	cross-session trajectory at Commit	judges ideas against history ("still conflates X as in an earlier session", "moved from what-is to why here") instead of scoring the session in isolation
Tutor	per-concept memory on lesson open	adapts what a lesson leans into vs. skims, per concept, per student
Graph Curator	prior context at COMMIT_PROJECT	filters "genuine engagement" from "passing mention" before spawning origin="exploration" nodes

flowchart TB
  subgraph READ["READ - before acting: query_prior_knowledge(mode=...)"]
    B["Brain Agent<br/>branches the curriculum structurally"]
    T["Tutor<br/>adapts each lesson per-concept"]
    E["Evaluator<br/>judges ideas against your trajectory"]
  end
  B & T & E --> J["Interaction Journal"]
  J -->|"Commit"| ADA["run_commit_adaptation()<br/>MemoryCandidate + MemoryPromotionGate"]
  ADA -->|"promoted"| PM[("Project memory<br/>Chroma project_memory")]
  ADA -->|"promoted (>=2 projects)"| COG[("Cognee research_profile<br/>~/.studybuddy/cognee")]
  ADA -->|"every decision"| LEDGER[("promotion_decisions.jsonl")]
  J -->|"Commit"| CUR["Graph Curator<br/>spawns exploration nodes"]
  CUR --> COG
  PM -.->|"feeds the next READ"| READ
  COG -.->|"feeds the next READ"| READ

Loading

Cognee itself: the exact write and read calls

Everything here is backend/app/services/student_memory.py, class StudentMemoryService. There is one Cognee dataset in use: PROFILE_DATASET = "research_profile".

Write. stage_promoted_candidate(candidate) is the only entry point MemoryPromotionGate output feeds into; it rejects any candidate whose destination != "student", then forwards to stage_profile_observation(project_id, text, kind, attribution, confidence, interaction_ids, evidence_ids). That method:

1. Collapses whitespace in text and truncates to 1000 chars to get statement.
2. Builds signal_id = f"sm_{sha256(f'{kind}\\0{statement.casefold()}')}" and a JSON payload {"schema": "student_profile_signal.v1", "signal_id", "kind", "statement", "attribution", "confidence" (clamped to [0,1]), "interaction_ids", "evidence_ids", "observed_at"}.
3. Calls the private _remember() helper, which runs cognee.remember(json_text, dataset_name="research_profile", session_id=project_id, self_improvement=False, temporal_cognify=True). If the installed Cognee build rejects temporal_cognify as a kwarg (TypeError), it retries once with that kwarg dropped.
4. On success, clears the class-level _profile_recall_cache dict so the next read is never served a stale rendering.

Flush. flush_project(project_id, strategy="improve") iterates a single-entry list [("research_profile", ensure_profile_dataset)] and calls _flush_dataset() per entry, which:

• Awaits cognee.improve(dataset="research_profile", session_ids=[project_id]) inside a class-level asyncio.Lock (_cognee_flush_lock), then sleeps COGNEE_FLUSH_SPACING_SECONDS = 1.0 before releasing it, so two flushes never hit Cognee back-to-back.
• With strategy="distill_then_improve", first calls cognee.session.distill_session(session_id=project_id, dataset=dataset) if that attribute exists on the installed Cognee build.
• Catches TypeError (Cognee API signature drift) and any other exception separately; both mark the operation degraded on the observability span and return False rather than raising.

Read. query_prior_knowledge(topic, project_id, mode="concept", *, cache_only=False, bypass_cache=False) where MemoryMode = Literal["concept", "profile", "project", "temporal"]:

• mode="project" never touches Cognee: it calls ProjectMemoryStore().search(project_id, topic, limit=8, consumer="project_memory") and renders each hit as f"- [{kind}; evidence={','.join(evidence_ids) or 'interaction'}] {statement}".
• For mode in {"profile", "concept"} (not bypass_cache), it first checks _profile_recall_cache, keyed by the whitespace-collapsed, casefolded, 240-char-clipped topic, with a _profile_recall_cache_ttl_seconds = 300.0 TTL.
• On a cache miss, _datasets_for_mode() resolves to ["research_profile"] for every non-project mode (there is only one dataset today), and _query_for_mode() substitutes the raw topic into a fixed synthetic query string per mode rather than passing it through verbatim, for example profile mode's literal query is f"student profile preferred name call me persona preferences learning style recurring strengths weaknesses {topic}".
• The actual call goes through _recall_with_fallback(): cognee.recall(query_text=..., query_type=SearchType.TEMPORAL if mode == "temporal" else SearchType.GRAPH_COMPLETION, datasets=[...], session_id=project_id or None, top_k=8, only_context=True, feedback_influence=0.35), retried up to 3 times on TypeError, first dropping only_context, then also dropping feedback_influence, to tolerate an installed Cognee version that doesn't accept one of those kwargs.
• Results are text-extracted, clipped, joined (top 5 chunks), and rendered as f'Prior {mode} memory for "{topic}":\\n' + clipped_context; profile/concept renderings are written back into _profile_recall_cache.
• DatabaseNotCreatedError / a "No data found in the system" message is caught and reported as memory_liveness="empty" (a fresh install, not a degradation). Any other exception is recorded on the span, and for mode="temporal" falls back to a local JSONL ledger (_query_temporal_ledger()) before returning empty.

Bootstrap and locality

Cerebras is aliased as an OpenAI-compatible LiteLLM provider for Cognee (app/main.py lifespan, app/services/cognee_bootstrap.py), llm_instructor_mode="json_schema_mode" is forced (Cerebras doesn't reliably support instructor's default TOOLS-mode extraction, so without this Gemma can emit markdown-fenced JSON that fails Cognee's Pydantic parse), and embeddings are forced local (fastembed, sentence-transformers/all-MiniLM-L6-v2, 384-dim, since Cerebras has no embeddings endpoint). Cognee's data root is pinned to ~/.studybuddy/cognee/ (LanceDB for vectors, SQLite for relational metadata); combined with the Chroma-backed layers also living under ~/.studybuddy/, none of it ever leaves the machine.

Memory tells you when it's not okay

memory_status() reports each layer's actual state (ready / empty / degraded / unavailable) instead of silently returning empty context and letting the app pretend nothing changed: a degraded Cognee dataset is visible, not hidden behind a quietly-empty answer.

Pull Cognee specifically out and the Brain Agent produces an identical tree on day 1 and day 10 for a returning student across projects. Project memory (Chroma) still makes a single project durable across its own sessions, though, and canonical evidence still grounds every citation. Together, these layers are what keep it from being a stateless wrapper around an LLM; Cognee is the piece responsible for the cross-project half of that claim.

---

Setup & Running

Click to expand

ResearchMate can be run two ways: Docker (the easiest route, only Docker itself is required) or a native setup with Python and Node directly (needed if you want the full Electron desktop shell). Both need the same API keys.

Docker (preferred)

The only requirement is Docker itself.

cp backend/.env.example backend/.env
# Edit backend/.env with your API keys, see Environment Variables below.
docker compose up --build

Open http://localhost:5173. This runs the same Vite frontend and FastAPI backend as npm run dev; it's also the exact workflow CI validates on pull requests and on master (it builds the images but does not publish them). The Electron desktop shell isn't containerized, so run npm run dev:electron natively for that.

Native setup

Prerequisites

Tool	Version	Install
Python	3.12+	python.org
uv	latest	pip install uv or docs.astral.sh/uv
Node.js	20+	nodejs.org

Quick Start

# 1. Clone
git clone https://github.com/solusops/ResearchMate.git
cd ResearchMate

# 2. Install dependencies
npm install
# Installs root + frontend Node workspaces and runs `uv sync --directory backend`.

# 3. API keys
cp backend/.env.example backend/.env
# Edit backend/.env:
#   CEREBRAS_API_KEY=csk-...       (required)
#   TAVILY_API_KEY=tvly-...        (optional -> enables Net Support mode)
#   YOUTUBE_API_KEY=...            (optional -> enables Deep Dive)

# 4. Launch
npm run dev

Open http://localhost:5173. The "Start Studying" button activates once the backend finishes loading (~10 to 15s on first run while the embedding model warms up).

Other run commands

Command	What it does
npm run dev	Starts Vite (localhost:5173) + uvicorn (127.0.0.1:8765) concurrently
npm run dev:docker	Same dev servers, bound to 0.0.0.0 for containers
npm run dev:electron	Full Electron desktop shell + Vite
npm run setup:node	Installs only Node dependencies, skipping backend postinstall
npm run setup:backend	Runs uv sync --directory backend
cd backend && uv run uvicorn app.main:app --reload --host 127.0.0.1 --port 8765	Backend only
cd frontend && npm run dev	Frontend only (browser, no Electron IPC)

Running Tests

ResearchMate ships with a large automated suite: 476 backend tests (backend/tests/, pytest) and 118 frontend tests across 28 files (frontend/src/**, Vitest).

# Full backend suite (476 tests)
cd backend
uv run pytest tests/ -v

# Single test file
uv run pytest tests/test_tutor_lesson.py -v

# Frontend tests (118 tests across 28 files)
cd frontend
npx vitest run

🔑 Environment Variables

Variable	Required	Description
CEREBRAS_API_KEY	✅	Cerebras Cloud API key for Gemma 4 inference
TAVILY_API_KEY	❌	Enables "Net Support" knowledge mode (web search fallback)
YOUTUBE_API_KEY	❌	Enables Deep Dive video search
ALLOWED_ORIGINS	❌	CORS origins (defaults to http://localhost:5173)
STT_MODEL_PATH	❌	Explicit whisper.cpp model file path. Defaults to pywhispercpp's local ggml-base.en-q5_1.bin path
STT_VAD_ENGINE	❌	Live backend VAD engine for dictation: none (default), ten, or silero
VOICE_LOG_TRANSCRIPTS	❌	When false, benchmark rows keep transcript length but omit full transcript text
VITE_BROWSER_VAD_ENABLED	❌	Frontend browser VAD UX toggle. Defaults to enabled; it never trims uploaded audio
DEPLOYMENT_ENV	❌	desktop (default) uncaps concurrency (e.g. 50 concurrent Cerebras calls) for local hardware. demo throttles internal semaphores severely (e.g. max 5 concurrent Cerebras calls) to survive constrained hosts like Hugging Face Spaces without OOM/429s

Local data layout (~/.studybuddy/)

Everything below lives under ~/.studybuddy/. Nothing leaves the machine except Cerebras inference calls and, optionally, Tavily/YouTube/scholar-provider lookups.

Path	Owner	What
projects/registry/	project_service.py	Project metadata registry
projects/uploads/{project_id}/	project_files.py	Per-project uploaded files, isolated per project
projects/intelligence/	project_intelligence.py	Auto-updated project brief, refreshed on Commit
projects/activity/	project_activity.py	Project activity feed (Control Room)
pdfs/{file_id}.pdf	projects.py / library.py	Content-addressed PDF cache, keyed by file SHA-256
graphs/{project_id}.json	graph_state.py	Live graph state (ongoing/completed nodes)
sessions/{project_id}.json	project_commit.py	Commit snapshot / Project History
evidence/	evidence_store.py	Canonical EvidenceStore (SQLite + FTS5)
regions/{project_id}/{document_id}.json	evidence_ingestion.py	Detected/manual figure & table regions
annotations/{document_id}.json	annotation_service.py	Margin notes
chroma/	chromadb_client.py	ChromaDB (paper_evidence, project_memory collections)
cognee/	student_memory.py	Cognee LanceDB + SQLite (research_profile dataset)
memory_integrity/	student_memory.py, memory_promotion.py	Promotion-decision ledger, pending memory staging, temporal ledger
citation_graphs/	citation_graph.py	Durable per-project citation graph
citation_exploration/	citation_exploration_store.py	Transient paper/author exploration cache
recommendations/, recommendation_clarifications/	recommendation_service.py	Paper recommendation + clarification cache
paper_metadata/	paper_metadata.py	Resolved provider metadata cache (OpenAlex/Semantic Scholar/arXiv)
cache/	output_cache.py	Deterministic LLM output cache
agent_control/	agent_control.py	Editable project/learner/skill settings (persona engine)
summaries/	summary_writer.py	Commit summary Markdown export
visual_lessons/	visual_lesson_store.py	/visualize composition/chart engine cache
models/, voice/	tts_service.py, voice_selection.py	Local STT/TTS model files and voice defaults
observability/	observability/config.py	SigNoz/OTel artifact export root

---

Voice Dictation

Current voice support on master is dictation-only. The Chat mic records audio, prepares a 16 kHz mono WAV in the browser, sends it through TRANSCRIBE_AUDIO (backend/app/websockets/handlers.py), and inserts the returned text into the current draft (frontend/src/components/study-tools/VoiceTool.tsx). It does not auto-send, interpret voice commands, or trigger an agent response.

The backend STT boundary is TranscriptionService.transcribe(...) -> STTResult (backend/app/services/transcription_service.py), backed by pywhispercpp / whisper.cpp, defaulting to the base.en model (ggml-base.en-q5_1.bin, q5_1 quantization, English, transcribe-only). Override the model file with the STT_MODEL_PATH environment variable.

The fuller voice work, including STT/TTS benchmark comparisons, voice presets, and the Pair Buddy Voice Buddy hands-free pipeline, lives on a separate, actively-developed branch rather than master: VoiceLab.

---

🧱 Architecture

Three processes at runtime, inside one Electron shell: a React renderer, a FastAPI backend hosting every agent and the RAG store, and two persistence layers (ChromaDB for content, Cognee for load-bearing memory). The only cloud call is grounded Gemma 4 inference on Cerebras; web sources are optional and per-turn.

flowchart TB
  subgraph SHELL["🖥  Electron Desktop Shell"]
    direction TB
    FE["React + Vite Renderer - :5173<br/>Project Start Window · PDF Reader · Knowledge Graph · Wiki · Chat · Pair Buddy · Citation Graph"]
    subgraph BE["FastAPI Backend - :8765"]
      WS["WebSocket dispatch (BUILD_GRAPH · CHAT_TURN · COMMIT_PROJECT · …)"]
      AG["Agent roster<br/>Brain · Tutor · Evaluator(idea-observer) · Graph Curator<br/>Net Research · Draft · Wiki · Senses · Modality Router"]
      RAG[("ChromaDB - content RAG (per-project)")]
      MEM["StudentMemoryService"]
    end
  end
  FE <-->|"WebSocket + REST"| BE
  AG --> CB["LLM<br/>strict JSON schema + streaming"]
  MEM --> COG[("Cognee (local)<br/>LanceDB + SQLite · ~/.studybuddy/cognee")]
  AG -.->|"optional, per-turn"| NET["🌐 Tavily · arXiv · YouTube"]

Loading

Engineering patterns behind the diagram

Six patterns recur across the codebase, each named for the file(s) that implement it:

Content-addressed document identity. Two hash-based identifiers drive caching and dedup: file_id (SHA-256 of one uploaded file's bytes, ChromaDBClient.file_hash, backend/app/rag/chromadb_client.py) and document_id (an order-independent combined hash of the whole file set in a project, ProjectService.derive_document_id, backend/app/services/project_service.py: sorted per-file hashes, joined, re-hashed). Uploading the same set of papers again, in any order, resolves to the same document_id, so the curriculum tree is replayed from ~/.studybuddy/graphs/doc_{document_id}.json (built by backend/app/agents/brain_agent.py, served by the BUILD_GRAPH handler in backend/app/websockets/handlers.py) instead of regenerated.

Multi-stage curriculum generation with cross-paper merge detection. The curriculum pipeline is three explicit stages, all in backend/app/agents/brain_agent.py: derive_root_and_sections() (coarse root + sections, each tagged with which source document(s) it draws from) → expand_section() (one parallel call per section, aware of sibling sections so it doesn't duplicate their topics) → cleanup_curriculum() (finds the same concept described differently across papers and merges those into one node tagged is_merged with a merge_summary). Every downstream consumer of a merged node (lessons, chat, Feynman) is told it's a merge so it attributes claims to the correct paper.

Ideas judged against sources, never students scored. The graph holds two node states, ongoing and completed (explicit, via a "Mark Completed" button), not a numeric mastery score, enforced in GraphStateManager.apply_node_patch() (backend/app/services/graph_state.py, mirrored in frontend/src/store/graphStore.ts). EvaluatorAgent.evaluate_session() (backend/app/agents/evaluator_agent.py) is an idea-observer that writes short, cited, prose observations about ideas and how they held up (cognee.remember()), judged against cross-session trajectory; there's no rubric lookup table to protect.

Lazy generation with a self-healing sandbox. LEARN_NODE (backend/app/websockets/handlers.py) returns lesson text only; the HTML5 visual generates on-demand when the student opens the Visual tab. The 3D/2D-animation shell path (backend/app/agents/shell_visual_engine.py) gets a real syntax preflight for free: the model's JS is passed as the final argument to new Function(...), so a parse failure throws a genuine browser SyntaxError, replacing an earlier Python-side compile() check that couldn't actually validate JavaScript. A client-side runtime error posts to POST /sandbox/repair (ShellVisualEngine.repair_setup_code / TutorAgent.repair_visual, dispatched from backend/app/routers/sandbox.py) for an automatic repair round-trip. Delivery is always a sandboxed iframe (frontend/src/components/panel/VisualSandbox.tsx): sandbox="allow-scripts", no allow-same-origin, srcDoc only, fully self-contained.

Structured output as the connective tissue. Every agent-to-agent and agent-to-frontend boundary is a strict=True JSON-schema Pydantic model, never free-text parsing. CerebrasClient._build_schema() (backend/app/agents/cerebras_client.py:90) inlines $refs and forces additionalProperties: false at every nesting level; CerebrasClient.structured_complete() (same file, :175) catches both json.JSONDecodeError and pydantic.ValidationError and retries once. A dozen independently-developed agents compose without any of them needing a bespoke parser for another agent's output.

Defense-in-depth over trust-the-first-answer. Several pipelines validate a generated artifact and fall back rather than assuming success, all in backend/app/agents/brain_agent.py: a cached curriculum graph is checked for structural validity and full document coverage before replay; cleanup_curriculum() rejects its own output in favor of the pre-cleanup tree if the pass would drop content; a blank/generic curriculum root label (_is_generic_label()) triggers a second, differently-framed generate_session_title() attempt via _fallback_topic_name() before ever falling back to filename-derived naming.

Detailed component map (auto-generated)

┌────────────────────────────────────────────────────────────────┐
│                        Electron Shell                          │
│  ┌────────────────────────┬────────────────────────────────┐   │
│  │     React Frontend     │       FastAPI Backend          │   │
│  │     (Vite, port 5173)  │       (uvicorn, port 8765)     │   │
│  │                        │                                │   │
│  │  PDFReader ◄──────────►│  WebSocket /ws/{project_id}    │   │
│  │  InfiniteWiki          │  ├─ BrainAgent (curriculum)    │   │
│  │  ChatTool              │  ├─ TutorAgent (lessons)       │   │
│  │  FlashcardTool         │  ├─ NetResearchAgent (chat)    │   │
│  │  QuizTool              │  ├─ StudyBuddyAgent (Feynman)  │   │
│  │  PairBuddyTool         │  ├─ ReportAgent (canvas)       │   │
│  │  ReportView            │  ├─ WikiAgent / InfinityWiki   │   │
│  │  VisualSandbox         │  ├─ EvaluatorAgent             │   │
│  │  KnowledgeGraph        │  ├─ SensesAgent (vision)       │   │
│  │  EvaluationView        │  └─ ModalityRouter             │   │
│  │                        │                                │   │
│  │  Zustand Stores ──────►│  REST Routers                  │   │
│  │  (session, context,    │  ├─ /library   (per-session    │   │
│  │   interaction, graph)  │  │              upload+history)│   │
│  │                        │  ├─ /session   (create/commit/ │   │
│  │                        │  │              clear/trajectory)  │
│  │                        │  ├─ /regions   (figure detect)  │  │
│  │                        │  ├─ /annotations (margin notes) │  │
│  │                        │  ├─ /sandbox   (visual repair)  │  │
│  │                        │  ├─ /review    (Cognee recall)  │  │
│  │                        │  └─ /api       (health, keys)   │  │
│  │                        │                                 │   │
│  │                        │  Services                       │   │
│  │                        │  ├─ ChromaDB (per-session RAG)  │   │
│  │                        │  ├─ StudentMemoryService(Cognee)│   │
│  │                        │  ├─ MemoryService (report/traj.)│   │
│  │                        │  ├─ project_files/project_commit│   │
│  │                        │  ├─ LayoutService (PyMuPDF)     │   │
│  │                        │  ├─ OutputCache / JournalService│   │
│  │                        │  └─ AnnotationService           │   │
│  └────────────────────────┴───────────────────────────── ───┘   │
│                              │                                  │
│              ┌───────────────┼────────────────┐                 │
│    ┌─────────▼──────────┐    │    ┌────────────▼───────────┐    │
│    │          LLM       │    │    │   Cognee (local)       │    │
│    │                    │    │    │   LanceDB + SQLite     │    │
│    │                    │    │    │   ~/.studybuddy/cognee/│    │
│    └────────────────────┘    │    └────────────────────────┘    │
│                              │ (optional)                       │
│                    ┌─────────▼──────────┐                       │
│                    │  Tavily / YouTube  │                       │
│                    │  / OpenAlex        │                       │
│                    │  (Net Support,     │                       │
│                    │   Deep Dive,       │                       │
│                    │   Further Reading) │                       │
│                    └────────────────────┘                       │
└──────────────────────────────────────────────────────────────────┘

RAG & evidence architecture (structured-evidence v3)

Three separate knowledge channels, never flattened into one blob. ContextComposer is the only code allowed to combine them, and it returns them as a labeled EvidenceBundle (source_evidence / project_memory / student_memory / citations / diagnostics):

1. Source evidence: what uploaded papers and bounded regions actually say. SQLite is canonical (EvidenceStore, FTS5 lexical index); ChromaDB (paper_evidence) is a searchable dense index, never the source of truth.
2. Project memory: what the student is investigating/deciding/noting inside one project. Project-filtered Chroma (project_memory).
3. Student memory: durable cross-project personalization. Cognee's research_profile dataset (see the Cognee internals section above).

Ingestion: AcademicDocumentParser (PyMuPDF4LLM + PyMuPDF) reconstructs reading order and section paths instead of splitting every N characters, producing three retrieval granularities: leaf evidence (paragraphs/formulas/tables/figures/captions), local windows (coherent adjacent evidence), and section cards (broad/graph-construction retrieval). Every EvidenceUnit gets a stable, deterministic evidence_id (hash of project_id + document_id + page + bbox + element_type + normalized_content), so re-parsing unchanged content is idempotent and graphs/memory/selections/citations can all anchor to the same durable ID.

Non-negotiable invariants:

• Every evidence/project-memory operation is scoped by project_id; the same PDF in two projects produces different evidence IDs.
• Citations are materialized from canonical EvidenceUnit records, never reconstructed from model prose or Chroma metadata alone.
• Explicit text selections, bounding boxes, and evidence IDs outrank semantic search.
• Assistant output alone cannot create persistent memory: project observations require verified evidence provenance; profile inference requires an explicit signal or cross-project recurrence.
• Cognee/telemetry failures are visible but nonfatal to the study request; canonical evidence corruption or unsafe deletion is fail-closed.
• Removing a paper or project cleans derived evidence before deleting the preserved source.

Citation graph & paper discovery architecture

Three deliberately separate surfaces, never conflated:

1. Durable project citation graph: uploaded project papers (paper:{file_id}, status="owned") and provider-verified citation edges between them. A durable edge requires provider-verified reference and matching reference text in the source PDF, never inferred from title similarity, search rank, or prose.
2. Transient exploration graph: external papers (external:{candidate_id}, identity resolved in priority order DOI → arXiv ID → Semantic Scholar ID → OpenAlex ID → normalized title/authors/year) surfaced while browsing a paper or author. Cached, but never project membership.
3. Paper recommendations: discovered/ranked external candidates from natural language (Chat/Pair Buddy). A recommendation becomes a project paper only after an explicit Download succeeds through the normal acquisition + ingestion path, never added to the durable graph just because it was displayed.

PaperMetadataService is the provider-neutral boundary merging OpenAlex, Semantic Scholar, and arXiv into one PaperMetadata record (identifiers, citation counts/references, destinations, open-access facts, field-level provenance/confidence); no raw provider response shape leaks past it. Google Scholar is a fallback destination for a human to inspect, never a citation authority or PDF source.

WebSocket event reference

All messages are { "type": str, "data": dict }. Dispatch lives in backend/app/websockets/handlers.py.

Event	What it does
BUILD_GRAPH	Streams the curriculum graph: root+sections, then parallel per-section expansion (GRAPH_NODE_ADDED/GRAPH_EDGE_ADDED) → GRAPH_BUILD_DONE
LEARN_NODE	RAG fetch scoped to the node's source paper(s) → streamed lesson (LESSON_TOKEN* → LESSON_DONE)
CHAT_TURN	RAG fetch → optional Net Support web research → streamed answer (CHAT_TOKEN* → CHAT_DONE)
CONTEXT_CARD_REQUEST	Infinite Wiki: resolves a page for the selected term → WIKI_PAGE → WIKI_DONE
WIKI_DEEPDIVE_REQUEST / WIKI_DEEPDIVE_SUMMARIZE	Deep Dive: YouTube search + transcript summarization
FLASHCARDS_REQUEST / QUIZ_REQUEST	Generate from question/content chunks → FLASHCARDS_READY / QUIZ_READY
FLASHCARD_GRADE / QUIZ_SUBMIT	Journal append (+ QUIZ_FEEDBACK for quiz)
STUDY_BUDDY_INIT / STUDY_BUDDY_TURN / STUDY_BUDDY_AUDIO	Pair Buddy/Feynman turns (text or transcribed audio)
TRANSCRIBE_AUDIO / TRANSCRIBE_AUDIO_CHUNK	Dictation-only STT: transcript is inserted into the draft, never auto-sent
TTS_START / TTS_CANCEL / TTS_PLAYBACK_STATUS	Voice output playback control
REPORT_COMPILE / REPORT_CLOSE	Report Canvas: process/pool annotations into a synthesized report; flush the per-PDF cluster on close
COMMIT_PROJECT	"Push": idea-observer evaluation (trajectory-aware) → NODE_ASSESSMENT; Graph Curator spawns exploration nodes (GRAPH_NODE_ADDED); project snapshot committed; Cognee push/flush + persona/citation-graph/intelligence refresh run in the background → EVALUATION_DONE
CLOSE_PROJECT	Final wrap: re-evaluates the session, writes the Markdown summary export → SESSION_COMPLETE
UPDATE_NODE_STATUS	Student manually marks a node ongoing/completed
STYLE_FEEDBACK	Explicit learner style preference → written to profile memory
CACHE_CLEAR	Dev: clears the deterministic output cache

---

Roadmap

ResearchMate is moving toward a full long-horizon research companion: Draft Review, Personal Graph, Viva, richer paper recommendation, and full Voice Buddy are planned or partially scaffolded.

The current v1 focuses on the working core: project-based paper reading, grounded Wiki and Chat, Pair Buddy, Citation Graph, Commit, Cognee-backed memory, and sandboxed visuals.

---

Tech Stack

Layer	Technology	Role
Desktop shell	Electron, Electron Forge	Packages and runs the app as a native desktop executable
Frontend	React 19, TypeScript, Vite, Zustand	Renderer, build tooling, state stores
Reading and graph surfaces	React PDF, lucide-react, React Flow + Dagre, D3, Three.js, Plotly.js, KaTeX, Mermaid	PDF rendering; knowledge/citation graph canvases (React Flow, laid out by Dagre); trusted selected-data transforms (D3); 3D scientific/molecular visuals (Three.js); lesson/report plots (Plotly.js); math notation (KaTeX); diagrams (Mermaid)
Backend API	Python 3.11, FastAPI, Uvicorn, Pydantic, WebSockets	Agent hosting, REST + WebSocket dispatch, structured contracts
Agent and LLM layer	Cerebras Cloud SDK, Gemma 4	Structured Pydantic outputs and streaming completions (contracts detailed below)
Retrieval and document ingestion	ChromaDB, PyMuPDF, PyMuPDF4LLM, LangChain Text Splitters, PyPDF, python-docx	Dense paper evidence and project memory storage (ChromaDB); page/text/figure/table/bbox extraction (PyMuPDF); document structure and Markdown parsing (PyMuPDF4LLM); bounded local-window evidence fallback (LangChain)
Long-term memory	Cognee 1.2.2, LanceDB, FastEmbed	Durable graph/vector student-profile memory (Cognee + LanceDB); local embeddings so retrieval and memory never need a cloud embedding endpoint (FastEmbed)
Web and media integrations	Tavily, OpenAlex-style scholar lookup, YouTube Data API, youtube-transcript-api	Net Support web citations, paper metadata/citation lookups, Deep Dive video search and transcripts
Speech and multimodal support	pywhispercpp / whisper.cpp, TEN VAD, Silero VAD, Pocket TTS, Piper, Gemma 4 vision	Default local STT engine; optional server-side voice-activity detection; local TTS providers behind the app's TTS boundary; figure/table vision description
Observability	OpenTelemetry, SigNoz	Instruments FastAPI and retrieval operations, exports traces and diagnostic signals
Testing and quality	pytest, pytest-asyncio, respx, Vitest, Testing Library, oxlint	476 backend tests, 118 frontend tests (see Running Tests)
Local-first storage	~/.studybuddy/	Project registry, uploads, annotations, graphs, summaries, Cognee memory, activity logs

Cerebras API & structured output contracts

• Model ID always pinned explicitly: gemma-4-31b (never omitted; the SDK may otherwise resolve a retired default). Vision calls (SensesAgent) must also use gemma-4-31b, not llama-4-scout.
• Context: 32K MCL (message context limit) / 65K MSL.
• Structured outputs: response_format.type = "json_schema", strict: true, additionalProperties: false at every nesting level. CerebrasClient._build_schema() inlines $refs (Cerebras strict mode rejects $ref) and fails fast locally, naming the offending field, if a schema uses a bare dict[str, X] (strict mode requires explicit properties; reshape to list[SomeKeyValueModel] instead).
• Image input is multimodal image_url content, base64 data URIs only; hosted URLs aren't supported.
• Reasoning is off by default; opt in per call with reasoning_effort: "low"|"medium"|"high".
• CerebrasClient.structured_complete() catches both json.JSONDecodeError and pydantic.ValidationError (Cerebras can return truncated/EOF'd JSON) and retries once.
• Every agent-to-agent and agent-to-frontend boundary is one of these strict Pydantic contracts, never free-text parsing, so a dozen independently-developed agents compose without any of them needing a bespoke parser for another agent's output.

---

Coded via AI-enhanced workflows

ResearchMate was implemented through a split AI-assisted engineering workflow. Claude Code was used primarily to structure and build the frontend; Codex was used primarily to structure and build the backend. Product direction, research workflow, privacy boundaries, and final technical decisions remained author-led.

Claude Code: frontend implementation

Claude Code helped turn the product and interaction design into the React/Electron implementation:

• composed the project start flow, reader workspace, graph surfaces, study tools, settings, and Control Room;
• built the PDF reader, margin-gutter annotations, text-selection context, region actions, and responsive draggable study-panel layout;
• structured React components, Zustand stores, browser hooks, WebSocket-facing UI state, loading states, and error/degraded states;
• implemented the citation graph, Pair Buddy, grounded Chat and Wiki, flashcard/quiz interactions, and scientific visual surfaces;
• iterated CSS, layout constraints, responsive behavior, and frontend tests until the interfaces matched the intended research workflow.

Codex: backend implementation

Codex helped structure the FastAPI application behind those interfaces:

• designed the service, router, agent, schema, and WebSocket-event boundaries that connect the desktop interface to the research pipeline;
• implemented paper ingestion, structured PDF evidence, SQLite/FTS5 canonical storage, Chroma dense retrieval, page/bounding-box citations, rebuilds, and safe deletion;
• built project-scoped memory, Cognee-backed learner profile memory, Commit-time adaptation, promotion gates, and visible memory liveness;
• implemented citation metadata, verified acquisition, transient exploration, recommendations, local transcription/VAD services, and OpenTelemetry/SigNoz instrumentation;
• traced failures across asynchronous handlers, Pydantic contracts, retrieval, memory, and provider integrations, then added or repaired backend tests around the corrected behavior.

Feature engineering across the stack

The work was carried through the codebase as connected feature slices rather than isolated UI or API generation.

Paper evidence and retrieval

The document path was engineered from upload through AcademicDocumentParser, canonical EvidenceStore, dense indexing, PaperEvidenceService, and grounded client citations. This required synchronizing parser output, evidence identifiers, retrieval packing, reader selection geometry, annotations, regions, and paper removal/rebuild behavior.

Project memory and Commit adaptation

The research-memory implementation connected interaction journals, MemoryPromotionGate, project memory, the Cognee research_profile, Commit review, persona updates, liveness state, and crash-safe append-only ledgers. The implementation separates paper facts from project observations and cross-project learner signals instead of treating all text as one memory store.

Research workspace and interaction contracts

Reader interactions, Wiki, Chat, Pair Buddy, Flashcards, Quiz, Citation Graph, recommendations, and visual tools were implemented through explicit REST and WebSocket payloads. The frontend state and backend Pydantic contracts were evolved together so interaction changes had matching service behavior and test coverage.

Local voice and operational visibility

The voice path was built around browser WAV preparation, local whisper.cpp transcription, optional guarded VAD, and structured benchmark telemetry. Retrieval and agent operations were instrumented with OpenTelemetry/SigNoz so failures and performance signals could be inspected outside the user-facing flow.

Implementation and qualification

Claude Code and Codex were used to inspect existing modules, propose a concrete change, implement it, run focused tests, diagnose failures, and revise the affected component or service. The resulting code includes backend tests for evidence, memory, citations, contracts, and services, plus frontend tests for reader behavior, study tools, state, and responsive interaction. This made the agents implementation environments for feature engineering and debugging, not just sources of snippets.

---

Privacy

• All student data stays local at ~/.studybuddy/; nothing goes to the cloud
• ChromaDB vector store, session memory, annotations, and summaries are all file-based
• The only outbound calls are to the Cerebras API (inference) and optionally Tavily (web search)