create_sample_docs.py

"""
Generate sample documents for the multi-agent Q&A system.
These cover 5 AI/ML topics to provide a realistic corpus for evaluation.
"""

import os

DOCS = {
    "machine_learning_basics.txt": """Machine Learning Fundamentals

Machine learning is a subset of artificial intelligence that enables systems to learn and improve from experience without being explicitly programmed. The core idea is that algorithms can identify patterns in data and make decisions with minimal human intervention.

Supervised Learning

Supervised learning is the most common paradigm in machine learning. In supervised learning, the algorithm learns from labeled training data, where each example consists of an input paired with the correct output. The model learns a mapping function from inputs to outputs, which it can then apply to unseen data.

There are two main types of supervised learning tasks. Classification involves predicting a discrete category or class label. Examples include spam detection (spam vs not spam), image recognition (cat vs dog), and sentiment analysis (positive vs negative). Regression involves predicting a continuous numerical value. Examples include house price prediction, stock price forecasting, and temperature prediction.

Common supervised learning algorithms include linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), and neural networks. The choice of algorithm depends on the nature of the data, the size of the dataset, and the complexity of the relationship between features and targets.

Unsupervised Learning

Unsupervised learning works with unlabeled data, meaning the algorithm must find structure and patterns on its own without predefined correct answers. The two main unsupervised tasks are clustering (grouping similar data points together) and dimensionality reduction (reducing the number of features while preserving important information).

Popular clustering algorithms include K-means, DBSCAN, and hierarchical clustering. For dimensionality reduction, Principal Component Analysis (PCA) and t-SNE are widely used. Unsupervised learning is valuable for exploratory data analysis, anomaly detection, and feature learning.

Model Evaluation

Evaluating model performance is critical to building reliable machine learning systems. For classification, common metrics include accuracy (fraction of correct predictions), precision (fraction of positive predictions that are truly positive), recall (fraction of actual positives that are correctly identified), and F1 score (harmonic mean of precision and recall). The confusion matrix provides a detailed breakdown of prediction outcomes.

For regression, common metrics include Mean Squared Error (MSE), Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), and R-squared (coefficient of determination).

Overfitting and Regularization

Overfitting occurs when a model learns the training data too well, including its noise and random fluctuations, and fails to generalize to new unseen data. Signs of overfitting include high training accuracy but low test accuracy.

Techniques to prevent overfitting include regularization (L1/Lasso and L2/Ridge penalties), cross-validation (splitting data into multiple train/test folds), early stopping (halting training when validation performance degrades), dropout (randomly disabling neurons during training), and data augmentation (artificially expanding the training dataset).

Gradient Descent

Gradient descent is the fundamental optimization algorithm used to train machine learning models. It works by iteratively adjusting model parameters in the direction that reduces the loss function. The learning rate controls the step size of each update.

Variants include batch gradient descent (uses all training data per update), stochastic gradient descent or SGD (uses one sample per update), and mini-batch gradient descent (uses a small batch per update). Adam, RMSprop, and AdaGrad are adaptive learning rate optimizers that adjust the step size for each parameter individually.

The gradient is the vector of partial derivatives of the loss function with respect to each parameter. By moving in the opposite direction of the gradient, the algorithm seeks to minimize the loss and find optimal parameter values.

Feature Engineering

Feature engineering is the process of using domain knowledge to create, transform, and select features that make machine learning algorithms work better. Good features can dramatically improve model performance even with simple algorithms.

Common techniques include one-hot encoding for categorical variables, normalization and standardization for numerical features, polynomial feature creation for capturing non-linear relationships, and interaction features that capture relationships between pairs of features. Feature selection methods like mutual information, chi-squared tests, and recursive feature elimination help identify the most informative features.
""",

    "transformer_architecture.txt": """The Transformer Architecture

The Transformer architecture, introduced in the landmark 2017 paper "Attention Is All You Need" by Vaswani et al., revolutionized natural language processing and has since been applied to computer vision, audio processing, and numerous other domains.

The Self-Attention Mechanism

The core innovation of the Transformer is the self-attention mechanism (also called scaled dot-product attention). Self-attention allows each position in a sequence to attend to all other positions, capturing dependencies regardless of distance. This is fundamentally different from recurrent neural networks (RNNs), which process sequences step by step and struggle with long-range dependencies.

In self-attention, each input token is projected into three vectors: a Query (Q), a Key (K), and a Value (V). The attention score between two tokens is computed as the dot product of the query of one token with the key of another, scaled by the square root of the key dimension. These scores are passed through a softmax function to produce attention weights, which are then used to compute a weighted sum of the value vectors.

The mathematical formulation is: Attention(Q, K, V) = softmax(QK^T / sqrt(d_k)) * V, where d_k is the dimension of the key vectors. The scaling factor prevents the dot products from growing too large, which would push the softmax into regions with very small gradients.

Multi-Head Attention

Instead of performing a single attention computation, the Transformer uses multi-head attention, which runs multiple attention operations in parallel. Each head operates on a different learned linear projection of the queries, keys, and values. The outputs of all heads are concatenated and linearly projected to produce the final output.

Multi-head attention allows the model to jointly attend to information from different representation subspaces at different positions. For example, one head might learn to attend to syntactic relationships while another captures semantic similarities. With h heads and model dimension d_model, each head operates on vectors of dimension d_model/h.

Positional Encoding

Since the Transformer processes all positions in parallel (unlike RNNs which inherently encode order), it needs explicit positional information. Positional encoding adds a signal to each input embedding that encodes its position in the sequence.

The original Transformer uses sinusoidal positional encodings with different frequencies for each dimension. The encoding for position pos and dimension i is: PE(pos, 2i) = sin(pos / 10000^(2i/d_model)) and PE(pos, 2i+1) = cos(pos / 10000^(2i/d_model)). This scheme allows the model to learn relative positions since PE(pos+k) can be expressed as a linear function of PE(pos).

More recent models use learned positional embeddings (BERT, GPT) or rotary position embeddings (RoPE, used in LLaMA and many modern LLMs).

Encoder-Decoder Structure

The original Transformer follows an encoder-decoder architecture. The encoder consists of a stack of identical layers, each containing a multi-head self-attention sublayer followed by a position-wise feed-forward network. Each sublayer has a residual connection and layer normalization.

The decoder also consists of a stack of identical layers, but with an additional cross-attention sublayer that attends to the encoder output. The decoder self-attention is masked to prevent positions from attending to future positions during autoregressive generation.

The feed-forward network in each layer consists of two linear transformations with a ReLU or GELU activation in between: FFN(x) = W2 * activation(W1 * x + b1) + b2. This typically expands the dimension by a factor of 4 (e.g., from 512 to 2048) and then projects back down.

Encoder-only models (like BERT) are used for tasks requiring bidirectional understanding such as classification and named entity recognition. Decoder-only models (like GPT) are used for autoregressive generation. Encoder-decoder models (like T5) are used for sequence-to-sequence tasks like translation and summarization.

Cross-Attention

Cross-attention (also called encoder-decoder attention) is the mechanism by which the decoder attends to the encoder output. In cross-attention, the queries come from the decoder while the keys and values come from the encoder. This allows the decoder to focus on relevant parts of the input sequence when generating each output token.

The distinction between self-attention and cross-attention is important. In self-attention, all three projections (Q, K, V) come from the same sequence. In cross-attention, Q comes from one sequence (the decoder) while K and V come from another (the encoder output). Self-attention captures relationships within a single sequence, while cross-attention captures relationships between two different sequences.

Training and Scaling

Transformers are typically trained with the Adam optimizer with a warm-up learning rate schedule. The original paper used 6 encoder and 6 decoder layers with d_model = 512 and 8 attention heads.

Modern large language models scale to billions of parameters with hundreds of layers. GPT-3 has 175 billion parameters, while models like PaLM and LLaMA push further. Scaling laws suggest that model performance improves predictably with increases in model size, dataset size, and compute budget.
""",

    "rag_systems.txt": """Retrieval-Augmented Generation (RAG)

Retrieval-Augmented Generation (RAG) is a technique that enhances large language model responses by grounding them in external knowledge retrieved at query time. Instead of relying solely on knowledge embedded in model weights during training, RAG systems retrieve relevant documents from a knowledge base and include them as context for the generation step.

Why RAG Matters

LLMs have several well-known limitations that RAG addresses directly. First, knowledge staleness: models only know information present in their training data, which has a cutoff date. RAG allows access to up-to-date information. Second, hallucination: models can generate plausible-sounding but incorrect information. By grounding responses in retrieved documents, RAG reduces hallucination. Third, domain specificity: general-purpose models may lack deep knowledge in specialized domains. RAG allows models to access domain-specific knowledge bases without fine-tuning.

The RAG Pipeline

A typical RAG pipeline consists of three main stages: indexing, retrieval, and generation.

During indexing, documents are processed and stored in a searchable format. The process involves document loading (reading files, web pages, or databases), chunking (splitting documents into smaller segments), embedding (converting chunks into vector representations using an embedding model), and storage (saving vectors in a vector database for efficient similarity search).

During retrieval, the user query is embedded using the same embedding model, and the most similar document chunks are found through vector similarity search. Common similarity metrics include cosine similarity, dot product, and Euclidean distance.

During generation, the retrieved chunks are combined with the original query and passed as context to the LLM, which generates a response grounded in the retrieved information.

Vector Embeddings and Similarity Search

Vector embeddings are dense numerical representations of text that capture semantic meaning. Similar texts produce similar vectors, enabling semantic search that goes beyond keyword matching.

Popular embedding models include OpenAI's text-embedding-ada-002, sentence-transformers (like all-MiniLM-L6-v2), and domain-specific models. The choice of embedding model significantly impacts retrieval quality. Models are typically evaluated on benchmarks like MTEB (Massive Text Embedding Benchmark).

Vector databases optimized for similarity search include FAISS (Facebook AI Similarity Search), Chroma, Pinecone, Weaviate, Qdrant, and Milvus. FAISS is particularly popular for local deployments due to its efficiency and the fact that it runs entirely in-process without requiring a separate server.

Chunking Strategies

How documents are split into chunks significantly affects RAG performance. Key considerations include chunk size (smaller chunks provide more precise retrieval but may lack context; larger chunks provide more context but may include irrelevant information), chunk overlap (overlapping chunks help ensure that information at chunk boundaries is not lost), and splitting strategy (splitting by paragraphs, sentences, or fixed character counts each has tradeoffs).

Common chunking approaches include fixed-size chunking with overlap, recursive character text splitting (which tries to split on natural boundaries like paragraphs and sentences before falling back to character counts), and semantic chunking (which groups sentences by semantic similarity). A typical configuration uses chunks of 500-1000 characters with 50-100 character overlap.

Challenges and Best Practices

The main challenges in RAG systems include retrieval quality (garbage in, garbage out: if retrieval returns irrelevant chunks, the generated answer suffers), context window limits (LLMs have finite context windows, so the number and size of retrieved chunks must be balanced), latency (embedding the query, searching the vector store, and generating the response all add latency), and evaluation (measuring RAG quality requires evaluating both retrieval relevance and generation accuracy).

Best practices include using hybrid search (combining vector search with keyword/BM25 search for better coverage), re-ranking retrieved results with a cross-encoder model, including metadata filtering to narrow the search space, implementing query decomposition for complex multi-part questions, and regularly evaluating retrieval performance on representative test queries.
""",

    "llm_agents.txt": """LLM Agents and Agentic Workflows

An LLM agent is a system where a large language model acts as the central reasoning engine, capable of making decisions, using tools, and taking actions to accomplish goals. Unlike simple chatbots that respond to single prompts, agents can plan multi-step strategies, execute actions, observe results, and adapt their approach.

What Makes an Agent

The defining characteristics of an LLM agent include autonomous decision-making (the agent decides what to do next based on the current state), tool use (the ability to call external functions, APIs, or tools to gather information or take actions), reasoning (the agent can break down complex tasks into subtasks and reason about how to approach them), and memory (maintaining context across multiple steps of execution).

An agent differs from a chatbot in a fundamental way. A chatbot takes a prompt and returns a response. An agent takes a goal and executes a sequence of actions to achieve it, adapting its strategy based on intermediate results.

The ReAct Pattern

The ReAct (Reasoning + Acting) pattern is one of the most influential frameworks for LLM agents. In ReAct, the agent alternates between reasoning steps (thinking about what to do) and action steps (executing a tool or operation). After each action, the agent observes the result and reasons about the next step.

A typical ReAct loop looks like: Thought (reason about the current state and what to do next), Action (choose and execute a tool), Observation (receive the result of the action), and then repeat until the task is complete. This explicit reasoning chain makes the agent's decision process transparent and debuggable.

Tool Use and Function Calling

Tool use is the mechanism by which agents interact with the external world. Tools can include web search APIs, code execution environments, database queries, file system operations, calculators, and any custom function.

Modern LLMs support function calling (or tool use) natively, where the model generates structured output specifying which function to call and with what arguments. The system executes the function and returns the result to the model for further reasoning.

LangChain provides a standardized interface for defining tools as Python functions with descriptions that the LLM can understand. Each tool has a name, a description explaining when and how to use it, and the function implementation. The agent framework handles the orchestration of selecting and calling tools.

Multi-Agent Systems

Multi-agent systems use multiple specialized agents that collaborate to solve complex tasks. Each agent has a defined role, specific tools, and a focused prompt. Coordination between agents happens through message passing and shared context.

Common multi-agent patterns include the supervisor pattern (a manager agent delegates tasks to worker agents), the pipeline pattern (agents process information sequentially, each adding its contribution), the debate pattern (multiple agents provide different perspectives and a judge agent synthesizes), and the collaborative pattern (agents work together with shared memory and tool access).

Multi-agent coordination requires careful design of communication protocols, shared state management, and conflict resolution. Frameworks like LangChain, CrewAI, and AutoGen provide abstractions for building multi-agent systems.

Benefits of multi-agent architectures include separation of concerns (each agent can be optimized for its specific task), scalability (new agents can be added without modifying existing ones), and modularity (agents can be tested and debugged independently).

Agent Memory

Memory is critical for agents that need to maintain state across multiple interactions or long-running tasks. There are several types of agent memory.

Short-term memory (also called working memory or context) is the information available in the current LLM context window. It includes the conversation history and any retrieved or computed information from the current session.

Long-term memory uses external storage (databases, vector stores) to persist information across sessions. This allows agents to remember user preferences, past decisions, and accumulated knowledge.

Episodic memory records specific interactions and their outcomes, allowing the agent to learn from experience. Semantic memory stores general knowledge and facts that the agent has acquired.

Effective memory management involves deciding what to remember, how to compress information to fit context limits, and when to retrieve stored memories.

Agent Evaluation

Evaluating agent performance is challenging because agents make sequences of decisions that are difficult to assess individually. Metrics include task completion rate (did the agent achieve the goal), efficiency (how many steps or tool calls were needed), accuracy (were the intermediate results and final output correct), and robustness (does the agent recover gracefully from errors).

Benchmarks like WebArena, AgentBench, and SWE-bench evaluate agents on realistic tasks like web navigation, coding, and multi-step reasoning.
""",

    "prompt_engineering.txt": """Prompt Engineering Techniques

Prompt engineering is the practice of designing and optimizing input prompts to elicit desired behaviors from large language models. As LLMs become more capable, prompt engineering has emerged as a critical skill for building effective AI applications.

Zero-Shot and Few-Shot Prompting

Zero-shot prompting provides the model with a task description but no examples. The model relies entirely on its pre-trained knowledge to understand and complete the task. For example: "Classify the following review as positive or negative: 'The food was amazing and the service was excellent.'"

Few-shot prompting includes a small number of input-output examples before the actual query. These examples demonstrate the expected behavior and output format. Research has shown that few-shot prompting significantly improves performance on many tasks, especially when the examples are diverse and representative. Typically 3-5 examples provide a good balance between performance and context usage.

The order and selection of examples matters. Studies have found that example ordering can cause performance variations of up to 30 percentage points. Best practices include using examples that cover different edge cases and arranging them in a logical order.

Chain-of-Thought Prompting

Chain-of-thought (CoT) prompting encourages the model to show its reasoning process step by step before arriving at an answer. This technique dramatically improves performance on tasks requiring multi-step reasoning, mathematical problem solving, and logical deduction.

The simplest form of CoT is adding "Let's think step by step" to the prompt. More effective approaches provide detailed reasoning examples. For instance, instead of asking "What is 23 x 47?" directly, a CoT prompt would show an example of breaking down a similar multiplication into steps.

Variants include zero-shot CoT (just adding "think step by step"), manual CoT (providing hand-crafted reasoning examples), auto-CoT (automatically generating reasoning chains), and tree-of-thought (exploring multiple reasoning paths and selecting the best).

Structured Output Prompting

When building applications, you often need the model to produce output in a specific format such as JSON, XML, or a structured template. Techniques for achieving reliable structured output include explicitly specifying the output format in the prompt, providing a schema or template, using output parsers to validate and retry, and leveraging function calling or tool use capabilities.

For JSON output, a common pattern is: "Respond with a JSON object containing the following fields: 'name' (string), 'category' (one of: tech, science, arts), 'summary' (string, max 50 words)." Providing an example of the expected JSON structure further improves reliability.

System Prompts and Role Setting

System prompts (also called system messages) set the overall behavior, personality, and constraints for the model. They are processed before user messages and establish the context for the entire conversation.

Effective system prompts include a clear role definition (what the model is and is not), behavioral guidelines (tone, style, level of detail), constraints (what topics to avoid, what format to use), and knowledge boundaries (what the model should and should not claim to know).

A well-designed system prompt acts as a behavioral contract between the developer and the model. It should be specific enough to guide behavior but flexible enough to handle diverse user inputs.

Prompt Optimization

Beyond manual prompt design, there are systematic approaches to optimizing prompts. A/B testing different prompt variants on representative queries helps identify the most effective formulations. Prompt chaining breaks complex tasks into sequential simpler prompts. Prompt templates with variable substitution enable reusable prompt patterns.

Important considerations include token efficiency (shorter prompts are cheaper and leave more room for output), robustness (good prompts work across diverse inputs, not just specific examples), and maintainability (clear, well-documented prompts are easier to debug and update).

Context Engineering

Context engineering goes beyond individual prompt design to consider the full context provided to the model. This includes selecting and ordering information in the context window, managing context window limits through summarization and compression, dynamically retrieving relevant context based on the query, and structuring the context to highlight the most important information.

In RAG systems, context engineering involves decisions about how many chunks to retrieve, how to order them (most relevant first vs most relevant last), and whether to include metadata alongside the content. Research has shown that models attend more strongly to information at the beginning and end of the context window.
""",
}


def main():
    os.makedirs("data", exist_ok=True)
    for filename, content in DOCS.items():
        path = os.path.join("data", filename)
        with open(path, "w", encoding="utf-8") as f:
            f.write(content.strip())
        print(f"Created {path} ({len(content):,} chars)")
    print(f"\n{len(DOCS)} documents created in data/")


if __name__ == "__main__":
    main()