Rain: Lightweight On-Device Agentic AI Framework

Rain is a Rust library designed to implement a lightweight, on-device agentic AI system based on the minimalist principles of Occam's razor. It is built specifically for resource-constrained edge environments, focusing on single-agent efficiency, semantic routing, and minimal overhead.

Core Architecture Layers

1. Routing Layer (Semantic Dispatch): Replaces generative LLM-based routing with Embedding-Based Semantic Routing. Deterministic requests route instantly at sub-penny costs, triggering the more expensive generative LLM only for ambiguous or novel tasks.
2. Cognitive Layer (Agent Role Triad): Adheres to the "Small Language Model (SLM) per Agent" pattern, defaulting to a single agent for sequential tasks. A single SLM uses conditional prompting to dynamically switch between three personas: Decision Maker, Generalist, and Advisor.
3. Memory Layer (Context Management): Prevents memory exhaustion and "context rot" using a three-layered pipeline (disk-based offloading, deterministic eviction, structured LLM summarization) and KV Cache Optimization (CrossKV and KVSwap).
4. Execution Layer (Thermal-Aware Scheduling): Manages hardware constraints and thermal throttling via a Deep Reinforcement Learning Scheduling (DRLS) controller.
5. Network Layer (Lightweight Coordination): Minimizes network processing power for multi-node collaboration using the Agent Communication Protocol (ACP), Ripple Effect Protocol (REP), and DNS AI agent Discovery (DNS-AID).

Integration with LLM Providers

Rain abstracts the underlying LLM provider, allowing seamless integration with:

• Ollama: Using the ollama_client_rs crate for local, on-device SLMs.
• Gemini: Using the gemini_client_rs crate for cloud-based fallback or complex reasoning tasks.

Interactive TUI

Rain includes a built-in terminal user interface (TUI) for interacting with the agent, viewing the execution trace, and monitoring the thermal-aware scheduler in real-time.

To run the TUI:

cargo run --bin rain

Quick Start (Library)

use rain::{AgentExecutor, RainConfig};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let config = RainConfig::default();
    let mut executor = AgentExecutor::from_config(config)?;

    let answer = executor.run("Summarize the key principles of Occam's razor.").await?;
    println!("{}", answer);
    Ok(())
}

See the examples/ directory for more advanced usage, including tool registration and semantic routing.

Features

• ollama (default): Enables the Ollama provider via ollama_client_rs.
• gemini: Enables the Google Gemini provider via gemini_client_rs.
• tracing: Enables structured logging via the tracing crate.

Crate Position in the Stack

Rain is the orchestration layer. It owns agentic execution, tool loops, context management, and multi-step workflows. The underlying transport clients (ollama_client_rs, gemini_client_rs) handle only serialization and HTTP — they do not own any agentic behavior.