🚀 npmai

By Sonu Kumar (Viral Boy)

npmai is a lightweight Python package designed to bridge the gap between users and open-source LLMs.

Connect with Ollama and 45+ other powerful models instantly—no installation, no login, and no API keys required, and help in development of RAG Agents without installing anything locally or on cloud and it is free without sigin or signup or any type of limit.

---

✨ Features

• 🔗 Zero Setup: No local Ollama installation or complex API signups needed.
• 🤖 Multi-Model Support: Execute prompts across 10+ open-source models simultaneously.
• 🧠 Built-in Memory: (New in v0.1.3) Native memory support—no need for external Agentic frameworks.
• 🕵️‍♂️🔍📑 RAG Frame-Work: no need to install Whisper or any model locally,no need to write code for the pdf,image,video,yt-video to text just use npmai
• 🔍📑 Vectorised Database Now you can store your any all type of files in vectorised form through npmai for free of cost unlimited time
• ⚡ Framework Ready: Fully compatible with LangChain, CrewAI, and other orchestration tools.
• 🛠️ Universal API: Access via Python, JavaScript, C++, Java, or C.
• 🔍 Tavily: Tavily integrated and also integrating MCP Servers and advance tools to make LLM more powerfull
• 📑 LARA:LARA we developed reaserch paper which addresses major problems of RAG industry.
• 🤖 Scalability:80640 requests per 24 hour we can easily handle on free computes neither we are paying anyone nor users or developers will pay anything to npmai.
• 🛠️Future Features: In v0.2.0 we are going to integrate LARA and Offline Ollama also but with integration of pool of MCP Servers and Tools that will advance a 1b Param Model and model will respond in a way you cannot imagine so keep patience we are working on it and soon we will launch it.

---

Achievment for npmai:-

We Achieved 1.2 Million+ installations this shows reliabity and trust on npmai

---

🖥️ Supported Models

Model Name	Description
llama3.2	Meta's latest powerful small model
gemma-2-instruct-9b	Google's high-performance open model
qwen-2.5-coder-7b	Alibaba's elite coding assistant
mistral-7b-instruct	Versatile and efficient instructor model
phi-3-medium	Microsoft's highly capable reasoning model
Falcon	From UAE ,TII
Baichuan-2	Baichuan from China
InternLM	From Sanghai AI Laboratory
Vicuna	From LMSYS Org
gemma3:12b	A latest AI Model by Google which has knowledge cutoff of 2026 also Latest AI Model in NPMAI ECOSYSTEM
gemma2:9b	Model by Google Deepmind also in NPMAI ECOSYSTEM
qwen3.5:9b	One of the latest and powerfull model with latest knowledge cutoff year 2026 now in NPMAI ECOSYSTEM
llama3.1:8b	Meta's core 8B general purpose engine optimized for multilingual logic
llama3.2:3b	Balanced high-performance edge text generation optimized for low RAM footprint
llama3.2:1b	Ultra-lightweight utility model ideal for fast token classification and structure tasks
qwen2.5:7b	Alibaba's foundational language asset with highly improved structured JSON adherence
qwen2-math:7b	Alibaba's official verified math release for complex algebraic step-by-step logic
qwen2.5vl	Official vision-language model trained for complex chart parsing and video analysis
phi3:3.8b	Microsoft's lightweight reasoning powerhouse focusing on mathematical logic puzzles
phi3:vision-128k	Compact multimodal variant built to extract structured tables out of visual layouts
gemma2:2b	Google's sub-3B model offering dense conversational performance on consumer hardware
openhermes	Time-tested community fine-tune excellent at general reasoning and chat instructions
yi	Highly optimized bilingual English/Chinese core developed by 01.AI
deepseek-coder:6.7b	DeepSeek's specialized coding layout optimized for code reasoning and repository scanning
codellama:7b	Meta's foundational software development assistant built for raw code synthesis
stable-code	Stability AI's low-latency developer companion built for fast auto-completions
granite3.1:2b	IBM's enterprise-grade language asset geared for strict corporate business logic
smollm:1.7b	Hugging Face's micro-agent tool fine-tuned for device-level orchestration layers
stablelm2	High-velocity text embeddings and lightweight text categorization suite
cniongolo/biomistral	Specialized variant constructed for medical terminology analysis and clinical extraction
meditron	Medical-focused framework optimized for healthcare response synthesis and guidance
hf.co/gaianet/FinGPT-MT-Llama-3-8B-LoRA-GGUF	Quantitative trading and market sentiment parser adapted for corporate earnings analysis
deepseek-math	High-tier open-source logical engine excelling at algorithmic structure tasks
wizardmath	Reinforcement-tuned model engineered for processing complex word-problem variables
llava	Versatile multimodal network connecting deep image contexts to multi-turn dialogues
moondream	Tiny, high-speed vision framework optimized for rapid edge visual classifications
adrienbrault/gorilla-openfunctions-v2	Functional automation system trained strictly to output exact external API mappings
command-r	Cohere's enterprise-grade framework designed natively for multi-step agent tool calling
deepseek-r1:7b	Open-weights reasoning architecture distilled for intense code, logic, and deep thought tasks
openbmb/minicpm-v2.6	Highly resilient multimodal setup built for localized smartphone and edge image parsing
gemma3:4b	Google's highly efficient compact variant with a 2026 knowledge cutoff
qwen2.5-coder:3b	Alibaba's light-footprint code generator engineered for inline IDE tool setups
olmo2	Fully open foundational training architecture matching standard 7B industry benchmarks
nemotron-mini	NVIDIA's highly responsive text model fine-tuned for rapid multi-turn chat applications
aya	Cohere's dedicated multilingual engine focusing on accurate low-resource language processing
phi4	High-tier mathematical and logical system optimized for complex contextual memory reasoning

---

Workflow:-

npmai

Rag

npmai Ecosystem:-

Here is the main core component of NPMAI ECOSYSTEM in Ecosystem following products are:-

1.NPM-Rag-A.I:-NPM Rag A.I is a beautiful, easy-to-use web application that lets you instantly create and talk to your own private or public knowledge bases using RAG (Retrieval-Augmented Generation).

Visit:-

2.NPM-Journalist:-You can raise voice to Government without getting traced,safe & secure journalism.

Visit:-

3.NPM-AutoCode-A.I:- Full Autonomous agent where A.I will write code to auotmate PC and execute debug and before execution of any code there will be a safety check,full secure.

Visit:-

4.NPM-Youtube-Automation:-Just give us Video and Thumbnail and go to sleep your video will automatically uploaded to Youtube with full meta-data including captions.(Future update:- Your video or any post will be uploaded to all social media platforms from Facebook to X to insta to all).

Visit:-

5.NPM-Debater-A.I:-You can enjoy debate of 4 AI models just enter topic and enjoy infinite debate arena.

Visit:-

6.NPM-Legal-A.I:-Legal Chatbot with specific models,processings for free supports all documents of users (Currently it only support Indian Laws).

Visit:-

7.NPM-Business-Analysis-A.I:-Business Analysis AI where you will explain your business and it will guide you in making plans (Future Updates:- NPM-Youtube-Automation and npmai-RAG will be integrated.)

Visit:-

8.NPM-Data-A.I:-It will analyse your Bank Account transaction history and it will give advice related to your financial future and conditions.

Visit:-

Note:- All projects are free and deployed and production ready.

⚙️ Installation

Install via pip in seconds:

pip install npmai
Use code with caution.

Tip for Python 3.13+: Use py -3.13 -m pip install npmai
💡 Quick Start (Python)
python
from npmai import Ollama

# Initialize the LLM
llm = Ollama()      

# Simple invocation
response = llm.invoke("What is the future of AI?", model="llama3.2")
print(response) 

🌐 API Usage (Other Languages)
If you aren't using Python, hit our global endpoint:
POST https://npmai-api.onrender.com
🟡 JavaScript
javascript
const response = await fetch("https://npmai-api.onrender.com", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    prompt: "Hello! Who are you?",
    model: "llama3.2",
    temperature: 0.4
  })
});
const data = await response.json();
console.log(data.response);

🔵 C++
cpp
nlohmann::json payload = {
    {"prompt", "Explain quantum physics."},
    {"model", "llama3.2"},
    {"temperature", 0.4}
};
auto res = cli.Post("/llm", payload.dump(), "application/json");

🆕 Latest Update: Version 0.1.9

In this update:-

1. npmai introduced a new distributed architechure that can handle 80K+ requests in 24 hour for free neither npmai will pay anything to anyone nor users or devs will pay anything to npmai.
2. npmai added change,Models,fallback_api paramters in Ollama class.
3. npmai removed the inheritence of LLM class from langchain.
4. npmai changed api parameter link in Ollama class.
5. npmai collabrated with developers who wanted to support us and we together launched 56 spaces which powers 80K+ requests in free of cost for 24hr daily.

version 0.1.8 --->> Integrated Supabase for long term storage of vectorised Documents,Added a method in Rag class of npmai sdk "vector_db_use", Updated parameters of Rag class to make compatible with new supabase integrations, Updated Docstrings for Rag class.

version 0.1.7 --->> Updated parameters of Rag class for sending multiple files of all type at once and also added a clear_memory method in Memory class to remove memory files and added docstrings description in every class Ollama,Memory,Rag.

version 0.1.6 --->> Added try and except for api hitting and added huggingface api endpoints as fallback.

version 0.1.5 --->> Just fixed some bugs and added link as a parameter in Rag class.

version 0.1.4 --->> Now you do not need to write code for RAG tools like pdf,image,video,audio,yt-video to text and no need to load whisper and other requirements locally no local process everything on cloud in free without any signup or singin or key hurdles.

Important Update in NPMAI RAG:-

🚀 NPMAI Update: Advanced RAG & Refine Architecture

We have officially upgraded the NPMAI Ecosystem to a more intelligent, cost-efficient, and "Product-Ready" pipeline. These updates move beyond basic RAG into High-Performance Agentic Retrieval.

---

🔍 1. Dynamic K-Context Retrieval (70% Coverage)

The Problem: Standard RAG systems use a fixed k value (e.g., k=4). This is inefficient—it provides too little context for large documents (missing facts) and too much "noise" for tiny documents (wasting tokens).

The Solution: I have engineered a Proportional Scaling Logic that calculates the optimal number of chunks to retrieve based on the actual density of your vectorized database.

• Logic: dynamic_k = max(1, int(total_chunks * 0.70))
• How it works:
  • Short Documents: If your database has only 2 chunks, the system retrieves only those 2.
  • Large PDFs: If your PDF generates 100 chunks, the system automatically scales up to retrieve 70 relevant chunks ($k=70$).
• The Impact: This ensures the AI always sees a statistically significant slice of the knowledge base, adapting perfectly to any document size.

---

🔄 2. Sliding Window Batch-Refinement (3-Chunk Window)

The Problem: Traditional "Refine" strategies process one chunk at a time. This is incredibly slow because it makes $N$ separate API calls. For a 30-chunk document, the user waits too long.

The Solution: I have implemented a Sliding Window Batch-Refine system that processes chunks in groups of 3 instead of 1.

• Logic: for i in range(0, total_chunks, 3):
• How it works:
  • Instead of making a single LLM call for every 1,000 characters, the system sends a batch of 3 related chunks (3,000 characters) in one go.
  • It uses the previous answer as a "Running Memory" to merge new information from the current 3-chunk batch.
• The Impact:
  • 3x Faster Execution: We have reduced total API latency by 66%.
  • Improved Coherence: The AI sees a broader context ($3,000$ chars vs $1,000$ chars), allowing it to spot connections between facts that are split across neighboring chunks.

---

☁️ 3. Infrastructure: Persistent Supabase Integration (v0.1.8)

We have successfully integrated Supabase Object Storage to move from temporary memory to Persistent Knowledge Bases.

• Vector Persistence: All .faiss and .pkl index files are now automatically uploaded to a secure Supabase bucket.
• Multi-Platform Access: This allows NPM-Rag-AI, NPM-AutoCode-AI, and the npmai SDK to share and load the same vectorized data from anywhere in the world.

---

Summary: These architectural changes make NPMAI one of the most efficient open-source RAG frameworks available for developers who need Speed + Accuracy without the high cost of standard 1-by-1 refinement.

⚠️ Important Notes:-

Please star our project on Github please.

🔗 Resources

Documentation: npmai.netlify.com

API Endpoint: https://npmaiecosystem-loadbalancer.hf.space/load_balancer Fallback API Endpoint: https://npmaiecosystem-loadbalancerfallback.hf.space/load_balancer

Developed with ❤️ to make AI accessible to everyone.

Developer and Maintainer:- Sonu Kumar

Thankyou Statement:-

So thankyou for Installations and Support of you people npmai PYPI Package achieved 1.2 Million+ Installations today and npmai is getting installed daily by 15k Installations.