I build autonomous AI systems: agents that plan, act, review their own work, and mostly run where the data already lives.
Two properties show up in most of what I ship. It makes decisions without a human in the loop, and it doesn't need to phone home to do it. A coding assistant with no API key. A statistics toolkit with no telemetry. A vulnerability scanner with two runtime dependencies. Local-first is a constraint I design against rather than a marketing angle, because the interesting problems start when you can't just call a hosted model for every decision.
Focus: agentic architectures, applied security, MLOps and drift, edge/on-device inference, retrieval over knowledge graphs
Principal engineer in applied AI and ML systems, with two decades in statistics and analytics and production ML and NLP since 2017. That work spans agentic infrastructure (Model Context Protocol servers, multi-provider LLM abstractions with OAuth 2.1), production NLP across the full lineage from lexicon scoring through LSTM/GRU to transformer fine-tuning, and applied causal inference rigorous enough to survive a CFO: synthetic control, difference-in-differences, ANOVA with post-hoc correction.
Independently, I've fine-tuned GPT-J and LLaMA-class models using both LoRA and full-parameter approaches on an 8×A100 DeepSpeed cluster, including debugging the training divergence and tokenization failures that never appear in the tutorials.
- Recognition: Brandon Hall Gold Award, Best Advance in AI for Learning
- Teaching: Guest lecturer on applied AI/ML, George Mason University CLO Certification Program (2020-2024, four cohorts)
- Education: M.S. Data Science, Southern Methodist University. B.B.A. Management Information Systems, Dallas Baptist University
| Project | What it does | Why it's not trivial |
|---|---|---|
| Agent-Builder | Design, test, and deploy AI agents against any LLM provider. CLI-first TypeScript monorepo. | 6 providers behind one interface, 5 agent patterns, DAG workflows, and an evaluation framework. Calling a model is easy. Making runs comparable across providers is the actual work. MCP compatible. |
| CodingAgent | A coding assistant that runs fully offline via Ollama. No API keys, no subscription, no data leaving the machine. | Every affordance you'd get from a hosted model has to be rebuilt against a local one: tool use, context management, and recovery when a smaller model returns something malformed. |
| Vulnerability-Scanner | Cross-platform scanner, pen tester, zero-day detector, and OS hardener for Linux, Windows, and macOS. | A five-phase pipeline with 14 CIS-aligned scanners and backup-first hardening. It changes system state, so every hardening step has to be reversible before it's applied. Two runtime dependencies. |
| Self-Healing-MLOps-Pipeline | Fraud-detection pipeline that detects data drift with Evidently AI and retrains itself. | Drift detection is the easy half. Deciding when retraining is justified, versus reacting to noise and quietly degrading the model, is the part that needs judgment. |
Eleven of my repos run an autonomous improvement loop I built on GitHub Actions. It does considerably more than bump dependencies:
- A scheduled agent reads the repo, picks one focused improvement, and opens a pull request.
- A second, stronger model reviews that PR, then either approves and auto-merges it or requests changes with specific feedback.
- If changes are requested, a third workflow dispatches an agent to address the review. It's explicitly instructed to verify each claim before implementing it, and to push back with file-and-line evidence when the reviewer is wrong. Two models agreeing to be wrong together is the failure mode worth designing against.
- A daily janitor closes proposals that stalled, and feeds those titles back into the prompt as "don't repeat these."
The guardrails matter here, because an agent with commit access fails in expensive ways. Writes stay confined to the repo root, with path traversal and dotfile paths rejected. Workflow files and lock files are off limits entirely. Git hooks are disabled during its commits. A rewrite that returns less than 60% of the original file gets treated as accidental deletion and discarded, however plausible the accompanying commit message reads. The fix loop is bounded to two rounds, so a disagreement can't ping-pong into an unbounded API bill.
It works. The loop has proposed, reviewed, revised, and merged its own changes across all eleven repos unattended.
Agents and autonomy
- DeepResearchAgent: cyclic research agent in LangGraph (Planner → Worker → Reviewer)
- Autonomous-SOC-Security-Agent: triages SIEM alerts and checks IP reputation, with human approval enforced before anything gets blocked
- PDA: answers real phone calls via Twilio, converses using Claude/Grok/Gemini, and sends call summaries to Telegram
Data, retrieval and ML
- GraphRAG-Compliance-Navigator: extracts compliance entities from legal text into a Neo4j knowledge graph, retrieved via hybrid vector and graph search
- Data-Science-Toolkit: 32+ interactive statistics and modeling tools in Streamlit, running 100% offline with zero telemetry
Edge and on-device
- Edge-Native-Vision-Copilot: real-time PPE violation detection via ONNX inference and acoustic sensor fusion, tuned for constrained hardware
- ToneAnalyzer: voice stress analysis and facial emotion detection running entirely client-side, shipped to web, desktop (Electron), and mobile (Capacitor)
Languages: Python, TypeScript AI/ML: LangGraph, Ollama, ONNX Runtime, TensorFlow.js, scikit-learn, Evidently AI, Feast Data: Neo4j, vector and graph hybrid retrieval, Streamlit Infra: GitHub Actions, Docker, Twilio
- LinkedIn: linkedin.com/in/scott-severance
- Website: scottseverance.net