On-device tool-routing models for the kAir iOS super-app. Fine-tunes a small open-weights base model (Qwen3.5-0.8B) on Apple Silicon to turn natural-language utterances into structured tool calls — fully on-device, no network round-trip.
⚠️ Research / experimental. This subsystem is in active development; the round-1 numbers below come from a small smoke-test eval (n=22) and are intentionally an end-to-end pipeline validation, not a generalization claim. Do not deploy.
Companion repository:
kAir— the Swift / SwiftUI iOS app that consumes these models.
kAir-models owns the decision layer of the kAir super-app. Given one natural-language utterance, it emits exactly one tool id from a closed catalog plus a structured argument object that conforms to a frozen JSON schema:
It explicitly does not own: chat shell, provider integrations, execution UI, or any freeform persona — those live in the kAir iOS app.
Five tools, frozen for Phase 1. Adding / renaming a tool is a coordinated change across tools.yaml, labels.yaml, the JSON schema, the GBNF grammar, and a re-run of the eval gates.
| Tool id | Purpose |
|---|---|
music.search_play |
Find music and play / queue it |
message.compose_send |
Draft and optionally send a message |
route.plan_navigate |
Plan a route, optionally start navigation |
search.web |
Free-form web search |
answer.direct |
Direct natural-language answer (no tool call) |
Round 1 vs the un-tuned Qwen3.5-0.8B base on the same 22-sample synthetic eval:
| Metric | Base (zero-shot) | Round 1 (LoRA) |
|---|---|---|
| JSON schema validity | 0.000 | 1.000 |
| Tool accuracy | 0.000 | 1.000 |
| Argument exact match | 0.000 | 1.000 |
| Macro-F1 across 5 tools | 0.000 | 1.000 |
| Clarification F1 (TP=15, FP=0, FN=0, TN=7) | 0.000 | 1.000 |
| Latency p50 / p95 (ms) | 100 / 100 | 74 / 109 |
Reading these numbers honestly:
- The 22-sample eval is a pipeline smoke test, not a generalization claim. The job of round 1 is to prove that JSON-schema-validated, schema-grammar-constrained decoding works end-to-end on Apple Silicon at the target latency envelope.
- The base model scores 0.0 on schema validity because it is un-tuned and emits free-form text rather than the schema-conformant JSON; this is the expected baseline.
- Held-out generalization metrics (round 2+, larger eval, replay-driven) are tracked separately and are not ready to publish.
See docs/eval_gates_split.md for the offline + replay gate definitions used at later rounds.
synthetic + replay-export data
│
▼
┌────────────────────────────┐
│ Stage 1: data preparation │
│ schema validation, dedup, │
│ stratified train/val/test │
└────────────────────────────┘
│
▼
┌──────────────────────────────────────┐
│ Stage 2: training (MLX-LM, LoRA/QLoRA)│
│ 18 rounds, 6 stages × 3 rounds │
│ Apple Silicon native │
└──────────────────────────────────────┘
│
▼
┌────────────────────────────┐
│ Stage 3: evaluation gates │
│ offline + replay │
│ schema, tool, args, F1 │
└────────────────────────────┘
│
▼
┌────────────────────────────────────────────┐
│ Three-layer artifact strategy │
│ │
│ ① Canonical Train (HF + LoRA) │
│ │ │
│ ▼ fuse │
│ ② Canonical Runtime (fused safetensors, │
│ │ quantized GGUF) │
│ ▼ export │
│ ③ Platform Packages │
│ ├─ iOS / macOS: Core ML .mlpackage │
│ ├─ Cross-plat: GGUF + llama.cpp │
│ └─ Windows: ONNX Runtime GenAI │
└────────────────────────────────────────────┘
Why three layers? Training format is optimized for iteration and resume; runtime format is optimized for on-device latency and binary size; platform packages handle the actual deployment quirks. Decoupling these means a base-model swap doesn't force a rewrite of the iOS adapter, and a quantization change doesn't invalidate prior training runs.
Three files form the contract between this repo and kAir. Any change to one requires a coordinated update to the others and a retraining or re-evaluation cycle.
| File | What it pins |
|---|---|
configs/global/tools.yaml |
The closed 5-tool catalog with per-tool argument types and surface availability |
configs/global/labels.yaml |
Tool labels, reason codes, and confidence thresholds |
data/schemas/router_sample.schema.json |
Training-sample + runtime-output JSON schema |
# Apple Silicon recommended (M-series). MLX falls back to CPU elsewhere.
git clone https://github.com/kairwang01/kAir-models.git
cd kAir-models
# Python 3.11+
python -m venv .venv && source .venv/bin/activate
pip install -e .
# Set up env
cp .env.example .env
# edit .env to set HF_TOKEN if you need gated repos
# Run the round-0 smoke test (FunctionGemma 270M, no training)
python base_inference_smoke.py
python check_integrity.py
# Inspect round-1 metrics (already committed for reference)
cat round1_metrics.json| Layer | Technology |
|---|---|
| Language | Python 3.11+ |
| Training | MLX-LM (Apple Silicon native) |
| Fine-tuning | LoRA / QLoRA |
| Base models | Qwen3.5-0.8B (main), FunctionGemma 270M (smoke), Qwen3-1.7B / Qwen2.5-1.5B-Instruct (stretch) |
| Constrained decoding | GBNF grammar via llama.cpp |
| Runtime export | safetensors → GGUF (llama.cpp) → Core ML (iOS / macOS) / ONNX (Windows) |
| Schema validation | jsonschema against frozen router_sample.schema.json |
.
├── configs/ Per-stage configuration
│ ├── global/ tools.yaml · labels.yaml · paths · platforms (frozen)
│ ├── models/ Per-base-model registry entries
│ ├── training/ 18-round schedule, LoRA / QLoRA configs, curricula
│ ├── eval/ Offline + replay gate definitions
│ └── runtime/ Inference / decoding configs
├── data/
│ ├── raw/ Synthetic, replay exports, hand-curated (gitignored)
│ ├── interim/ Cleaned, deduped, stratified (gitignored)
│ ├── processed/ Train / val / test / hardcases.jsonl (gitignored)
│ └── schemas/ router_sample, tool_call, replay_case (frozen)
├── prompts/
│ ├── synthetic/ Synthetic data generation prompts
│ └── labeling/ Labeling prompts
├── tools/
│ ├── definitions/ Per-platform tool exposure
│ └── grammar/ GBNF grammars for constrained decode
├── scripts/ Bootstrap + per-round shell wrappers
├── artifacts/ Adapters, fused, GGUF, ONNX, Core ML, reports
├── docs/ Technical design, eval gates, runtime contracts
├── round1_metrics.json Round-1 smoke-test results (n=22)
├── base_metrics_raw.json Pre-tuning baseline on the same eval
├── pyproject.toml
├── LICENSE MIT
└── README.md
| Stage | Status |
|---|---|
| Frozen P0 contracts (tools, labels, schema) | ✅ Landed |
| Round-0 smoke (FunctionGemma 270M) | ✅ Passes |
| Round-1 LoRA on Qwen3.5-0.8B | ✅ Passes the smoke eval (n=22) |
| Rounds 2–18 against larger held-out evals | ⏳ Not started |
| Core ML export to iOS app | ⏳ Pipeline scaffolded, not exercised |
| Replay-driven continual eval | ⏳ Schema only |
Kair Wang (@kairwang01)
kAir-models 拥有 kAir super-app 的决策层。给一句自然语言，它输出一个封闭目录里的 tool id，外加一个符合冻结 JSON schema 的结构化参数对象：
// 输入
"放点学习用的音乐"
// 输出（router 产出，schema 校验）
{
"tool": "music.search_play",
"args": { "mood": "study", "play_now": true },
"needs_clarification": false,
"confidence": 0.94
}它不负责：聊天主壳、provider 接入细节、execution surface UI、自由对话人格——这些都在 kAir iOS app 那边。
5 个 tool，Phase 1 冻结。新增 / 改名 / 删除一个 tool 都属于联动改动：要同时改 tools.yaml、labels.yaml、JSON schema、GBNF grammar，并重跑 eval gate。
| Tool id | 用途 |
|---|---|
music.search_play |
搜索并播放 / 排队音乐 |
message.compose_send |
起草并可选发送消息 |
route.plan_navigate |
规划路线，可选开始导航 |
search.web |
自由网页搜索 |
answer.direct |
直接自然语言回答（不调工具） |
Round 1 vs 未微调的 Qwen3.5-0.8B base，跑同一个 22 样本合成 eval：
| 指标 | Base (zero-shot) | Round 1 (LoRA) |
|---|---|---|
| JSON schema 合规率 | 0.000 | 1.000 |
| 工具命中准确率 | 0.000 | 1.000 |
| 参数完全匹配 | 0.000 | 1.000 |
| 5 tool 宏 F1 | 0.000 | 1.000 |
| Clarification F1（TP=15, FP=0, FN=0, TN=7） | 0.000 | 1.000 |
| 延迟 p50 / p95 (ms) | 100 / 100 | 74 / 109 |
怎么诚实地读这些数字：
- 22 样本 eval 是流水线烟测，不是泛化能力主张。Round 1 的任务是端到端验证：JSON schema 合规、grammar 约束解码、Apple Silicon 上跑得动、延迟达标。
- Base 模型在 schema 合规率上是 0.0，因为它没微调过、输出的是自由文本而非 schema 形 JSON；这是预期的基线。
- 留出泛化指标（Round 2+、更大 eval、replay 驱动）是另一条评估线，还没到可以发布的程度。
后续 round 用的离线 + replay gate 定义见 docs/eval_gates_split.md。
合成数据 + 真实回放数据
│
▼
┌────────────────────────────┐
│ Stage 1: 数据准备 │
│ schema 校验、去重、 │
│ 分层 train/val/test │
└────────────────────────────┘
│
▼
┌──────────────────────────────────────┐
│ Stage 2: 训练 (MLX-LM, LoRA/QLoRA) │
│ 18 轮训练，6 stage × 3 round │
│ Apple Silicon 原生 │
└──────────────────────────────────────┘
│
▼
┌────────────────────────────┐
│ Stage 3: 评估 gate │
│ offline + replay │
│ schema、tool、args、F1 │
└────────────────────────────┘
│
▼
┌────────────────────────────────────────────┐
│ 三层产物策略 │
│ │
│ ① 训练标准产物 (HF + LoRA) │
│ │ │
│ ▼ fuse │
│ ② 运行时标准产物 (fused safetensors、 │
│ │ 量化 GGUF) │
│ ▼ export │
│ ③ 平台包 │
│ ├─ iOS / macOS: Core ML .mlpackage │
│ ├─ 跨平台: GGUF + llama.cpp │
│ └─ Windows: ONNX Runtime GenAI │
└────────────────────────────────────────────┘
为什么分三层？ 训练格式优化的是迭代速度和断点续训；运行时格式优化的是端侧延迟和包体；平台包处理的是部署细节。解耦这三层意味着：换基座不用重写 iOS adapter，改量化方案也不会让之前的训练 run 作废。
3 个文件构成本仓库与 kAir 之间的契约。改其中任何一个都要联动改另外两个，并触发一次重训练或重评估。
| 文件 | 锁定了什么 |
|---|---|
configs/global/tools.yaml |
封闭的 5-tool 目录，含每个 tool 的参数类型和 surface 可用性 |
configs/global/labels.yaml |
Tool 标签、reason code、置信度阈值 |
data/schemas/router_sample.schema.json |
训练样本 + 运行时输出的 JSON schema |
# 推荐 Apple Silicon (M 系列)。MLX 在其他平台会回落到 CPU。
git clone https://github.com/kairwang01/kAir-models.git
cd kAir-models
# Python 3.11+
python -m venv .venv && source .venv/bin/activate
pip install -e .
# 配置环境变量
cp .env.example .env
# 如需访问 gated 仓库，编辑 .env 设置 HF_TOKEN
# 跑 round-0 烟测 (FunctionGemma 270M, 无训练)
python base_inference_smoke.py
python check_integrity.py
# 查看 round-1 指标（已提交在仓库里，可直接看）
cat round1_metrics.json| 层 | 技术 |
|---|---|
| 语言 | Python 3.11+ |
| 训练 | MLX-LM（Apple Silicon 原生） |
| 微调 | LoRA / QLoRA |
| 基座模型 | Qwen3.5-0.8B（主力）、FunctionGemma 270M（烟测）、Qwen3-1.7B / Qwen2.5-1.5B-Instruct（备选） |
| 约束解码 | GBNF grammar via llama.cpp |
| 运行时导出 | safetensors → GGUF (llama.cpp) → Core ML (iOS / macOS) / ONNX (Windows) |
| Schema 校验 | jsonschema，对齐冻结的 router_sample.schema.json |
.
├── configs/ 各阶段配置
│ ├── global/ tools.yaml · labels.yaml · paths · platforms (冻结)
│ ├── models/ 基座模型注册表
│ ├── training/ 18 轮训练计划、LoRA / QLoRA 配置、课程
│ ├── eval/ 离线 + replay gate 定义
│ └── runtime/ 推理 / 解码配置
├── data/
│ ├── raw/ 合成、回放导出、人工标注（gitignored）
│ ├── interim/ 清洗、去重、分层（gitignored）
│ ├── processed/ train / val / test / hardcases.jsonl（gitignored）
│ └── schemas/ router_sample、tool_call、replay_case（冻结）
├── prompts/
│ ├── synthetic/ 合成数据生成 prompt
│ └── labeling/ 打标 prompt
├── tools/
│ ├── definitions/ 各平台 tool 暴露
│ └── grammar/ GBNF grammar，约束解码用
├── scripts/ 启动 + 各 round shell wrapper
├── artifacts/ adapter、fused、GGUF、ONNX、Core ML、报告
├── docs/ 技术设计、eval gate、runtime 契约
├── round1_metrics.json Round-1 烟测指标 (n=22)
├── base_metrics_raw.json 同一 eval 上的微调前基线
├── pyproject.toml
├── LICENSE MIT
└── README.md
| 阶段 | 状态 |
|---|---|
| P0 冻结契约（tools、labels、schema） | ✅ 已落地 |
| Round-0 烟测（FunctionGemma 270M） | ✅ 通过 |
| Round-1 LoRA on Qwen3.5-0.8B | ✅ 在烟测 eval (n=22) 上通过 |
| Rounds 2–18 在更大留出 eval 上 | ⏳ 未启动 |
| Core ML 导出到 iOS app | ⏳ 流水线已搭建，尚未跑通 |
| Replay 驱动的持续评估 | ⏳ 仅有 schema |
Kair Wang (@kairwang01)