Skip to content

Commit e18d371

Browse files
0xharkiratclaude
andcommitted
docs: wake word detection section, updated roadmap
- architecture.mdx: added Wake Word Detection section - overview.mdx: added Wake step to how-it-works, updated roadmap blurb - roadmap.mdx: wake word moved to In Progress Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
1 parent 6dd101f commit e18d371

3 files changed

Lines changed: 9 additions & 3 deletions

File tree

content/docs/hark/architecture.mdx

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,12 @@ Main (Dart) -> HarkMainApi (Kotlin) -> Overlay relay -> HarkOverlayFlu
5757
4. The overlay renders chat bubbles with app icons, a keyboard/mic toggle, and auto-activates the mic on open.
5858
5. On dismiss, the overlay engine is kept warm for the next invocation.
5959

60+
## Wake word detection
61+
62+
Hark supports hands-free activation via the wake phrase "Hey Hark". Wake word detection runs on-device using [openWakeWord](https://github.com/dscripka/openWakeWord) (Apache 2.0) with ONNX Runtime for inference. A custom-trained model (201KB) listens continuously for the trigger phrase and auto-starts the microphone when detected. The wake word engine is integrated through Pigeon bindings in `hark_platform`, consistent with the rest of the native bridge.
63+
64+
AudioRecord is a shared resource on Android, so the wake word listener and STT cannot run simultaneously. When the wake phrase is detected, the wake word engine stops its audio stream before STT begins listening. Once the voice command completes, the wake word engine restarts. This mutual exclusion is handled automatically in the platform plugin. The wake word engine persists even when the app is backgrounded, so detection continues across app switches. Background service support (Phase 2) will allow detection when the app is fully closed.
65+
6066
## Two-stage resolution
6167

6268
### Stage 1 - Intent selection (EmbeddingGemma 308M)

content/docs/hark/overview.mdx

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@ Hark is the open-source assistant built on [OACP](/docs/oacp/what-is-oacp). It d
2020

2121
## How it works at a glance
2222

23+
0. **Wake** - on-device wake word detection listens for "Hey Hark" and activates the assistant hands-free.
2324
1. **Listen** - on-device speech-to-text captures your voice command.
2425
2. **Discover** - scans installed apps for OACP capability manifests via Android `ContentProvider`.
2526
3. **Resolve** - two-stage on-device AI pipeline matches your command to the right app action and extracts parameters.
@@ -92,4 +93,4 @@ flutter run
9293

9394
## Roadmap
9495

95-
The short version: the protocol and Kotlin SDK ship today, and the lightweight assistant overlay is shipped. Hark is focused next on self-hosted inference, better speech input, wake word activation, and chat persistence. See the [roadmap](/docs/roadmap) for the tracked priorities.
96+
The short version: the protocol and Kotlin SDK ship today, and the lightweight assistant overlay is shipped. Wake word detection ("Hey Hark") is in progress with in-app listening working now and a background service coming next. Hark is also focused on self-hosted inference, better speech input, and chat persistence. See the [roadmap](/docs/roadmap) for the tracked priorities.

content/docs/roadmap.mdx

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -35,14 +35,13 @@ Other SDKs are not documented as installable products until there is code to ins
3535

3636
### In progress
3737

38+
- **Wake word** - "Hey Hark" on-device detection using openWakeWord with ONNX Runtime. Phase 1 (in-app listening, auto-mic activation) is working. Phase 2 (background service for detection when the app is fully closed) is next.
3839
- **Self-hosted inference** - connect Hark to Ollama or LM Studio on a local network for unlimited context and `OACP.md` consumption
3940
- **BYOK cloud fallback** - bring-your-own-key for OpenAI, Gemini, Anthropic when no local model is available
4041
- **Better STT** - evaluate whisper.cpp and sherpa-onnx for fully on-device speech recognition
4142
- **Disambiguation UI** - show top candidates as chips when scores are close and learn from user feedback
4243

4344
### Planned
44-
45-
- **Wake word** - "Hey Hark" activation without touching the phone
4645
- **Personalization** - assistant learns user preferences and per-app vocabulary over time
4746
- **Gemma 4 single-model pipeline** - once `flutter_gemma` supports it, collapse the two-stage pipeline into a single call
4847
- **iOS** - exploring feasibility

0 commit comments

Comments
 (0)