Problem
Voice I/O is spread across four skills that each re-solve a slice of the same transport:
voice-walk — browser page, Web Speech API in, speechSynthesis out, persistent headless sessionvoice-command — Pebble watch, speech-to-text in, ntfy JSON envelope outquick-command — one-shot headless dispatch, text onlyntfy-ping — push notification out
Each owns its own answer to: how does speech become text, how does text become audio, how is a session addressed and kept alive, how does a link or reply reach the phone. Fixing any one of those fixes it for exactly one skill.
The first real voice-walk field test (2026-07-15) made the cost concrete. Six of seven defects were transport, not brainstorm logic, and none were specific to walking:
- TTS dies on screen lock. The page speaks via
SpeechSynthesisUtterance; Chrome Android suspends the Web Speech API when the tab backgrounds. Not a setting — the API itself. Phone must stay awake and foregrounded, defeating the premise. Proper fix is server-side TTS → <audio> + MediaSession, which is architecture, not a patch.
- Long-press on the record button selects its text. Hold-to-talk fights Android's own long-press gesture. Toggle removes the conflict.
- Truncated dictation, two independent causes. The selection popup fires
pointercancel instead of pointerup, which nothing handles. Separately, the send path waits a hardcoded 250ms after rec.stop() before reading the transcript — a race with a magic number that silently drops the tail when the final result finalizes slower. Should send on rec.onend.
- Missing space makes TTS read punctuation aloud. Text deltas are joined with
"".join(chunks), so when a tool_use block sits between two text blocks the texts concatenate with no separator and TTS pronounces doing.Confirmed as "doing dot Confirmed".
- Greeting is discarded. The warm-up turn loads context and greets, then throws the output away, so the first real turn is wasted and the greeting has to be re-asked. Should be a Start button.
- Session link is not tappable in the ntfy app — long-press copies the whole message. (ntfy likely has click/action headers for this; unverified.)
- Latency was fine (~1.3s warm turns), which is where most of the original design effort went.
Result: zero brainstorm content captured. The whole transcript is the user fighting the tool.
Proposal
Design a shared voice capability for Clayde — the transport layer the skills sit on top of, so a fix lands once. Rough scope, to be settled in design:
- Audio out — server-side TTS, MediaSession, survives screen lock. Replaces per-skill
speechSynthesis.
- Speech in — capture semantics (toggle vs hold), reliable end-of-utterance, jargon handling.
- Session lifecycle — warm session, addressing, start/stop, crash recovery, transcript persistence.
- Delivery — how a session link or a reply reaches a phone or a watch, tappably.
Per-skill concerns stay per-skill: what the agent is for (brainstorm an issue, run a command), its system prompt, its artifact handling.
Notes
Not scheduled — deferred deliberately as a larger design task. Filing so the field-test evidence and the capability argument aren't lost.
Item 1 alone changes the architecture, so this wants a design pass, not seven patches to voice-walk.
Problem
Voice I/O is spread across four skills that each re-solve a slice of the same transport:
voice-walk— browser page, Web Speech API in,speechSynthesisout, persistent headless sessionvoice-command— Pebble watch, speech-to-text in, ntfy JSON envelope outquick-command— one-shot headless dispatch, text onlyntfy-ping— push notification outEach owns its own answer to: how does speech become text, how does text become audio, how is a session addressed and kept alive, how does a link or reply reach the phone. Fixing any one of those fixes it for exactly one skill.
The first real
voice-walkfield test (2026-07-15) made the cost concrete. Six of seven defects were transport, not brainstorm logic, and none were specific to walking:SpeechSynthesisUtterance; Chrome Android suspends the Web Speech API when the tab backgrounds. Not a setting — the API itself. Phone must stay awake and foregrounded, defeating the premise. Proper fix is server-side TTS →<audio>+ MediaSession, which is architecture, not a patch.pointercancelinstead ofpointerup, which nothing handles. Separately, the send path waits a hardcoded 250ms afterrec.stop()before reading the transcript — a race with a magic number that silently drops the tail when the final result finalizes slower. Should send onrec.onend."".join(chunks), so when a tool_use block sits between two text blocks the texts concatenate with no separator and TTS pronouncesdoing.Confirmedas "doing dot Confirmed".Result: zero brainstorm content captured. The whole transcript is the user fighting the tool.
Proposal
Design a shared voice capability for Clayde — the transport layer the skills sit on top of, so a fix lands once. Rough scope, to be settled in design:
speechSynthesis.Per-skill concerns stay per-skill: what the agent is for (brainstorm an issue, run a command), its system prompt, its artifact handling.
Notes
Not scheduled — deferred deliberately as a larger design task. Filing so the field-test evidence and the capability argument aren't lost.
Item 1 alone changes the architecture, so this wants a design pass, not seven patches to
voice-walk.