Skip to content

SPEC.md

Latest commit

 

History

History
240 lines (200 loc) · 12.8 KB

SPEC.md

File metadata and controls

240 lines (200 loc) · 12.8 KB
Raw

herald — self-hosted iMessage bridge with TUI

One Go module, one binary, two roles:

  • Server (herald serve) — runs on a Mac. Reads ~/Library/Messages/chat.db (SQLite, read-only), sends messages by shelling out to osascript, and exposes a small REST + WebSocket API. Intended to be reached only over Tailscale. --mock swaps the Mac backend for an in-memory fake so the whole system runs on Linux.
  • Client (herald, the default command) — a bubbletea TUI that talks to the server. The same binary/TUI runs on Linux (over the tailnet) and on the Mac itself (pointing at localhost).

Everything is pure Go (modernc.org/sqlite, no cgo) so the darwin/arm64 server binary cross-compiles from Linux: GOOS=darwin GOARCH=arm64 go build.

Repo layout & ownership

cmd/herald/            main: flag parsing, config, wiring        (integration)
internal/protocol/     wire types — WRITTEN, DO NOT MODIFY
internal/store/        Store/Sender interfaces — WRITTEN, DO NOT MODIFY
internal/store/macdb/  chat.db reader implementing store.Store
internal/store/mock/   in-memory Store + Sender for dev/demo
internal/typedstream/  attributedBody → plain text extractor
internal/send/         osascript Sender implementation
internal/server/       HTTP + WebSocket API on top of Store/Sender
internal/client/       Go client for the API (REST + reconnecting WS)
internal/tui/          bubbletea app on top of internal/client
docs/research/         reference docs for chat.db / typedstream / sending

Rules for implementers:

  • Touch only your own package (plus its _test.go files). internal/protocol and internal/store/store.go are frozen contracts.
  • Do not edit go.mod/go.sum. Available deps: bubbletea v1, bubbles, lipgloss, modernc.org/sqlite, github.com/coder/websocket, stdlib. Need something else → say so in your report instead of adding it.
  • Your package must compile alone (go build ./internal/<pkg>) and have unit tests for the logic that can be tested without a Mac.
  • Read the relevant docs/research/*.md before writing code.

Config & flags

Config dir: os.UserConfigDir()/herald/ (i.e. ~/.config/herald on Linux, ~/Library/Application Support/herald on macOS).

Server (herald serve):

  • --listen (default 127.0.0.1:8787) — bind address. README documents binding to the Tailscale IP.
  • --db (default ~/Library/Messages/chat.db) — chat.db path.
  • --mock — use the mock backend (works on any OS).
  • Token: read from <configdir>/token; if absent, generate 32 hex chars, write the file (0600), and log it once. --token overrides.

Client (herald):

  • --server URL and --token; defaults read from <configdir>/client.json ({"server_url": "...", "token": "..."}). On first run with flags provided, offer nothing fancy — just use flags; herald login --server URL --token T writes client.json.

HTTP API (all under /v1, all require Authorization: Bearer <token>)

  • GET /v1/ping → protocol.PingResponse.
  • GET /v1/chats?limit=50&offset=0 → protocol.ChatsResponse. Ordered by last activity desc. limit capped at 200. Each chat carries unread_count and last_unread (per message.is_read; tapbacks, event rows, and retracted messages excluded). is_read flips only when the chat is read on an Apple device — the server never writes chat.db — so clients keep their own read watermark and flag a chat only when unread_count > 0 and last_unread postdates it. Each chat also carries junk (Apple's junk bucket: chat.is_filtered = 2 or chat.is_blackholed) and unknown_sender (no participant resolves to a contact; without a contact source, chat.is_filtered = 1). junk wins: a junk chat is never also unknown_sender. category is "transactions" or "promotions" when iOS's SMS filter tagged the sender — recovered from the handle suffix it bakes into chat.db identifiers ((smsft…) / (smsfp…), e.g. 86434(smsft_fi)); display names strip the suffix.
  • GET /v1/contacts?q=&limit=0 → protocol.ContactsResponse. Contacts from the server's contact source (AddressBook), ordered by name, handles in send-ready form (usable as SendRequest.To). q filters by name/handle substring, case-insensitive; a digits-only reading of q also matches phone handles. limit <= 0 or absent = no cap. Backends without contacts return {"contacts":[]}, not an error.
  • GET /v1/chats/{guid}/messages?before=0&limit=50 → protocol.MessagesResponse. Ascending by date within the page; before pages backwards by RowID (0 = latest). guid is URL-escaped by the client (it contains ; and +).
  • POST /v1/send body protocol.SendRequest → 200 protocol.SendResponse or 4xx/5xx protocol.ErrorResponse. Validation: text non-empty (whitespace-only counts as empty); exactly one of chat_guid / to.
  • GET /v1/events — WebSocket (coder/websocket). Server → client frames are protocol.Event JSON text messages: every new message from Store.Watch (fan-out to all connected clients), plus {"type":"ping"} every 30s. Client frames are ignored. Auth via the same Bearer header on the handshake request.
  • Unknown auth → 401 before any handler logic. Non-2xx bodies are always protocol.ErrorResponse.

Server internals (internal/server)

server.New(store store.Store, sender store.Sender, token, version, backendName string) *Server with func (s *Server) Handler() http.Handler and func (s *Server) Run(ctx, listenAddr) error. One goroutine consumes Store.Watch and fans out to subscribers; WS write errors drop that subscriber. Slow subscribers must not block the fan-out (small buffered channel per subscriber, drop-oldest or disconnect on overflow — disconnect is fine, client reconnects). Log requests + watch errors with log/slog.

macdb backend (internal/store/macdb)

macdb.Open(path string) (*DB, error) implementing store.Store. Follow docs/research/chatdb.md exactly (dates, WAL read-only open, queries). Text resolution: message.text when non-empty else typedstream.ExtractText(attributedBody). Render tapbacks per research doc. Watch = poll MAX(message.ROWID) every 1s (config const), emit new rows joined to their chat. Never write to the db; open with mode=ro and query_only pragma. Unit tests: build a tiny fixture chat.db in-memory (create the needed tables yourself in the test, insert rows, point the reader at it) — schema DDL for fixtures is in the research doc.

Sender (internal/send)

send.NewOSAScript() store.Sender. Follow docs/research/sending.md: pass text and target via on run argv (no string interpolation), map common osascript failures to readable errors, support chat-guid sends and direct-handle sends. Runs osascript via exec; on non-darwin return a clear "server must run on macOS" error at Send time (construction succeeds so the package is testable anywhere). Unit-test the argv construction and error mapping with a fake exec (inject the command runner).

Mock backend (internal/store/mock)

mock.New() *Mock implementing both store.Store and store.Sender. Seed ~6 chats (mix of DMs/groups) and a few dozen messages with believable timestamps. Send appends the outgoing message and emits it on Watch; a background goroutine (started by Start(ctx), stopped by ctx) emits a scripted incoming reply into a random chat every 15–25s so the TUI demo feels alive. Deterministic under test (allow injecting the tick interval).

Client library (internal/client)

client.New(baseURL, token string) *Client with methods mirroring the API: Ping, Chats(limit, offset), Messages(guid, before, limit), Send(req), and Events(ctx) (<-chan protocol.Event, <-chan error) — a WebSocket consumer that reconnects with capped exponential backoff (1s→30s) forever until ctx is done, and surfaces connection state changes as synthetic events: {"type":"_connected"} / {"type":"_disconnected"} (client-side only, not from the wire). Unit tests against httptest.Server + a real coder/websocket handler.

TUI (internal/tui)

tui.Run(c *client.Client) error. bubbletea v1 + bubbles + lipgloss. Layout:

┌─ chats (30 cols) ─┬─ messages ────────────────────────────┐
│ ▍Mom        12:01 │  scrollback viewport                  │
│  Group: fam 11:48 │  [Mom] hey                            │
│  +1555…     Tue   │  [me] omw                             │
│                   ├───────────────────────────────────────┤
│                   │ compose textarea (3 lines)            │
└───────────────────┴───────────────────────────────────────┘
 status bar: ● connected · server · keys hint
  • Chat list: bubbles/list, filterable (/), shows display name + relative time.
  • Messages: viewport; render [sender] text with lipgloss (me vs them colors, timestamps on hover-less TUI: dim HH:MM prefix). Attachments render as ⎘ name (mime). Wrap to width.
  • Compose: bubbles/textarea, 3 lines. Enter sends; Shift+Enter is not reliably detectable in terminals, so: Enter sends, Ctrl+J inserts newline.
  • Keys: Tab cycles focus (list ↔ compose), j/k or arrows scroll when viewport focused, PgUp loads older page (before=oldest RowID), n prompts for a new recipient handle (simple textinput modal) then sends via to, r manual refresh, f cycles the sender bucket (all-but-junk → known → unknown → transactions → promotions → junk, skipping empty buckets; junk, unknown-sender, and categorized rows render dim in the mixed view), q / Ctrl+C quits (q and f only when compose not focused).
  • Live events: new message in the open chat appends + autoscrolls (only if already at bottom); other chats bump in the list with a ● unread marker (cleared on open). _connected/_disconnected flip the status-bar indicator.
  • Unread markers: seeded at startup and on every chat-list fetch from the server's unread_count/last_unread, gated by a per-chat read watermark persisted in <configdir>/readstate.json (opening a chat, or a message arriving while it is open, advances the watermark). The watermark exists because the server cannot mark chat.db read: without it, chats read only in the TUI would be re-flagged by every fetch and every restart. Reading a chat on an Apple device (is_read flips → unread_count 0) clears its marker here on the next fetch.
  • Errors (send failure, fetch failure) show in the status bar, red, until next event.
  • Must handle terminal resize.

Testability: keep Update/View pure over an injected client interface (define a narrow api interface in the tui package so tests can fake it); test key flows with bubbletea's Program-less Update calls (table tests on Msg → Model transitions). No golden-file rendering tests required.

cmd/herald (integration)

Subcommands: none (default = TUI), serve, login, version. Plain flag package with a subcommand switch, no cobra. Wire: mock vs macdb by flag; send.NewOSAScript() vs the mock as Sender. Graceful shutdown on SIGINT/SIGTERM.

Frozen entry-point signatures

Cross-package compilation depends on these exact signatures; do not deviate:

// internal/typedstream
func ExtractText(b []byte) string // never panics; "" when unparseable

// internal/store/macdb
func Open(path string) (*DB, error) // *DB implements store.Store

// internal/store/mock
func New() *Mock                      // implements store.Store AND store.Sender
func (m *Mock) Start(ctx context.Context)

// internal/send
func NewOSAScript() store.Sender

// internal/server
func New(st store.Store, snd store.Sender, token, version, backend string) *Server
func (s *Server) Handler() http.Handler
func (s *Server) Run(ctx context.Context, addr string) error

// internal/client
func New(baseURL, token string) *Client
func (c *Client) Ping(ctx context.Context) (protocol.PingResponse, error)
func (c *Client) Chats(ctx context.Context, limit, offset int) ([]protocol.Chat, error)
func (c *Client) Messages(ctx context.Context, chatGUID string, beforeRowID int64, limit int) ([]protocol.Message, error)
func (c *Client) Send(ctx context.Context, req protocol.SendRequest) error
func (c *Client) Events(ctx context.Context) (<-chan protocol.Event, <-chan error)

// internal/tui
func Run(c *client.Client) error

Conventions

Go 1.25. gofmt clean, go vet clean. Errors: wrap with %w, messages lower-case. Logging: log/slog (server side only; the TUI must never write to stdout/stderr while running — route errors into the UI). No global state. Contexts flow from main. Keep it small: this is a personal tool, not a product — prefer the direct implementation over abstraction, but never at the cost of correctness around concurrency or the frozen contracts.