Skip to content

opendb-ai/openWorkspace

Repository files navigation

openWorkspace

English | 简体中文

A durable local workspace for ephemeral compute — develop against local SQLite and a plain filesystem, while the platform transparently handles persistence, migration, and recovery.

Status License Docs


What is openWorkspace?

Cloud compute is ephemeral: pods get rescheduled, spot instances get reclaimed, sandboxes are short-lived. Meanwhile, the most natural storage abstraction for developers — and especially for AI agents — is still a local working directory:

/workspace/
  app.db        # SQLite — structured data
  files/        # plain filesystem — unstructured data

openWorkspace gives you exactly that, and nothing more visible. Behind the scenes, the platform:

  • Persists every committed state to object storage as an atomic, versioned generation
  • Migrates the workspace across compute nodes with a seconds-level pause
  • Recovers from node crashes to the last committed generation — without touching the dead node
  • Hides all of it: no S3 buckets, no distributed locks, no replication settings

In one sentence: it wraps cloud-native complexity into a single-node programming model.

Why not just use a distributed filesystem?

openWorkspace EFS / CephFS / JuiceFS
Mental model Local SQLite + local files Shared mount + consistency/locking semantics
Small writes, fsync, rename Local NVMe speed Network round-trips
Ephemeral compute (agents, sandboxes) State follows the compute Many nodes share one volume
Snapshot / fork / rollback / time travel First-class (content-addressed generations) Not visible at this layer
Multi-node concurrent writers ❌ Single writer by design ✅ Their home turf
Full POSIX ❌ Documented subset ✅ Mostly

openWorkspace is not a DFS replacement. It trades multi-node sharing and full POSIX for a radically simpler developer experience — the right trade for AI agents, notebooks, sandboxes, and serverless apps.

Core design

                ┌─────────────────────────┐
                │      Control Plane       │
                │  workspace / lease mgmt  │
                └────────────┬────────────┘
                             │ lease (single writer, epoch-fenced)
┌────────────────────────────▼────────────────────────────┐
│            Compute Node (microVM sandbox)                │
│   user process ──► /workspace (local NVMe)               │
│        ▲                          │                      │
│        │ fail-stop on lease loss  │                      │
│   ┌────┴──────────────────────────▼─────────────────┐    │
│   │ Sync Agent: shadow-WAL SQLite replication +      │    │
│   │ Merkle-manifest file sync + backpressure         │    │
│   └──────────────────────────┬───────────────────────┘    │
└──────────────────────────────┼────────────────────────────┘
                               │
              ┌────────────────▼────────────────┐
              │ Object store (immutable chunks)  │
              │ Metadata DB (generations, lease) │
              └─────────────────────────────────┘

Key decisions (see the architecture doc for full rationale):

  • Single-writer model. One active node per workspace, enforced by an epoch-fenced lease with local fail-stop. No multi-master, no merge conflicts — by design.
  • Unified generation. Every commit atomically references (sqlite_state, file_manifest_root). A recovery point is always a mutually consistent cut across SQLite and files.
  • Shadow-WAL SQLite replication (Litestream-style). The sync agent owns checkpointing; WAL frames are checksum-verified. We deliberately do not reinvent this layer.
  • Content-addressed file storage with directory-level Merkle manifests — incremental commits, structural sharing, and cheap fork/rollback for free.
  • Honest durability semantics. Default RPO ≈ 1s; workspace.sync() gives a zero-loss-window durable commit for critical writes. Sync lag and restore-drill results are user-visible.

Consistency guarantees (MVP)

We promise:

  • Single writer per workspace; a node that loses its lease is force-stopped
  • Planned handoff never loses data
  • Crash recovery restores the last committed generation — SQLite and files as one consistent cut
  • workspace.sync() returning success ⇒ data is durable
  • Corruption is never acceptable (verified by continuous automated restore drills)

We do not promise:

  • Concurrent multi-node writes
  • Runtime cross-resource transactions spanning SQLite and files
  • Durability of the last ~1s of unsynced writes after a hard crash (unless sync() was called)

Documentation

All design documents are currently in Chinese (zh-CN):

Document Contents
PRD Product requirements: users, stories, features (P0–P2), milestones, metrics, risks
Architecture Full technical design: consistency model, lease protocol, WAL replication, manifests, recovery, security
Business Model Five-layer monetization loop, competitive landscape, moat, GTM
Architecture Review Deep review record: 4 P0 findings + 13 revisions (all applied in v1.1)

Status & roadmap

Current stage: design (v1.1). Implementation has not started.

Milestone Scope Acceptance
M1 — Storage loop shadow WAL + file sync + unified generation + lease/fail-stop Lossless handoff; consistent crash recovery; restore drills pass; sync() semantics verified
M2 — Runtime loop sandboxed compute + pre-pull handoff + one-click deploy ≤1 GB workspace handoff pause < 5s (P95)
M3 — AI loop LLM access + agent runs Agents resume across nodes; fail-stop prevents double execution
M4 — Monetization metering + Free/Pro tiers + abuse detection Free → paid funnel live

Post-MVP: CoW-snapshot capture, lazy hydration (handoff time decoupled from workspace size), microVM memory snapshots, read replicas.

Prior art & acknowledgments

This design stands on well-proven ground: Litestream / LiteFS (SQLite WAL replication to object storage), Cloudflare Durable Objects + SQLite (single-writer WAL streaming), restic (content-defined chunking), and the sandbox-persistence work of Replit, E2B, and Modal.

Contributing

The project is at the design stage — the most valuable contributions right now are design reviews and prior-art pointers. Open an issue to discuss.

License

Apache License 2.0

About

A durable local workspace for ephemeral compute — develop against local SQLite and a plain filesystem, while persistence, migration, and recovery happen transparently.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors