Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 10 additions & 6 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,13 +58,17 @@ Document what you ruled out in PR comments so reviewers do not chase the wrong d

## Project overview

VMP (Video Monetization Platform) is a subscription-gated HLS video streaming platform. npm workspaces monorepo with three packages:
VMP (Video Monetization Platform) is a subscription-gated HLS video streaming platform. npm workspaces monorepo — see [README.md](README.md) for the full package table. Core packages:

| Package | Path | Runtime |
|---|---|---|
| `@vmp/api` | `packages/api` | Cloudflare Worker (JS) — REST API, auth, Stripe, push, thumbnails |
| `@vmp/web` | `packages/web` | Nuxt 4 / Vue 3 frontend (TypeScript) — Cloudflare **Worker** SSR (`wrangler.workers.toml`) |
| `@vmp/shared` | `packages/shared` | Shared TS types |
| `@vmp/api` | `packages/api` | Cloudflare Worker (TypeScript) — REST API, auth, Stripe, push, thumbnails |
| `@vmp/web` | `packages/web` | Nuxt 4 / Vue 3 frontend (TypeScript) — Cloudflare **Worker** SSR (`packages/web/wrangler.workers.toml`) |
| `@vmp/shared` | `packages/shared` | Shared TypeScript types |
| `@vmp/storage` | `packages/storage` | Pluggable object storage (R2 / S3-compatible) |
| `@vmp/payments` | `packages/payments` | Payment provider registry (Stripe, legacy Qerko) |
| `@vmp/api-node` | `packages/api-node` | Deno Deploy backup API (Postgres + S3 adapters) |
| `@vmp/media-pipeline` | `packages/media-pipeline` | Media VM: SVT Encore + Shaka HLS → R2 |

### Infrastructure

Expand All @@ -73,7 +77,7 @@ VMP (Video Monetization Platform) is a subscription-gated HLS video streaming pl
| Video/asset storage | Cloudflare R2 |
| API + auth backend | Cloudflare Workers |
| Database | Cloudflare D1 (SQLite) |
| Config format | `wrangler.json` (not `.toml`) |
| Config format | `wrangler.json` for `@vmp/api` (exception: `@vmp/web` uses `wrangler.workers.toml`) |
| Frontend | Nuxt 4 on Cloudflare Workers (`vmp-web-worker-dev` / `vmp-web-worker-prod`) + `@vmp/api` API Worker |
| Email | Brevo Transactional API |
| Payments | Stripe (card, PayPal, SEPA via Checkout); optional legacy provider for grandfathered subs |
Expand Down Expand Up @@ -104,7 +108,7 @@ Migrations live in `packages/api/migrations/` — always add a new numbered file

### Auth system (DO NOT rewrite)

Fully implemented in `packages/api/src/auth.js`. Key exports:
Fully implemented in `packages/api/src/auth.ts`. Key exports:
- `handleRequestMagicLink` — `POST /api/auth/magic-link`
- `handleVerifyMagicLink` — `GET /api/auth/verify?token=`
- `handleRefreshToken` — `POST /api/auth/refresh`
Expand Down
13 changes: 12 additions & 1 deletion DEPLOYMENT.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,16 @@
# Deployment and Environment Strategy

Operational deploy guide for VMP. For product architecture and package overview, see [README.md](README.md). For agent/dev workflow, see [AGENTS.md](AGENTS.md).

## Contents

- [Unified environment variables](#unified-environment-variables)
- [Multi-domain setup](#multi-domain-setup)
- [CI/CD flow](#cicd-flow)
- [Fresh infrastructure bootstrap runbook](#fresh-infrastructure-bootstrap-runbook)
- [Rollback notes (staging/production)](#rollback-notes-stagingproduction)
- [Livestream notes (Media over QUIC)](#livestream-notes-media-over-quic)

## Unified environment variables

Use the root `.env.example` as the single source template for all runtime values.
Expand Down Expand Up @@ -128,7 +139,7 @@ Use this when staging/production D1, KV, and/or R2 were intentionally reset.
- D1 database
- KV namespace(s)
- R2 bucket(s)
- Update Worker/Page bindings to point to recreated resources.
- Update Worker bindings to point to recreated resources.

1. Restore required secrets (per environment)

Expand Down
208 changes: 100 additions & 108 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,147 +1,139 @@
# VMP (Video Monetization Platform)

VMP is a Cloudflare-based video subscription platform with a Worker API, Nuxt web app, and shared TypeScript types.
Subscription-gated HLS video streaming: Cloudflare Worker API + Nuxt 4 web app, with optional Deno backup API and a media VM for transcoding.

## Contents

- [Architecture](#architecture)
- [Monorepo packages](#monorepo-packages)
- [Documentation map](#documentation-map)
- [Deployment model (high level)](#deployment-model-high-level)
- [Documentation](#documentation)
- [Prerequisites](#prerequisites)
- [Local setup](#local-setup)
- [Manual deploy commands](#manual-deploy-commands)
- [Runtime secrets/vars](#runtime-secretsvars)
- [Optional MediaConvert pipeline](#optional-mediaconvert-pipeline-additive-to-local-rclone-flow)
- [API docs (`docs/`)](#api-docs-docs)
- [Notes](#notes)
- [Local development](#local-development)
- [Deploy](#deploy)
- [Secrets and configuration](#secrets-and-configuration)
- [Media encoding](#media-encoding)

## Architecture

| Layer | Technology |
| --- | --- |
| API | Cloudflare Worker (`@vmp/api`) — REST, auth, Stripe, push, RSS |
| Web | Nuxt 4 / Vue 3 (`@vmp/web`) — Cloudflare **Workers** SSR (`vmp-web-worker-dev` / `vmp-web-worker-prod`) |
| Database | Cloudflare D1 (SQLite); Postgres shim on Deno backup |
| Object storage | Cloudflare R2 (pluggable via `@vmp/storage`) |
| Payments | Stripe (+ optional legacy Qerko via `@vmp/payments`) |
| Email / push | Brevo transactional, Web Push (VAPID) |
| Transcoding | `@vmp/media-pipeline` on a media VM (SVT Encore + Shaka → R2) |
| Backup API | `@vmp/api-node` on Deno Deploy (same handlers, Postgres + S3) |

Cloudflare **Pages** (`vmp-fe`) is deprecated. Do not attach production hostnames to Pages.

```text
Browser ──► @vmp/web (Worker SSR)
@vmp/api (Worker) ──► D1 / R2 / KV
├── Stripe webhooks, Brevo, Web Push
└── pipeline-status ◄── @vmp/media-pipeline (VM)
```

## Monorepo packages

| Package | Path | Role |
| --- | --- | --- |
| `@vmp/api` | [`packages/api`](packages/api) | Cloudflare Worker API + D1/R2/KV integrations |
| `@vmp/web` | [`packages/web`](packages/web) | Nuxt 4 frontend deployed to Cloudflare Pages |
| `@vmp/shared` | [`packages/shared`](packages/shared) | Shared TypeScript contracts |
| `@vmp/api-node` | [`packages/api-node`](packages/api-node) | Deno Deploy backup API (Postgres + S3 adapters) — see [README](packages/api-node/README.md) |
| `@vmp/media-pipeline` | [`packages/media-pipeline`](packages/media-pipeline) | Media VM: SVT Encore transcoding + Shaka HLS + R2 — see [README](packages/media-pipeline/README.md) |
| `@vmp/offloading` | [`packages/offloading`](packages/offloading) | R2↔Garage hot/cold tier orchestration — see [README](packages/offloading/README.md) |
| `@vmp/moq-probe` | [`packages/moq-probe`](packages/moq-probe) | MoQ broadcast diagnostic probe — see [README](packages/moq-probe/README.md) |
| `@vmp/api` | [`packages/api`](packages/api) | Primary Cloudflare Worker API |
| `@vmp/web` | [`packages/web`](packages/web) | Nuxt 4 frontend (Workers SSR) |
| `@vmp/shared` | [`packages/shared`](packages/shared) | Shared TypeScript types |
| `@vmp/storage` | [`packages/storage`](packages/storage) | Pluggable object storage (R2 / S3-compatible) — [README](packages/storage/README.md) |
| `@vmp/payments` | [`packages/payments`](packages/payments) | Payment provider registry (Stripe, legacy Qerko) — [README](packages/payments/README.md) |
| `@vmp/api-node` | [`packages/api-node`](packages/api-node) | Deno Deploy backup API — [README](packages/api-node/README.md) |
| `@vmp/media-pipeline` | [`packages/media-pipeline`](packages/media-pipeline) | Media VM: Encore + Shaka HLS + R2 — [README](packages/media-pipeline/README.md) |
| `@vmp/offloading` | [`packages/offloading`](packages/offloading) | R2 ↔ Garage hot/cold tiering — [README](packages/offloading/README.md) |
| `@vmp/moq-probe` | [`packages/moq-probe`](packages/moq-probe) | MoQ live broadcast diagnostic probe — [README](packages/moq-probe/README.md) |

Core API and web packages do not ship separate READMEs; see [AGENTS.md](AGENTS.md) for architecture, auth, schema, and local dev.
Core API/web packages do not ship separate READMEs; see [AGENTS.md](AGENTS.md) for auth, D1 schema, roles, and agent workflow.

## Documentation map
## Documentation

| Document | Description |
| Document | Audience |
| --- | --- |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

The new Documentation table links [docs/README.md](docs/README.md) as "Index of API notes and historical design docs", but that file does not exist in the repository and is not being created in this PR. Any reader following the link will hit a 404.

| [AGENTS.md](AGENTS.md) | Canonical agent/dev guide: git workflow, D1 schema, auth, secrets, roadmap |
| [DEPLOYMENT.md](DEPLOYMENT.md) | Environment variables, CI/CD, multi-domain deploy |
| [packages/api-node/README.md](packages/api-node/README.md) | Deno Deploy API backup, Postgres replication ingest |
| [packages/media-pipeline/README.md](packages/media-pipeline/README.md) | Encore transcoding orchestration, supervisor, webhooks, TTP logging |
| [packages/media-pipeline/MIGRATION.md](packages/media-pipeline/MIGRATION.md) | Cutover guide from legacy `podcast-host` |
| [packages/media-pipeline/systemd/README.md](packages/media-pipeline/systemd/README.md) | `vmp-supervisor` systemd unit install and ops |
| [packages/offloading/README.md](packages/offloading/README.md) | Garage compose, demote/promote scripts |
| [packages/offloading/DEPLOYMENT.md](packages/offloading/DEPLOYMENT.md) | Docker/Compose deployment for offloading |
| [packages/moq-probe/README.md](packages/moq-probe/README.md) | Live MoQ probe usage and recorder recommendations |
| [.cursor/README.md](.cursor/README.md) | Cursor workspace git/deploy policy for agents |
| [docs/pills-external-update-api.md](docs/pills-external-update-api.md) | Pills external update API |

## Deployment model (high level)

- Pushes to `main` run staging deploy in [`.github/workflows/deploy.yml`](.github/workflows/deploy.yml): API Worker + web Worker (`vmp-web-worker-dev`).
- Version tags (`v*.*.*`) run production deploy: API Worker + web Worker (`vmp-web-worker-prod`).
- Frontend is **Cloudflare Workers only** (Nuxt SSR). Pages is deprecated.
- Deploy pipeline fails fast on type-checking before build/deploy:
- `@vmp/shared` `tsc --noEmit`
- `@vmp/api` `tsc --noEmit`
- `@vmp/web` `nuxi prepare && nuxi typecheck`

See [DEPLOYMENT.md](DEPLOYMENT.md) for env templates, secrets, and domain overrides.
| [AGENTS.md](AGENTS.md) | **Canonical** architecture, git workflow, secrets, local Cloud setup, roadmap |
| [DEPLOYMENT.md](DEPLOYMENT.md) | CI/CD, env vars, smoke checks, bootstrap / rollback |
| [docs/README.md](docs/README.md) | Index of API notes and historical design docs |
| [packages/web/docs/workers-deploy-env.md](packages/web/docs/workers-deploy-env.md) | Web Worker build-time env vars |
| Package READMEs under `packages/*` | Package-specific ops (pipeline, api-node, storage, …) |

## Prerequisites

1. Node.js 20+ and npm 10+.
2. Cloudflare account with:
- Two Workers per environment (API + Nuxt web frontend)
- D1 database
- R2 bucket
- KV namespaces
3. Repository secrets configured for CI deploy:
- `CLOUDFLARE_API_TOKEN_STAGING`
- `CLOUDFLARE_ACCOUNT_ID_STAGING`
- `CLOUDFLARE_API_TOKEN_PROD`
- `CLOUDFLARE_ACCOUNT_ID_PROD`
1. Node.js 20+ and npm 10+ (`packageManager` in root `package.json`).
2. Cloudflare account with Workers, D1, R2, and KV (per environment).
3. CI deploy secrets: `CLOUDFLARE_API_TOKEN_{STAGING,PROD}` and `CLOUDFLARE_ACCOUNT_ID_{STAGING,PROD}`.

## Local setup
## Local development

1. Install dependencies:
- `npm ci`
2. Build service worker helper from TS source:
- `npm run build:sw-push --workspace=@vmp/web`
3. Type-check everything:
- `npm run typecheck`
```bash
npm ci
npm run typecheck

Full local dev (API on `:8787`, web on `:3000`, D1 migrations): [AGENTS.md → Cursor Cloud-specific instructions](AGENTS.md#cursor-cloud-specific-instructions).
# API (Wrangler, port 8787) — needs packages/api/.dev.vars
npm run dev --workspace=@vmp/api

## Manual deploy commands
# Web (Nuxt, port 3000)
API_URL=http://localhost:8787 npm run dev --workspace=@vmp/web
```

These are useful for controlled/manual rollouts outside GitHub Actions.
Apply D1 migrations locally before serving data:

1. API deploy:
- `npm run deploy:api`
2. Web deploy (from `packages/web` after `npm run build`):
- Staging Worker: `npm run deploy --workspace=@vmp/web`
- Production Worker: `npm run deploy:prod --workspace=@vmp/web`
```bash
cd packages/api
for f in $(ls -1 migrations/*.sql | sort); do
npx wrangler d1 execute video-subscription-db --local --file="$f"
done
```

## Runtime secrets/vars
Seed videos start as drafts. To publish them for the homepage:

Use Wrangler secrets for sensitive values (never commit secrets). Required values are documented in [AGENTS.md](AGENTS.md) and [DEPLOYMENT.md](DEPLOYMENT.md) (JWT, Stripe, Brevo, VAPID, RSS, TOTP encryption, etc.).
```bash
npx wrangler d1 execute video-subscription-db --local \
--command="UPDATE videos SET publish_status = 'published', published_at = CURRENT_TIMESTAMP WHERE publish_status = 'draft';"
```

### Optional MediaConvert pipeline (additive to local rclone flow)
Full Cloud-agent local notes (secrets, gotchas): [AGENTS.md → Cursor Cloud-specific instructions](AGENTS.md#cursor-cloud-specific-instructions).

The existing local/watchfolder/rclone pipeline remains the default and is unchanged.
An optional cloud path can be enabled for direct source uploads + AWS Elemental MediaConvert.
## Deploy

| Trigger | What deploys |
| --- | --- |
| Push to `main` | Staging: API Worker + web Worker (`vmp-web-worker-dev`) via [`.github/workflows/deploy.yml`](.github/workflows/deploy.yml) |
| Tag `v*.*.*` | Production: API Worker + web Worker (`vmp-web-worker-prod`) |
| Push / PR (Deno git integration) | `@vmp/api-node` preview/production build on Deno Deploy — **not** from `deploy.yml`. PR check status is **pending until** `deploy/tjm/vmp` clears; maintainer log review on [console.deno.com](https://console.deno.com) is required when that check stays red. |

Required API env vars (Worker):
Manual commands:

- `MEDIA_CONVERT_ENABLED=1`
- `AWS_REGION` (for S3 + MediaConvert, e.g. `eu-central-1`)
- `AWS_ACCESS_KEY_ID`
- `AWS_SECRET_ACCESS_KEY`
- `AWS_SESSION_TOKEN` (optional, only when using temporary credentials)
- `MEDIA_CONVERT_ENDPOINT` (account-specific endpoint from MediaConvert console)
- `MEDIA_CONVERT_ROLE_ARN` (IAM role MediaConvert assumes)
- `MEDIA_CONVERT_INPUT_BUCKET` (S3 source uploads)
- `MEDIA_CONVERT_OUTPUT_BUCKET` (S3 transcode outputs)
- `MEDIA_CONVERT_INPUT_PREFIX` (optional, default `mediaconvert-input`)
- `MEDIA_CONVERT_OUTPUT_PREFIX` (optional, default `mediaconvert-output`)
- `MEDIA_CONVERT_MAX_UPLOAD_MB` (optional, default `4096`)
- `MEDIA_CONVERT_PRICE_HD_PER_MIN` (optional, default `0.015`; rough estimator)
```bash
npm run deploy:api
npm run deploy --workspace=@vmp/web # staging Worker
npm run deploy:prod --workspace=@vmp/web # production Worker
```

AWS setup checklist:
Deploy gates typecheck `@vmp/shared`, `@vmp/storage`, `@vmp/api`, and `@vmp/web` before build. Details: [DEPLOYMENT.md](DEPLOYMENT.md).

1. Create/choose two S3 prefixes/buckets for input and output.
2. Create a MediaConvert IAM service role (`MEDIA_CONVERT_ROLE_ARN`) with read access to input and write access to output.
3. Create an IAM principal for the Worker credentials with:
- `mediaconvert:CreateJob`, `mediaconvert:GetJob`
- `iam:PassRole` scoped to `MEDIA_CONVERT_ROLE_ARN`
- `s3:PutObject` on input prefix
- `s3:GetObject` on output object prefix
- `s3:ListBucket` on output bucket (bucket-level action), scoped by `s3:prefix` condition to the output prefix
4. In MediaConvert console, copy the account endpoint into `MEDIA_CONVERT_ENDPOINT`.
5. Run migration `packages/api/migrations/0017_media_convert_jobs.sql`.
**Never push feature work directly to `main`** — use a branch + pull request (autodeploy + CodeRabbit). See [AGENTS.md → Git workflow](AGENTS.md#git-workflow-mandatory--read-first).

Notes:
## Secrets and configuration

- Current cloud profile is H.264/HLS with 720p only, fps capped at 30.
- Architecture is rendition-based and supports future expansion (480p/1080p/4K, alternate codecs).
- Usage/cost values are approximate normalized-minute estimates, not billing-grade accounting.
- Template: root [`.env.example`](.env.example). Copy to `.env.staging` / `.env.production` as needed.
- Local API secrets: `packages/api/.dev.vars` (never commit).
- Required Worker secrets (JWT, Stripe, Brevo, VAPID, RSS, TOTP, …): listed in [AGENTS.md](AGENTS.md) and [DEPLOYMENT.md](DEPLOYMENT.md).
- Prices, plan names, and limits live in D1 `admin_settings` — not hardcoded.

## API docs (`docs/`)
## Media encoding

- [Pills external update API](docs/pills-external-update-api.md)
Transcoding runs on a **media VM** via [`@vmp/media-pipeline`](packages/media-pipeline/README.md):

## Notes
1. Watchfolder intake → SVT Encore encode
2. encore-packager (Shaka) → fMP4 HLS ladder uploaded to R2
3. HMAC callback to `POST /api/admin/videos/:id/pipeline-status`

- API entrypoint is TypeScript (`packages/api/src/index.ts`) referenced by `packages/api/wrangler.json`.
- Worker/service scripts that must remain JavaScript at runtime (for browser/service-worker execution) are generated from TypeScript sources during build.
- Media encoding on a VM is handled by [`@vmp/media-pipeline`](packages/media-pipeline/README.md) (SVT Encore + orchestrator); optional Deno backup API by [`@vmp/api-node`](packages/api-node/README.md).
There is **no** AWS Elemental MediaConvert admin upload/transcode UI in this repo anymore. The historical `media_convert_jobs` D1 table remains: playback and offline-download code may still read completed **Bunny Stream** rows (`provider = 'bunnystream'`, `bunny_playback_url`) as an alternate HLS entrypoint. Do not drop that table without a migration that replaces those reads.
29 changes: 29 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# Documentation index

Start with the [repository README](../README.md) and [AGENTS.md](../AGENTS.md). This folder holds API notes and historical design write-ups.

## Current

| Document | Description |
| --- | --- |
| [pills-external-update-api.md](pills-external-update-api.md) | `POST /api/pills/update` external API contract |
| [console-errors.md](console-errors.md) | Common browser console messages and how to interpret them |
| [i18n-prep.md](i18n-prep.md) | Per-instance UI locale (`NUXT_PUBLIC_UI_LOCALE`) and translation workflow |

Related (outside `docs/`):

| Document | Description |
| --- | --- |
| [../DEPLOYMENT.md](../DEPLOYMENT.md) | CI/CD, env vars, smoke checks |
| [../packages/web/docs/workers-deploy-env.md](../packages/web/docs/workers-deploy-env.md) | Web Worker build-time environment |
| [../packages/web/docs/workers-pages-compatibility.md](../packages/web/docs/workers-pages-compatibility.md) | Workers vs deprecated Pages routing notes |

## Archive (historical / planning)

These are kept for context. They are **not** the source of truth for current behavior.

| Document | Notes |
| --- | --- |
| [archive/admin-homescreen-layout-redesign.md](archive/admin-homescreen-layout-redesign.md) | Homescreen editor redesign plan |
| [archive/offline-downloads-roadmap.md](archive/offline-downloads-roadmap.md) | Offline downloads milestones (M1–M6 shipped) |
| [archive/stripe-express-checkout-investigation.md](archive/stripe-express-checkout-investigation.md) | Stripe Express Checkout investigation notes |
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
# Admin homescreen layout — redesign plan

This document is a **plan only** (no implementation in the current PR). It captures the target experience described for the admin “homescreen layout” editor.
> **Archived.** Historical planning note. Not the source of truth for current admin UI behavior. See [docs/README.md](../README.md).

This document is a **plan only**. It captures the target experience described for the admin “homescreen layout” editor.

## Problem with the current admin UI

Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Offline Downloads — Architecture & Roadmap

> **Archived.** Milestones below were used during implementation. For current behavior, prefer code under `packages/api` / `packages/web` and [docs/README.md](../README.md).

Status: **M1–M6 implemented** (API + client storage, playback, UI, revalidation).

## Goals
Expand Down
Loading
Loading