Turn a rough idea or a finished story into a story summary, character prompts, and numbered cinematic scene prompts — each formatted for AI image/video generation tools (Midjourney, Veo, Runway, Flow, etc.).
FrameCraft is a lean, single-page pipeline tool. Text goes in; a structured, copy-ready storyboard comes out. It does not generate images or video itself, and it has no accounts or database — everything lives in memory for the session.
The pipeline runs in four stages, streamed into the UI as each resolves so you see results progressively instead of waiting for the whole run.
| Stage | What it does | Model tier | Notes |
|---|---|---|---|
| 1 — Story expansion | Expands your idea into a full story sized for N scenes | quality, temp ~0.8 | The creative backbone |
| 2 — Breakdown | Extracts characters (visible prompt + hidden bible) and the scene outline in one call | fast, temp ~0.35 | One call so character names and charactersPresent always match |
| 3 — Scene expansion | Writes the full cinematic prompt for each scene, in parallel | quality, temp ~0.88 | Capped concurrency; each call gets only the characters present in that scene |
| 4 — Continuity notes | Six production-reference notes (character, lighting, costume, environment, tone, timeline) | fast, temp ~0.3 | Runs once all scenes exist |
Reliability details:
- Content-repair retry — every scene is checked (≥7 lines, restates style + aspect ratio, names all present characters). If it fails, the scene is re-prompted once with the specific problems named.
- Transient-error retry — network/rate-limit/timeout errors retry separately with exponential backoff + jitter.
- Thinking-aware token budgets — Gemini 2.5 models draw reasoning tokens from the same budget as the answer, so each stage sets a generous
maxOutputTokenswith a cappedthinkingBudget.
npm installCreate .env.local in the project root (it is git-ignored — never commit your key):
Option A — Gemini (default)
LLM_PROVIDER=gemini
GEMINI_API_KEY=your_key_here
MODEL_FAST=gemini-2.5-flash # Stages 2 & 4
MODEL_QUALITY=gemini-2.5-flash # Stages 1 & 3
SCENE_CONCURRENCY=4 # parallel Stage 3 calls
MAX_RETRIES=2 # transient-error retry cap
LLM_RPM=5 # requests/minute the pipeline paces itself toGet a Gemini API key at aistudio.google.com/apikey.
Free tier note:
gemini-2.5-prois not available on Google's free tier (limit: 0). Usegemini-2.5-flashforMODEL_QUALITYon the free tier. If you enable billing, setMODEL_QUALITY=gemini-2.5-profor higher-quality scene prose.The free tier also caps
gemini-2.5-flashat 5 requests/minute, and a single generation makes far more than 5 calls (story + breakdown + one call per scene + continuity). The pipeline proactively paces every call to stay withinLLM_RPM(default 5), so generation intentionally slows down rather than failing with a quota error — expect ~1 request every 12s on the default free-tier setting, so higher scene counts take longer. RaiseLLM_RPMif you're on a paid tier with a higher quota.
Option B — Grok / xAI
LLM_PROVIDER=xai
XAI_API_KEY=your_key_here
MODEL_FAST=grok-4.20-non-reasoning # Stages 2 & 4 (defaults if unset)
MODEL_QUALITY=grok-4.20-reasoning # Stages 1 & 3 (defaults if unset)
SCENE_CONCURRENCY=4
MAX_RETRIES=2
LLM_RPM=5 # lower this once you know your actual Grok rate limitGet an xAI API key at console.x.ai. MODEL_FAST/MODEL_QUALITY are optional — each provider has its own sensible defaults, so you only need to set them to override.
Switching providers is just changing
LLM_PROVIDER(and the matching API key) — no code changes. The "thinking" knob differs per provider under the hood (Gemini takes a token budget, Grok takes areasoningEffortlevel of none/low/medium/high), but that's handled internally per stage.
npm run devOpen http://localhost:3000, type an idea, pick your config, and hit Generate.
| Command | Description |
|---|---|
npm run dev |
Start the local dev server |
npm run build |
Production build |
npm start |
Serve the production build |
npm run lint |
ESLint (must pass before a PR) |
npm test |
Vitest unit tests (pipeline, mocked LLM) |
Set in the UI before generating:
- Input mode — rough idea or pasted full story
- Scene count — 3–15
- Visual style — preset or custom
- Aspect ratio — preset or custom width/height
- Tone, age group, shot intensity
- Reference images planned — when on, character prompts focus on costume/silhouette/energy instead of detailed facial description
Results support copy-per-card, regenerate-per-card (character or scene), copy-all, and JSON/TXT download.
- Next.js (App Router) + TypeScript — API routes double as the backend, with native streaming
- Vercel AI SDK (
ai) +@ai-sdk/google/@ai-sdk/xai— provider-abstracted LLM calls withOutput.object()structured output; swapping providers is an env-var change (LLM_PROVIDER=gemini|xai) - Zod — every LLM response is validated before it touches UI state
- p-limit — caps Stage 3 concurrency
- Zustand — session state
- Tailwind CSS + a small set of shadcn-style primitives — dark cinematic theme
- sonner — toast notifications
- Vitest — unit tests
app/
page.tsx # the single page
api/
story/expand/route.ts # Stage 1
breakdown/route.ts # Stage 2 (characters + outline)
scenes/generate/route.ts # Stage 3 fan-out (streams NDJSON)
scenes/[id]/regenerate/route.ts
characters/[id]/regenerate/route.ts
continuity/route.ts # Stage 4
components/ # ConfigForm, ResultsPanel, cards, ui/ primitives
lib/
llm/
client.ts # model routing + per-stage tuning
generate.ts # structured call + transient retry wrapper
stages.ts # the four pipeline stages
pipeline.ts # end-to-end orchestration
prompts/ # per-stage prompt templates
validate.ts # scene rule checker
retry.ts # exponential backoff + jitter
schemas/ # Zod schemas mirroring types/
export/ # toJson, toTxt
store/useProjectStore.ts # Zustand store + streaming logic
types/index.ts # shared types
- No API key is ever bundled into client code — all LLM calls run in server-side API routes.
- Gemini and xAI/Grok are supported out of the box via
LLM_PROVIDER; adding another AI SDK provider (OpenAI, Anthropic, etc.) is a small addition tolib/llm/client.ts, not a rewrite. - Out of scope by design: image/video generation, auth, database, i18n, landing pages.