Annotate UI problems in the browser, export a precise prompt for your coding agent.
A Manifest V3 Chrome extension that turns "this button looks wrong" into a concise, well-located task description for Cursor / Claude Code / Copilot — so the agent knows where the problem is and what to fix.
English · 简体中文
Loop mode turns UI2Prompt from a one-shot prompt exporter into a live, collaborative fix pipeline. You keep annotating UI problems in the browser; one or more coding agents (Claude Code, Cursor, …) continuously claim → fix → self-review each problem and report progress back onto the page — and when a fix has several valid options, the agent asks you a multiple-choice question you answer right in the popup, without blocking its other work.
A tiny, zero-dependency local broker is the shared source of truth between the extension and the agents. It hands each task to exactly one agent at a time (atomic lock), so you can run many agents in parallel with no collisions. The agent drives it through a skill that auto-starts the broker for you — no MCP wiring, no servers to babysit.
flowchart LR
H["🧑 Human<br/>annotates UI problems"] -->|push task| B[("🗂️ Loop broker<br/>auto-started · atomic locks")]
B -->|next · locked| A1["🤖 Agent 1<br/>(ui2prompt-loop skill)"]
B -->|next · locked| A2["🤖 Agent N"]
A1 -->|edit source| S["📦 your app / demo"]
A1 -->|fixed · reviewed| B
A1 -.->|ask · non-blocking| B
B -.->|answer in popup| H
B -->|live status on markers| H
The agents never stop on their own: when the queue is empty they wait ~60s and poll again, looping until you end the session. Markers update live through in-progress → AI-fixed → AI-reviewed, and you close each item with Confirm / Reject.
- Install the browser extension (see Install below).
- Install the loop skill into your coding agent — one line, works for Claude Code and Cursor:
From a clone instead:
curl -fsSL https://github.com/cocbin/MarkUI2Prompt/releases/latest/download/install.sh | bashnpm run install-skill.
That's it. Now use it:
- In the extension, click the ↻ Loop button, toggle Enable loop mode, and Copy the agent prompt.
- Paste the prompt into your agent (Claude Code, Cursor, …) and let it run. The skill auto-starts the local broker and the agent begins claiming and fixing — start a second agent in another terminal any time for more throughput.
- Annotate problems as usual. Watch the agents fix them live, answer any multiple-choice questions in the popup, and Confirm the results.
Try it on the demo:
npm run demoserves a deliberately-buggy, multi-tab settings app at http://localhost:5179. Annotate its issues, paste the prompt into an agent, and watch them get fixed. This loop was verified end-to-end with a real Claude Code agent fixing all five planted bugs and marking each AI-reviewed.
Prefer MCP? (optional)
The repo also ships an MCP server (server/mcp.mjs) exposing the same operations as native tools. Point your agent at .mcp.json (e.g. claude --mcp-config .mcp.json) and start the broker with npm run broker. The skill above is the recommended path because it needs no MCP config and starts the broker itself.
Telling an agent "fix the spacing on the dashboard" rarely works — it can't see your screen and it guesses at the wrong file. UI2Prompt closes that loop:
- Point at the broken element on the real page.
- Describe the problem in one sentence.
- Export a compact prompt that pins each problem to a Vue component / source file, a stable CSS selector, and — as a fallback — an annotated screenshot.
Every exported item has exactly two jobs: locate it and state the problem. Nothing else competes for the model's attention.
| Area | What you get |
|---|---|
| 🔁 Loop mode | Human annotates, one or more coding agents continuously claim → fix → self-review through a local broker (MCP or HTTP). Non-blocking multiple-choice questions are answered in the popup; multi-agent runs are collision-free via atomic task locks. See Loop mode. |
| Tab-aware markers | Markers remember the dialog/tab they were created under. Switch tabs and a marker hides itself instead of lingering at the wrong spot; Locate automatically switches back to the recorded tab first. |
| Annotation mode | Hover to highlight any element, click to drop a numbered marker, write the problem. The host page's own keyboard shortcuts are suppressed so your typing never triggers the underlying app. |
| Smart locators | Selectors prefer id → semantic data-* (e.g. data-type) → name/aria-label/title → meaningful classes. Each selector is verified to match exactly one element; unstable positional selectors are graded weak and left out of the prompt so the agent is never misled. |
| Vue source mapping | Detects the Vue component, its full component path, and the real source file (__file, e.g. src/components/.../Widget.vue) — the strongest possible "where". React component names are detected too. |
| Draggable toolbar | A floating status bar shows mode + annotation count, with Screenshot, Exit + shot, and Exit. Drag it by the grip handle when it covers what you're annotating. Toggle mode with ⌘/Ctrl + M, exit with Esc. |
| Reference picker | While writing a note, click Reference element to pick any other element and insert its semantic path (Vue path preferred) into your description. |
| Fix-verification workflow | open → fixed · pending → confirmed / reopened with full history. After a fix, markers re-bind via selector → xpath → coordinates; a rejected fix reopens the item with your reason recorded. |
| Focused view | Resolved annotations (fixed / confirmed) are hidden from the page and screenshots by default, and the panel opens on the Open tab — so you only see what still needs work. Toggle this in Settings. |
| Annotated screenshot | Each annotation is connected to its element with a red arrow and a translucent numbered label that never buries the underlying UI. Trigger it from the toolbar, from Exit + shot, or from the panel — a self-contained fallback for when a selector can't be resolved. |
| Localized prompt export | Page title + URL, then one line per problem: status, description, and best location. Copy or download the current page or all pages (merged into one file or split per page). The exported prompt follows your selected language. |
| Built-in guide | A step-by-step walkthrough — from entering annotation mode to handing the prompt to an agent — lives behind the ? button in the panel. |
| Polished UI | A layered, editor-style neutral theme with light / dark / system modes, five UI languages (English, 简体中文, 繁體中文, 日本語, 한국어), and crisp inline SVG icons throughout. |
| Persistence | Annotations are stored per-URL and survive reloads (chrome.storage.local in the extension; IndexedDB in injected/page contexts). SPA route changes are tracked automatically. |
A six-step tour from spotting a UI problem to seeing your agent fix it. The same guide lives behind the ? button in the panel.
Click Annotate in the top-right corner, or press ⌘/Ctrl + M. A floating toolbar appears at the bottom of the page.
Hover to highlight an element, click it, then describe the UI problem in the popover. Use Reference element to insert another element's selector.
Review annotations in the panel. Move each through Open → Fixed · pending → Confirmed, and locate, edit or delete as needed.
Use Screenshot to render an annotated image as a fallback, then Copy page or Download to export the Agent prompt.
Paste the exported prompt (with the screenshot when needed) into a coding Agent such as Claude Opus 4.8 1M Max.
The Agent locates the source from the prompt and applies the fix. Return to the page and use Confirm / Reject to verify the result.
| Delete with confirmation | Settings |
|---|---|
 |
 |
- Download
ui2prompt-dist.zipfrom the Releases page and unzip it. - Open
chrome://extensionsand enable Developer mode (top-right). - Click Load unpacked and select the unzipped
dist/folder.
npm install
npm run build # outputs the extension to dist/Then load the dist/ folder via Load unpacked as above.
Install the agent skill once (Claude Code + Cursor); the broker then auto-starts on demand:
curl -fsSL https://github.com/cocbin/MarkUI2Prompt/releases/latest/download/install.sh | bash
# or from a clone: npm run install-skill- Click the toolbar icon → Start annotating, or press
⌘/Ctrl + M(orAlt + Shift + A). The cursor becomes a crosshair and the floating toolbar appears. - Hover to highlight an element, click it, and describe the problem. Optionally click Reference element to insert another element's path. Save → a numbered marker appears.
- Click a marker to view details: change status, edit, locate, or delete. Select a marker and press
Deletefor a quick, confirmed removal (Enterconfirms). - After the agent applies a fix, reload the page — markers re-bind automatically. If an element is gone, the marker degrades to its last coordinates and is flagged.
- For items in fixed · pending, click Confirm or Reject (with a reason) to close the loop.
- From the panel footer, use Copy / Download for the current page or all pages, and Screenshot to capture the annotated image. The exported text matches your selected language. Press
Esc(or the toolbar Exit) to leave annotation mode — exiting no longer screenshots automatically; use Exit + shot when you want both.
New to it? Open the panel and click the ? button for an illustrated walkthrough.
| Shortcut | Action |
|---|---|
⌘/Ctrl + M |
Toggle annotation mode |
Alt + Shift + A |
Toggle annotation mode (global command) |
Delete / Backspace |
Delete the selected annotation (with confirmation) |
Enter |
Confirm the delete prompt |
Esc |
Close the current popover, or exit annotation mode |
While annotation mode is active, the host page's own keyboard shortcuts are suppressed.
UI2Prompt was specified and implemented from a single structured brief — see docs/ui-annotation-plugin-prompt.md — and built end-to-end with the Claude Opus 4.8 1M Max coding agent. That document is worth a read on its own: it's a worked example of the kind of precise, well-scoped prompt this extension is designed to help you produce.
src/
├── shared/ # Environment-agnostic core
│ ├── constants.js annotation.js id.js
│ ├── db.js backends.js store.js # IndexedDB + chrome.storage backends
│ ├── router.js # message router (background + harness)
│ ├── prompt.js loop.js # locale-aware prompt + loop task/prompt builders
│ ├── i18n.js locales/ # en / zh-CN / zh-TW / ja / ko
│ ├── settings.js theme.js icons.js # theme + locale prefs, design tokens, SVG
├── background/ # Service worker: single source of truth, badge, screenshot + download, loop bridge
├── content/ # Page-side engine
│ ├── index.js # orchestration: messages, SPA routing, hotkey, settings, loop poll
│ ├── annotator.js # annotation mode, shortcut blocking, reference picker
│ ├── capture.js locator.js # selector/XPath/bbox + quality grading
│ ├── ui-context.js tab-path.js dom-model.js # dialog/tab context, tab-aware locate, 4-layer DOM
│ ├── vue-detect.js framework-bridge.js main-world.js # cross-world detection
│ └── overlay/ # overlay.js / marker.js / editor.js / toolbar.js / snapshot.js
├── popup/ # management UI (html/css/js + api.js + render.js + menus.js + dialogs.js + loop-panel.js)
server/ # Loop broker (run locally, not part of the extension)
├── broker.mjs # zero-dep HTTP task broker: queue, atomic locks, questions, persistence
├── store.mjs # in-memory task/question store + JSON persistence
└── mcp.mjs # optional stdio MCP server proxying the broker for coding agents
skills/ui2prompt-loop/ # the installable agent skill (this is what users install)
├── SKILL.md # loop protocol the agent follows
├── loop.mjs # self-contained CLI: auto-starts the broker, claim/fix/review/ask/answer
└── broker.mjs # broker bundled into one zero-dep file (built from server/*.mjs)
demo/ # deliberately-buggy multi-tab demo app (npm run demo)
Key ideas:
- Dual-world content script — the
ISOLATEDworld owns UI, storage, and messaging; theMAINworld reads framework internals (__vueParentComponent,__file) that are invisible to isolated scripts. They communicate viapostMessage. - Single source of truth — the background owns
chrome.storage.local; content and popup read/write through messages. Injected page contexts fall back to IndexedDB, so the engine is independently testable. - High-performance overlay —
requestAnimationFramebatches marker positioning,MutationObserverre-binds on DOM changes, and a Shadow DOM isolates styles without blocking page interaction.
npm run build # build the extension to dist/
npm run watch # incremental rebuild on change
npm run harness # build dist/popup-dev.html to preview the popup UI in a normal tab
npm run serve # static-serve dist/ at http://localhost:5180 (open /popup-dev.html)
npm run broker # start the loop broker at http://127.0.0.1:8787 (normally auto-started by the skill)
npm run mcp # run the optional stdio MCP server (only if you use .mcp.json)
npm run skill # bundle the broker into skills/ui2prompt-loop and zip it to dist-skill/
npm run install-skill # build the skill and install it into ~/.claude/skills and ~/.cursor/skills
npm run demo # serve the buggy demo app at http://localhost:5179The harness drives the real store/router through a chrome.* shim and is not shipped in the extension build. The server/ broker, the skills/ package, and the demo/ app are local tools for loop mode and are excluded from the extension build.
- Iframe content isn't reachable. Only top-frame elements can be annotated (
all_frames: false). <canvas>/WebGL internals (charts, maps) are opaque — the locator targets the nearest semantic wrapper element, and the annotated screenshot serves as the visual fallback.- Selector stability depends on the app. When an element exposes no
id, semantic attribute, or meaningful class, the selector is gradedweakand intentionally left out of the prompt; rely on the Vue source mapping and the screenshot instead. - Source-file mapping needs dev metadata.
__fileis present in Vue dev builds; production builds that strip it fall back to the component name. - Single device-pixel screenshots. Capture uses
captureVisibleTab— the current viewport at the current zoom.
Issues and PRs are welcome. Please keep modules small and cohesive, prefer root-cause fixes, and run npm run build (lint-clean) before submitting.
MIT © UI2Prompt contributors.