Skip to content

[mcp] Add AI Render tool to MCP bridge so agents can complete the four-image render workflow end-to-end #1

Description

@kfarr

Problem

The RENDER_WITH_3DSTREET.md workflow produces four images per location at a single locked camera pose:

  1. before-raw — geo 3D tiles only, envelopes hidden (snapshot)
  2. before-renderAI Render upscale of (1) to photorealistic
  3. massing — same camera, envelopes visible (snapshot)
  4. future-renderAI Render img2img on (3), boxes → buildings

The doc's "Driving via MCP" section invites an LLM agent to run the whole workflow, but the bridge currently only exposes takeSnapshot. There is no tool to invoke the AI Render panel (Nano Banana Pro / img2img). I confirmed this on 3dstreet-mcp paired to scene a7a99cb6-b8bf-438f-ac26-8f4cda3949c7 — a tools/list against the bridge surfaces snapshot, scene/entity read+edit, camera focus, lat/lon, undo/redo, but nothing for AI Render.

This breaks an explicit goal of the integration: an agent can do steps 1 and 3, but has to hand off to a human for steps 2 and 4, which forces a human in the loop on every location and removes most of the value of MCP-driving the workflow.

Proposed tool surface

A single tool, e.g. aiRender, mirroring the AI Render panel:

aiRender({
  mode: "txt2img" | "img2img",     // img2img is what the workflow needs
  sourceImage: "<id-or-data-uri>", // result of takeSnapshot, for img2img
  prompt: string,
  model?: string,                  // default: "nano-banana-pro"
  caption?: string,
})
→ { imageId, url | dataUri, model, prompt }

Minimum to unblock the workflow:

  • img2img against the most recent snapshot (so takeSnapshotaiRender chains cleanly)
  • Returns the rendered image to the chat the same way takeSnapshot does
  • Surfaces model + prompt in the response so agents can log them

Nice to have (not blockers):

  • Pluggable model param (the doc mentions experimenting with other models)
  • Persistent image IDs so the agent can refer back to them later in the session

Repro

claude mcp add 3dstreet -- npx -y 3dstreet-mcp
# in 3dstreet.app: AI Assistant → /mcp → Reconnect (green)

Then ask the agent to run the workflow end-to-end. It will get to before-raw, then have nothing to call for the upscale step.

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status
    To Do - High Prio

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions