Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 11 additions & 11 deletions docs/motion-planning/3d-scene/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,11 @@ description: "Visualize your machine's frame system, geometries, and point cloud
---

The **3D SCENE** tab renders your machine's frame system as an interactive 3D visualization on your machine's page in the [Viam app](https://app.viam.com).
Frame configuration is otherwise invisible: a JSON translation of `{x: 50, y: 0, z: 110}` tells you nothing about whether the gripper actually sits where the arm needs it. The **3D SCENE** tab makes that spatial relationship visible so you can catch misconfigurations before a motion plan fails.
It shows the spatial relationships your frame configuration describes: the offset you configured as `{x: 50, y: 0, z: 110}` renders as a gripper 50 mm forward and 110 mm above the arm flange, so you can catch a misplaced frame before a motion plan fails.

The tab reads your machine's configuration and, when the machine is online, connects for live data. Each component's frame appears as a set of coordinate axes positioned by its translation and orientation relative to its parent frame. Attached geometries render as translucent shapes, and point clouds from depth cameras render as colored point sets.
The tab reads your machine's configuration and, when the machine is online, connects for live data. Each component's frame appears as a set of coordinate axes positioned by its translation and orientation relative to its parent frame.

## The interface
## Interface

The tab has four areas, each doing a distinct job: the **viewport** renders the scene; the **World panel** and **Details panel** select and inspect entities; the **Dashboard toolbar** changes how the viewport renders.

Expand All @@ -29,7 +29,7 @@ Click a row to select the entity; its details appear in the Details panel.
The panel is draggable and anchors to the top-right of the viewport by default.
It includes:

- **world position** (mm) and **world orientation** (deg, as an orientation vector `x / y / z / th`): the entity's absolute pose in the world frame. Read-only.
- **world position** (mm) and **world orientation** (an orientation vector: `x / y / z` unit-vector components, `th` in degrees): the entity's absolute pose in the world frame. Read-only.
- **parent frame**: which frame this entity is a child of. Editable when the entity is a configurable frame.
- **local position** (mm) and **local orientation** (deg): pose relative to the parent frame. Editable for configurable frames; these correspond to the `translation` and `orientation` in your frame configuration.
- **geometry**: four buttons (`None` / `Box` / `Sphere` / `Capsule`) plus **dimensions** (`x / y / z` for Box, `r / l` for Capsule, `r` for Sphere, all in mm).
Expand All @@ -39,13 +39,13 @@ Entities that can be removed (for example, dropped PCD files) also show a **Remo

**Dashboard toolbar** (top-center): Visible buttons, left to right:

- **Orthographic / Perspective** toggle — switch between an orthographic view (no foreshortening) and a perspective view. Keyboard: `C`.
- **Add frames** opens a floating panel listing components that do not yet have a frame; click a component and then **Add frame** (singular) to attach a default frame to it. See [Edit frames visually](/motion-planning/3d-scene/edit-frames/).
- **Measurement** (ruler icon) activate to measure distance between two points you pick in the viewport. Click the icon again to clear.
- **Measurement settings** (sliders icon next to the ruler) toggle `x`, `y`, or `z` under **Enabled axes** to constrain the second point to the enabled axes of the first.
- **AI Scene Builder** opens a prompt panel for editing frames with natural language. See [Edit frames with AI](/motion-planning/3d-scene/edit-frames/#edit-frames-with-ai).
- **Logs** shows a count badge for errors/warnings from the scene renderer.
- **Settings** (gear icon) opens the Settings panel.
- **Orthographic / Perspective** toggle: switches between an orthographic view (no foreshortening) and a perspective view. Keyboard: `C`.
- **Add frames**: opens a floating panel listing components that do not yet have a frame; click a component and then **Add frame** (singular) to attach a default frame to it. See [Edit frames visually](/motion-planning/3d-scene/edit-frames/).
- **Measurement** (ruler icon): activate to measure distance between two points you pick in the viewport. Click the icon again to clear.
- **Measurement settings** (sliders icon next to the ruler): toggle `x`, `y`, or `z` under **Enabled axes** to constrain the second point to the enabled axes of the first.
- **AI Scene Builder**: opens a prompt panel for editing frames with natural language. See [Edit frames with AI](/motion-planning/3d-scene/edit-frames/#edit-frames-with-ai).
- **Logs**: shows a count badge for errors/warnings from the scene renderer.
- **Settings** (gear icon): opens the Settings panel.

## Navigation controls

Expand Down
4 changes: 2 additions & 2 deletions docs/motion-planning/3d-scene/calibrate-frame-offsets.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ type: "docs"
description: "Verify and adjust the spatial relationship between components using the 3D scene and measurement tool."
---

When you configure a camera on an arm, or a sensor on a base, the frame system needs the exact translation and orientation between the two components. A 15 mm error in a camera offset places a detected object 15 mm off; the arm then reaches for the wrong spot, or the point cloud sits behind the table instead of on it. The **3D SCENE** tab lets you verify offsets visually and measure distances directly, so you can catch these errors before they produce bad motion.
When you configure a camera on an arm, or a sensor on a base, the frame system needs the exact translation and orientation between the two components. A 15 mm error in a camera offset places a detected object 15 mm off, and the arm reaches for the wrong spot. The **3D SCENE** tab lets you verify offsets visually and measure distances directly, so you can catch these errors before you run a plan.

## Prerequisites

Expand Down Expand Up @@ -50,7 +50,7 @@ Select a component and compare its coordinate axes in 3D to the physical compone
- A camera's Z axis (blue) should point forward along the optical axis.
- An arm's axes should follow the manufacturer's convention.

If the axes are rotated relative to what you expect, adjust the orientation vector in the frame configuration. The Details panel displays it as (`x`, `y`, `z`, `th` in degrees).
If the axes are rotated relative to what you expect, adjust the orientation vector in the frame configuration. The Details panel displays it as (`x`, `y`, `z` as unit-vector components and `th` in degrees).

### 5. Verify with a point cloud

Expand Down
12 changes: 6 additions & 6 deletions docs/motion-planning/3d-scene/edit-frames.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,12 @@ title: "Edit frames visually"
weight: 50
layout: "docs"
type: "docs"
description: "Add, edit, and attach geometry to frames directly in the 3D scene instead of editing JSON configuration."
description: "Add, edit, and attach geometry to frames directly in the 3D scene."
---

The **3D SCENE** tab can serve as a configuration editor: you can add, move, re-parent, and reshape frames without writing JSON.

Visual editing is most useful while you are still figuring out where things go. Typing coordinates into JSON and reloading the 3D view to check them is slow; editing in the viewport and seeing the result immediately is faster. The trade-off is that the visual editor writes the same JSON fields through a smaller surface area, so it is less suited to bulk changes or cross-machine-part frames. Changes flow back to the machine configuration, and the app surfaces an unsaved-changes banner on the CONFIGURE tab where you save them with **Save** or `⌘/Ctrl+S`.
Visual editing is most useful while you are still figuring out where things go. Edits render in the viewport as you type, so you can position a frame by eye and read off the values. The trade-off is that the visual editor edits only a subset of the frame JSON (translation, orientation vector, and a single geometry), so it is less suited to bulk changes or cross-machine-part frames.

## Prerequisites

Expand All @@ -30,7 +30,7 @@ You can then reposition it using the Details panel.
1. Select the component in the **World** panel on the upper-left, or by clicking it in the 3D viewport.
2. The Details panel (upper-right) shows the entity's current values. There is no edit-mode toggle; for any configurable frame, the **local position** and **local orientation** fields are editable inputs.
3. Edit the position values (`x`, `y`, `z` in mm) to set the translation relative to the parent frame.
4. Edit the orientation values (`x`, `y`, `z`, `th` in degrees) to set the orientation as an orientation vector.
4. Edit the orientation values (`x`, `y`, `z` unit-vector components and `th` in degrees) to set the orientation as an orientation vector.

Changes appear immediately in the 3D viewport as you type.
The values you enter here correspond directly to the `translation` and `orientation` fields in the frame JSON configuration.
Expand Down Expand Up @@ -86,10 +86,10 @@ After the AI applies changes, save or discard buttons appear in the **3D SCENE**

Visual editing covers most cases, but a few are faster in JSON:

- **Bulk changes** (renaming many frames, regenerating a layout) JSON
- **Bulk changes** (renaming many frames, regenerating a layout): JSON
edits are easier in a text editor.
- **Frames that reference components on a different machine part**
- **Frames that reference components on a different machine part**:
the visual editor's parent dropdown only shows local frames.
- **Complex orientations** (rotations expressed in `axis_angles` or
`quaternion` rather than `ov_degrees`)the visual editor surfaces
`quaternion` rather than `ov_degrees`): the visual editor shows
only the orientation vector form.
15 changes: 9 additions & 6 deletions docs/motion-planning/3d-scene/set-up-obstacle-avoidance.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ type: "docs"
description: "Visualize and adjust obstacle geometry so the motion planner routes around physical objects."
---

The motion planner avoids obstacles only if it knows about them, and it knows about them only as geometries you define: boxes, spheres, capsules, or cylinders positioned in the frame system. That definition is invisible in JSON. A box specified as `{x: 800, y: 1200, z: 20}` at some parent-relative translation either covers the table or it doesn't, and you can't tell which from the numbers. The **3D SCENE** tab lets you see what the planner sees, so you can check coverage before running a plan.
The motion planner avoids only the obstacles you define as geometries: boxes, spheres, capsules, or cylinders positioned in the frame system. The **3D SCENE** tab draws each geometry at its configured position and size, so you can see whether a box specified as `{x: 800, y: 1200, z: 20}` actually covers the table before you run a plan.

## Prerequisites

Expand Down Expand Up @@ -35,6 +35,8 @@ Click an obstacle in the scene or in the World panel to see its details:
- **local position** (mm): the geometry's center relative to its parent frame.
- **parent frame**: which frame the geometry is attached to.

Cylinders configured in JSON render in the scene; the geometry buttons cover box, sphere, and capsule only.

### 3. Compare geometry to physical objects

Orbit the scene to view obstacles from different angles.
Expand Down Expand Up @@ -67,15 +69,16 @@ After defining obstacles, run through this checklist in the **3D SCENE** tab:

## Dynamic obstacles

Static obstacles in configuration cover fixed workspace objects. For objects
that move, pass geometry at runtime through the `WorldState` parameter of the
`Move` request. See
The scene draws only geometries from saved configuration; obstacles passed
in a `Move` call's `WorldState` are not drawn. Static obstacles in
configuration cover fixed workspace objects. For objects that move, pass
geometry at runtime through the `WorldState` parameter of the `Move`
request. See
[Static vs dynamic obstacles](/motion-planning/obstacles/overview/#static-vs-dynamic-obstacles)
for the distinction and
[Plan collision-free paths](/motion-planning/obstacles/avoid-obstacles/)
for code examples.

Dynamic obstacles passed to a single `Move` call are not drawn in the **3D
SCENE** tab. To verify a dynamic obstacle's position visually, log its pose
To verify a dynamic obstacle's position visually, log its pose
from your code or temporarily add a static geometry with the same dimensions
to a component frame.
13 changes: 6 additions & 7 deletions docs/motion-planning/3d-scene/verify-point-cloud-alignment.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ aliases:

Depth cameras produce point clouds: sets of 3D points that represent the surfaces the camera sees. The **3D SCENE** tab renders those points in your frame system, so you can check two things at once: the camera is producing usable data, and the data lines up with the rest of the workspace.

Misalignment usually means one of two things: the camera's frame offset is wrong, or the camera itself has a problem. Finding that out now, before a motion plan runs or an ML detector ships, costs minutes; finding it out later costs a day of debugging a downstream pipeline.
Misalignment usually means one of two things: the camera's frame offset is wrong, or the camera itself has a problem. Catch it before a motion plan runs or you train an ML detector on the data.

## Prerequisites

Expand Down Expand Up @@ -62,15 +62,14 @@ Adjust the camera's configuration, mounting angle, or lighting conditions to add

## What's next

The motion planner does not consume raw point clouds directly: a vision
service with a 3D segmenter (such as the
The motion planner accepts bounded geometries, not raw point clouds: a
vision service with a 3D segmenter (such as the
[`obstacles_pointcloud` module](https://app.viam.com/module/viam/obstacles-pointcloud))
turns point clouds into bounded 3D objects, and you feed those to the
planner.
turns point clouds into bounded 3D objects that you pass to the planner.

- [Pick an object](/motion-planning/move-an-arm/pick-an-object/):
uses `GetObjectPointClouds` to localize the target the arm should
grasp on a single `Move` call.
uses `GetObjectPointClouds` to localize the grasp target, then moves
to it with a single `Move` call.
- [Define obstacles](/motion-planning/obstacles/): the geometry types
the motion planner accepts, including the
[`WorldState.obstacles`](/motion-planning/obstacles/avoid-obstacles/)
Expand Down
19 changes: 12 additions & 7 deletions docs/motion-planning/move-an-arm/constraints.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
linkTitle: "Configure constraints"
title: "Configure motion constraints"
weight: 5
weight: 25
layout: "docs"
type: "docs"
description: "Restrict how the arm moves between poses using linear, orientation, and collision constraints."
Expand Down Expand Up @@ -52,9 +52,9 @@ Forces the end effector to maintain a consistent orientation throughout the
motion. Use this when the end effector must stay level or keep a fixed
orientation (for example, carrying a liquid).

| Parameter | Type | Description |
| ---------------------------- | ---------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `orientation_tolerance_degs` | float (required) | Maximum orientation deviation, in degrees, for orientations that fall outside the start-to-goal box. A value of 0 rejects any deviation outside that box. |
| Parameter | Type | Description |
| ---------------------------- | --------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `orientation_tolerance_degs` | float (optional, default 0) | Maximum orientation deviation, in degrees, for orientations that fall outside the start-to-goal box. A value of 0 rejects any deviation outside that box. |

The planner checks each orientation vector component (`OX`, `OY`, `OZ`,
`Theta`) against the start and goal independently. If every component of
Expand Down Expand Up @@ -83,8 +83,9 @@ short move gets a tight tolerance; a long move gets a proportionally larger one.
### CollisionSpecification

Allows specific pairs of frames to collide during planning. By default, the
planner rejects any path where any two frames collide. CollisionSpecification
lets you whitelist specific pairs.
planner rejects any path where any two non-adjacent frames collide.
CollisionSpecification lets you list specific pairs the planner allows to
collide.

| Parameter | Type | Description |
| --------- | ------------------- | ---------------------------------------------------- |
Expand Down Expand Up @@ -130,7 +131,11 @@ and the failure rate.

- **Tight tolerances** (small `line_tolerance_mm` or `orientation_tolerance_degs`)
increase planning time and may cause the planner to fail if no path exists
within the tolerance.
within the tolerance. A `LinearConstraint` also switches planning to
Cartesian-step subdivision, and tolerances below 10 mm or 10 degrees disable
the cBiRRT fallback entirely: each step then needs a direct straight-line IK
solution, or planning fails. See
[How motion planning works](/motion-planning/how-planning-works/).
- **Start with larger tolerances** and tighten only as needed. A 10 mm linear
tolerance is easier to satisfy than a 1 mm tolerance.
- **Combining constraints** multiplies the difficulty. Use the minimum set of
Expand Down
14 changes: 8 additions & 6 deletions docs/motion-planning/move-an-arm/move-by-joint-positions.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ commanded configuration makes the arm swing through the table or your
workspace fixture, the arm will swing through the table. Joint-space is for
configurations you have already verified safe.

## Before you start
## Prerequisites

- A configured arm component and an SDK client.
- You know the joint angles you want. For a 6-DOF arm, this is six
Expand Down Expand Up @@ -178,10 +178,11 @@ from Python; the arm uses its module's default speed profile.
| `max_acc_degs_per_sec2` | `double` (optional) | Uniform acceleration cap across every joint, in degrees per second squared. |
| `max_vel_degs_per_sec_joints` | `[]double` (repeated) | Per-joint velocity caps. Length must match the arm's degrees of freedom. Overrides the uniform cap when set. |
| `max_acc_degs_per_sec2_joints` | `[]double` (repeated) | Per-joint acceleration caps. Length must match the arm's degrees of freedom. Overrides the uniform cap when set. |
| `max_tcp_speed` | `double` (optional) | Maximum speed of the tool center point in meters per second. The arm moves as fast as possible up to this limit. |
| `max_tcp_speed` | `double` (optional) | Caps the tool center point's speed, in meters per second. Unset means no cap. |

All fields are optional ceilings. Any combination may be set. Every
constraint that is set is respected at every point along the trajectory.
All fields are optional ceilings. Any combination may be set. Each cap
you set applies along the whole trajectory; the arm module is responsible
for enforcing it.
Per-joint fields take precedence over global fields. Pass `nil`
options to use the module's default motion profile.

Expand Down Expand Up @@ -218,8 +219,9 @@ logger.Infof("joint positions (radians): %v", current)
{{% /tab %}}
{{< /tabs >}}

Pair `GetJointPositions` with `MoveToJointPositions` to capture a pose
by hand (teach-by-demonstration) and replay it programmatically.
Pair `GetJointPositions` with `MoveToJointPositions` to capture a
configuration by hand (teach-by-demonstration) and replay it
programmatically.

## Joint-space moves compared to motion.Move

Expand Down
Loading
Loading