Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 21 additions & 14 deletions docs/motion-planning/frame-system/arm-fixed-camera.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ world

Unlike a wrist-mounted camera, this camera is a child of the world frame, not the arm.
The camera frame stays fixed in space when the arm moves.
`table-surface` is workspace obstacle geometry, configured separately; see [Define obstacles](/motion-planning/obstacles/).

## Steps

Expand Down Expand Up @@ -96,17 +97,17 @@ For a camera mounted 200 mm to the right, 300 mm forward, and 800 mm above the w
"translation": { "x": 200, "y": 300, "z": 800 },
"orientation": {
"type": "ov_degrees",
"value": { "x": 1, "y": 0, "z": 0, "th": 180 }
"value": { "x": 0, "y": 0, "z": -1, "th": 0 }
}
}
```

An overhead camera's lens points at the floor. In the default camera
frame, +z points out of the lens, so an unrotated overhead camera has
its +z pointing up rather than down at the workspace. The 180-degree
rotation around x flips the camera's +z to point downward, matching the
lens's actual aim, so 2D image coordinates map intuitively to world
positions.
In an orientation vector, `(x, y, z)` is the direction the camera's +z
axis points in the parent frame, and the camera's +z axis is the lens.
An overhead camera's lens points at the floor, so set the pointing
vector to `(0, 0, -1)`: straight down at the workspace. If the image
needs to rotate to line up with your world axes, use `th` to spin the
camera about the lens axis.

**Tripod-mounted camera at an angle:**
For a camera on a tripod 500 mm to the left, 600 mm forward, and 700 mm above the origin, tilted 45 degrees downward:
Expand All @@ -117,12 +118,16 @@ For a camera on a tripod 500 mm to the left, 600 mm forward, and 700 mm above th
"translation": { "x": -500, "y": 600, "z": 700 },
"orientation": {
"type": "ov_degrees",
"value": { "x": 1, "y": 0, "z": 0, "th": -45 }
"value": { "x": 0.7, "y": 0, "z": -0.7, "th": 0 }
}
}
```

A negative angle around the x axis tilts the camera's view downward from the horizontal.
Set `(x, y, z)` to the direction the lens points; Viam normalizes the
vector for you. `(0.7, 0, -0.7)` aims the lens 45 degrees below
horizontal, facing the +x direction. To aim at a different part of the
bench, change the horizontal components: for example,
`(0.5, -0.5, -0.7)` faces between +x and -y.

Click **Save**.

Expand All @@ -131,8 +136,10 @@ Click **Save**.
For the arm, jog in **CONTROL** along +x, +y, +z and watch the physical
direction. For the camera, open the camera stream in **CONTROL** and move a
known object (a pen, a ruler) in the physical +x direction; the object should
move in the +x direction in the camera's image coordinates. If either is
wrong, rotate the relevant component's `orientation` until it matches.
move along one consistent axis in the camera's image. If it moves along the
wrong image axis or in the wrong direction, adjust `th` (which spins the
image about the lens axis) or the pointing vector until image motion matches
physical motion.

### 6. Visualize the frame system

Expand Down Expand Up @@ -160,9 +167,9 @@ Change the parent to `"world"` so the camera frame remains fixed in space.

{{< expand "Overhead camera orientation looks wrong in the visualizer" >}}

For a camera pointing straight down, the rotation should be 180 degrees around the x axis: `"value": { "x": 1, "y": 0, "z": 0, "th": 180 }`.
This flips the camera's z axis to point downward.
If the camera is rotated in the horizontal plane as well (not aligned with the x or y axis), you may need to combine rotations or use a different orientation type.
For a camera pointing straight down, set the pointing vector to `(0, 0, -1)`: `"value": { "x": 0, "y": 0, "z": -1, "th": 0 }`.
In an orientation vector, `(x, y, z)` is the direction the lens points, so point it wherever the lens actually aims.
If the image appears rotated in the visualizer, adjust `th`, which spins the camera about the lens axis.

{{< /expand >}}

Expand Down
46 changes: 25 additions & 21 deletions docs/motion-planning/frame-system/arm-gripper-camera.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,11 @@ aliases:
---

A table-mounted arm, a gripper bolted to the end effector, and a camera
clamped near the wrist is the most common manipulation setup. The camera moves
clamped near the wrist is a common manipulation setup. The camera moves
with the arm, so its view is always centered on wherever the arm is reaching.
The motion service and vision pipelines do not know that on their own; you
have to declare three frame relationships: arm-to-world, gripper-to-arm, and
camera-to-arm. This guide walks through all three.
You declare three frame relationships (arm-to-world, gripper-to-arm, and
camera-to-arm) so the frame system can compute where the gripper and camera
sit as the arm moves. This guide walks through all three.

## Frame hierarchy

Expand All @@ -28,6 +28,7 @@ world

The arm is a direct child of the world frame.
The gripper and camera are both children of the arm, so they move with the arm automatically as it changes position.
`table-surface` is workspace obstacle geometry, configured separately; see [Define obstacles](/motion-planning/obstacles/).

## Steps

Expand All @@ -39,7 +40,7 @@ base means the arm's translation is `(0, 0, 0)` and every measurement
starts from the arm. Using a table corner gives you a visible landmark
but requires measuring arm-base-to-corner first.

Every frame on the machine is defined relative to this point, so you
Every frame on the machine ultimately resolves to this point, so you
will refer to it every time you add or adjust a frame.

### 2. Add a frame to the arm
Expand Down Expand Up @@ -100,10 +101,10 @@ In the sidebar, click your gripper component to open its card. On the card, clic

#### Pick where the gripper frame origin sits

A point near the center of the gripper jaws is usually the most
convenient frame origin. The point you pick here is what `translation`
measures _to_, and what gets moved to a target pose when you later call
the motion service.
The tool center point, between the tips of the gripper jaws, is usually
the most convenient frame origin. The point you pick here is what
`translation` measures _to_, and what gets moved to a target pose when
you later call the motion service.

#### Configure the frame

Expand All @@ -112,8 +113,8 @@ the offset in mm from the arm's end effector to the gripper frame
origin, and `orientation` to the gripper's rotation relative to the arm.

For a parallel-jaw gripper that bolts directly to the arm's end effector
with the frame origin at the jaw tip, the offset is the gripper's body
length. 120 mm is typical:
with the frame origin at the tool center point between the jaw tips, the
offset is the gripper's body length. 120 mm is typical:

```json
{
Expand All @@ -132,17 +133,17 @@ gripper above, use `z: 170`.

A wrong `translation.z` produces silent failures: motion plans validate,
then the physical gripper tip lands short or long of the target, or the
planner returns "outside workspace" for poses the arm can clearly reach.
planner fails to find a plan for poses the arm can clearly reach.

Click **Save**.

#### Check whether the gripper has a kinematics file

Some grippers ship with a kinematics file that describes the position of
the jaws as they open and close, along with collision geometry for the
jaw linkages. If the gripper has one, the motion planner already knows
the gripper's volume and you do not need to add collision geometry to
the gripper frame.
Some gripper modules include a kinematics file that describes the position
of the jaws as they open and close, along with collision geometry for the
jaw linkages. If the gripper has one, the motion planner already has the
gripper's collision geometry, so leave the gripper frame's `geometry`
field empty.

Call `GetKinematics` on the gripper (or check the module source). If
the call returns kinematics data, verify the gripper renders as expected
Expand Down Expand Up @@ -173,19 +174,22 @@ For a camera mounted 30 mm to the side and 60 mm above the end effector:
}
```

If the camera is tilted downward (for example, angled 30 degrees toward the gripper), add a rotation around the x axis:
If the camera is tilted (for example, angled 30 degrees toward the gripper jaws), set `(x, y, z)` to the direction the lens points in the arm's end effector frame:

```json
{
"parent": "my-arm",
"translation": { "x": 0, "y": 30, "z": 60 },
"orientation": {
"type": "ov_degrees",
"value": { "x": 1, "y": 0, "z": 0, "th": -30 }
"value": { "x": 0, "y": -0.5, "z": 0.87, "th": 0 }
}
}
```

`(0, -0.5, 0.87)` tips the lens 30 degrees away from the tool axis, back
toward the gripper jaws. Viam normalizes the vector for you.

Click **Save**.

### 6. Visualize the frame system
Expand Down Expand Up @@ -217,8 +221,8 @@ If the parent is `"world"`, the frame stays fixed in space when the arm moves.
{{< expand "Camera image appears rotated or flipped" >}}

The camera's orientation in the frame system must match its physical mounting orientation.
If the camera is mounted upside down, add a 180-degree rotation around the z axis.
If the image is mirrored, check whether you need to rotate around the x or y axis.
If the image appears rotated (for example, the camera is mounted upside down), adjust `th`, which spins the camera about the lens axis: `th: 180` corrects an upside-down mounting.
If image motion runs along the wrong axis, re-check the pointing vector `(x, y, z)` against the direction the lens actually aims.

{{< /expand >}}

Expand Down
59 changes: 31 additions & 28 deletions docs/motion-planning/frame-system/mobile-base-arm.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,7 @@ moves within reach of the base. The frame system represents this by
parenting the arm to the base rather than to the world. Every
arm-attached component (the gripper, a wrist camera) then inherits the
base's motion, while navigation sensors mounted directly on the base
form a parallel subtree. This guide builds the full hierarchy: world to
base to (arm to (gripper, wrist camera), navigation sensors).
form a parallel subtree. This guide builds that full hierarchy.

## Frame hierarchy

Expand All @@ -30,21 +29,21 @@ world
└── my-lidar
```

The hierarchy reflects what physically moves with what. The base's parent is
the world, so when the base drives, every frame below it shifts with it. The
arm's parent is the base, so arm motion moves the gripper and wrist camera
but not the navigation sensors. This separation keeps the lidar's reading
about the floor consistent while the arm reaches for something.
The hierarchy reflects what physically moves with what. The arm's parent is
the base, so arm motion moves the gripper and wrist camera but not the
navigation sensors. This separation keeps lidar readings fixed to the base
while the arm reaches for something.

## Steps

### 1. Choose your world frame

For a mobile base, the world frame origin is the base center at machine
start. Unlike a table-mounted arm where you pick a fixed physical
landmark, the frame system tracks the base's motion from this initial
origin, so you do not mark anything physically. All component positions
are defined relative to this point.
For a mobile base, configure the base with zero translation so the
world frame origin sits at the base center. The frame system holds this
configured relationship; tracking the base's actual position as it
drives is the job of SLAM or a movement sensor, not the frame system.
All component offsets on the machine are
measured from this point.

### 2. Add a frame to the base

Expand Down Expand Up @@ -119,17 +118,19 @@ If there is an adapter plate between the arm and gripper, set the z translation
```

Adjust the translation values to match the camera's actual mounting position relative to the arm's end effector.
If the camera is tilted, add a rotation. For example, tilted 30 degrees downward:
If the camera is tilted, set `(x, y, z)` to the direction the lens points in the arm's end effector frame. For example, tilted 30 degrees from the tool axis back toward the gripper jaws:

```json
{
"orientation": {
"type": "ov_degrees",
"value": { "x": 1, "y": 0, "z": 0, "th": -30 }
"value": { "x": 0, "y": -0.5, "z": 0.87, "th": 0 }
}
}
```

In an orientation vector, `(x, y, z)` is the direction the camera's +z axis (the lens) points; Viam normalizes the vector for you.

Click **Save** after adding each frame.

### 5. Add frames to navigation sensors
Expand All @@ -144,12 +145,16 @@ Navigation sensors are children of the base, not the arm.
"translation": { "x": 0, "y": 200, "z": 120 },
"orientation": {
"type": "ov_degrees",
"value": { "x": 0, "y": 0, "z": 1, "th": 0 }
"value": { "x": 0, "y": 1, "z": 0, "th": 0 }
}
}
```

**LIDAR mounted on top of the base.** In the sidebar, click your LIDAR component to open its card, then click **Frame**:
Point `(x, y, z)` where the lens aims. A base's forward direction is +y,
so `(0, 1, 0)` faces the camera forward; the identity orientation
`(0, 0, 1)` would aim the lens at the ceiling.

**Lidar mounted on top of the base.** In the sidebar, click your lidar component to open its card, then click **Frame**:

```json
{
Expand All @@ -170,9 +175,8 @@ Click **Save** after adding each frame.

1. Open the **3D SCENE** tab.
2. Confirm the tree structure: arm, gripper, and wrist camera under the base; navigation sensors as direct children of the base.
3. **Jog the arm from the CONTROL tab — the single clearest test that your hierarchy is correct.** Only the arm subtree should move; the base and navigation sensors should stay put.
4. **Drive the base a short distance.** Every frame should shift together.
5. Measure a known physical offset (base center to lidar, for example) and compare to the translation values in your config.
3. **Jog the arm from the CONTROL tab.** This is the clearest test that your hierarchy is correct: only the arm subtree should move; the base and navigation sensors should stay put. (The visualizer shows frames parented to the base at the base's configured pose, even while the base drives.)
4. Measure a known physical offset (base center to lidar, for example) and compare to the translation values in your config.

### 7. Verify with TransformPose

Expand Down Expand Up @@ -201,21 +205,20 @@ If the gripper's parent is the base, its position will not account for the arm's

{{< expand "Navigation camera and wrist camera report conflicting object positions" >}}

These cameras have different parents (base and arm respectively), so their raw coordinates are in different reference frames.
The navigation camera's parent is the base and the wrist camera's parent is the arm, so their raw coordinates are in different reference frames.
Always use `TransformPose` to convert positions to a common frame (such as the world frame) before comparing them.
Also verify that each camera's translation and orientation offsets are accurate.

{{< /expand >}}

{{< expand "Arm subtree shifts unexpectedly when base rotates" >}}
{{< expand "Arm sits in the wrong place relative to the base in the 3D scene" >}}

This is expected. When the base rotates in place, anything mounted off-center
sweeps through an arc; the further off-center, the longer the arc. An arm
mounted forward of the base's rotation center will end up 100 mm to the side
after a 90-degree base turn if it is 100 mm forward of center. If the shift
does not match that geometry, the arm's `translation` offset from the base is
probably wrong; measure from the base's rotation center (usually the wheel
axis midpoint) rather than from a corner of the base chassis.
The arm's `translation` offset from the base is probably measured from the
wrong point. Measure from the base's rotation center (usually the wheel axis
midpoint) rather than from a corner of the base chassis. This matters on the
physical robot too: when the base turns in place, anything mounted off the
rotation center sweeps through an arc, so an offset measured from the wrong
point puts every downstream frame in the wrong place by that same error.

{{< /expand >}}

Expand Down
Loading
Loading