Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions docs/build-modules/deploy-a-module.md
Original file line number Diff line number Diff line change
Expand Up @@ -357,6 +357,15 @@ This means the binary was compiled for the wrong architecture. For example, you

{{< /expand >}}

{{< expand "Module fails to start or keeps restarting" >}}

- Read the module's lifecycle state from [`GetMachineStatus`](/reference/apis/robot/#getmachinestatus). A module in `STATE_UNHEALTHY` reports the cause in its `error` field and the time it entered that state in `last_updated`.
- Check `consecutive_failures`. A count that keeps climbing indicates a restart loop, such as a module failing repeatedly on a syntax error.
- For the state list and field details, see [Module status states](/build-modules/module-reference/#module-status-states).
- Check the module logs in the **LOGS** tab for the full traceback.

{{< /expand >}}

{{< expand "Module works locally but fails after deployment" >}}

- Check for hard-coded paths, missing environment variables, or dependencies installed on your machine but not in the build environment.
Expand Down
43 changes: 43 additions & 0 deletions docs/build-modules/module-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,49 @@ If a module process crashes, `viam-server` automatically restarts it:
If the module keeps crashing, `viam-server` retries indefinitely. Check the
**LOGS** tab for crash tracebacks.

### Module status states

While the machine runs, `viam-server` tracks every configured module through a
fixed set of lifecycle states, from download through running. The
[`GetMachineStatus`](/reference/apis/robot/#getmachinestatus) method reports the
current state of each module in a `ModuleStatus` message, so you can tell whether
a module is still downloading, starting up, running, or failing.

`ModuleStatus.state` is one of the following values. Each maps to what the module
process is doing at that point:

| State | What the module is doing |
| ------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `STATE_UNSPECIFIED` | The state is unknown or unset. |
| `STATE_PENDING` | The module is configured, but `viam-server` has not yet started it, for example because a required package is still downloading. Next state: `STATE_STARTING`. |
| `STATE_STARTING` | The module process has spawned, and `viam-server` is waiting for it to send a `Ready` response and register its models. Next state: `STATE_READY`. |
| `STATE_READY` | The module has sent its `Ready` response, registered its models, and is running. If it exits with an error, it moves to `STATE_UNHEALTHY`. |
| `STATE_UNHEALTHY` | The module failed to start, exited unexpectedly, or is restarting after a crash. Any failure in another state moves to this state. |
| `STATE_CLOSING` | The module process is shutting down, because you removed the module or because it is restarting during a reconfigure. On a restart it moves to `STATE_STARTING`. |

This per-module state is separate from the machine-level state (`initializing` or
`running`) that `GetMachineStatus` also reports. For how the machine-level state
changes when a module crashes, see [Crash recovery](#crash-recovery).

Alongside `state`, each `ModuleStatus` reports:

- `module_name`: the configured name of the module.
- `last_updated`: the time of the module's most recent state transition. Use it to
see how long the module has been in its current state.
- `error`: the error that caused the module to enter `STATE_UNHEALTHY`. The error
persists until the module returns to `STATE_READY`, where it is empty.
- `consecutive_failures`: the number of times the module has entered
`STATE_UNHEALTHY` since it was last ready. This count resets to zero when the
module reaches `STATE_READY` or you reconfigure it. A count that keeps climbing
indicates a module stuck in a restart loop, such as a Python module with a syntax
error.

To diagnose a module that failed to start, read its `ModuleStatus` from
`GetMachineStatus`. A module in `STATE_UNHEALTHY` reports the cause in `error` and
the time it entered that state in `last_updated`. A rising `consecutive_failures`
count indicates a restart loop, where the module fails and retries repeatedly. For
the full crash traceback, check the **LOGS** tab.

### Communication

By default, modules communicate with `viam-server` over a Unix domain socket
Expand Down
Loading