Author: Alejandro Meza Tudela
An end-to-end implementation of a massively parallelized, manager-based reinforcement learning environment using NVIDIA Isaac Lab and Proximal Policy Optimization (PPO) via the RSL-RL library. This project traces the incremental development of a differential drive TurtleBot3 robot—starting from basic high-speed straight-line locomotion to 2D LiDAR-based obstacle navigation in procedural environments.
The initial baseline environment trains the TurtleBot3 on a flat, obstacle-free ground plane to achieve stable forward velocity control while minimizing excessive yaw spinning.
Figure 1: Parallel vectorized instances of TurtleBot3 learning basic forward velocity control on a flat plane.
The environment scales up to procedurally generated grid terrains featuring static box obstacles per environment cell. The TurtleBot3 is augmented with a 360° single-channel 2D LiDAR raycaster to perceive spatial obstacles directly within its observation space.
Figure 2: Parallel vectorized instances navigating procedurally generated box obstacle terrains using 2D LiDAR raycasting.
The obstacle terrain is replaced with a procedurally generated maze, unique per environment. The TurtleBot3 gains a forward-facing RGB camera whose frames a lightweight V-JEPA-style encoder compresses into a latent vector, fused with velocity and LiDAR in a custom PPO actor-critic, with a LiDAR-based collision penalty added so the agent learns to traverse the maze without colliding.
The TurtleBot3 (Burger) by ROBOTIS is a widely adopted, open-source, differential-drive mobile robot chassis.
- Kinematics: Differential drive system driven by independent wheel joint actuators.
-
Actuation Type: Velocity-controlled joints (
JointVelocityActionCfg) mapping continuous actions to target wheel angular velocities ($[v_{\text{left}}, v_{\text{right}}]$ ).
-
Action Space: Continuous 2D vector for differential wheel velocity commands (scale factor
$= 5.0$ ). -
Observation Space:
-
Baseline: Base linear velocity (
$v$ ), angular velocity ($\omega$ ), and joint velocities ($\dot{q}$ ). -
LiDAR Variant: Adds a 360° horizontal raycast distance vector (36 beams at
$10^\circ$ resolution, max distance$2.0\text{m}$ ).
-
Baseline: Base linear velocity (
-
Terrain & Raycasting:
$8 \times 8$ procedurally generated sub-terrain mesh of repeated static boxes ($0.4\text{m} \times 0.4\text{m} \times 0.8\text{m}$ ) with a$1.0\text{m}$ clear spawn platform at the center of each cell.
The environment is built using Isaac Lab's modular ManagerBasedRLEnv structure:
isaaclab_tasks/manager_based/turtlebot3/
├── __init__.py # Environment registration and Gym hook
├── turtlebot3_asset_cfg.py # Simulation scene details & physical actuator configurations
└── turtlebot3_env_cfg.py # Complete MDP formulation (Observations, Actions, Rewards, LiDAR & Terrain)
- GPU: NVIDIA GeForce RTX 4070 Ti Super