climb-cv | climbing with computer vision

read the journal for ongoing updates and development!

concept

the main concept of this project is to create a package of computer vision tools for climbing.

• this project aims to explore the potential of computer vision in climbing and provide a platform for climbers to leverage visual data to enhance the climbing technology.
• by analyzing visual data, we can gain insights into climbing techniques, developing consistent routesetting, and even predict climbing routes based on visual cues.

implementations / tech

stack is compartmentalized into 3 distinct modules:

1. Video streaming and processing

OpenCV for image processing and analysis

2. Pose estimation and analysis

MediaPipe for pose estimation Data points are extracted and filtered to remove noise and improve accuracy Results are rendered in 3d space using matplotlib

3. Climb hold prediction and visualization

YOLO 26 model for object detection; fork to existing dataset here Ultralytics

requirements

• python 3.10+
• a webcam
• optional (macOS only): macbook M2 or newer + swift compiler, for the lid angle sensor

climb-cv runs on macOS, Windows, and Linux. The core pipeline (video, pose estimation, hold detection) is cross-platform. The only mac-only feature is the lid angle sensor, which reads the laptop hinge angle via a Swift helper; it is automatically disabled on Windows and Linux.

installation

install the package (and its dependencies) from the repo root or directly from GitHub:

pip install -e .
# or
pip install git+https://github.com/copypastin/climb-cv

this exposes the climbcv package so you can from climbcv.climbcv import climbcv from anywhere. the bundled pose and hold-detection models are included with the package, so they work after a normal pip install.

landmark exports are saved to ./data by default when you stop a run. pass output_dir=... if you want them written somewhere else.

run the samples from the repo root if you want the local assets and demo scripts, e.g. python src/sample_1.py.

windows

climb-cv works on Windows out of the box. A few platform notes:

• the correct camera backend (DirectShow / Media Foundation) is selected automatically. no configuration needed.

• MediaPipe's pip build does not ship a GPU delegate on Windows, so pose estimation automatically falls back to the CPU delegate. everything still runs; it just uses the CPU.

• the mac lid angle sensor is disabled automatically. no Swift compiler is required.

• Windows uses the spawn process-start method, so any script that creates a climbcv instance must guard its entry point:

  if __name__ == "__main__":
      main()

  the bundled src/sample_1.py and src/sample_2.py already do this.

from PowerShell:

python -m venv .venv
.venv\Scripts\Activate.ps1
pip install -e .
python src\sample_1.py

ways to contribute / future goals

• implementations of physics calculations (like center of gravity, velocity) using scipy
• visualization of climbing movements in a relative space rather than absolute space
• improve the accuracy of pose estimation and analysis
• expand the dataset of climb holds for better object detection

additional tech & references

• Crux Lab Reconstruct
• LidAngleSensor