Skip to content

PyVizAST Prerelease 0.7.0

Pre-release
Pre-release

Choose a tag to compare

@ChidcGithub ChidcGithub released this 11 Mar 14:24
· 42 commits to main since this release

New Feature: Webcam Gesture Control

Hand gesture recognition for 2D/3D AST visualization control using MediaPipe.

Supported Gestures:

  • Thumb Up: Zoom in
  • Thumb Down: Zoom out
  • Closed Fist: Pan mode (grab and drag)
  • Open Palm: Quick tap = select, Hold = reset view
  • Pointing Up: Select node
  • Victory (V sign): Rotate mode
  • Two Hands Pinch: Pinch to zoom, move both hands to pan

Components:

  • GestureService.js: Core gesture recognition service using MediaPipe GestureRecognizer
  • GestureControl.js: UI component with camera preview, status indicator, gesture guide
  • GestureControl.css: Styling for gesture control overlay

Features:

  • Toggle button in header (default OFF)
  • Real-time camera preview with gesture overlay
  • Status indicator (loading/ready/running/error/stopped)
  • Visual gesture guide with action mappings
  • Support for both 2D (Cytoscape) and 3D (Three.js) visualizations

Dependencies:

  • Added @mediapipe/tasks-vision for gesture recognition

Known Issues:

  • Gesture recognition may be unstable in certain lighting conditions
  • Future improvements planned: stability filtering, confidence threshold adjustment, gesture cooldown

Files Added:

  • frontend/src/utils/GestureService.js
  • frontend/src/components/GestureControl.js
  • frontend/src/components/GestureControl.css

Files Modified:

  • frontend/src/components/Header.js - Added gesture toggle button
  • frontend/src/App.js - Added gesture state management
  • frontend/src/components/ASTVisualizer.js - Added gesture handling for 2D
  • frontend/src/components/ASTVisualizer3D.js - Added gesture handling for 3D
  • frontend/package.json - Added MediaPipe dependency