feat: support HRNet DarkPose pose estimation (w32/w48, COCO body & wholebody)#258
Conversation
Signed-off-by: wep21 <[email protected]>
Signed-off-by: wep21 <[email protected]>
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
- Add per-model hub_owner/hub_repo overrides to ORTConfig so a model can fetch weights from a GitHub release other than the default jamjamjon/assets, while preserving the existing dtype (fp16/etc.) candidate resolution. - Point HRNet configs at the wep21/assets 'hrnet' release. - Rename weight files to the usls/assets hyphen convention with a '-dark' qualifier so the DarkPose variant is identifiable (e.g. hrnet-w32-coco-256x192-dark.onnx). - Fix hrnet_w32_133 to 256x192 input to match the available wholebody weight (w48 wholebody stays 384x288). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
HRNet (DarkPose) inference resultsAll weights are fetched on-demand from the Source image: COCO body (17 keypoints)COCO-WholeBody (133 keypoints)
Verified on CPU (macOS). All six configs fetch, run, and annotate successfully. The |
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
|
@jamjamjon Hi, is it applicable to add this new pose heatmap model? Currently, onnx is fetched from my assets repo. |
…hub approach Adopts the HRNet DarkPose pose estimation model (w32/w48, COCO body & wholebody, 256x192/384x288) contributed by @wep21 in PR jamjamjon#258. Key adjustments to the original PR: - Rejected the per-model `hub_owner`/`hub_repo` config fields to keep `ORTConfig` lean and consistent with the existing architecture. - Instead, HRNet variants now use explicit GitHub release URLs pointing to `wep21/assets`. - Fixed a URL bug in `hrnet_w48_17_384()` where the w48 variant incorrectly referenced a w32 ONNX file. - Extended `ORTConfig::try_commit()` so that explicit GitHub release URLs also support automatic dtype variant lookup (e.g. `-fp16`, `_fp16`, `.fp16`). Previously this only worked for bare filenames; now both bare filenames and full URLs benefit from it. This keeps the configuration API surface minimal while fully supporting third-party model hosting and dtype-aware downloads.
Hi @wep21, thanks for the great PR! The HRNet model is definitely applicable and the code quality is solid. I merged it with a few adjustments:
The ONNX files stay in your |






Summary
Adds HRNet top-down heatmap-based pose estimation, supporting COCO body (17 keypoints) and COCO-WholeBody (133 keypoints) with the
w32/w48backbones at256x192and384x288input resolutions.What's included
src/models/vision/hrnet/impl.rs—HRNetmodel. Reuses the top-down affine preprocessing (person crop → fixed input size), and decodes the model's heatmap output[batch, nk, h, w]via per-keypoint argmax plus quarter-pixel sub-pixel refinement (shift0.25toward the larger neighbor), then maps heatmap coordinates back to the crop usingcenter/scale.src/models/vision/hrnet/config.rs—Config::hrnet*()builders coveringw32/w48× body/wholebody ×256x192/384x288.src/models/vision/hrnet/mod.rs, module registration and re-export insrc/models/vision/mod.rs.examples/pose-estimation/hrnet.rs+main.rs—hrnetsubcommand with--width w32|w48,--is-coco,--hires, and--model(local file override). Uses a YOLO person detector for the top-down crops.Verification
Built clean (
cargo build --libandcargo build --example pose-estimation) and ran end-to-end against a sampleend2end.hrnet_w48COCO-WholeBody ONNX (input[1,3,384,288]→ output[1,133,96,72]) onassets/bus.jpg; rendered skeletons align accurately on all detected people.🤖 Generated with Claude Code