chore(g1/23dof): host motion npz on HuggingFace, keep git history clean #pr 691#698
Closed
LeeLeno wants to merge 13 commits into
Closed
chore(g1/23dof): host motion npz on HuggingFace, keep git history clean #pr 691#698LeeLeno wants to merge 13 commits into
LeeLeno wants to merge 13 commits into
Conversation
…ort-prone Jacobian test Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <[email protected]>
… backend Two-part fix for the G1 23-DoF box_tracking ValueError 'operands could not be broadcast together with shapes (N,16) (23,)': 1. BoxMotionLoader fallback: remove erroneous '-7' from n_robot_joints calc when joint_names is absent. 23-DoF motion data joint_pos is pure robot joints (23) without floating base, so subtracting 7 truncated to 16. 2. MotrixBackend get_dof_pos/get_dof_vel: use actuator-level joint indices (_actuator_joint_pos/vel_indices) instead of body.get_joint_dof_pos/vel() which returned wrong dim for 23-DoF model. Also includes pre-existing uncommitted updates: - Register motrix backend for G1Walk23DofFlat/Rough and G1WBTObs23Dof - Add pose_weights to g1_23dof_walk_flat motrix config - Update support matrix (23-DoF tasks marked Tested/Registered for motrix) - Add g1_23dof_walk_rough mujoco config - Ignore .sisyphus/ directory
- Remove G1_23DOF_INIT.md from tracking (internal doc, now gitignored) - Ignore internal docs: G1_23DOF_INIT.md, G1_23DOF_PR_CHECKLIST.md, APPO_HARDWARE_TIMING_ANALYSIS.md - Add conf/offpolicy/task/flashsac/g1_23dof_walk_flat/motrix.yaml (FlashSAC 23-DoF motrix config, trained and verified)
- Run generate_support_matrix.py to update 23-DoF support status - Ignore log/ directory (training checkpoints/ONNX exports)
The 18 g1 23-DoF AMP motion clips under motions/g1/23dof_amp/ are now hosted on the unilabsim/unilab-motions HF dataset and pulled on demand, mirroring how g1/x2 motions and x2 meshes are already handled. Their blobs are purged from this branch's history; .gitignore is extended to cover nested motion subdirectories so they cannot be re-committed, and .gitkeep placeholders preserve the directory layout for HF downloads. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Wrap long assert message to satisfy repo ruff line-length config. Cherry-picked script from fork was authored against a looser style.
Collaborator
|
澄清和 #691 的关系，原pr的处理方式 |
Collaborator
Author
|
#691 涉及分支无权限，故fork作者仓库后另提pr。 |
|
@LeeLeno @TATP-233 这个 PR 在做 filter-branch 时，commit 的 author 信息变成了 LeeLeno，同时我本机 git 配置误绑定为 "hero"，没绑定 GitHub 账号。这样合并后 GitHub 无法把我的 contribution 归到我名下。 麻烦 @LeeLeno 在分支上跑一下 filter-branch 把 commit authors 修正成我的 GitHub 账号（Kennyp-Chen [email protected]），然后 force push。合进来后 contribution 记录就正常了，谢谢！ 如果不方便操作的话，告诉我，我来处理也可以。 |
Collaborator
Author
|
@Kennyp-Chen 直接拉取我fork仓库中feat/g1-23dof-hf-motions，修正信息后force push到你的pr |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & Why
G1 23-DoF motion tracking 的参考轨迹（npz）此前直接提交进 git，导致仓库历史膨胀。本 PR 将全部 motion npz 迁移到 HuggingFace 数据集
unilabsim/unilab-motions，运行时按需下载，并保证 git 历史中零 npz blob。Changes
motions/g1/；env 通过resolve_motion_files()（src/unilab/assets/hub.py）在文件缺失时自动从 HF 下载。git rev-list --objects HEAD | grep '\.npz'→ 0）。.gitignore收严：新增src/unilab/assets/motions/g1/**/*.npz递归规则，防止子目录下的 npz 再次被误提交。scripts/convert_motion_29dof_to_23dof.py（29→23 DoF：移除 6 个 RM 关节 + body 重映射），记录 23-DoF 轨迹的生成来源。Runtime download
env config 中的
motion_file指向ASSETS_ROOT_PATH / "motions" / "g1" / "<name>_23dof.npz"；MotionLoader→resolve_motion_files()逻辑：本地存在即用，否则从unilabsim/unilab-motions拉取对应相对路径（支持HF_ENDPOINT镜像回退到官方端点）。下载物被.gitignore覆盖，不污染工作树。HF asset state
motions/g1/新增 5 个*_23dof.npz（均校验为合法 23-DoF：joint_pos23 列、25 bodies；box 轨迹含 object 数据）。23dof_amp/locomotion 文件。Validation
make test-all的check阶段（ruff format + ruff check + mypy + pyright）全部通过：Success: no issues found in 222 source files/ pyright0 errors, 14 warnings。MotionLoader（内部resolve_motion_files→ HF 下载 →np.load）逐一加载 5 个 task 默认 motion，全部成功从 HF 拉回并解析（帧数 715/225/198/602/725，DoF=23，box 轨迹含 object 数据）；下载物被.gitignore覆盖，工作树保持干净。uv run train --algo ppo --task g1_23dof_flip_tracking --sim motrix完成参数解析 → Hydra 配置组装 → 后端选择，停在后端初始化门槛（本地未装 motrix/mujoco extra，环境依赖问题，非代码回归）。git rev-list --objects HEAD | grep '\.npz'计数为 0（含 nan_dump 在内任何.npzblob 均为 0）；相对main的 diff 中 npz 文件数为 0。mujoco出现 collection error（3 个 test 模块 import 阶段失败），与本 PR 改动无关（改动仅涉及scripts/脚本、conf/、env 代码与.gitignore，未触碰相关测试模块）。