feat(customizer): add docs and skills for customizer rl backend#495
Merged
Conversation
Contributor
|
🌿 Preview your docs: https://nvidia-preview-299-dpo-docs-anubhutiv.docs.buildwithfern.com/nemo-platform |
anubhutivyas
force-pushed
the
aalgo-292-dpo-support/anubhutiv
branch
4 times, most recently
from
July 1, 2026 00:02
80cd9ab to
d9dc55f
Compare
anubhutivyas
force-pushed
the
299-dpo-docs/anubhutiv
branch
from
July 1, 2026 00:03
3f60480 to
4dffd49
Compare
gabwow
requested changes
Jul 1, 2026
gabwow
left a comment
gabwow
left a comment
Contributor
There was a problem hiding this comment.
I think this can be improved with a little clean up
gabwow
approved these changes
Jul 2, 2026
Contributor
|
LGTM |
anubhutivyas
changed the base branch from
aalgo-292-dpo-support/anubhutiv
to
main
July 2, 2026 17:03
anubhutivyas
force-pushed
the
299-dpo-docs/anubhutiv
branch
from
July 2, 2026 17:39
d3282da to
d220ea1
Compare
anubhutivyas
enabled auto-merge
July 2, 2026 17:39
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (17)
✅ Files skipped from review due to trivial changes (10)
🚧 Files skipped from review as they are similar to previous changes (4)
📝 WalkthroughWalkthroughAdds a DPO tutorial notebook and expands NeMo Customizer docs and tests to cover the ChangesDPO/RL Documentation and Tutorial
Possibly related PRs
Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 5
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/customizer/tutorials/dpo-customization-job.ipynb`:
- Around line 302-304: The persisted resource names are using too short a random
suffix, which can cause collisions across reruns in shared workspaces. Update
the suffix generation in the notebook setup cells that define job_suffix,
OUTPUT_NAME, DEPLOYMENT_CONFIG_NAME, and DEPLOYMENT_NAME to use a longer
UUID-derived suffix with more entropy. Keep the naming pattern the same, but
increase the suffix length consistently wherever these resource names are
constructed.
In
`@plugins/nemo-customizer/src/nemo_customizer/skills/nemo-customizer/references/batch-sizing.md`:
- Around line 145-150: The markdown table in the batch-sizing reference has a
broken row in the troubleshooting section because the Loss not moving entry is
being split into too many cells. Update that row in the table so it has only the
expected columns, using the existing troubleshooting entry near the Unsloth
guidance, and move the example/extra detail out of the table if needed to keep
the table rendering correctly.
In
`@plugins/nemo-customizer/src/nemo_customizer/skills/nemo-customizer/references/dataset-formats.md`:
- Around line 3-5: The dataset-format intro mixes the SFT and DPO filename
contracts, so separate the backend guidance into distinct sentences in the
dataset-formats reference. Update the text around the shared upload instructions
to clearly distinguish automodel/unsloth usage of train.jsonl and
validation.jsonl from the rl/DPO requirement for training.jsonl and
validation.jsonl, using the existing backend names and dataset field references
to keep each contract unambiguous.
In
`@plugins/nemo-customizer/src/nemo_customizer/skills/nemo-customizer/references/hf-conversion.md`:
- Line 58: The portability claim is too broad: in the hf-conversion doc, narrow
the statement around the `to_sft` flow so it only says the
chat-template/messages path is backend-agnostic. Update the wording near the SFT
backend note to reflect that the converted JSONL may be reused for both
Automodel and Unsloth only when produced through the chat-template/messages
path, and avoid implying `to_sft` itself is Unsloth-compatible.
In
`@plugins/nemo-customizer/src/nemo_customizer/skills/nemo-customizer/references/hyperparameters-rl.md`:
- Line 45: The `hyperparameters-rl.md` table entry for `max_steps` currently
reads like a hard rule in the `max_steps` description, but it should be a
smoke-job caveat instead. Update the wording in the `max_steps` row under the
hyperparameters reference so `epochs` is not presented as invalid when used with
`max_steps`, and make sure the `max_steps`/`epochs` guidance in this section
matches the smoke fixture behavior without implying JSON validation rejection.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 5064df78-fda1-40b3-829f-aea048090b8d
📒 Files selected for processing (17)
docs/customizer/tutorials/dpo-customization-job.ipynbplugins/nemo-customizer/src/nemo_customizer/skills/nemo-customizer/SKILL.mdplugins/nemo-customizer/src/nemo_customizer/skills/nemo-customizer/references/batch-sizing.mdplugins/nemo-customizer/src/nemo_customizer/skills/nemo-customizer/references/dataset-formats.mdplugins/nemo-customizer/src/nemo_customizer/skills/nemo-customizer/references/hf-conversion.mdplugins/nemo-customizer/src/nemo_customizer/skills/nemo-customizer/references/hyperparameters-automodel.mdplugins/nemo-customizer/src/nemo_customizer/skills/nemo-customizer/references/hyperparameters-rl.mdplugins/nemo-customizer/src/nemo_customizer/skills/nemo-customizer/references/hyperparameters-unsloth.mdplugins/nemo-customizer/src/nemo_customizer/skills/nemo-customizer/references/hyperparameters.mdplugins/nemo-customizer/src/nemo_customizer/skills/nemo-customizer/references/integrations-setup.mdplugins/nemo-customizer/src/nemo_customizer/skills/nemo-customizer/references/reporting.mdplugins/nemo-customizer/src/nemo_customizer/skills/nemo-customizer/references/rl-kubernetes-runtime.mdplugins/nemo-customizer/src/nemo_customizer/skills/nemo-customizer/references/troubleshooting.mdplugins/nemo-customizer/src/nemo_customizer/skills/nemo-customizer/tests.jsonplugins/nemo-rl/tests/fixtures/integrations_wandb_mlflow.jsonplugins/nemo-rl/tests/fixtures/minimal_dpo.jsonplugins/nemo-rl/tests/test_contract_job_inputs.py
Contributor
|
Signed-off-by: anubhutiv <[email protected]>
anubhutivyas
force-pushed
the
299-dpo-docs/anubhutiv
branch
from
July 2, 2026 19:18
d220ea1 to
db6e2d1
Compare
arpitsardhana
pushed a commit
that referenced
this pull request
Jul 9, 2026
Signed-off-by: anubhutiv <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary by CodeRabbit