Skip to content

Decouple Megatron paths and centralize chat-template defaults#686

Open
Kovbo wants to merge 4 commits into
mainfrom
kovbo/megatron-service-avoid-unsloth-import
Open

Decouple Megatron paths and centralize chat-template defaults#686
Kovbo wants to merge 4 commits into
mainfrom
kovbo/megatron-service-avoid-unsloth-import

Conversation

@Kovbo
Copy link
Copy Markdown
Collaborator

@Kovbo Kovbo commented May 20, 2026

Summary

  • stop importing ART's Unsloth training stack from art.megatron.service at module import time, so Megatron service helpers can load in Megatron-only environments
  • keep create_identity_lora usable without TRL/vLLM side imports by using a local GC/CUDA cleanup helper
  • remove the Unsloth/unsloth_zoo dependency from SFT tokenization by adding ART-owned response-only label masking in art.preprocessing.response_masking
  • centralize chat-template thinking defaults and make the helper types compatible with ty when kwargs are passed to tokenizer.apply_chat_template

Testing

  • uv run --with ruff ruff check src/art/megatron/service.py
  • uv run ruff check src/art/utils/chat_template.py src/art/preprocessing/tokenize.py src/art/preprocessing/response_masking.py tests/unit/test_preprocessing_tokenize.py
  • uv run ty check src/art/utils/chat_template.py src/art/preprocessing/tokenize.py src/art/tinker/server.py
  • PYTHONPATH=/tmp/art-serverless-fix/src:/tmp/art-serverless-fix/tests uv run pytest /tmp/art-serverless-fix/tests/unit/test_preprocessing_tokenize.py -q
  • PYTHONPATH=/tmp/art-serverless-fix/src uv run python - <<'PY'
    from art.megatron.service import create_identity_lora
    print(create_identity_lora.__name__)
    PY
  • created identity LoRA for Qwen/Qwen3-30B-A3B-Instruct-2507 with rank 1

@Kovbo Kovbo requested a review from bradhilton May 20, 2026 21:37
@Kovbo Kovbo marked this pull request as draft May 20, 2026 21:37
@Kovbo Kovbo requested review from FurtherAI and removed request for FurtherAI and bradhilton May 20, 2026 21:37
@Kovbo Kovbo force-pushed the kovbo/megatron-service-avoid-unsloth-import branch from 26edd52 to a2019a6 Compare May 21, 2026 00:10
@Kovbo Kovbo changed the title Avoid Unsloth import in Megatron service Fix Megatron service imports and Qwen3 identity LoRA export May 21, 2026
@Kovbo Kovbo force-pushed the kovbo/megatron-service-avoid-unsloth-import branch from a2019a6 to 8014898 Compare May 21, 2026 18:47
@Kovbo Kovbo changed the title Fix Megatron service imports and Qwen3 identity LoRA export Avoid Unsloth import in Megatron service May 21, 2026
@Kovbo Kovbo force-pushed the kovbo/megatron-service-avoid-unsloth-import branch from 3e9733c to 7aa3be3 Compare May 21, 2026 23:37
@Kovbo Kovbo force-pushed the kovbo/megatron-service-avoid-unsloth-import branch 2 times, most recently from 824d471 to 63945e6 Compare May 22, 2026 01:29
@Kovbo Kovbo force-pushed the kovbo/megatron-service-avoid-unsloth-import branch from 63945e6 to ad1d5a6 Compare May 22, 2026 18:08
@Kovbo Kovbo changed the title Avoid Unsloth import in Megatron service Decouple Megatron SFT paths from Unsloth imports May 22, 2026
@Kovbo Kovbo changed the title Decouple Megatron SFT paths from Unsloth imports Decouple Megatron paths and centralize chat-template defaults May 22, 2026
@Kovbo Kovbo requested review from FurtherAI and angkywilliam May 23, 2026 00:24
@Kovbo Kovbo marked this pull request as ready for review May 23, 2026 00:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant