v0.1.5

[Luosuu]

Highlights

• Many enhancement on NPU thanks to community contribution, including NPU fused ops and CIs
• Memory-efficient DCP to HuggingFace checkpoint conversion script by @TimYangst
• Support FlashAttention3 by @Coach257 @Luosuu

What's Changed

• [model] fix: quick fix by @Coach257 in [model] fix: quick fix #260
• [misc] chore: add auto label github flow & pr title check by @FoolPlayer in #[misc] chore: add auto label github flow & pr title check #264
• [docs] feat: quick start doc for Ascend NPU by @Crystal-jiang in [docs] feat: quick start doc for Ascend NPU #257
• [dist] fix: the issue where NPU does not support the PreSumMul operation by @Feng0w0 in [dist] fix: the issue where NPU does not support the PreSumMul operation #263
• [misc] fix: fix auto-label for forked pr by @FoolPlayer in [misc] fix: fix auto-label for forked pr #277
• [chore] fix: The missing weights_path in build_parallelize_model prevents weight loading when using fsdp2 by @Feng0w0 in [chore] fix: The missing weights_path in build_parallelize_model prevents weight loading when using fsdp2 #275
• [ci] fix: handle pull_request_target event in auto_label workflow by @Ziyi-Wang in [ci] fix: handle pull_request_target event in auto_label workflow #279
• [misc] fix: fix auto-label logic & add tag by @FoolPlayer in [misc] fix: fix auto-label logic & add tag #281
• [misc] fix: fix check pr workflow by @FoolPlayer in [misc] fix: fix check pr workflow #282
• [ops]feat: support npu fused moe by @onehaitao in [ops]feat: support npu fused moe #269
• [model] fix: calculate norm by foreach by @heidongxianhua in [model] fix: calculate norm by foreach #274
• [feat]Add environment variables that may lead to performance optimization on NPU by @Feng0w0 in [feat]Add environment variables that may lead to performance optimization on NPU #276
• [dist] chore: refactor torch dcp to support multiple storage backends by @Ziyi-Wang in [dist] chore: refactor torch dcp to support multiple storage backends #278
• [config] feat: add qwen3_vl flops estimation by @yfchen-byted in [config] feat: add qwen3_vl flops estimation #220
• [config] fix: restrict Python version to 3.11 and fix PyTorch CUDA dependency resolution by @TimYangst in [config] fix: restrict Python version to 3.11 and fix PyTorch CUDA dependency resolution #287
• [dist]fix:fix a bug in the PreMulSum patch by @Feng0w0 in [dist]fix:fix a bug in the PreMulSum patch #285
• [ci,config] chore: migrate to uv dependency management with enhanced installation guide by @TimYangst in [ci,config] chore: migrate to uv dependency management with enhanced installation guide #232
• [dist] chore: make execute_save public to support multiple DCP backends by @Ziyi-Wang in [dist] chore: make execute_save public to support multiple DCP backends #288
• [data] fix: video utils by @Coach257 in [data] fix: video utils #294
• [ci] fix: uv by @Coach257 in [ci] fix: uv #292
• [dist] fix: remove redundant code (async_op) in ulysses by @KKZ20 in [dist] fix: remove redundant code (async_op) in ulysses #295
• [data] fix: interleave iterable dataset sharding by @ValMystletainn in [data] fix: interleave iterable dataset sharding #291
• [misc] fix: print log twice bug by @pjgao in [misc] fix: print log twice bug #290
• [data] fix: fix annotation by @Fazziekey in [data] fix: fix annotation #296
• [docs] chore: update docs with readthedocs by @FoolPlayer in [docs] chore: update docs with readthedocs #286
• [docs] feat: support Readthedocs on npu by @Crystal-jiang in [docs] feat: support Readthedocs on npu #299
• [misc] fix: del veomni_patch in helper by @Coach257 in [misc] fix: del veomni_patch in helper #298
• [docs] chore: add icon & fix cite by @FoolPlayer in [docs] chore: add icon & fix cite #303
• [misc] fix: clean vescale by @Coach257 in [misc] fix: clean vescale #302
• [ops] feat: Optimize Qwen3-Moe Performance on Ascend NPU with Patches by @zhihaofang1017 in [ops] feat: Optimize Qwen3-Moe Performance on Ascend NPU with Patches #167
• [model, ops] feat: support fa3 by @Luosuu in [model, ops] feat: support fa3 #305
• [ci] chore: add dockerfile and migrate to run ci with uv cache by @FoolPlayer in [ci] chore: add dockerfile and migrate to run ci with uv cache  #300
• [misc] fix: fix mfu for vlm by @heidongxianhua in [misc] fix: fix mfu for vlm #301
• [ckpt, data] fix: fix merge dcp & fix chatml by @FoolPlayer in [ckpt, data] fix: fix merge dcp & fix chatml #310
• [ops] feat: veomni_fa_function by @Coach257 in [ops] feat: veomni_fa_function #304
• [config] refactor: simplify NPU extras with separate aarch64 variant by @TimYangst in [config] refactor: simplify NPU extras with separate aarch64 variant #312
• [config] fix: remove torch extra from transformers dependency by @TimYangst in [config] fix: remove torch extra from transformers dependency #313
• [ops] fix: mv qwen3moe patch by @Coach257 in [ops] fix: mv qwen3moe patch #314
• [logging] fix: fix duplicata log print in wandb case by @FoolPlayer in [logging] fix: fix duplicata log print in wandb case #315
• [ops] fix: rm ALL_FLASH_ATTENTION_FUNCTIONS by @Coach257 in [ops] fix: rm ALL_FLASH_ATTENTION_FUNCTIONS #316
• [misc] feat: use MODELING_BACKEND to control force_hf_load by @Coach257 in [misc] feat: use MODELING_BACKEND to control force_hf_load #311
• [misc] fix: support enable_full_determinism without enable CUDA_LAUNCH_BLOCKING=1 by @FoolPlayer in [misc] fix: support enable_full_determinism without enable CUDA_LAUNCH_BLOCKING=1 #318
• [model] feat: patch LOSS_MAPPING by @Coach257 in [model] feat: patch LOSS_MAPPING  #321
• [model] feat: fused_moe_forward by @Coach257 in [model] feat: fused_moe_forward #325
• [docs] feat: add NPU Overview doc and qwen3 training guide doc by @Crystal-jiang in [docs] feat: add NPU Overview doc and qwen3 training guide doc #319
• [config] feat: Set the default value of use_wandb to false by @Crystal-jiang in [config] feat: Set the default value of use_wandb to false #324
• [docs] fix: fix npu doc suffix py by @Crystal-jiang in [docs] fix: fix npu doc suffix py #328
• [ci] test: add e2e ci test by @FoolPlayer in [ci] test: add e2e ci test #320
• [model] feat: support async ulysses for qwen3vl dense by @yyytiancai in [model] feat: support async ulysses for qwen3vl dense #307
• [model] fix: logits when labels None by @Coach257 in [model] fix: logits when labels None #332
• [misc] feat: env param by @Coach257 in [misc] feat: env param #329
• [docs] refactor: Update README by @FoolPlayer in [docs] refactor: Update README #334
• [ops] feat: Optimize Qwen3 Performance on Ascend NPU with Patches by @Crystal-jiang in [ops] feat: Optimize Qwen3 Performance on Ascend NPU with Patches #308
• [data] fix: Add sorted() to os.listdir() result in dataset.py to ensure consistent file order by @Crystal-jiang in [data] fix: Add sorted() to os.listdir() result in dataset.py to ensure consistent file order #331
• [docs] fix: update readthedoc by @FoolPlayer in [docs] fix: update readthedoc #335
• [ops] feat: Optimize Wan2.1 Performance on Ascend NPU with Patches by @zbian99 in [ops] feat: Optimize Wan2.1 Performance on Ascend NPU with Patches #336
• [dist] fix: loss when sp enabled by @Coach257 in [dist] fix: loss when sp enabled #339
• [misc] fix: vl token count by @Coach257 in [misc] fix: vl token count #341
• [ckpt] fix: checkpointer kwargs by @Coach257 in [ckpt] fix: checkpointer kwargs #343
• [ops] refactor: abstract commonly used NPU fused operators into ops module by @zhihaofang1017 in [ops] refactor: abstract commonly used NPU fused operators into ops module #347
• [optim] fix: support fused/foreach adamw on ascend by @onehaitao in [optim] fix: support fused/foreach adamw on ascend #349
• [docs] feat: add wan2.1 training guide doc and mock dataset generator by @zbian99 in [docs] feat: add wan2.1 training guide doc and mock dataset generator #340
• [model] fix: fix config usage in modeling_qwen3_vl by @yfchen-byted in [model] fix: fix config usage in modeling_qwen3_vl #353
• [misc] fix: bugfix for memory snapshot by @onehaitao in [misc] fix: bugfix for memory snapshot #351
• [ci] fix: add skip remote path test by @FoolPlayer in [ci] fix: add skip remote path test #356
• [data] fix: interleave_dataset duplication by @Coach257 in [data] fix: interleave_dataset duplication #358
• [ci] fix: change network setting by @FoolPlayer in [ci] fix: change network setting #360
• [ci] feat: Add Dockerfile and workflow for Ascend image by @phdddd in [ci] feat: Add Dockerfile and workflow for Ascend image #364
• [model] fix: repeat all2all in Wan2.1 ulysses. by @TKONIY in [model] fix: repeat all2all in Wan2.1 ulysses. #333
• [ci] fix: Npu fix by @phdddd in [ci] fix: Npu fix #368
• [ci] fix: fix_npu_proxy by @FoolPlayer in [ci] fix: fix_npu_proxy #370
• [ci] test: Npu docker auto build by @phdddd in [ci] test: Npu docker auto build #372
• [ops] fix: RoPE fused op caused freqs shape bug by @zbian99 in [ops] fix: RoPE fused op caused freqs shape bug #355
• [ci] fix: sync npu unit tests from gpu unit tests by @onehaitao in [ci] fix: sync npu unit tests from gpu unit tests #359
• [ckpt, ci] fix: refactor merge_dcp_to_hf.py to remove internal dependencies and add checkpoint verification by @TimYangst in [ckpt, ci] fix: refactor merge_dcp_to_hf.py to remove internal dependencies and add checkpoint verification #272
• [ckpt, ci, docs] feat: memory-efficient DCP to HuggingFace checkpoint conversion by @TimYangst in [ckpt, ci, docs] feat: memory-efficient DCP to HuggingFace checkpoint conversion #374
• [ci] fix: fix async ulysses test by @Crystal-jiang in [ci] fix: fix async ulysses test #357
• [ops] feat: add chunkloss characteristics for NPU by @zhihaofang1017 in [ops] feat: add chunkloss characteristics for NPU #361
• [model] feat: Add ascend fused operators for Qwen3VL by @phdddd in [model] feat: Add ascend fused operators for Qwen3VL #323
• [dist, data] fix: init parallel state in data collator post init to avoid worker processing getting single process state by @Luosuu in [dist, data] fix: init parallel state in data collator post init to avoid worker processing getting single process state #383
• [model, ops] feat: add Qwen3 sequence classification model and loss for embedding classification. by @yiwzhao in [model, ops] feat: add Qwen3 sequence classification model and loss for embedding classification. #322

New Contributors

• @yfchen-byted made their first contribution in [config] feat: add qwen3_vl flops estimation #220
• @yyytiancai made their first contribution in [model] feat: support async ulysses for qwen3vl dense #307
• @TKONIY made their first contribution in [model] fix: repeat all2all in Wan2.1 ulysses. #333

Full Changelog: v0.1.4...v0.1.5

---

This discussion was created from the release v0.1.5.