v0.6.2

Release

• framework 0.6.2: patch version, add compatibility for peft >= 0.15
• peft 0.4.2: patch version, add compatibility for peft>=0.15, bnb >=0.45.1
• foak 0.7.0: minor version, enable GraniteMoeHybridForCausalLM
• moe 0.4.0: minor version, enable GraniteMoeHybridForCausalLM

What's Changed

• fix: Update QuantLinear import to GPTQLoraLinear by @Luka-D in #146
• feat: Support granite 4 preview architecture for MoE kernels, EP, and fast kernels by @kmehant in #143

New Contributors

• @Luka-D made their first contribution in #146

Full Changelog: v0.6.1.4...v0.6.2

v0.6.1.4

Release

foak 0.6.1: patch version, Extend support to GraniteMoeSharedForCausalLM architecture

What's Changed

Extend support to GraniteMoeSharedForCausalLM architecture by @kmehant in #126

Full Changelog: v0.6.1.3...v0.6.1.4

v0.6.1.3

Release

moe 0.3.1: patch version, filter out router layer for lora, use element_size

What's Changed

• fix: use element_size by @kmehant in #140
• fix: filter out router so all-linear can be tuned for LoRA target modules by @willmj in #142

Full Changelog: v0.6.1.2...v0.6.1.3

v0.6.1.2

Release

moe 0.3.0: minor version, lora tuning for accelerated MoE

What's Changed

• feat: lora for accelerated MoE - limited by @willmj in #141

Full Changelog: v0.6.1.1...v0.6.1.2

v0.6.1.1

Release

moe 0.2.0: minor version, allow for MoE kernels irrespective of use of EP, remove external library dependency, allow multi-node training

What's Changed

• Add code of conduct by @spzala in #125
• Extend support to GraniteMoeSharedForCausalLM architecture by @kmehant in #126
• fix: nit return sd by @kmehant in #131
• fix: Allow for multi node training for accelerated moe by @kmehant in #129
• fix(deps): Copy KHD imports into scattermoe_utils by @willmj in #127
• chore: Move dcp to safetensors to a wrapper function for downstream usage by @kmehant in #133
• fix: checkpoint utils for single shard by @willmj in #137
• feat: Allow for MoE kernels (Scatter MoE) irrespective of use of EP by @kmehant in #136

New Contributors

• @spzala made their first contribution in #125

Full Changelog: v0.6.1...v0.6.1.1

v0.6.1

Release

• framework 0.6.1: patch version, allow for custom loss instead of forward patching
• peft 0.4.1: patch version, add shard_checkpoint method which is removed in transformers>4.45
• foak 0.6.0: minor version, fix cross entropy for transformers>4.45 by adding custom loss
• moe 0.1.1: patch version, move model loader functionality to augmentation

What's Changed

• feat: move model loader functionality to augmentation by @willmj in #119
• fix: cross entropy for transformers>4.45 by @anhuong in #123
• fix: liger fails to run loss with new param by @anhuong in #124

Full Changelog: v0.6.0...v0.6.1

v0.6.0

Release

• framework 0.6.0: minor version, fix function name 'requires_augmentation'
• peft 0.4.0: minor version, fix function name 'requires_augmentation'
• foak 0.5.0: minor version, fix function name 'requires_augmentation'
• aadp 0.2.0: minor version, fix function name 'requires_augmentation'

What's Changed

• fix: function name 'requires_agumentation' to 'requires_augmentation' by @willmj in #118

Full Changelog: https://github.com/foundation-model-stack/fms-acceleration/commits/v0.6.0

v0.5.0.1

Release

• foak v0.4.1: patch version. fixes for CI builds

What's Changed

• Decouple Filter MP Rules function from cuda imports by @fabianlim in #117

Full Changelog: v0.5.0...v0.5.0.1

New Mixture-of-Experts Plugin

Release

• framework v0.5.0: minor version. Updated to manage new moe plugin.
• moe v0.1.0: new plugin. Released mixture-of-experts plugin with ScatterMoE kernels.
• peft v0.3.5: patch version. Fixed Autocast warnings (#113).
• foak v0.4.0: minor version. Add support for Liger fused-ce (#93), fixes for Fused Ops (dropout and activation).

What's Changed

• Fix Dropout in Fused LoRA Operations by @fabianlim in #102
• Add ExpertParallel Mixture-of-Experts Plugin by @fabianlim in #99
• Disable MLP Fused Ops if Not SwiGLU, Depracted Fast Quantized Peft Plugin, Update Benchmarks by @fabianlim in #106
• fix: requirements file path in error by @willmj in #111
• fix: Deprecation Warnings in AutoCast API by @Abhishek-TAMU in #113
• feat: add liger kernel with fused cross entropy loss by @anhuong in #93
• feat: Checkpoint utils safetensors by @willmj in #116

New Contributors

• @Abhishek-TAMU made their first contribution in #113
• @anhuong made their first contribution in #93

Full Changelog: v0.4.0.4...v0.4.0.5

v0.4.0.4

Release

• peft v0.3.4: patch version. Address #90 for AutoGPTQ when certain parameters require resizing.
• foak v0.3.3: patch version. Address bug introduced in #90 where the grad accum hooks were overwritten.

What's Changed

• Fix Issue with Resizing Parameters on the Meta Device in Low CPU Mem Mode by @fabianlim in #96
• model: Add granite GPTQ model by @willmj in #95

New Contributors

• @willmj made their first contribution in #95

Full Changelog: v0.4.0.3...v0.4.0.4