Skip to content

Commit 72e9533

Browse files
committed
feat: update docs
Signed-off-by: Mehant Kammakomati <mehant.kammakomati2@ibm.com>
1 parent 61f7c13 commit 72e9533

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -804,7 +804,7 @@ Notes:
804804
* Notes on Fast MoE
805805
- `--fast_moe` is an integer value that configures the amount of expert parallel sharding (ep_degree).
806806
- `world_size` must be divisible by the `ep_degree`
807-
- Running fast moe modifies the state dict of the model, and must be post-processed using [checkpoint utils](https://github.com/foundation-model-stack/fms-acceleration/blob/main/plugins/accelerated-moe/src/fms_acceleration_moe/utils/checkpoint_utils.py) to run inference (HF, vLLM, etc.).
807+
- Running fast moe modifies the state dict of the model, and must be post-processed which happens automatically and the converted checkpoint can be found at `hf_converted_checkpoint` folder within every saved checkpoint directory. Alternatively, we can perform similar option manually through [checkpoint utils](https://github.com/foundation-model-stack/fms-acceleration/blob/main/plugins/accelerated-moe/src/fms_acceleration_moe/utils/checkpoint_utils.py) script.
808808
- The typical usecase for this script is to run:
809809
```
810810
python -m fms_acceleration_moe.utils.checkpoint_utils \

0 commit comments

Comments
 (0)