[Bug]: regression due to the MOE refactor: AttributeError: 'FP8EPShardingInfo' object has no attribute 'mlp_type'

[MrGeva]

System Info

on latest main with the minusA fix: #10509 getting this error:
AttributeError: 'FP8EPShardingInfo' object has no attribute 'mlp_type'

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

na

Expected behavior

na

actual behavior

na

additional notes

na

Before submitting a new issue...

• Make sure you already searched for relevant issues, and checked the documentation and examples for answers to frequently asked questions.

[MrGeva]

fixed in #10512