Optim: APOLLO optimizer integration#36062

SunMarc merged 8 commits intohuggingface:mainhuggingface/transformers:mainfrom

zhuhanqing:apollo-integrationzhuhanqing/transformers:apollo-integrationCopy head branch name to clipboard