This repository was archived by the owner on Oct 19, 2024. It is now read-only.

[WIP] OPT training improvements#858

Open

zhisbug wants to merge 18 commits intomainfrom

Commits on Jan 3, 2023

add some dev essentials
zhisbug
committed

Commits on Jan 17, 2023

finetuning works [skip ci]
zhisbug
committed

Commits on Jan 19, 2023

Commits on Jan 25, 2023

Commits on Jan 26, 2023

switch to use broadcast [skip ci]
Hao Zhang
committed
expand the dataset size 10x
Hao Zhang
committed
do not destroy collective group
Hao Zhang
committed

Commits on Jan 30, 2023

175B script for 32 node
Hao Zhang
committed

Commits on Feb 1, 2023

Commits on Feb 3, 2023

update
zhisbug
committed
reproduct bug
zhisbug
committed

Commits on Feb 6, 2023

Commits on Feb 20, 2023

update opt finetuning weight init code
zhisbug
committed