Finetuning#258

Open

neo2478 wants to merge 36 commits intoopenai:masterfrom

neo2478:finetuning

Commits on Mar 3, 2019

Add finetuning code.
Neil Shepperd
committed
chmod +x
Neil Shepperd
committed
Add finetuning instructions
Neil Shepperd
committed
Fix sample generation with batch_size greater than 1.
Neil Shepperd
committed

Commits on Mar 18, 2019

Commits on Mar 19, 2019

Merge pull request #2 from tlkh/finetuning
nshepperd
authored
autoformat
Neil Shepperd
committed
Combine input text files with <|endoftext|> delimiter to ensure there are enough contiguous tokens to sample from.
Neil Shepperd
committed

Commits on Mar 20, 2019

Write losses to summary file for tensorboard.
Neil Shepperd
committed
Add learning rate as command line flag.
Neil Shepperd
committed
Use argparse instead of fire in train.py.
Neil Shepperd
committed

Commits on Mar 21, 2019

Fix encode.py
Neil Shepperd
committed
Add gradient accumulation with default of 5 minibatches
Neil Shepperd
committed

Commits on Mar 25, 2019

Merge remote-tracking branch 'origin/master' into finetuning
Neil Shepperd
committed

Commits on May 2, 2019

Turn off gradient accumulation by default, it shouldn't be needed.
Neil Shepperd
committed

Commits on May 4, 2019

Add gradient checkpointing and another optimization necessary to allow training the 345M model.
Neil Shepperd
committed
Add "validation" loss calculation.
Neil Shepperd
committed

Commits on May 5, 2019

Add toposort to requirements
Tenoke
committed

Commits on May 6, 2019

Merge pull request #3 from Tenoke/finetuning
Neil Shepperd
authored

Commits on May 14, 2019

Add option to use SGD for optimizer
Neil Shepperd
committed
Record learning rate in tensorboard logs
Neil Shepperd
committed
Add text in README for --optimizer flag
Neil Shepperd
committed
Reduce default learning rate of train.py.
Neil Shepperd
committed
Merge remote-tracking branch 'origin/master' into finetuning
Neil Shepperd
committed

Commits on May 15, 2019

New feature: add noise to network inputs to regularize against overreacting to typos.
Neil Shepperd
committed
Add top-p sampling
Neil Shepperd
committed
Add top_p to interactive_conditional_samples.py and generate_unconditional_samples.py.
Neil Shepperd
committed
fix typo in top_p
Neil Shepperd
committed
Fix top_p sampling for batch_size>1
Neil Shepperd
committed

Commits on Aug 15, 2019

Updated README.md
biranchi2018
authored

Commits on Aug 27, 2019

Merge pull request #22 from biranchi2018/biranchi2018-patch-1
N Shepperd
authored
Add note to install cudnn, re https://github.com/nshepperd/gpt-2/issues/8
Neil Shepperd
committed
Add flag to set encoding for text reading and writing, defaulting to utf-8.
Neil Shepperd
committed

Commits on Aug 13, 2020

Replace Dockerfile with a fixed one
Roberto Moratore
committed