Skip to content
This repository was archived by the owner on Apr 29, 2021. It is now read-only.

Learning-rate warmup + polynomial schedule #22

Draft
julianmack wants to merge 269 commits into
base: master
Choose a base branch
from
Draft

Learning-rate warmup + polynomial schedule #22

julianmack wants to merge 269 commits into from

Conversation

@julianmack
Copy link
Contributor

@julianmack julianmack commented Jan 15, 2020

Added learning-rate warmup scheduler and polynomial scheduler.

This currently blocked by PRs:

@julianmack
Updated installation docs
b6074d3
@julianmack
Updated default config for rnnt
a106765
@julianmack
Added rnnt example usage notebook
40a403b
@julianmack
Added del of variables - preventing/reducing cuda memory leak
2a83e95
@julianmack
RNNT decoding callback is no longer evaluated on first epoch by default
017ad19
@julianmack
Changes to reduce (RNN-T) training memory consumption. All activation…
da3204d 
…s now inplace=True
@julianmack
Changed rnnt default lr
35b9b7f
@julianmack
Added test for rnnt model. Not passing
c928870
@julianmack
Forward pass of rnnt test working
b32ec6d
@julianmack
Added fc layers in rnnt encoder and updated rnnt test
7a2da9e
@julianmack
Updated rnnt docstrings
0fb8742
@julianmack
RNN-T now accepts stacked context frames in channel dim as input
ec1b2cc
@julianmack
Added fc2 to default rnnt config
89bbf43
@julianmack
Halved rnnt fc2 output_size to more closely reflect deepspeech_intern…
67211a3 
…al model definition
@julianmack
Added rnnt builder docstrings
3dc71ef
@julianmack
Made corrections to docstrings
b284f64
@julianmack
Updated docstrings and altered decoder file structure
1c13f3d
@julianmack
Added rnnt decoder test. Currently fails
ccd658d
@julianmack
Small formatting changes and updated decoder test (not passing)
abffeb5
@julianmack
Removed user-specific files
28c7dac
@julianmack
Removed whitespace
f04d054
@julianmack
Removed unnecessary addition
926dadd
@julianmack
Removed whitespace
220bba1
@julianmack
removed mistaken copy-paste
d339eca
@julianmack
Updated docstrings for StackTime
6aa0b1e
@julianmack
Removed whitespace and made more readable
1450a3c
@julianmack
Altered docstrings and whitespace
2000726
@julianmack
Updated comments and other pylint requests
90706f6
@julianmack
Updated rnn_t_base formatting
b1b09b0
@julianmack
Added ability to eval WER less frequently
eef0dd5
@julianmack
Updated Saver() docstrings
9b5a6e0
@julianmack
Merge branch 'state_dict' into rnnt_lr_scheduler
85c27d4
@julianmack
Added step command to scheduler load_state_dict()
c29edb1
@julianmack
Fixed failing test re state_dict loading
14be29d
@julianmack
Added warmup to lr scheduler
2104797
@julianmack
Added docstrings
24d525f
@julianmack
Refactored lr_scheduler folder
3ec5a6f
@julianmack
Added polynomial lr schedule
49f5af4
@julianmack
Updated warmup scheduler to make kwarg passing explicit
0df92bb
@julianmack
Added poly lr and updated rnn_t config
babe696
@julianmack
Removed min multiple from warmup proto
7ce117a
@julianmack
Updated config to give current best
e904d62
@julianmack
Fixed failing test by adding lr_warmup and polynomial_lr protos
1bde1c0
@julianmack
Added lr_scheduler builder tests and refactored lr_scheduler builder
c68d23b
@julianmack
Fixed bug in ConstantLR
7a0cd62
@julianmack
Added lr_scheduler tests
fe36657
@julianmack
Added check for expected frequency
5228c9e
@julianmack
Made general scheduler class and updated tests
ee9b67d
@julianmack
Added lr warmup tests
07e849d
@julianmack
Added lr_scheduler docs
afc1910
@julianmack
Deleted profile notebook
a655550
@julianmack
Added full state_dict test
2aa31a6
@julianmack
Removed duplicate test to reduce suite test-time
c9afddf
@julianmack julianmack changed the base branch from master to rnnt January 15, 2020 17:03
@julianmack julianmack changed the base branch from rnnt to master January 15, 2020 17:03
This was referenced Jan 15, 2020
Draft
Open
@julianmack julianmack changed the base branch from master to rnnt January 28, 2020 15:17
@julianmack julianmack changed the base branch from rnnt to master January 28, 2020 15:18
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

No reviews

Assignees

No one assigned

Labels

blocked

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@julianmack