Skip to content

1.5B is finally here!

Pre-release
Pre-release

Choose a tag to compare

View all tags
@ksjae ksjae released this 22 Sep 06:39
· 7 commits to master since this release
v0.2-beta
f010360
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Verified
Learn about vigilant mode.

All releases will be available from my server. If Google drive gives too much hassle(e.g. headless linux), just wget from there.

v0.2-beta is only trained on ~700MB web corpus, on v3-8 TPUs for 36 hours(about 80K steps)

My server : here
Google Drive : here

From this release, I'm switching to home-brew code. This makes the model unable to be loaded in HF Transformers. I'll make a pull request later this year(lots of paperwork involved).
For demo/generation, check out the train code

Assets 2
Loading