v0.1.1

github-actions released this 13 Jun 16:09

· 18 commits to main since this release

1159886

Most of the changes in this release focused on improving the training and evaluation code. If you are mainly using pretrained lenses, this should not affect you too much.

Changes

The evaluation sub-command now produces json files, evaluating for a certain number of tokens rather than steps, and the command line interface has been improved. (#92)
Training now supports check pointing to allow for saving lenses during training and resuming training if it is interrupted (#95).
Training can now be done in 8 bits though this does not currently combine with fspd (#88, #94)

Bug Fixes

Slow tokenizers can now work correctly when installing with the [slow_tokenizers] optional dependency (#91).
Lens hashes that were broken by a previous change have been removed and should no longer produce warnings (#99, https://huggingface.co/spaces/AlignmentResearch/tuned-lens/discussions/39)

Assets 2