v0.1.0 Release

[Praful932]

This project started off as adding a small adapter over scikit-learn to do hyperparameter search over LLMs but evolved into a more comprehensive module, encompassing the features below.

• Add in generation_seed, tfs & top_a as part of HF model.generate function
• Cache to store optimal batch size for a specific inference configuration
• Stopping criteria which stops on multiple sequences that works even for a batch of input

The future will be powered by S(Small)LMs that run on edge devices and sampling techniques (generation parameters) will play a huge role in proving their effectiveness. This belief is reaffirming to see when new sampling techniques keep getting adding to transformers and llama-3 itself comes with a predefined set of generation parameters instead of the default setting.

llmsearch v0.1.0 still has a long way to go. There are certain deficiencies of the package that need to be addressed so that users using it get more value out of it. For example the way the package is currently structured where it uses scikit-learn to treat the generation parameters as a hyperparameter search is not optimal as there is no particular fit method. Then there is the reproducibility problem, where a particular generation parameter set is selected when it has a better score (which could itself be a artifact of reproducibility).

Excited to make this open source and gather feedback from the community.

The v1.0.0 aims to remedy these issues and some more.