StabilityAI StableLM

[levicki]

In case you missed it:

https://github.com/stability-AI/stableLM/

Maybe support for their models can be added?

[henk717]

They should already be supported since its NeoX based models, the models are generally considered awful though so I am not sure how much sense it makes to include in the menu. None of our tuners like them, and they perform much worse than the existing models such as Pythia.

[levicki]

@henk717 I thought they might be a good base for further tuning with specific material, but I will defer to your judgement on whether they are actually any good. Thanks for letting me know.

[henk717]

That part is already fully supported, but the tuners estimate it would take months of tuning just to get on the level of Pythia.