Feature Request: Add support for GLM4-9B and related models #509
Description
Prerequisites
- I am running the latest code. Mention the version if possible as well.
- I carefully followed the README.md.
- I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
- I reviewed the Discussions, and have a new and useful enhancement to share.
Feature Description
Recently llama.cpp has got support for ChatGLM models, namely GLM4-9B and ChatGLM3-6B, you can see more details here: ggml-org/llama.cpp#8031
It would be nice if llamafile can sync with upstream such that we can use these models with llamafile as well.
Motivation
The models are pretty powerful and at least on benchmarks, the GLM4-9B beats LLAMA3-8B in many benchmarks, making it an ideal model for those with weaker machines or cheaper inference.
Possible Implementation
Merging with upstream should solve the problem.