Model Support

[dreaming-panda]

• Qwen 2.5 series #7
• Gemma 2 series #12
• GLM-4-9B series #19
• Mistral 7B v0.3
• mistralai/Ministral-8B-Instruct-2410
• mistralai/Mistral-Small-Instruct-2409
• Phi 4 14B
• ibm-granite/granite-3.2-8b-instruct-preview #18
• baichuan-inc/Baichuan-M1-14B-Instruct #13

1. Create independent files for model layers and models.
2. Search for chat/code template and add in umbrella/template.py.
3. Register the models you write in umbrella/models/auto_model.py
4. To run the models, we can use python generate.py --model <model you add> --template <template you add>. generate.py is in ./example/
5. Compare the results with huggingface. The example is attached. You need to adjust the model, input to make sure they are the same as you run the models with generate.py.
6. Submit a PR to let me review your code before merging.

[dreaming-panda]

# Load model directly
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, Qwen2ForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-3B-Instruct")
model = Qwen2ForCausalLM.from_pretrained("Qwen/Qwen2.5-3B-Instruct", torch_dtype=torch.float16, _attn_implementation="eager").to("cuda:0")
text = "Tell me what you know about Reinforcement Learning in 100 words."
input_ids = tokenizer.encode(text=text, return_tensors="pt").to("cuda:0")

output = model.generate(input_ids, do_sample=False, max_new_tokens=128)
print(tokenizer.decode(output[0], skip_special_tokens=True))