[BUG] AttributeError: 'MiniCPMForCausalLM' object has no attribute 'generate' #1038
Description
是否已有关于该错误的issue或讨论？ | Is there an existing issue / discussion for this?
- 我已经搜索过已有的issues和讨论 | I have searched the existing issues / discussions
该问题是否在FAQ中有解答？ | Is there an existing answer for this in FAQ?
- 我已经搜索过FAQ | I have searched FAQ
当前行为 | Current Behavior
Traceback (most recent call last):
File "/app/MiniCPM_V2.py", line 39, in
res, context, _ = model.chat(
^^^^^^^^^^^
File "/root/.cache/huggingface/modules/transformers_modules/MiniCPM-V-2/modeling_minicpmv.py", line 375, in chat
res, vision_hidden_states = self.generate(
^^^^^^^^^^^^^^
File "/root/.cache/huggingface/modules/transformers_modules/MiniCPM-V-2/modeling_minicpmv.py", line 308, in generate
result = self._decode(model_inputs["inputs_embeds"], tokenizer, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.cache/huggingface/modules/transformers_modules/MiniCPM-V-2/modeling_minicpmv.py", line 212, in _decode
output = self.llm.generate(
^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/torch/nn/modules/module.py", line 1940, in getattr
raise AttributeError(
AttributeError: 'MiniCPMForCausalLM' object has no attribute 'generate'
期望行为 | Expected Behavior
import torch
import pandas as pd
from PIL import Image
from datasets import load_from_disk
from transformers import AutoModel, AutoTokenizer
path = "/app/"
dataset = load_from_disk(path + "train")
print(dataset)
print('='*20)
model = AutoModel.from_pretrained('/app/models/MiniCPM-V-2', trust_remote_code=True, torch_dtype=torch.bfloat16)
model = model.to(device='cuda', dtype=torch.bfloat16)
tokenizer = AutoTokenizer.from_pretrained('/app/models/MiniCPM-V-2', trust_remote_code=True)
model.eval()
image = Image.open('xxx.png').convert('RGB')
question = 'What is in the image?'
msgs = [{'role': 'user', 'content': question}]
res, context, _ = model.chat(
image=image,
msgs=msgs,
context=None,
tokenizer=tokenizer,
sampling=True,
temperature=0.7
)
print(res)
复现方法 | Steps To Reproduce
Can the NVIDIA GeForce RTX 5090 be used with MiniCPM-V2?
运行环境 | Environment
NVIDIA GeForce RTX 5090
- OS:Ubuntu 22.04
- Python:3.11.13
- Transformers: 4.55.4
- PyTorch:2.7.1+cu128
- CUDA (`python -c 'import torch; print(torch.version.cuda)'`):12.8备注 | Anything else?
No response