Got error when run demo 'LlamaConfig' object has no attribute 'num_key_value_heads'

[ffiioonnaa]

hi, thanks for your work! I got an error when run demo：

VTG-LLM/vtgllm/models/modeling_llama.py:219
AttributeError: 'LlamaConfig' object has no attribute 'num_key_value_heads'

[gyxxyg]

Thank you for your attention. The error may be related to the version of the transformers library you are using. Could you kindly provide the specific version of the transformers library, as well as any additional details about your environment?

[ffiioonnaa]

tokenizers 0.19.1
transformers 4.41.2

Python 3.9.7
_libgcc_mutex 0.1 main
_openmp_mutex 4.5 1_gnu
absl-py 1.4.0 pypi_0 pypi
accelerate 0.30.1 pypi_0 pypi
antlr4-python3-runtime 4.9.3 pypi_0 pypi
av 10.0.0 pypi_0 pypi
beautifulsoup4 4.12.2 pypi_0 pypi
braceexpand 0.1.7 pypi_0 pypi
brotlipy 0.7.0 py39h27cfd23_1003
ca-certificates 2021.10.26 h06a4308_2
cachetools 5.3.0 pypi_0 pypi
certifi 2021.10.8 py39h06a4308_2
cffi 1.15.0 py39hd667e15_1
charset-normalizer 2.0.4 pyhd3eb1b0_0
conda 4.11.0 py39h06a4308_0
conda-content-trust 0.1.1 pyhd3eb1b0_0
conda-package-handling 1.7.3 py39h27cfd23_1
contourpy 1.0.7 pypi_0 pypi
cryptography 36.0.0 py39h9ce1e76_0
cycler 0.11.0 pypi_0 pypi
decord 0.6.0 pypi_0 pypi
einops 0.8.0 pypi_0 pypi
filelock 3.12.0 pypi_0 pypi
fonttools 4.38.0 pypi_0 pypi
fsspec 2024.6.0 pypi_0 pypi
gdown 4.7.1 pypi_0 pypi
google-auth 2.16.1 pypi_0 pypi
google-auth-oauthlib 0.4.6 pypi_0 pypi
grpcio 1.51.3 pypi_0 pypi
huggingface-hub 0.23.2 pypi_0 pypi
idna 3.3 pyhd3eb1b0_0
imageio 2.28.0 pypi_0 pypi
imageio-ffmpeg 0.4.8 pypi_0 pypi
importlib-metadata 6.0.0 pypi_0 pypi
importlib-resources 5.12.0 pypi_0 pypi
iopath 0.1.10 pypi_0 pypi
jinja2 3.1.4 pypi_0 pypi
joblib 1.4.2 pypi_0 pypi
kiwisolver 1.4.4 pypi_0 pypi
ld_impl_linux-64 2.35.1 h7274673_9
libffi 3.3 he6710b0_2
libgcc-ng 9.3.0 h5101ec6_17
libgomp 9.3.0 h5101ec6_17
libstdcxx-ng 9.3.0 hd4cf53a_17
markdown 3.4.1 pypi_0 pypi
markdown-it-py 2.2.0 pypi_0 pypi
markupsafe 2.1.2 pypi_0 pypi
matplotlib 3.7.0 pypi_0 pypi
mdurl 0.1.2 pypi_0 pypi
mpmath 1.3.0 pypi_0 pypi
ncurses 6.3 h7f8727e_2
networkx 3.2.1 pypi_0 pypi
numpy 1.24.2 pypi_0 pypi
nvidia-cublas-cu12 12.1.3.1 pypi_0 pypi
nvidia-cuda-cupti-cu12 12.1.105 pypi_0 pypi
nvidia-cuda-nvrtc-cu12 12.1.105 pypi_0 pypi
nvidia-cuda-runtime-cu12 12.1.105 pypi_0 pypi
nvidia-cudnn-cu12 8.9.2.26 pypi_0 pypi
nvidia-cufft-cu12 11.0.2.54 pypi_0 pypi
nvidia-curand-cu12 10.3.2.106 pypi_0 pypi
nvidia-cusolver-cu12 11.4.5.107 pypi_0 pypi
nvidia-cusparse-cu12 12.1.0.106 pypi_0 pypi
nvidia-nccl-cu12 2.20.5 pypi_0 pypi
nvidia-nvjitlink-cu12 12.5.40 pypi_0 pypi
nvidia-nvtx-cu12 12.1.105 pypi_0 pypi
oauthlib 3.2.2 pypi_0 pypi
omegaconf 2.3.0 pypi_0 pypi
opencv-python 4.10.0.82 pypi_0 pypi
openssl 1.1.1m h7f8727e_0
packaging 23.0 pypi_0 pypi
pandas 2.2.2 pypi_0 pypi
peft 0.11.1 pypi_0 pypi
pillow 9.4.0 pypi_0 pypi
pip 21.2.4 py39h06a4308_0
portalocker 2.8.2 pypi_0 pypi
protobuf 3.20.3 pypi_0 pypi
psutil 5.9.8 pypi_0 pypi
pyasn1 0.4.8 pypi_0 pypi
pyasn1-modules 0.2.8 pypi_0 pypi
pycosat 0.6.3 py39h27cfd23_0
pycparser 2.21 pyhd3eb1b0_0
pygments 2.14.0 pypi_0 pypi
pyopenssl 21.0.0 pyhd3eb1b0_1
pyparsing 3.0.9 pypi_0 pypi
pysocks 1.7.1 py39h06a4308_0
python 3.9.7 h12debd9_1
python-dateutil 2.8.2 pypi_0 pypi
pytz 2024.1 pypi_0 pypi
pyyaml 6.0 pypi_0 pypi
readline 8.1.2 h7f8727e_1
regex 2024.5.15 pypi_0 pypi
requests 2.27.1 pyhd3eb1b0_0
requests-oauthlib 1.3.1 pypi_0 pypi
rich 13.3.1 pypi_0 pypi
rsa 4.9 pypi_0 pypi
ruamel_yaml 0.15.100 py39h27cfd23_0
safetensors 0.4.3 pypi_0 pypi
scikit-learn 1.5.0 pypi_0 pypi
scipy 1.13.1 pypi_0 pypi
sentencepiece 0.2.0 pypi_0 pypi
setuptools 58.0.4 py39h06a4308_0
six 1.16.0 pyhd3eb1b0_0
soupsieve 2.4.1 pypi_0 pypi
sqlite 3.37.0 hc218d9a_0
sympy 1.12.1 pypi_0 pypi
tensorboard 2.12.0 pypi_0 pypi
tensorboard-data-server 0.7.0 pypi_0 pypi
tensorboard-plugin-wit 1.8.1 pypi_0 pypi
tensorboardx 2.6 pypi_0 pypi
threadpoolctl 3.5.0 pypi_0 pypi
timm 1.0.3 pypi_0 pypi
tk 8.6.11 h1ccaba5_0
tokenizers 0.19.1 pypi_0 pypi
torch 2.3.0 pypi_0 pypi
torchvision 0.18.0 pypi_0 pypi
tqdm 4.62.3 pyhd3eb1b0_1
transformers 4.41.2 pypi_0 pypi
triton 2.3.0 pypi_0 pypi
typing-extensions 4.12.1 pypi_0 pypi
tzdata 2024.1 pypi_0 pypi
urllib3 1.26.7 pyhd3eb1b0_0
webdataset 0.2.86 pypi_0 pypi
werkzeug 2.2.3 pypi_0 pypi
wheel 0.37.1 pyhd3eb1b0_0
xz 5.2.5 h7b6447c_0
yaml 0.2.5 h7b6447c_0
zipp 3.14.0 pypi_0 pypi
zlib 1.2.11 h7f8727e_4

[gyxxyg]

I tried again and did not encounter that error. Furthermore, I examined the Transformers codebase, and it appears that the 'num_key_value_heads' attribute is indeed present in the LLamaConfig class. It's a bit hard to figure out what's going on with the information we have. Maybe try setting up your environment the same to 'requirements-v100.txt' file can help.