-
Notifications
You must be signed in to change notification settings - Fork 576
Description
Hi,
Error occured in 2nd iteraton of generate() (2nd iteration appears due to need for accessing Cache is more from 2nd)
Exact error: AttributeError: 'StaticCache' object has no attribute 'max_batch_size'. Did you mean: 'batch_size'?
Error shown in the terminal:
generation = model.generate(input_ids=input_ids, prompt_input_ids=prompt_input_ids, attention_mask=attention_mask, prompt_attention_mask=prompt_attention_mask,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:_Develop_dev\px3.pixi\envs\default\Lib\site-packages\torch\utils_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "D:_Develop_dev\px3.pixi\envs\default\Lib\site-packages\parler_tts\modeling_parler_tts.py", line 3494, in generate
model_kwargs["past_key_values"] = self._get_cache(
^^^^^^^^^^^^^^^^
File "D:_Develop_dev\px3.pixi\envs\default\Lib\site-packages\parler_tts\modeling_parler_tts.py", line 3278, in _get_cache
or cache_to_check.max_batch_size != max_batch_size
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:_Develop_dev\px3.pixi\envs\default\Lib\site-packages\torch\nn\modules\module.py", line 1928, in getattr
raise AttributeError(
AttributeError: 'StaticCache' object has no attribute 'max_batch_size'. Did you mean: 'batch_size'?
Flag set in the code before any call to generate() was made:
model.generation_config.cache_implementation = "static"
Could you please help by giving a fix, if it only parler-tts code, even source code fix is ok if exact changes are provided will apply locally and build
Thank you in advance