Skip to content

Fix needed: AttributeError: 'StaticCache' object has no attribute 'max_batch_size'. Did you mean: 'batch_size'? #214

@textbc1

Description

@textbc1

Hi,
Error occured in 2nd iteraton of generate() (2nd iteration appears due to need for accessing Cache is more from 2nd)

Exact error: AttributeError: 'StaticCache' object has no attribute 'max_batch_size'. Did you mean: 'batch_size'?

Error shown in the terminal:
generation = model.generate(input_ids=input_ids, prompt_input_ids=prompt_input_ids, attention_mask=attention_mask, prompt_attention_mask=prompt_attention_mask,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:_Develop_dev\px3.pixi\envs\default\Lib\site-packages\torch\utils_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "D:_Develop_dev\px3.pixi\envs\default\Lib\site-packages\parler_tts\modeling_parler_tts.py", line 3494, in generate
model_kwargs["past_key_values"] = self._get_cache(
^^^^^^^^^^^^^^^^
File "D:_Develop_dev\px3.pixi\envs\default\Lib\site-packages\parler_tts\modeling_parler_tts.py", line 3278, in _get_cache
or cache_to_check.max_batch_size != max_batch_size
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:_Develop_dev\px3.pixi\envs\default\Lib\site-packages\torch\nn\modules\module.py", line 1928, in getattr
raise AttributeError(
AttributeError: 'StaticCache' object has no attribute 'max_batch_size'. Did you mean: 'batch_size'?

Flag set in the code before any call to generate() was made:
model.generation_config.cache_implementation = "static"

Could you please help by giving a fix, if it only parler-tts code, even source code fix is ok if exact changes are provided will apply locally and build

Thank you in advance

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions