ChatVertexAI max_output_tokens length stuck at 128 tokens

[westn]

I'm trying to get a longer output with ChatVertexAI, but it seems to cut off the text earlier than expected.
Is anyone else experiencing the same issue? Or am I doing something wrong?
It seems stuck with 128 tokens.

chat = ChatVertexAI(
        temperature=0.28,
        max_output_tokens=1000,
        top_p=0.95,
        top_k=40,
        model_name="chat-bison@001",
    )
    return chat.predict("Tell me a 1000 word story about milk")

Results in

Once upon a time, there was a little girl named Mary who lived on a farm. She loved all the animals on the farm, but her favorite was the cow named Bessie. Bessie was a gentle cow who gave lots of milk. Mary loved to help her mother milk Bessie every morning.

One day, Mary's mother told her that they were going to have to sell Bessie. Mary was very sad. She didn't want to lose her best friend. But her mother explained that they needed the money to buy food for the other animals.

The next day, a man came to the farm

[westn]

#5049
For anyone looking into this, this is a known bug. But no issue connected to it, only a discussion in the original PR