Is there standard method to update the some parameters like max_tokens, temp., top_p, and so on before invoking and without reinitialization of llm whether api or local?

[moataz-kemetai]

Hi, again, Is there any way to pass the parameters an Ollama llm should have, such as num_predict, through the _generate, generate, or invoke function, such as the top_p, and temperature, or must this be done on llm creation? If not, how can I change the configuration parameters of an llm? Is there a function for this?

Thanks!

I second this. I would like to configure a model once but potentially call it multiple times with differing temperatures or max tokens depending on each circumstance. For example, lower max tokens for conversation history summarization, but unbounded for response to the user.

Originally posted by @brbarnett in #19718 (reply in thread)