Custom context size for local models

[gelim]

Hello,

I'm stumbling on the 4096 context size limit on local models served via litellm.

Is there a way/plan to support custom context size limit?

Cheers

[danny-avila]

Yes, this week there will be a solution for this.

[krrishdholakia]

hey @danny-avila how did you fix this?

[gelim]

Ping @danny-avila :)

[danny-avila]

Sorry forgot to close this discussion.

It's implemented here:

You can use presets now for this