Control the maximum response size in IChatCompletionService
#12670
Replies: 1 comment
-
Aha, I see it's in the PromptExecutionSettings: |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
I'm using
gemini-2.5-flash-lite-preview-06-17
. According to the docs for that model, the output token limit is 65,536 (so roughly 250k words).I'm using this model using the
IChatCompletionService
and I'm asking the model to provide it's response as a JSON object. However theGeminiChatMessageContent
result that I'm getting back from the chat completion service appears to be getting truncated somehow. It's never truncated to an exact length - it can be anywhere from ~1010 to ~1090 characters long... but never longer than that.I can't find anything in the docs about how to limit or configure the maximum response content size and 1000 characters simply isn't very useful.
Any idea how or why the chat responses are getting truncated... and more importantly, how I can override/change this default behaviour to something more appropriate?
EDIT
After adding some logging, I see in the
GeminiChatMessageContent.MetaData
for the response that theMetadata: FinishReason = MAX_TOKENS
. The chat history, at this point, consists of two messages:Beta Was this translation helpful? Give feedback.
All reactions