Clarification on Input/Output Length Parameters for gpt-4-1106-preview and gpt-4-0125-preview Models

[MBaltz]

I'm not sure if the guide and the actual code match up, especially about how much data the gpt-4-1106-preview and gpt-4-0125-preview models can handle. The guide says both models can deal with the same amount of data at once. But, looking at the code, it seems there's a difference in their settings.

BetterChatGPT/src/constants/chat.ts

Lines 50 to 51 in ecad41f

	'gpt-4-1106-preview': 128000,
	'gpt-4-0125-preview': 4096,

---

Version / Description / Context

gpt-4-0125-preview
Description: The latest GPT-4 model intended to reduce cases of “laziness” where the model doesn’t complete a task. Returns a maximum of 4,096 output tokens.
Context window: 128,000 tokens

gpt-4-1106-preview
Description: GPT-4 Turbo model featuring improved instruction following, JSON mode, reproducible outputs, parallel function calling, and more. Returns a maximum of 4,096 output tokens. This preview model is not yet suited for production traffic.
Context window: 128,000 tokens

Reference:
https://platform.openai.com/docs/models/gpt-4-and-gpt-4-turbo

[MBaltz]

This PR approach that:
#525

[kaz-on]

I'm interested in this issue.

This value seems to have been discussed in #521.
And I encountered the same problem as @almagest21's comment (#521 (comment)) regarding "Max Token" when using gpt-4-0125-preview.

It appears there's confusion due to a discrepancy between what "Max Token" is described as and how it's actually utilized.

"Max Token" is described in this application as "The maximum number of tokens to generate in the chat completion."

BetterChatGPT/public/locales/en/model.json

Lines 5 to 6 in ecad41f

	"label": "Max Token",
	"description": "The maximum number of tokens to generate in the chat completion. The total length of input tokens and generated tokens is limited by the model's context length."

However, in practice, the max_tokens parameter in the API calls is set to undefined here:

BetterChatGPT/src/api/api.ts

Line 52 in ecad41f

max_tokens: undefined,

and here:

BetterChatGPT/src/api/api.ts

Line 105 in ecad41f

max_tokens: undefined,

Instead, "Max Token" is utilized as a parameter for limitMessageTokens function, which is meant to limit the number of input tokens.

BetterChatGPT/src/hooks/useSubmit.ts

Lines 73 to 77 in ecad41f

	const messages = limitMessageTokens(
	chats[currentChatIndex].messages,
	chats[currentChatIndex].config.max_tokens,
	chats[currentChatIndex].config.model
	);

So modelMaxToken needs to be set to the value in the Context Window and should be 128000 in gpt-4-0125-preview.
I think the proper approach is to match the description with the actual behavior, but it is not clear to me which is correct, the description or the behavior.