Set limit of text for response

[parth-patel2023]

Hello,
I am building a document based Langchain app using the OpenAI where I want to set the limit or do customization for the response text.

Is there any way which can help me to achieve this ?

[dosubot[bot]]

🤖

Hello,

Yes, you can set or customize the response text limit in a document-based LangChain app using OpenAI. This can be achieved by adjusting the max_tokens attribute of the BaseOpenAI class. This attribute determines the maximum number of tokens to generate in the completion.

Here is how you can do it:

openai_model = BaseOpenAI(max_tokens=500)

In the above code, the maximum number of tokens is set to 500. This will limit the response text to 500 tokens. If you want to remove the limit, you can set max_tokens to -1:

openai_model = BaseOpenAI(max_tokens=-1)

This will return as many tokens as possible given the prompt and the model's maximal context size.

The default value of max_tokens in the BaseOpenAI class in the LangChain codebase is 256. You can find this information in the libs/langchain/langchain/llms/openai.py file.

I hope this helps! If you have any other questions, feel free to ask.

Best,
Dosu

Sources

• libs/langchain/langchain/llms/openai.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[parth-patel2023]

Great,

Is there possible with the Cohere implementation instead of OpenAI ?

[dosubot[bot]]

🤖

Yes, you can set a limit or customize the response text in a document-based LangChain application using the Cohere language model, similar to adjusting the max_tokens attribute of the BaseOpenAI class in the OpenAI implementation.

In the Cohere class of the LangChain application, the max_tokens attribute is used to set the maximum number of tokens to generate in the completion. This attribute can be adjusted to limit or customize the response text.

Here is the relevant code snippet:

class Cohere(LLM):
    """Cohere large language models."""

    max_tokens: int = 256
    """Denotes the number of tokens to predict per generation."""

In this code, max_tokens is set to 256 by default. You can adjust this value to set a limit or customize the response text in your LangChain application. For example, if you want to limit the response text to 100 tokens, you can set max_tokens to 100. If you want to return as many tokens as possible given the prompt and the model's maximal context size, you can set max_tokens to -1.

The max_tokens attribute in the Cohere class of the LangChain application can be adjusted by setting it directly on an instance of the Cohere class.

Here is an example of how to do it:

from langchain.llms import Cohere

# Create an instance of the Cohere class
cohere = Cohere(model="gptd-instruct-tft", cohere_api_key="my-api-key")

# Adjust the max_tokens attribute
cohere.max_tokens = 500

In the provided context, the max_tokens attribute is defined in the Cohere class. This attribute is a class attribute and its default value is 256. You can adjust this value by setting a new value to the max_tokens attribute of an instance of the Cohere class.

Please let me know if you need further assistance.

Sources

• Agent does not use max_tokens parameter from ChatOpenAI
• libs/langchain/langchain/llms/openai.py
• libs/langchain/langchain/llms/cohere.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.