diff --git a/pages/generative-apis/troubleshooting/fixing-common-issues.mdx b/pages/generative-apis/troubleshooting/fixing-common-issues.mdx index 2240ad6c8b..5ca250e7d0 100644 --- a/pages/generative-apis/troubleshooting/fixing-common-issues.mdx +++ b/pages/generative-apis/troubleshooting/fixing-common-issues.mdx @@ -122,6 +122,45 @@ Below are common issues that you may encounter when using Generative APIs, their - Graph across time should be empty ``` +## Previous messages are not taken into account by the model + +### Causes +- Previous messages are not sent to the model +- The content sent exceeds the maximum context window for this model + +### Solution +- LLM models are completely "stateless" and thus do not store previous messages or conversations. For example, when building a chatbot application, each time a new message is sent by the user, all preceding messages in the conversation need to be sent through the API payload. Example payload for multi-turn conversation: +```python +from openai import OpenAI + +client = OpenAI( + base_url="https://api.scaleway.ai/v1", + api_key=os.getenv("SCW_SECRET_KEY") +) + +response = client.chat.completions.create( + model="llama-3.1-8b-instruct", + messages=[ + { + "role": "user", + "content": "What is the solution to 1+1= ?" + }, + { + "role": "assistant", + "content": "2" + }, + { + "role": "user", + "content": "Double this number" + } + ] +) + +print(response.choices[0].message.content) +``` +This snippet will output the model response, which is `4`. +- When exceeding maximum context window, you should receive a `400 - BadRequestError` detailing context length value you exceeded. In this case, you should reduce the size of the content you send to the API. + ## Best practices for optimizing model performance ### Input size management @@ -135,4 +174,4 @@ Below are common issues that you may encounter when using Generative APIs, their ### Debugging silent errors - For cases where no explicit error is returned: - Verify all fields in the API request are correctly named and formatted. - - Test the request with smaller and simpler inputs to isolate potential issues. \ No newline at end of file + - Test the request with smaller and simpler inputs to isolate potential issues.