Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 40 additions & 1 deletion pages/generative-apis/troubleshooting/fixing-common-issues.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,45 @@ Below are common issues that you may encounter when using Generative APIs, their
- Graph across time should be empty
```

## Previous messages are not taken into account by the model

### Causes
- Previous messages are not sent to the model
- The content sent exceeds the maximum context window for this model

### Solution
- LLM models are completely "stateless" and thus do not store previous messages or conversations. For example, when building a chatbot application, each time a new message is sent by the user, all preceding messages in the conversation need to be sent through the API payload. Example payload for multi-turn conversation:
```python
from openai import OpenAI

client = OpenAI(
base_url="https://api.scaleway.ai/v1",
api_key=os.getenv("SCW_SECRET_KEY")
)

response = client.chat.completions.create(
model="llama-3.1-8b-instruct",
messages=[
{
"role": "user",
"content": "What is the solution to 1+1= ?"
},
{
"role": "assistant",
"content": "2"
},
{
"role": "user",
"content": "Double this number"
}
]
)

print(response.choices[0].message.content)
```
This snippet will output the model response, which is `4`.
- When exceeding maximum context window, you should receive a `400 - BadRequestError` detailing context length value you exceeded. In this case, you should reduce the size of the content you send to the API.

## Best practices for optimizing model performance

### Input size management
Expand All @@ -135,4 +174,4 @@ Below are common issues that you may encounter when using Generative APIs, their
### Debugging silent errors
- For cases where no explicit error is returned:
- Verify all fields in the API request are correctly named and formatted.
- Test the request with smaller and simpler inputs to isolate potential issues.
- Test the request with smaller and simpler inputs to isolate potential issues.