You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: pages/generative-apis/troubleshooting/fixing-common-issues.mdx
+40-1Lines changed: 40 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -122,6 +122,45 @@ Below are common issues that you may encounter when using Generative APIs, their
122
122
- Graph across time should be empty
123
123
```
124
124
125
+
## Previous messages are not taken into account by the model
126
+
127
+
### Causes
128
+
- Previous messages are not sent to the model
129
+
- The content sent exceeds maximum context window for this model
130
+
131
+
### Solution
132
+
- LLM models are completely "stateless" and thus do not store previous messages or conversations. For example, when building a chatbot application, each time a new message is sent by the user, all preceding messages in the conversation needs to be sent through the API payload. Example payload for multi-turn conversation:
133
+
```python
134
+
from openai import OpenAI
135
+
136
+
client = OpenAI(
137
+
base_url="https://api.scaleway.ai/v1",
138
+
api_key=os.getenv("SCW_SECRET_KEY")
139
+
)
140
+
141
+
response = client.chat.completions.create(
142
+
model="llama-3.1-8b-instruct",
143
+
messages=[
144
+
{
145
+
"role": "user",
146
+
"content": "What is the solution to 1+1= ?"
147
+
},
148
+
{
149
+
"role": "assistant",
150
+
"content": "2"
151
+
},
152
+
{
153
+
"role": "user",
154
+
"content": "Double this number"
155
+
}
156
+
]
157
+
)
158
+
159
+
print(response.choices[0].message.content)
160
+
```
161
+
This snippet will output the model response, which is `4`.
162
+
- When exceeding maximum context window, you should receive a `400 - BadRequestError` detailing context length value you exceeded. In this case, you should reduce the size of the content you send to the API.
163
+
125
164
## Best practices for optimizing model performance
126
165
127
166
### Input size management
@@ -135,4 +174,4 @@ Below are common issues that you may encounter when using Generative APIs, their
135
174
### Debugging silent errors
136
175
- For cases where no explicit error is returned:
137
176
- Verify all fields in the API request are correctly named and formatted.
138
-
- Test the request with smaller and simpler inputs to isolate potential issues.
177
+
- Test the request with smaller and simpler inputs to isolate potential issues.
0 commit comments