Skip to content

Commit ee9a4bf

Browse files
authored
feat(genapi): update faq explaining how multi-turn conversation works with llm
1 parent c1b227c commit ee9a4bf

File tree

1 file changed

+40
-1
lines changed

1 file changed

+40
-1
lines changed

pages/generative-apis/troubleshooting/fixing-common-issues.mdx

Lines changed: 40 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -122,6 +122,45 @@ Below are common issues that you may encounter when using Generative APIs, their
122122
- Graph across time should be empty
123123
```
124124
125+
## Previous messages are not taken into account by the model
126+
127+
### Causes
128+
- Previous messages are not sent to the model
129+
- The content sent exceeds maximum context window for this model
130+
131+
### Solution
132+
- LLM models are completely "stateless" and thus do not store previous messages or conversations. For example, when building a chatbot application, each time a new message is sent by the user, all preceding messages in the conversation needs to be sent through the API payload. Example payload for multi-turn conversation:
133+
```python
134+
from openai import OpenAI
135+
136+
client = OpenAI(
137+
base_url="https://api.scaleway.ai/v1",
138+
api_key=os.getenv("SCW_SECRET_KEY")
139+
)
140+
141+
response = client.chat.completions.create(
142+
model="llama-3.1-8b-instruct",
143+
messages=[
144+
{
145+
"role": "user",
146+
"content": "What is the solution to 1+1= ?"
147+
},
148+
{
149+
"role": "assistant",
150+
"content": "2"
151+
},
152+
{
153+
"role": "user",
154+
"content": "Double this number"
155+
}
156+
]
157+
)
158+
159+
print(response.choices[0].message.content)
160+
```
161+
This snippet will output the model response, which is `4`.
162+
- When exceeding maximum context window, you should receive a `400 - BadRequestError` detailing context length value you exceeded. In this case, you should reduce the size of the content you send to the API.
163+
125164
## Best practices for optimizing model performance
126165

127166
### Input size management
@@ -135,4 +174,4 @@ Below are common issues that you may encounter when using Generative APIs, their
135174
### Debugging silent errors
136175
- For cases where no explicit error is returned:
137176
- Verify all fields in the API request are correctly named and formatted.
138-
- Test the request with smaller and simpler inputs to isolate potential issues.
177+
- Test the request with smaller and simpler inputs to isolate potential issues.

0 commit comments

Comments
 (0)