Commit 40a4e9a
committed
Resolves asking for cluster ID when it's known.
By examining the last 100 runs of assisted-chat eval tests in the CI the issue appeared 4 times.
The description of the issue:
The LLM asks for cluster ID even if the cluster was created with the previous message in the same conversation.
Example:
Query: Using the ID of the cluster you just created, get the Discovery ISO download URL for cluster 'eval-test-singlenode-d0xsqi07'
Response: I cannot use the cluster name to get the Discovery ISO download URL. I need the cluster ID. The cluster ID is 12e392cb-82e3-43e1-9923-d627b6476f43. Would you like me to get the Discovery ISO download URL for you?
Possible root cause:
a) the resolution logic in lines 125-129 and the instruction in line 124 never said that the model should not verify the cluster ID or any information which can be learned from tool calls with the user after getting it from the tool calls.
a) line 124 instructed the model to acquire the needed information via tool calls without saying where the logic was located. It might caused uncertainity in the model1 parent 67ce640 commit 40a4e9a
1 file changed
+5
-3
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
322 | 322 | | |
323 | 323 | | |
324 | 324 | | |
325 | | - | |
326 | | - | |
| 325 | + | |
| 326 | + | |
327 | 327 | | |
328 | | - | |
| 328 | + | |
329 | 329 | | |
330 | 330 | | |
| 331 | + | |
| 332 | + | |
331 | 333 | | |
332 | 334 | | |
333 | 335 | | |
| |||
0 commit comments