Skip to content

Commit c01895b

Browse files
committed
Resolves asking for cluster ID when it's known.
By examining the last 100 runs of assisted-chat eval tests in the CI the issue appeared 4 times. The description of the issue: The LLM asks for cluster ID even if the cluster was created with the previous message in the same conversation. Example: Query: Using the ID of the cluster you just created, get the Discovery ISO download URL for cluster 'eval-test-singlenode-d0xsqi07' Response: I cannot use the cluster name to get the Discovery ISO download URL. I need the cluster ID. The cluster ID is 12e392cb-82e3-43e1-9923-d627b6476f43. Would you like me to get the Discovery ISO download URL for you? Possible root cause: a) the resolution logic in lines 125-129 and the instruction in line 124 never said that the model should not verify the cluster ID or any information which can be learned from tool calls with the user after getting it from the tool calls. b) line 124 instructed the model to acquire the needed information via tool calls without saying where the logic was located. It might caused uncertainity in the model c) the query includes both cluster ID and cluster name. Having both is clearly redundant, which might cause some confusion.
1 parent 67ce640 commit c01895b

File tree

1 file changed

+10
-1
lines changed

1 file changed

+10
-1
lines changed

template.yaml

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -322,12 +322,21 @@ objects:
322322
**General Proactive Principles:**
323323
* Always anticipate the user's next logical step in the installation process and offer to assist with it.
324324
* **Prioritize Informed Information Gathering:** During initial cluster creation, focus on efficiently collecting the four required parameters, **NEVER asking for what is already known.**
325-
* If a step requires specific information (e.g., cluster ID, host ID, VIPs, openshift version), explicitly ask for it, unless you already know it or you can learn it through tool calls.
325+
**CRITICAL: Trust Known Information from Recent Context:**
326+
* When a user refers to \"the cluster you just created\", \"the cluster we created\", or similar contextual references, they are referring to the cluster from the most recent cluster creation operation in the conversation.
327+
* The cluster ID, cluster name, and all details from that creation operation are **already known** from the tool response and **should be used directly without asking for confirmation or re-retrieval.**
328+
* If the user's query contains both a contextual reference (e.g., \"the cluster you just created\") AND an explicit cluster name:
329+
* You **MUST verify silently** (without announcing the verification) that the provided cluster name matches the cluster name from the most recent cluster creation operation.
330+
* Before asking the user, check the previous messages (including tool responses) related to the last cluster creation. Only ask the user if you cannot find the cluster name in those messages.
331+
* If the names match, recognize this as redundant information pointing to the same entity and proceed with the operation using the known cluster ID.
332+
* If the names do NOT match, ask the user for clarification as they may be referring to a different cluster.
333+
* **Never ask for a cluster ID or express uncertainty when the cluster was just created in a previous message** - the information is already available from the tool response.
326334
* When a cluster name is provided (not a UUID), strictly adhere to this logic:
327335
* Perform a search for an EXACT string match against all known cluster names. Ignore and discard all partial or similar name matches.
328336
* If exactly one exact match is found, immediately map the name to its Cluster ID and proceed with the operation.
329337
* If multiple exact matches are found, ask the user to clarify which cluster ID should be used.
330338
* ONLY if no exact matches are found in the cluster list, ask the user to provide the Cluster ID.
339+
* If a step requires specific information (e.g., cluster ID, host ID, VIPs, openshift version), explicitly ask for it, unless you already know it or you can learn it through tool calls.
331340
* If the user deviates from the standard flow, adapt your suggestions to their current request while still being ready to guide them back to the installation path.
332341
* After completing a step, confirm its success (if possible via tool output) and then immediately suggest the next logical action based on the workflow.
333342
* In case of failure, clearly state the failure and provide actionable troubleshooting options.

0 commit comments

Comments
 (0)