-
Notifications
You must be signed in to change notification settings - Fork 45
Description
When working with large and complex GraphQL schemas, the model can sometimes struggle to generate accurate queries based solely on introspection and search tools. Our schema, for instance, has several non-obvious patterns and relationships that are difficult for the model to infer on its own.
Based on our experiments, providing the model with specific hints or instructions about the schema's structure and common query patterns dramatically improves the accuracy and reliability of the generated GraphQL queries.
I'd like to use a mechanism to inject this custom guidance into the model's context, if such mechanism exists. Otherwise to propose it:
Allow users to override the default descriptions for the built-in tools, especially the execute tool. By modifying its description, we could embed our hints directly into the context the model uses when it considers executing a GraphQL query.
For example, we could change the execute tool's description from its default to something like:
"Executes a GraphQL query. IMPORTANT HINT: To get a user's active cart, always query the 'carts' connection with a filter, like 'carts(where: { status: ACTIVE })'. The 'shoppingCart' field is deprecated."
Example Use Case
Imagine a schema where fetching a user's active shopping cart requires a specific filter:
Naive/Incorrect Query: query { user(id: "123") { shoppingCart { id } } }
Correct Query: query { user(id: "123") { carts(where: { status: ACTIVE }) { id } } }
Without guidance, the model might default to the simpler but incorrect field. A custom hint could explicitly guide it to use the correct pattern, preventing errors and improving the user experience.