Skip to content

Providing custom instructions/hints for GraphQL query generation #346

@michalpie

Description

@michalpie

When working with large and complex GraphQL schemas, the model can sometimes struggle to generate accurate queries based solely on introspection and search tools. Our schema, for instance, has several non-obvious patterns and relationships that are difficult for the model to infer on its own.

Based on our experiments, providing the model with specific hints or instructions about the schema's structure and common query patterns dramatically improves the accuracy and reliability of the generated GraphQL queries.

I'd like to use a mechanism to inject this custom guidance into the model's context, if such mechanism exists. Otherwise to propose it:

Allow users to override the default descriptions for the built-in tools, especially the execute tool. By modifying its description, we could embed our hints directly into the context the model uses when it considers executing a GraphQL query.

For example, we could change the execute tool's description from its default to something like:
"Executes a GraphQL query. IMPORTANT HINT: To get a user's active cart, always query the 'carts' connection with a filter, like 'carts(where: { status: ACTIVE })'. The 'shoppingCart' field is deprecated."

Example Use Case
Imagine a schema where fetching a user's active shopping cart requires a specific filter:

Naive/Incorrect Query: query { user(id: "123") { shoppingCart { id } } }

Correct Query: query { user(id: "123") { carts(where: { status: ACTIVE }) { id } } }

Without guidance, the model might default to the simpler but incorrect field. A custom hint could explicitly guide it to use the correct pattern, preventing errors and improving the user experience.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions