Full text search: Restrict search to only input or output #8257
Replies: 5 comments 1 reply
-
|
this would be a highly anticipated feature for us, since we monitor many topics our users mention to the agent, however we only want to count instances where the user mentioned the topic and not the chatbot by itself. |
Beta Was this translation helpful? Give feedback.
-
|
Agreed, this would be extremely useful, having the ability to search for specific string patterns within the output, as well as to add those particular output (matching patterns) to the human annotation queue |
Beta Was this translation helpful? Give feedback.
-
|
Is this something you would take a PR for @jannikmaierhoefer ? |
Beta Was this translation helpful? Give feedback.
-
|
Additionally, it would be helpful to also specify what part of the input or output the user wants to search on (user message, tool, system message). This would allow to exclude the system prompt from the search results. |
Beta Was this translation helpful? Give feedback.
-
|
++ This would be extremely useful! @xstraven @maxdeichmann may I please also request that instead of just searching in "Output" field, it is also possible to search by the values in a specific path, i.e. when using structured outputs (so restricting search to look in "Output.foo")? |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
Describe the feature or potential improvement
For some debugging cases, it would be very helpful to restrict the full-text search to only generation inputs or outputs. For instance, if my system prompt instructs the LLM to call a tool using the tool name (e.g. ), but the LLM sometimes fails to make a tool call and instead just returns the name of the tool as string.
Additional information
No response
Beta Was this translation helpful? Give feedback.
All reactions