You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Enhance MultiTurnSample Validator to Support Multiple Tool Calls (#2017)
- Fixes: #1995
The current `validate_user_input` method has a strict validation rule
that requires each `ToolMessage` to be immediately preceded by an
`AIMessage` with `tool_calls`. This prevents valid conversation patterns
where:
1. Multiple `ToolMessage` instances appear in sequence
2. `ToolMessage` instances appear after `ToolMessage` types as long as
an `AIMessage` appeared earlier in the conversation
### Changes
This PR modifies the validation logic to:
1. Track whether we've seen an `AIMessage` at any point in the
conversation
2. Allow a `ToolMessage` to follow either an `AIMessage` or another
`ToolMessage`
### Example
The provided sample demonstrates this pattern with:
```python
from ragas.dataset_schema import MultiTurnSample
from ragas.messages import HumanMessage, AIMessage, ToolMessage, ToolCall
sample_input = [
HumanMessage(
content="Can you provide me with details about Einstein's theory of relativity?"
),
AIMessage(
content="Got it! Let me fetch more details from 'General Theory of Relativity by A. Einstein'.",
tool_calls=[
ToolCall(
name="document_retrieve",
args={"document": "General Theory of Relativity by A. Einstein"},
),
ToolCall(
name="document_retrieve",
args={"document": "A. Einstein biography"},
),
],
),
ToolMessage(
content="Found relevant documents: 1. Relativity: The Special and the General Theory, 2. General Theory of Relativity by A. Einstein."
),
ToolMessage(content="Found relevant documents: 1. A. Einstein biography"),
AIMessage(content="I found some documents on Einstein's theory of relativity..."),
]
sample = MultiTurnSample(user_input=sample_input)
```
0 commit comments