Allow LangSmith evaluators to access tool calls

[alexnodeland]

Checked

• I searched existing ideas and did not find a similar one
• I added a very descriptive title
• I've clearly described the feature request and motivation for it

Feature request

It would be useful to allow LangSmith evaluators to assess tool calls. Currently if a user is using the prompt playground with function calling and runs that against examples that exist in a data set, the prompt playground automatically appends a StrOutputParser to the LLM response, and extracts the output from the response content. Then, when building an evaluator, when selecting the Submission I cannot select tool_calls, only output from the StrOutputParser. I would like to be able to build an evaluator that looks at the tool_calls attribute of the response, and compares that to an expectation, defined in the dataset.

Motivation

Essentially, I just want to be able to build evaluators for function calling that leverage the prompt playgrounds and datasets features in LangSmith directly

Proposal (If applicable)

No response