Skip to content

Conversation

@gabor-openai
Copy link
Contributor

Summary

EvalsAPI Images Inputs notebook

  1. Fix: "grading_results" construction: it's .sample, not ["sample"]
  2. Improve: Add "score" and "passed" to the dataframe for easier interpretability

Motivation

The cell I've changed previously errored out with:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[18], line 11
      3 if run.status == "completed" or run.status == "failed": # check if the run is finished
      4     output_items = list(client.evals.runs.output_items.list(
      5         run_id=run.id, eval_id=eval_object.id
      6     ))
      7     df = pd.DataFrame({
      8             "prompt": [item.datasource_item["prompt"]for item in output_items],
      9             "reference": [item.datasource_item["reference"] for item in output_items],
     10             "model_response": [item.sample.output[0].content for item in output_items],
---> 11             "grading_results": [item.results[0]["sample"]["output"][0]["content"]
     12                                 for item in output_items]
     13         })
     14     display(df)
     15     break

TypeError: 'Result' object is not subscriptable

Add score and passed fields to DataFrame so for better interpretability
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant