Skip to content

Commit b0fa1cb

Browse files
committed
Create eval harness
1 parent 532de67 commit b0fa1cb

File tree

6 files changed

+428
-7
lines changed

6 files changed

+428
-7
lines changed

.gitignore

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -209,4 +209,5 @@ __marimo__/
209209
pgdata/
210210
uploads/
211211
notes.md
212-
qdrant_store/
212+
qdrant_store/
213+
.vscode/

eval_harness.py

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
import pandas as pd
2+
from ai import review_application
3+
4+
def run_input(i, job_description):
5+
print(f"Processing row {i}")
6+
return review_application(job_description)
7+
8+
df = pd.read_csv("test/job_rewriting_tests.csv")
9+
results = [run_input(*row) for row in df.itertuples()]
10+
summaries = [result.overall_summary for result in results]
11+
revised_descriptions = [result.revised_description for result in results]
12+
df["Summary"] = summaries
13+
df["Fixed Description"] = revised_descriptions
14+
df.to_csv("eval_output.csv")

0 commit comments

Comments
 (0)