Skip to content

Commit c932d29

Browse files
committed
eval driven system design cookbook updates merge-2
1 parent 45c09f1 commit c932d29

File tree

1 file changed

+2
-22
lines changed

1 file changed

+2
-22
lines changed

examples/partners/eval_driven_system_design/receipt_inspection.ipynb

Lines changed: 2 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -112,7 +112,6 @@
112112
"source": [
113113
"## Project Lifecycle\n",
114114
"\n",
115-
<<<<<<< HEAD
116115
"Not every project will proceed in the same way, but projects generally have some \n",
117116
"important components in common.\n",
118117
"\n",
@@ -122,10 +121,8 @@
122121
"represents the ongoing nature of problem understanding - uncovering more about\n",
123122
"the customer domain will influence every step of the process. We wil examine \n",
124123
"several of these iterative cycles of refinement in detail below. \n",
125-
=======
126124
"Not every project will proceed in the same way, but projects generally have some common\n",
127125
"important components.\n",
128-
>>>>>>> origin/main
129126
"\n",
130127
"### 1. Understand the Problem\n",
131128
"\n",
@@ -145,18 +142,15 @@
145142
"It's very rare that a real-world project will start with all the data necessary to get\n",
146143
"to a satisfactory solution, much less to establish confidence.\n",
147144
"\n",
148-
<<<<<<< HEAD
149145
"In our case, we're going to assume that we have a decent sample of system *inputs*, \n",
150146
"in the form of but receipt images, but start without any fully annotated data. We find \n",
151147
"this is a not-unusual situation when automating an existing process. Instead, \n",
152148
"we'll walk through the process of building that out as we go along by collaborating with\n",
153149
"domain experts, and make our evals progressively more comprehensive.\n",
154-
=======
155150
"In our case, we're going to assume that we have a decent sample of system *inputs*\n",
156151
"(here, photographs of receipts), but start without any fully annotated data. We'll walk\n",
157152
"through the process of incrementally expanding our test and training sets as we go along\n",
158153
"and make our evals progressively more comprehensive.\n",
159-
>>>>>>> origin/main
160154
"\n",
161155
"### 3. Build an End-to-End V0 System\n",
162156
"\n",
@@ -414,11 +408,7 @@
414408
"cell_type": "markdown",
415409
"metadata": {},
416410
"source": [
417-
<<<<<<< HEAD
418-
"![Walmart_image](../../../images/Supplies_20240322_220858_Raven_Scan_3_jpeg.rf.50852940734939c8838819d7795e1756.jpg)"
419-
=======
420411
"<img src=\"../../../images/Supplies_20240322_220858_Raven_Scan_3_jpeg.rf.50852940734939c8838819d7795e1756.jpg\" alt=\"Walmart_image\" width=\"400\"/>"
421-
>>>>>>> origin/main
422412
]
423413
},
424414
{
@@ -521,8 +511,8 @@
521511
"source": [
522512
"### Action Decision\n",
523513
"\n",
524-
<<<<<<< HEAD
525-
"Next, we need to close the loop and get to an actual decision based on receipts. \n",
514+
"Next, we need to close the loop and get to an actual decision based on receipts. This\n",
515+
"looks pretty similar, so we'll present the code without comment.\n",
526516
"\n",
527517
"Ordinarily one would start with the most capable model - `o3`, at this time - for a \n",
528518
"first pass, and then once correctness is established experiment with different models\n",
@@ -536,12 +526,8 @@
536526
"that we get a few reasoning errors we can use to illustrate the means of addressing\n",
537527
"them when they occur.\n",
538528
"\n",
539-
"Otherwise, this is pretty similar to the last, so we'll present the code without \n",
540-
"further comment."
541-
=======
542529
"Next, we need to close the loop and get to an actual decision based on receipts. This\n",
543530
"looks pretty similar, so we'll present the code without comment."
544-
>>>>>>> origin/main
545531
]
546532
},
547533
{
@@ -930,14 +916,11 @@
930916
"metadata": {},
931917
"source": [
932918
"After you run that eval you'll be able to view it in the UI, and should see something\n",
933-
<<<<<<< HEAD
934919
"like the below. \n",
935920
"\n",
936921
"(Note, if you have a Zero-Data-Retention agreement, this data is not stored\n",
937922
"by OpenAI, so will not be available in this interface.)\n",
938-
=======
939923
"like:\n",
940-
>>>>>>> origin/main
941924
"\n",
942925
"![Summary UI](../../../images/partner_summary_ui.png)\n",
943926
"\n",
@@ -1667,11 +1650,8 @@
16671650
"ARE NOT TRAVEL-RELATED, THEN IT MUST BE AUDITED.\n",
16681651
"```\n",
16691652
"\n",
1670-
<<<<<<< HEAD
16711653
"4. We added three examples, JSON input/output pairs wrapped in XML tags.\n",
1672-
=======
16731654
"3. We added three examples, JSON input/output pairs wrapped in XML tags.\n",
1674-
>>>>>>> origin/main
16751655
"\n",
16761656
"With our prompt revisions, we'll regenerate the data to evaluate and re-run the same\n",
16771657
"eval to compare our results:"

0 commit comments

Comments
 (0)