|
112 | 112 | "source": [
|
113 | 113 | "## Project Lifecycle\n",
|
114 | 114 | "\n",
|
115 |
| -<<<<<<< HEAD |
116 | 115 | "Not every project will proceed in the same way, but projects generally have some \n",
|
117 | 116 | "important components in common.\n",
|
118 | 117 | "\n",
|
|
122 | 121 | "represents the ongoing nature of problem understanding - uncovering more about\n",
|
123 | 122 | "the customer domain will influence every step of the process. We wil examine \n",
|
124 | 123 | "several of these iterative cycles of refinement in detail below. \n",
|
125 |
| -======= |
126 | 124 | "Not every project will proceed in the same way, but projects generally have some common\n",
|
127 | 125 | "important components.\n",
|
128 |
| ->>>>>>> origin/main |
129 | 126 | "\n",
|
130 | 127 | "### 1. Understand the Problem\n",
|
131 | 128 | "\n",
|
|
145 | 142 | "It's very rare that a real-world project will start with all the data necessary to get\n",
|
146 | 143 | "to a satisfactory solution, much less to establish confidence.\n",
|
147 | 144 | "\n",
|
148 |
| -<<<<<<< HEAD |
149 | 145 | "In our case, we're going to assume that we have a decent sample of system *inputs*, \n",
|
150 | 146 | "in the form of but receipt images, but start without any fully annotated data. We find \n",
|
151 | 147 | "this is a not-unusual situation when automating an existing process. Instead, \n",
|
152 | 148 | "we'll walk through the process of building that out as we go along by collaborating with\n",
|
153 | 149 | "domain experts, and make our evals progressively more comprehensive.\n",
|
154 |
| -======= |
155 | 150 | "In our case, we're going to assume that we have a decent sample of system *inputs*\n",
|
156 | 151 | "(here, photographs of receipts), but start without any fully annotated data. We'll walk\n",
|
157 | 152 | "through the process of incrementally expanding our test and training sets as we go along\n",
|
158 | 153 | "and make our evals progressively more comprehensive.\n",
|
159 |
| ->>>>>>> origin/main |
160 | 154 | "\n",
|
161 | 155 | "### 3. Build an End-to-End V0 System\n",
|
162 | 156 | "\n",
|
|
414 | 408 | "cell_type": "markdown",
|
415 | 409 | "metadata": {},
|
416 | 410 | "source": [
|
417 |
| -<<<<<<< HEAD |
418 |
| - "" |
419 |
| -======= |
420 | 411 | "<img src=\"../../../images/Supplies_20240322_220858_Raven_Scan_3_jpeg.rf.50852940734939c8838819d7795e1756.jpg\" alt=\"Walmart_image\" width=\"400\"/>"
|
421 |
| ->>>>>>> origin/main |
422 | 412 | ]
|
423 | 413 | },
|
424 | 414 | {
|
|
521 | 511 | "source": [
|
522 | 512 | "### Action Decision\n",
|
523 | 513 | "\n",
|
524 |
| -<<<<<<< HEAD |
525 |
| - "Next, we need to close the loop and get to an actual decision based on receipts. \n", |
| 514 | + "Next, we need to close the loop and get to an actual decision based on receipts. This\n", |
| 515 | + "looks pretty similar, so we'll present the code without comment.\n", |
526 | 516 | "\n",
|
527 | 517 | "Ordinarily one would start with the most capable model - `o3`, at this time - for a \n",
|
528 | 518 | "first pass, and then once correctness is established experiment with different models\n",
|
|
536 | 526 | "that we get a few reasoning errors we can use to illustrate the means of addressing\n",
|
537 | 527 | "them when they occur.\n",
|
538 | 528 | "\n",
|
539 |
| - "Otherwise, this is pretty similar to the last, so we'll present the code without \n", |
540 |
| - "further comment." |
541 |
| -======= |
542 | 529 | "Next, we need to close the loop and get to an actual decision based on receipts. This\n",
|
543 | 530 | "looks pretty similar, so we'll present the code without comment."
|
544 |
| ->>>>>>> origin/main |
545 | 531 | ]
|
546 | 532 | },
|
547 | 533 | {
|
|
930 | 916 | "metadata": {},
|
931 | 917 | "source": [
|
932 | 918 | "After you run that eval you'll be able to view it in the UI, and should see something\n",
|
933 |
| -<<<<<<< HEAD |
934 | 919 | "like the below. \n",
|
935 | 920 | "\n",
|
936 | 921 | "(Note, if you have a Zero-Data-Retention agreement, this data is not stored\n",
|
937 | 922 | "by OpenAI, so will not be available in this interface.)\n",
|
938 |
| -======= |
939 | 923 | "like:\n",
|
940 |
| ->>>>>>> origin/main |
941 | 924 | "\n",
|
942 | 925 | "\n",
|
943 | 926 | "\n",
|
|
1667 | 1650 | "ARE NOT TRAVEL-RELATED, THEN IT MUST BE AUDITED.\n",
|
1668 | 1651 | "```\n",
|
1669 | 1652 | "\n",
|
1670 |
| -<<<<<<< HEAD |
1671 | 1653 | "4. We added three examples, JSON input/output pairs wrapped in XML tags.\n",
|
1672 |
| -======= |
1673 | 1654 | "3. We added three examples, JSON input/output pairs wrapped in XML tags.\n",
|
1674 |
| ->>>>>>> origin/main |
1675 | 1655 | "\n",
|
1676 | 1656 | "With our prompt revisions, we'll regenerate the data to evaluate and re-run the same\n",
|
1677 | 1657 | "eval to compare our results:"
|
|
0 commit comments