You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: notebooks/tutorials/intro_to_guidance.ipynb
+19Lines changed: 19 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -581,6 +581,25 @@
581
581
"gpt35 + experts(query='What is the meaning of life?')"
582
582
]
583
583
},
584
+
{
585
+
"cell_type": "markdown",
586
+
"metadata": {},
587
+
"source": [
588
+
"## Streaming\n",
589
+
"\n",
590
+
"Often you want to get the results of a generation as it is happening so you update an interface. You can do this programmatically using the `.stream()` method of model objects. This creates a `ModelStream` that you can use to accumulate updates. These updates don't get executed until you interate over then `ModelStream` object. When you iterate over the object you get lots of partially completed model objects as the guidance program is executed."
591
+
]
592
+
},
593
+
{
594
+
"cell_type": "code",
595
+
"execution_count": null,
596
+
"metadata": {},
597
+
"outputs": [],
598
+
"source": [
599
+
"for part in llama2.stream() + qa_bot(query):\n",
600
+
" part # do something with the partially executed lm"
0 commit comments