Skip to content

Commit 1d01c35

Browse files
authored
docs: azure openai quickstart (#155)
Added documentation for azure-openai as alternative quickstart shoutout to @Techneaux for helping test this
1 parent 16407ea commit 1d01c35

File tree

5 files changed

+532
-20
lines changed

5 files changed

+532
-20
lines changed

β€Ždocs/guides/llms.ipynbβ€Ž

Lines changed: 18 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,9 @@
1212
"- [Completion LLMs Supported](https://api.python.langchain.com/en/latest/api_reference.html#module-langchain.llms)\n",
1313
"- [Chat based LLMs Supported](https://api.python.langchain.com/en/latest/api_reference.html#module-langchain.chat_models)\n",
1414
"\n",
15-
"This guide will show you how to use another or LLM API for evaluation."
15+
"This guide will show you how to use another or LLM API for evaluation.\n",
16+
"\n",
17+
"> **Note**: If your looking to use Azure OpenAI for evaluation checkout [this guide](./quickstart-azure-openai.ipynb)"
1618
]
1719
},
1820
{
@@ -29,7 +31,7 @@
2931
},
3032
{
3133
"cell_type": "code",
32-
"execution_count": 5,
34+
"execution_count": null,
3335
"id": "a6d96660",
3436
"metadata": {},
3537
"outputs": [],
@@ -57,34 +59,34 @@
5759
"id": "f1fdb48b",
5860
"metadata": {},
5961
"source": [
60-
"Now initialise `Faithfulness` with `gpt4`"
62+
"Now lets replace the `llm` used in faithfulness with `gpt4`"
6163
]
6264
},
6365
{
6466
"cell_type": "code",
65-
"execution_count": 9,
67+
"execution_count": 2,
6668
"id": "307321ed",
6769
"metadata": {},
6870
"outputs": [],
6971
"source": [
70-
"from ragas.metrics import Faithfulness\n",
72+
"from ragas.metrics import faithfulness\n",
7173
"\n",
72-
"faithfulness_gpt4 = Faithfulness(name=\"faithfulness_gpt4\", llm=gpt4, batch_size=3)"
74+
"faithfulness.llm = gpt4"
7375
]
7476
},
7577
{
7678
"cell_type": "markdown",
7779
"id": "1930dd49",
7880
"metadata": {},
7981
"source": [
80-
"That's it!\n",
82+
"That's it! faithfulness will now be using GPT-4 under the hood for evaluations.\n",
8183
"\n",
8284
"Now lets run the evaluations using the example from [quickstart](../quickstart.ipnb)."
8385
]
8486
},
8587
{
8688
"cell_type": "code",
87-
"execution_count": 6,
89+
"execution_count": 3,
8890
"id": "62c0eadb",
8991
"metadata": {},
9092
"outputs": [
@@ -98,7 +100,7 @@
98100
{
99101
"data": {
100102
"application/vnd.jupyter.widget-view+json": {
101-
"model_id": "c55f09ffe1094e6190c255c09c0eb141",
103+
"model_id": "9fb581d4057d4e70a0b70830b2f5f487",
102104
"version_major": 2,
103105
"version_minor": 0
104106
},
@@ -120,7 +122,7 @@
120122
"})"
121123
]
122124
},
123-
"execution_count": 6,
125+
"execution_count": 3,
124126
"metadata": {},
125127
"output_type": "execute_result"
126128
}
@@ -135,31 +137,31 @@
135137
},
136138
{
137139
"cell_type": "code",
138-
"execution_count": 10,
140+
"execution_count": 5,
139141
"id": "c4396f6e",
140142
"metadata": {},
141143
"outputs": [
142144
{
143145
"name": "stdout",
144146
"output_type": "stream",
145147
"text": [
146-
"evaluating with [faithfulness_gpt4]\n"
148+
"evaluating with [faithfulness]\n"
147149
]
148150
},
149151
{
150152
"name": "stderr",
151153
"output_type": "stream",
152154
"text": [
153-
"100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 10/10 [15:38<00:00, 93.84s/it]\n"
155+
"100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 2/2 [22:28<00:00, 674.38s/it]\n"
154156
]
155157
},
156158
{
157159
"data": {
158160
"text/plain": [
159-
"{'faithfulness_gpt4': 0.6594}"
161+
"{'faithfulness': 0.7237}"
160162
]
161163
},
162-
"execution_count": 10,
164+
"execution_count": 5,
163165
"metadata": {},
164166
"output_type": "execute_result"
165167
}
@@ -168,7 +170,7 @@
168170
"# evaluate\n",
169171
"from ragas import evaluate\n",
170172
"\n",
171-
"result = evaluate(fiqa_eval[\"baseline\"], metrics=[faithfulness_gpt4])\n",
173+
"result = evaluate(fiqa_eval[\"baseline\"], metrics=[faithfulness])\n",
172174
"\n",
173175
"result"
174176
]

0 commit comments

Comments
Β (0)