Skip to content

Commit 897163b

Browse files
vazirimvazirim
authored
pdl notebook (#142)
Signed-off-by: Mandana Vaziri <[email protected]>
1 parent 7676870 commit 897163b

File tree

1 file changed

+392
-0
lines changed

1 file changed

+392
-0
lines changed

‎examples/notebooks/pdl.ipynb

Lines changed: 392 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,392 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"id": "50f4ce4a",
6+
"metadata": {},
7+
"source": [
8+
"# Prompt Declaration Language\n",
9+
"\n",
10+
"Prompt engineering is difficult: minor variations in prompts have large impacts on the output of LLMs and prompts are model-dependent. In recent years <i> prompt programming languages </i> have emerged to bring discipline to prompt engineering. Many of them are embedded in an imperative language such as Python or TypeScript, making it difficult for users to directly interact with prompts and multi-turn LLM interactions.\n",
11+
"\n",
12+
"The Prompt Declaration Language (PDL) is a YAML-based declarative approach to prompt programming, where prompts are at the forefront. PDL facilitates model chaining and tool use, abstracting away the plumbing necessary for such compositions, enables type checking of the input and output of models, and is based on LiteLLM to support a variety of model providers. PDL has been used with RAG, CoT, ReAct, and an agent for solving SWE-bench. PDL is [open-source](https://github.com/IBM/prompt-declaration-language) and works well with watsonx.ai and Granite models.\n",
13+
"\n",
14+
"You can use PDL stand-alone or from a Python SDK or, as shown here, in a notebook via a notebook extension. In the cell output, model-generated text is rendered in green font, and tool-generated text is rendered in purple font."
15+
]
16+
},
17+
{
18+
"cell_type": "code",
19+
"execution_count": null,
20+
"id": "bfc303da",
21+
"metadata": {},
22+
"outputs": [],
23+
"source": [
24+
"! pip install prompt-declaration-language\n",
25+
"! pip install 'prompt-declaration-language[examples]'"
26+
]
27+
},
28+
{
29+
"cell_type": "code",
30+
"execution_count": 2,
31+
"id": "e25a6874-54d9-4167-82ed-ab2f4fdc0a6f",
32+
"metadata": {},
33+
"outputs": [],
34+
"source": [
35+
"%load_ext pdl.pdl_notebook_ext"
36+
]
37+
},
38+
{
39+
"cell_type": "markdown",
40+
"id": "b2234ce9",
41+
"metadata": {},
42+
"source": [
43+
"## Model call\n",
44+
"\n",
45+
"In PDL, the user specifies step-by-step the shape of data they want to generate. In the following, the `text` construct indicates a text block containing a prompt and a model call. Implicitly, PDL builds a background conversational context (list of role/content) which is used to make model calls. Each model call uses the context built so far as its input prompt."
46+
]
47+
},
48+
{
49+
"cell_type": "code",
50+
"execution_count": 15,
51+
"id": "f3c62df1-0347-4711-acd7-3892cfd5df30",
52+
"metadata": {},
53+
"outputs": [
54+
{
55+
"name": "stdout",
56+
"output_type": "stream",
57+
"text": [
58+
"What is the meaning of life?\n",
59+
"\u001b[32mThe meaning of life is a philosophical question that has been debated by many thinkers throughout history. There is no one definitive answer, as the answer may vary depending on one's personal beliefs, values, and experiences.\n",
60+
"\u001b[0m"
61+
]
62+
}
63+
],
64+
"source": [
65+
"%%pdl --reset-context\n",
66+
"description: Model call\n",
67+
"text: \n",
68+
"- \"What is the meaning of life?\\n\"\n",
69+
"- model: replicate/ibm-granite/granite-8b-code-instruct-128k\n",
70+
" parameters:\n",
71+
" stop_sequences: \"!\"\n",
72+
" include_stop_sequence: true"
73+
]
74+
},
75+
{
76+
"cell_type": "markdown",
77+
"id": "c9d405f8",
78+
"metadata": {},
79+
"source": [
80+
"## Model chaining\n",
81+
"Model chaining can be done by simply adding to the list of models to call declaratively. Since this cell has the `%%pdl` cell magic without `--reset-context`, it executes in the context created by the previous cell."
82+
]
83+
},
84+
{
85+
"cell_type": "code",
86+
"execution_count": 17,
87+
"id": "d7149b3f",
88+
"metadata": {},
89+
"outputs": [
90+
{
91+
"name": "stdout",
92+
"output_type": "stream",
93+
"text": [
94+
"\n",
95+
"Say it like a poem\n",
96+
"\u001b[32mThe meaning of life, a question so profound,\n",
97+
"A mystery that has puzzled men and women for so long,\n",
98+
"A path that we must tread, a goal to reach,\n",
99+
"A journey that will bring us joy and pain,\n",
100+
"\n",
101+
"A road that twists and turns, a fork in the road,\u001b[0m\u001b[32m\n",
102+
"Where we must choose, which way to go,\n",
103+
"A decision that we must make, with our souls at stake,\n",
104+
"A choice that will shape our destiny,\n",
105+
"\n",
106+
"The meaning of life, a question so grand,\n",
107+
"A goal that we must strive for, to find,\n",
108+
"A purpose that gives our hearts meaning,\n",
109+
"A reason to live, a\u001b[0m\u001b[32m reason to die,\n",
110+
"\n",
111+
"A journey that will take us far, a journey that will bring,\n",
112+
"A new understanding of the world we live in,\n",
113+
"A new perspective on life, a new way of thinking,\n",
114+
"A new path to follow, a new way to live,\n",
115+
"\n",
116+
"The meaning of life, a question so deep,\n",
117+
"A mystery that will never be solved,\n",
118+
"A journey that will\u001b[0m\u001b[32m take us far, a journey that will bring,\n",
119+
"A new understanding of the world we live in,\n",
120+
"\n",
121+
"A road that twists and turns, a fork in the road,\n",
122+
"Where we must choose, which way to go,\n",
123+
"A decision that we must make, with our souls at stake,\n",
124+
"A choice that will shape our destiny\u001b[0m\u001b[32m,\n",
125+
"\n",
126+
"The meaning of life, a question so grand,\n",
127+
"A goal that we must strive for, to find,\n",
128+
"A purpose that gives our hearts meaning,\n",
129+
"A reason to live, a reason to die,\n",
130+
"\n",
131+
"A journey that will take us far, a journey that will bring,\n",
132+
"A new understanding of the world we live in,\n",
133+
"A new perspective on life, a new way of thinking,\n",
134+
"A new\u001b[0m\u001b[32m path to follow, a new way to live,\n",
135+
"\n",
136+
"The meaning of life, a question so deep,\n",
137+
"A mystery that will never be solved,\n",
138+
"A journey that will take us far, a journey that will bring,\n",
139+
"A new understanding of the world we live in,\n",
140+
"\n",
141+
"A road that twists and turns, a fork in the road,\n",
142+
"Where we must choose, which way to go\u001b[0m\u001b[32m,\n",
143+
"A decision that we must make, with our souls at stake,\n",
144+
"A choice that will shape our destiny,\n",
145+
"\n",
146+
"The meaning of life, a question\u001b[0m\n",
147+
"\n",
148+
"What is the most important verse in this poem?\n",
149+
"\u001b[32mThe most important verse in this poem is the first one: \"The meaning of life, a question so profound.\" This line sets the tone for the entire poem and emphasizes the central theme of the question of what gives life meaning. It also highlights the idea that the answer to this question is not straightforward\u001b[0m\u001b[32m and may vary depending on one's personal beliefs and experiences.\n",
150+
"\u001b[0m"
151+
]
152+
}
153+
],
154+
"source": [
155+
"%%pdl\n",
156+
"text:\n",
157+
"- \"\\nSay it like a poem\\n\"\n",
158+
"- model: replicate/ibm-granite/granite-8b-code-instruct-128k\n",
159+
"- \"\\n\\nWhat is the most important verse in this poem?\\n\"\n",
160+
"- model: replicate/ibm-granite/granite-8b-code-instruct-128k"
161+
]
162+
},
163+
{
164+
"cell_type": "markdown",
165+
"id": "86d5a0e1-606e-400a-90ac-6aa650e2eb1e",
166+
"metadata": {},
167+
"source": [
168+
"## Chat templates\n",
169+
"\n",
170+
"The following example shows a full-fledged chatbot. In PDL roles are high level annotations and PDL takes care of applying the appropriate chat templates. This example illustrates the use of control structures such as the repeat-until block and reading from files or stdin with the read block. The chatbot repeatedly prompts the user for a query, which it submits to a model, and stops when the query is quit."
171+
]
172+
},
173+
{
174+
"cell_type": "code",
175+
"execution_count": 18,
176+
"id": "455b2dbc-69fb-4164-9b8b-5817b3f33e9b",
177+
"metadata": {},
178+
"outputs": [
179+
{
180+
"name": "stdout",
181+
"output_type": "stream",
182+
"text": [
183+
"You are Granite, an AI language model developed by IBM in 2024. You are a cautious assistant. You carefully follow instructions. You are helpful and harmless and you follow ethical guidelines and promote positive behavior.Type `quit` to exit this chatbot.\n"
184+
]
185+
},
186+
{
187+
"name": "stdin",
188+
"output_type": "stream",
189+
"text": [
190+
">>> What is APR?\n"
191+
]
192+
},
193+
{
194+
"name": "stdout",
195+
"output_type": "stream",
196+
"text": [
197+
"\u001b[32mAPR stands for Annual Percentage Rate. It is a measure of the total cost of borrowing money, including interest and fees, expressed as a yearly rate. It is commonly used in the lending industry to compare the cost of different loans and credit products.\n",
198+
"\u001b[0m\n",
199+
"\n"
200+
]
201+
},
202+
{
203+
"name": "stdin",
204+
"output_type": "stream",
205+
"text": [
206+
">>> Say it like I'm 5 years old\n"
207+
]
208+
},
209+
{
210+
"name": "stdout",
211+
"output_type": "stream",
212+
"text": [
213+
"\u001b[32mThe meaning of life is like a big, big, big question mark. It's a question that has been asked for as long as people can remember, and it's still a question that people don't always know the answer to. Some people think the answer is to have fun and make friends, while others think the answer is to work hard\u001b[0m\u001b[32m and be smart. But no matter what the answer is, the question of what gives life meaning is a question that will always be with us.\n",
214+
"\u001b[0m\n",
215+
"\n"
216+
]
217+
},
218+
{
219+
"name": "stdin",
220+
"output_type": "stream",
221+
"text": [
222+
">>> quit\n"
223+
]
224+
},
225+
{
226+
"name": "stdout",
227+
"output_type": "stream",
228+
"text": [
229+
"\u001b[32mThank you for chatting with me! If you have any more questions or need further assistance, feel free to ask.\n",
230+
"\u001b[0m"
231+
]
232+
}
233+
],
234+
"source": [
235+
"%%pdl\n",
236+
"text:\n",
237+
"- role: system\n",
238+
" content: You are Granite, an AI language model developed by IBM in 2024. You are a cautious assistant. You carefully follow instructions. You are helpful and harmless and you follow ethical guidelines and promote positive behavior.\n",
239+
"- \"Type `quit` to exit this chatbot.\\n\"\n",
240+
"- repeat:\n",
241+
" text:\n",
242+
" - read:\n",
243+
" message: \">>> \"\n",
244+
" def: query\n",
245+
" contribute: [context]\n",
246+
" - model: replicate/ibm-granite/granite-8b-code-instruct-128k\n",
247+
" until: ${ query == 'quit'}\n",
248+
" join:\n",
249+
" with: \"\\n\\n\"\n",
250+
"role: user\n"
251+
]
252+
},
253+
{
254+
"cell_type": "markdown",
255+
"id": "7e1bc1a2",
256+
"metadata": {},
257+
"source": [
258+
"## Chat templates\n",
259+
"\n",
260+
"The first call to the model in the above program submits the following prompt. PDL takes care of applying the appropriate chat templates and tags, and builds the background context implicitly. Chat templates make your program easier to port across models, since you do not need to specify control tokens by hand. All the user has to do is list the models they want to chain, PDL takes care of the rest.\n",
261+
"\n",
262+
"```\n",
263+
"<|start_of_role|>system<|end_of_role|>You are Granite, an AI language model developed by IBM in 2024. You are a cautious assistant. You carefully follow instructions. You are helpful and harmless and you follow ethical guidelines and promote positive behavior.<|end_of_text|>\n",
264+
"<|start_of_role|>user<|end_of_role|>Type `quit` to exit this chatbot.\n",
265+
"What is APR?<|end_of_text|><|start_of_role|>assistant<|end_of_role|>\n",
266+
"```"
267+
]
268+
},
269+
{
270+
"cell_type": "markdown",
271+
"id": "cfa00cfc",
272+
"metadata": {},
273+
"source": [
274+
"## Data pipeline\n",
275+
"\n",
276+
"The following program shows a common prompting pattern: read some data, formulate a prompt using that data, submit to a model, and evaluate. In this program, we formulate a prompt for code explanation. The program first defines two variables: `code`, which holds the data we read, and `truth` for the ground truth. It then prints out the source code, formulates a prompts with the data, and calls a model to get an explanation. Finally, a Python code block uses the Levenshtein text distance metric and evaluate the explanation against the ground truth. This pipeline can similarly be applied to an entire data set to produce a jsonl file."
277+
]
278+
},
279+
{
280+
"cell_type": "code",
281+
"execution_count": 19,
282+
"id": "7f6c323b-ad1a-4434-8732-bc19c5c47883",
283+
"metadata": {},
284+
"outputs": [
285+
{
286+
"name": "stdout",
287+
"output_type": "stream",
288+
"text": [
289+
"\n",
290+
"@SuppressWarnings(\"unchecked\")\n",
291+
"public static Map<String, String> deserializeOffsetMap(String lastSourceOffset) throws IOException {\n",
292+
" Map<String, String> offsetMap;\n",
293+
" if (lastSourceOffset == null || lastSourceOffset.isEmpty()) { \n",
294+
" offsetMap = new HashMap<>(); \n",
295+
" } else {\n",
296+
" offsetMap = JSON_MAPPER.readValue(lastSourceOffset, Map.class); \n",
297+
" }\n",
298+
" return offsetMap;\n",
299+
"}\n",
300+
"\n",
301+
"\u001b[32mThe function `deserializeOffsetMap` is a method that takes a string `lastSourceOffset` as input and returns a `Map` of `String` keys and `String` values. The function is used to deserialize a JSON string into a `Map` object.\n",
302+
" \n",
303+
"\n",
304+
" The function first checks if the `lastSourceOffset` is null or empty.\u001b[0m\u001b[32m If it is, it creates a new `HashMap` object and assigns it to the `offsetMap` variable. If the `lastSourceOffset` is not null or empty, the function uses the `JSON_MAPPER` object to read the JSON string and deserialize it into a `Map` object. The `JSON_MAPPER` object is assumed to be a pre-defined\u001b[0m\u001b[32m object that is used for JSON serialization and deserialization.\n",
305+
" \n",
306+
"\n",
307+
" Finally, the function returns the `offsetMap` object.\n",
308+
"\u001b[0m\n",
309+
"\n",
310+
"EVALUATION:\n",
311+
"The similarity (Levenshtein) between this answer and the ground truth is:\n",
312+
"\u001b[35m0.31163434903047094\u001b[0m"
313+
]
314+
}
315+
],
316+
"source": [
317+
"%%pdl --reset-context\n",
318+
"description: Code explanation example\n",
319+
"defs:\n",
320+
" CODE:\n",
321+
" read: ./data.yaml\n",
322+
" parser: yaml\n",
323+
" TRUTH:\n",
324+
" read: ./ground_truth.txt\n",
325+
"text:\n",
326+
"- \"\\n${ CODE.source_code }\\n\"\n",
327+
"- model: replicate/ibm-granite/granite-8b-code-instruct-128k\n",
328+
" def: EXPLANATION\n",
329+
" input: |\n",
330+
" Here is some info about the location of the function in the repo.\n",
331+
" repo: \n",
332+
" ${ CODE.repo_info.repo }\n",
333+
" path: ${ CODE.repo_info.path }\n",
334+
" Function_name: ${ CODE.repo_info.function_name }\n",
335+
"\n",
336+
"\n",
337+
" Explain the following code:\n",
338+
" ```\n",
339+
" ${ CODE.source_code }```\n",
340+
"- |\n",
341+
"\n",
342+
"\n",
343+
" EVALUATION:\n",
344+
" The similarity (Levenshtein) between this answer and the ground truth is:\n",
345+
"- def: EVAL\n",
346+
" lang: python\n",
347+
" code: |\n",
348+
" import textdistance\n",
349+
" expl = \"\"\"\n",
350+
" ${ EXPLANATION }\n",
351+
" \"\"\"\n",
352+
" truth = \"\"\"\n",
353+
" ${ TRUTH }\n",
354+
" \"\"\"\n",
355+
" result = textdistance.levenshtein.normalized_similarity(expl, truth)"
356+
]
357+
},
358+
{
359+
"cell_type": "markdown",
360+
"id": "61c40266",
361+
"metadata": {},
362+
"source": [
363+
"## Conclusion\n",
364+
"\n",
365+
"Since prompts are at the forefront, PDL makes users more productive in their trial-and-error with LLMs. Try it!\n",
366+
"\n",
367+
"https://github.com/IBM/prompt-declaration-language"
368+
]
369+
}
370+
],
371+
"metadata": {
372+
"kernelspec": {
373+
"display_name": "Python 3 (ipykernel)",
374+
"language": "python",
375+
"name": "python3"
376+
},
377+
"language_info": {
378+
"codemirror_mode": {
379+
"name": "ipython",
380+
"version": 3
381+
},
382+
"file_extension": ".py",
383+
"mimetype": "text/x-python",
384+
"name": "python",
385+
"nbconvert_exporter": "python",
386+
"pygments_lexer": "ipython3",
387+
"version": "3.12.5"
388+
}
389+
},
390+
"nbformat": 4,
391+
"nbformat_minor": 5
392+
}

0 commit comments

Comments
 (0)