Capturing new results for Run Examples (#957)

jgchn · web-flow · commit 81dfb1a285f5 · 2025-06-04T11:28:04.000-04:00
Signed-off-by: Jing Chen &lt;jing.chen2@ibm.com&gt;
diff --git a/docs/contrib.md b/docs/contrib.md
@@ -135,5 +135,5 @@ Note: Your PR should always set `update_results: false` before merging, though t
 Here's a preview of the current configuration file for Run Examples:
 
 ```yaml
---8<-- "../tests/test_examples_run.yaml"
-```
+--8<-- "tests/test_examples_run.yaml"
+```
diff --git a/examples/demos/granite_io_hallucinations.pdl b/examples/demos/granite_io_hallucinations.pdl
@@ -1,3 +1,4 @@
+description: GraniteIO halluncination example
 defs:
   doc:
     data:
diff --git a/examples/demos/react_fun.pdl b/examples/demos/react_fun.pdl
@@ -1,4 +1,4 @@
-description: Function definition for react
+description: Function definition for react pattern
 defs:
   react_inner:
     function:
diff --git a/examples/tutorial/code_python.pdl b/examples/tutorial/code_python.pdl
@@ -1,4 +1,4 @@
-description: Hello world showing call to python code
+description: Hello world showing call to Python code
 text:
 - "Hello, "
 - lang: python
diff --git a/examples/tutorial/independent.pdl b/examples/tutorial/independent.pdl
@@ -1,4 +1,4 @@
-description: Independent example -- hello * ( (' in French'*g) + (' en Espanol'*g)) * Goodbye! * g 
+description: Independent example --- hello * ( (' in French'*g) + (' en Espanol'*g)) * Goodbye! * g 
 text:
   - hello
   - context: independent
diff --git a/examples/tutorial/independent_docs.pdl b/examples/tutorial/independent_docs.pdl
@@ -1,3 +1,4 @@
+description: Independent documents
 defs:
   doc1: "Hello world!"
   doc2: "Goodbye world!"
diff --git a/examples/tutorial/test.pdl b/examples/tutorial/test.pdl
@@ -1,3 +1,4 @@
+description: test
 defs:
   m1: 
     content: doc1
diff --git a/pdl-live-react/src-tauri/tests/cli/model-input-nested.pdl b/pdl-live-react/src-tauri/tests/cli/model-input-nested.pdl
@@ -1,3 +1,4 @@
+description: Nesting inputs to model calls
 text:
   - role: user
     content: answer as if you live in europe
diff --git a/tests/results/examples/demos/react_fun.0.result b/tests/results/examples/demos/react_fun.0.result
@@ -0,0 +1 @@
+
diff --git a/tests/results/examples/tutorial/code_python.1.result b/tests/results/examples/tutorial/code_python.1.result
@@ -0,0 +1 @@
+Hello, r
diff --git a/tests/results/examples/tutorial/independent.1.result b/tests/results/examples/tutorial/independent.1.result
@@ -0,0 +1,6 @@
+hello in French!Bonjour! (Hello!) en Espanol!¡Hola! ¿En qué puedo ayudarte hoy? (Hello! How can I assist you today?)Which language did I use last?You used Spanish in your previous message. Here's a quick summary:
+
+- Your first message was in English.
+- The second message was in Spanish.
+
+So, the languages you've used are English and Spanish.
diff --git a/tests/results/examples/tutorial/independent_docs.2.result b/tests/results/examples/tutorial/independent_docs.2.result
@@ -0,0 +1,11 @@
+Hello world!Goodbye world!Compare and contrast the above documents.The two texts provided are identical, as they both read "Hello world!" and "Goodbye world!". Since there's no difference in content or context, I can't perform a formal comparison and contrast. However, if we were to analyze them based on their purpose and tone:
+
+1. Purpose:
+   - The first text ("Hello world!") serves as an introduction or greeting. It's often used in programming contexts (like Python) to demonstrate that the code is running without errors. In a broader context, it could be a friendly "hello" message sent over digital communication channels.
+   - The second text ("Goodbye world!") functions as a farewell or sign-off. This phrase is typically used when ending a conversation or interaction, either in person or digitally.
+
+2. Tone:
+   - Hello world! carries a neutral to friendly tone, implying warmth and openness for the upcoming interaction. It's more inviting than simply stating one's presence.
+   - Goodbye world!, on the other hand, has a slightly more formal or polite tone. While it doesn't necessarily convey sadness or disconnection, it does signal that the conversation is coming to an end and may imply respect for the recipient's time or attention.
+
+In summary, while these two texts are identical in content, they differ primarily by their purpose (greeting vs farewell) and tone (neutral-friendly vs slightly formal).
diff --git a/tests/results/examples/tutorial/test.0.result b/tests/results/examples/tutorial/test.0.result
@@ -0,0 +1,25 @@
+Without specific content from doc1 and doc2, I can't provide a detailed comparison. However, I'll outline how you might compare two documents based on common elements:
+
+1. **Title**: Check if both documents have titles that accurately represent their contents.
+
+2. **Introduction/Summary**: Compare the introduction or summary sections to see if they convey similar purposes (e.g., what the document is about, its main points).
+
+3. **Content Sections**:
+   - **Headings and Subheadings**: Look at how these are used in both documents. Do they serve the same purpose? Are there any differences in structure or depth of topics covered under each heading?
+   - **Paragraphs/Sections**: Analyze the content within paragraphs or sections. Do they cover similar ground, or do they diverge into different areas?
+
+4. **Tone and Style**: Assess the tone (formal, informal, persuasive, etc.) and style (use of language, sentence structure). Are these consistent across both documents?
+
+5. **Data/Information Presentation**: If applicable, compare how data or information is presented (tables, graphs, lists, etc.). Is it formatted similarly in both documents?
+
+6. **References/Citations**: Check if citations or references are present and if they follow the same citation style (e.g., APA, MLA).
+
+7. **Conclusion/Summary**: Compare how each document concludes its main points. Do they reach similar conclusions?
+
+8. **Visual Elements**: If images, charts, or diagrams are included, assess their relevance and whether they serve the same purpose in both documents.
+
+9. **Length**: Note if one document is significantly longer or shorter than the other for a given topic.
+
+10. **Audience/Purpose**: Consider what each document aims to achieve (e.g., inform, persuade, entertain). Are these purposes similarly conveyed?
+
+To get a more precise comparison, you would need to have access to the actual content of doc1 and doc2. If you provide those documents or their summaries, I can offer a more detailed analysis.[{{'role': 'user', 'content': 'doc1', 'defsite': 'text.0.message'},{'role': 'user', 'content': 'doc2', 'defsite': 'text.1.message'}},{'role': 'user', 'content': 'Compare the docs', 'defsite': 'text.2.message'}]
diff --git a/tests/results/pdl-live-react/src-tauri/tests/cli/model-input-nested.1.result b/tests/results/pdl-live-react/src-tauri/tests/cli/model-input-nested.1.result
@@ -0,0 +1,5 @@
+{'role': 'user', 'content': 'answer as if you live in europe', 'defsite': 'text.0.message'}{'role': 'user', 'content': 'what is the fastest animal where i live?', 'defsite': 'text.1.text.0.message'}As an assistant living in Europe, I'd be happy to share that the fastest land animal native to this continent is actually the Greyhound dog breed. These dogs are renowned for their incredible speed and agility on the ground. They can reach top speeds of around 45-48 miles per hour (72-77 kilometers per hour), making them one of the fastest animals in Europe.
+
+However, if we consider wild animals, the Cheetah is the fastest land animal globally and can be found in parts of Africa, including regions near the European border with North African countries like Libya and Tunisia. Cheetahs typically reach speeds up to 60-70 miles per hour (97-113 kilometers per hour), but they are not native to Europe.
+
+For a truly European animal, the Greyhound remains the fastest land creature in our region.
diff --git a/tests/test_examples_run.yaml b/tests/test_examples_run.yaml
@@ -20,6 +20,7 @@ skip:
   - pdl-live-react/demos/demo1.pdl
   - pdl-live-react/demos/demo2.pdl
   - pdl-live-react/src-tauri/tests/cli/read-stdin.pdl
+  - examples/demos/granite_io_hallucinations.pdl
   - examples/granite-io/granite_io_hallucinations.pdl
   - examples/granite-io/granite_io_openai.pdl
   - examples/granite-io/granite_io_thinking.pdl

Original file line number	Diff line number	Diff line change
`@@ -1,3 +1,4 @@`
	`1`	`+description: GraniteIO halluncination example`
`1`	`2`	`defs:`
`2`	`3`	`doc:`
`3`	`4`	`data:`
Original file line number	Diff line number	Diff line change
`@@ -1,4 +1,4 @@`
`1`		`-description: Function definition for react`
	`1`	`+description: Function definition for react pattern`
`2`	`2`	`defs:`
`3`	`3`	`react_inner:`
`4`	`4`	`function:`
Original file line number	Diff line number	Diff line change
`@@ -1,4 +1,4 @@`
`1`		`-description: Hello world showing call to python code`
	`1`	`+description: Hello world showing call to Python code`
`2`	`2`	`text:`
`3`	`3`	`- "Hello, "`
`4`	`4`	`- lang: python`
Original file line number	Diff line number	Diff line change
`@@ -1,4 +1,4 @@`
`1`		`-description: Independent example -- hello * ( (' in French'g) + (' en Espanol'g)) * Goodbye! * g`
	`1`	`+description: Independent example --- hello * ( (' in French'g) + (' en Espanol'g)) * Goodbye! * g`
`2`	`2`	`text:`
`3`	`3`	`- hello`
`4`	`4`	`- context: independent`
Original file line number	Diff line number	Diff line change
`@@ -1,3 +1,4 @@`
	`1`	`+description: Independent documents`
`1`	`2`	`defs:`
`2`	`3`	`doc1: "Hello world!"`
`3`	`4`	`doc2: "Goodbye world!"`
Original file line number	Diff line number	Diff line change
`@@ -1,3 +1,4 @@`
	`1`	`+description: test`
`1`	`2`	`defs:`
`2`	`3`	`m1:`
`3`	`4`	`content: doc1`
Original file line number	Diff line number	Diff line change
`@@ -1,3 +1,4 @@`
	`1`	`+description: Nesting inputs to model calls`
`1`	`2`	`text:`
`2`	`3`	`- role: user`
`3`	`4`	`content: answer as if you live in europe`