tests: disable non-deterministic granite-io tests (#983)

Signed-off-by: Louis Mandel <[email protected]>

Author: mandel

examples/prompt_library/gsm8k_cot.pdl

@@ -2,7 +2,7 @@ description: Demo of CoT pattern
 defs:
   cot:
     import: ../../contrib/prompt_library/CoT
-  model: watsonx_text/ibm/granite-3-8b-instruct
+  model: ollama/granite3.2:8b
   demonstrations:
     data:
       - question: |-

examples/prompt_library/gsm8k_react.pdl

@@ -4,7 +4,7 @@ defs:
     import: ../../contrib/prompt_library/ReAct
   tools:
     import: ../../contrib/prompt_library/tools
-  model: watsonx_text/ibm/granite-3-8b-instruct
+  model: ollama/granite3.2:8b
   demonstrations:
     data:
       - - question: Noah charges $60 for a large painting and $30 for a small painting.  Last month he sold eight large paintings and four small paintings.  If he sold twice as much this month, how much is his sales for this month?

examples/prompt_library/gsm8k_rewoo.pdl

@@ -4,7 +4,7 @@ defs:
     import: ../../contrib/prompt_library/ReWoo
   tools:
     import: ../../contrib/prompt_library/tools
-  model: watsonx_text/ibm/granite-3-8b-instruct
+  model: ollama/granite3.2:8b
   demonstrations:
     data:
       - - question: Noah charges $60 for a large painting and $30 for a small painting. Last month he sold eight large paintings and four small paintings. If he sold twice as much this month, how much is his sales for this month?

tests/results/examples/prompt_library/gsm8k_cot.1.result

@@ -0,0 +1,3 @@
+Answer the questions to the best of your abilities.
+
+Result: {'answer': "Jacob earns $6 per hour. \n\nJake earns thrice what Jacob does, so Jake earns 3 * $6 = $18 per hour.\n\nJake works 8 hours a day for 5 days, so he works a total of 8 * 5 = 40 hours in those 5 days.\n\nTherefore, Jake's earnings for those 5 days are 40 * $18 = $720.\n\nThe answer is $720."}

tests/results/examples/prompt_library/gsm8k_react.1.result

@@ -0,0 +1 @@
+Result: {'answer': 'Jake earns $720 in 5 days working 8 hours a day.'}

tests/results/examples/prompt_library/gsm8k_rewoo.1.result

@@ -0,0 +1 @@
+Result: {'answer': "Jake earns $18 per hour (3 times Jacob's $6 per hour). In a day, working 8 hours, Jake earns $144 ($18/hour * 8 hours). Over 5 days, Jake earns $720 ($144/day * 5 days)."}

tests/results/examples/prompt_library/gsm8k_rewoo.2.result

@@ -0,0 +1 @@
+Result: {'answer': '720 dollars.'}

tests/results/examples/react/react_call.3.result

@@ -0,0 +1,18 @@
+ How many years ago was the discoverer of the Hudson River born? Keep in mind we are in 2025. When searching avoid using the words discovery or birthday.
+
+To find out how many years ago Henry Hudson was born without using the words 'discovery' or 'birthday', I will use the Search tool. 
+
+Action:
+[{"name": "Search", "arguments": {"topic": "Henry Hudson"}}]
+Observation: Henry Hudson (c. 1565 – disappeared 23 June 1611) was an English sea explorer and navigator during the early 17th century, best known for his explorations of present-day Canada and parts of the Northeastern United States.
+In 1607 and 1608, Hudson made two attempts on behalf of English merchants to find a rumoured Northeast Passage to Cathay via a route above the Arctic Circle. In 1609, he landed in North America on behalf of the Dutch East India Company and explored the region around the modern New York metropolitan area. Looking for a Northwest Passage to Asia on his ship Halve Maen ("Half Moon"), he sailed up the Hudson River, which was later named after him, and thereby laid the foundation for Dutch colonization of the region. His contributions to the exploration of the New World were significant and lasting. His voyages helped to establish European contact with the native peoples of North America and contributed to the development of trade and commerce. 
+On his final expedition, while still searching for the Northwest Passage, Hudson became the first European to see Hudson Strait and the immense Hudson Bay. In 1611, after wintering on the shore of James Bay, Hudson wanted to press on to the west, but most of his crew mutinied. The mutineers cast Hudson, his son, and six others adrift; what then happened to the Hudsons and their companions is unknown.
+According to this information, Henry Hudson was born around 1565. As we are in 2025, we can calculate how many years ago that was by subtracting 1565 from 2025. 
+
+Action:
+[{"name": "Calc", "arguments": {"expr": "2025 - 1565"}}]
+Observation: 460
+Henry Hudson was born approximately 460 years ago, as calculated from the year 2025.
+
+Action:
+[{"name": "Finish", "arguments": {}}]

tests/test_examples_run.yaml

@@ -20,6 +20,9 @@ skip:
   - pdl-live-react/demos/demo2.pdl
   - pdl-live-react/src-tauri/tests/cli/read-stdin.pdl
   - examples/granite-io/granite_io_transformers.pdl
+  - examples/granite-io/granite_io_thinking.pdl
+  - examples/granite-io/granite_io_object.pdl
+  - examples/granite-io/granite_io_hallucinations.pdl
   - examples/optimizer/mbpp.pdl
   - examples/optimizer/fever.pdl
   - examples/optimizer/gsm8k.pdl