Skip to content

Commit 8562c42

Browse files
committed
Add runtime tests
Signed-off-by: Louis Mandel <[email protected]>
1 parent e475470 commit 8562c42

11 files changed

+113
-16
lines changed

examples/demos/granite_io_hallucinations.pdl

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
description: GraniteIO halluncination example
1+
description: GraniteIO hallucination example
22
defs:
33
doc:
44
data:
@@ -48,6 +48,8 @@ text:
4848
- ${ doc }
4949
controls:
5050
hallucinations: true
51+
generate_inputs:
52+
temperature: 0.0
5153
modelResponse: output
5254
- "\nHallucinations:\n"
5355
- for:

examples/granite-io/granite_io_hallucinations.pdl

Lines changed: 3 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
description: GraniteIO hallucination example
12
defs:
23
doc:
34
data:
@@ -47,7 +48,8 @@ text:
4748
- ${ doc }
4849
controls:
4950
hallucinations: true
50-
citations: true
51+
generate_inputs:
52+
temperature: 0.0
5153
modelResponse: output
5254
- "\nHallucinations:\n"
5355
- for:
@@ -58,12 +60,3 @@ text:
5860
- "\nSentence: ${ hallucination.response_text }"
5961
join:
6062
with: "\n"
61-
- "\n\nCitations:\n"
62-
- for:
63-
citation: ${ output.results[0].next_message.citations }
64-
repeat:
65-
text:
66-
- "Citation: ${ citation.context_text }"
67-
- "\nSentence: ${ citation.response_text }"
68-
join:
69-
with: "\n"
Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
defs:
2+
io_proc:
3+
lang: python
4+
code: |
5+
from granite_io import make_backend, make_io_processor
6+
model_name = "granite3.2:2b"
7+
backend = make_backend("openai", { "model_name": model_name })
8+
result = make_io_processor(model_name, backend=backend)
9+
text:
10+
- "Hello!\n"
11+
- processor: ${ io_proc }
12+
parameters:
13+
generate_inputs:
14+
temperature: 0.0

examples/granite-io/granite_io_openai.pdl

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,3 +3,6 @@ text:
33
- processor:
44
model: "granite3.2:2b"
55
backend: openai
6+
parameters:
7+
generate_inputs:
8+
temperature: 0.0

examples/granite-io/granite_io_thinking.pdl

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,11 +6,13 @@ text:
66
backend: openai
77
parameters:
88
thinking: true
9+
generate_inputs:
10+
temperature: 0.0
911
modelResponse: outputs
1012
contribute: []
1113
- |
1214

1315
>> Thoughts:
1416
${ outputs.results[0].next_message.reasoning_content }
1517
>> Response:
16-
${ response }
18+
${ response }
Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
Did Faith Hill take a break from recording after releasing her second album, It Matters to Me?Yes, following the release of her second album, It Matters to Me in 1995, Faith Hill took a three-year hiatus from recording. This break was taken to rest from four years of extensive touring and to start a family with her then-partner, Tim McGraw.
2+
Hallucinations:
3+
Hallucination Risk: low
4+
Sentence: Yes, following the release of her second album, It Matters to Me in 1995, Faith Hill took a three-year hiatus from recording.
5+
Hallucination Risk: high
6+
Sentence: This break was taken to rest from four years of extensive touring and to start a family with her then-partner, Tim McGraw.
Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
Did Faith Hill take a break from recording after releasing her second album, It Matters to Me?Yes, following the release of her second album, It Matters to Me in 1995, Faith Hill took a three-year hiatus from recording. This break was taken to rest from four years of extensive touring and to start a family with her then-partner, Tim McGraw.
2+
Hallucinations:
3+
Hallucination Risk: low
4+
Sentence: Yes, following the release of her second album, It Matters to Me in 1995, Faith Hill took a three-year hiatus from recording.
5+
Hallucination Risk: high
6+
Sentence: This break was taken to rest from four years of extensive touring and to start a family with her then-partner, Tim McGraw.
Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
Hello!
2+
Hello! It seems like you've greeted me without asking a question or providing context. I'm here to help answer your questions or engage in a conversation on various topics. How can I assist you today? Let's chat about anything you're interested in, from general knowledge to specific queries. What would you like to discuss?
Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
Hello!
2+
Hello! It seems like you've greeted me without asking a question or providing context. I'm here to help answer your questions or engage in a conversation on various topics. How can I assist you today? Let's chat about anything you're interested in, from general knowledge to specific queries. What would you like to discuss?
Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
Find the fastest way for a seller to visit all the cities in their region
2+
>> Thoughts:
3+
1. **Identify the Problem**: The goal is to find the most efficient route for a seller to visit every city in their region, considering factors like distance and time. This problem can be framed as a variant of the Traveling Salesman Problem (TSP), where the objective is to minimize the total distance traveled while visiting each city exactly once and returning to the origin city.
4+
5+
2. **Data Collection**: To solve this, we need data on:
6+
- The number of cities in the region. Let's denote this as 'n'.
7+
- The geographical coordinates (latitude and longitude) for each city. These are essential for calculating distances between any two points.
8+
- A method to estimate travel times between cities based on distance or actual road networks, if available.
9+
10+
3. **Algorithm Selection**: For large datasets, exact algorithms like the Concorde TSP Solver can be computationally expensive. In such cases, approximation algorithms or heuristics are often used due to their efficiency and practicality for real-world problems:
11+
- **Nearest Neighbor Algorithm**: Start at an arbitrary city, then at each step, visit the nearest unvisited city until all cities have been visited. This is simple but may not yield the optimal solution.
12+
- **2-Opt or 3-Opt Improvements**: These are local search heuristics that improve upon a near-optimal tour by iteratively swapping segments of the route to reduce total distance. They're faster than exact algorithms for large datasets and often provide solutions close to the optimal one.
13+
14+
4. **Implementation Steps**:
15+
- Input: List of cities with their coordinates, and an initial city (let's say City 1).
16+
- Step 1: Calculate the distance matrix using Haversine formula or a similar method based on latitudes and longitudes.
17+
- Step 2: Apply the chosen heuristic algorithm to generate an initial tour.
18+
- Step 3: Refine this tour using 2-Opt or 3-Opt improvements until no further reduction in total distance is possible, indicating an optimal or near-optimal solution.
19+
20+
5. **Output**: The seller's route should be a list of cities in the order they would visit, along with estimated travel times between consecutive stops if available. This could look like:
21+
22+
```
23+
[City 1, City 2, ..., City n, City 1]
24+
Travel Time (City i to City i+1) for all i from 1 to n-1
25+
```
26+
27+
6. **Considerations**:
28+
- The solution will be a near-optimal tour due to the computational limitations of heuristics on large datasets.
29+
- If real-time traffic data is available, it could further refine travel times and potentially reduce overall journey time by optimizing for current conditions.
30+
- For very large regions or frequent trips, dynamic routing systems that recalculate routes based on real-time traffic might be more practical.
31+
>> Response:
32+
To find the fastest route for a seller to visit all cities in their region, we'll employ a combination of geographical data processing and an efficient heuristic algorithm due to the large scale of this problem. Here's how you can approach it:
33+
34+
1. **Data Preparation**:
35+
- Gather a list of 'n' cities with their respective coordinates (latitude and longitude). Let's denote these as `Cities = [C1, C2, ..., Cn]`, where each city is represented by its geographical data `(lat_i, lon_i)`.
36+
- If real-time traffic data or road network information is available for your region, incorporate this into the calculation to get more accurate travel times.
37+
38+
2. **Distance Calculation**:
39+
- Use a distance formula like the Haversine formula to compute the great-circle distance between each pair of cities:
40+
```
41+
d(C_i, C_j) = 2 * R * arcsin(√((sin(Δlat/2))^2 + cos(lat_i) * cos(lat_j) * (sin(Δlon/2))^2))
42+
```
43+
where `R` is the Earth's radius, and `Δlat` and `Δlon` are the differences in latitudes and longitudes between cities `C_i` and `C_j`.
44+
45+
3. **Tour Generation**:
46+
- Start at an arbitrary city (let’s choose City 1).
47+
- Use a nearest neighbor algorithm to iteratively select the next unvisited city with the smallest distance from the current city:
48+
```
49+
Tour = [City 1]
50+
while len(Tour) < n:
51+
i = argmin(distance[j, Tour[-1]]) for j in range(len(Cities)) if j not in Tour
52+
Tour.append(Cities[i])
53+
```
54+
55+
4. **Route Optimization**:
56+
- Apply a 2-Opt or 3-Opt heuristic to improve the tour:
57+
- Start with an initial tour generated by the nearest neighbor algorithm.
58+
- For each pair of consecutive cities (i, i+1) in the tour:
59+
- If swapping these two cities reduces total distance (`d(Tour[i], Tour[i+1]) + d(Tour[i+1], Tour[i+2]) < d(Tour[i], Tour[i+2])`), perform a 2-Opt swap.
60+
- Repeat this process until no further improvements can be made, indicating an optimal or near-optimal solution.
61+
62+
5. **Output**:
63+
- The seller's route will be a list of cities in the order they should visit: `[City 1, City 2, ..., City n]`.
64+
- Include estimated travel times between consecutive stops if available. If real-time traffic data is used, these would represent current conditions and could vary based on traffic.
65+
66+
6. **Considerations**:
67+
- For very large regions or frequent trips, consider using dynamic routing systems that recalculate routes in real-time based on current traffic conditions for more accurate travel times.
68+
- If the number of cities is extremely large (e.g., thousands), you might need to employ distributed computing techniques or specialized algorithms designed for massive TSP instances.
69+
70+
By following these steps, the seller can efficiently plan a route that minimizes total travel time while visiting all cities in their region. This approach balances computational efficiency with practicality for real-world applications.
71+

0 commit comments

Comments
 (0)