Skip to content

Commit b0f0d8a

Browse files
jgchnjgchn
committed
Halluncination demo wip
Signed-off-by: Jing Chen <[email protected]>
1 parent c73078a commit b0f0d8a

File tree

1 file changed

+166
-154
lines changed

1 file changed

+166
-154
lines changed

‎examples/intrinsics/demo-hallucination.pdl‎

Lines changed: 166 additions & 154 deletions
Original file line numberDiff line numberDiff line change
@@ -6,160 +6,172 @@
66
# The model responds to a query and provides a hallucination score
77

88
text:
9-
109
# Builds granite3-rag:8b
10+
# - contribute: [context]
11+
# lang: command
12+
# code: echo "Hello"
1113
- contribute: []
12-
lang: python
14+
lang: command
1315
code: |
14-
import os
15-
import shutil
16-
import subprocess
17-
import huggingface_hub
18-
import ollama
19-
20-
granite3_rag_model = 'granite3-rag:8b'
21-
22-
# Check if rag model exists
23-
found = False
24-
result = ""
25-
local_ollama_models = ollama.list().models
26-
for model in local_ollama_models:
27-
if model.model == granite3_rag_model:
28-
found = True
29-
result = f"Model {granite3_rag_model} found in Ollama, skipping build"
30-
break
31-
32-
if not found:
33-
# Install dependencies
34-
# Define the repository and directory
35-
llama_cpp_repo = "https://github.com/ggerganov/llama.cpp.git"
36-
llama_cpp_dir_name = "llama.cpp"
37-
38-
# Clone the repository if it doesn't already exist
39-
if not os.path.isdir(llama_cpp_dir_name):
40-
subprocess.run(["git", "clone", "--depth", "1", llama_cpp_repo], check=True)
41-
42-
# Install the required packages
43-
subprocess.run([
44-
"pip", "install",
45-
"git+https://github.com/ibm-granite-community/utils.git",
46-
"huggingface_hub",
47-
"langchain_community",
48-
"langchain_ollama",
49-
"langchain-milvus",
50-
"docling",
51-
"-r", f"{llama_cpp_dir_name}/requirements.txt"
52-
], check=True)
53-
54-
# Ollama pull base model
55-
granite3_base_model = 'granite3-dense:8b'
56-
ollama.pull(granite3_base_model)
57-
58-
# HG: download lora adapter
59-
lora_folder = huggingface_hub.snapshot_download(repo_id="ibm-granite/granite-rag-3.0-8b-lora")
60-
base_folder = huggingface_hub.snapshot_download(repo_id="ibm-granite/granite-3.0-8b-instruct", allow_patterns="*.json")
61-
lora_gguf = "granite-rag-3.0-8b-lora-fp16.gguf"
62-
subprocess.run(f"python3 llama.cpp/convert_lora_to_gguf.py --outtype f16 --outfile {lora_gguf} --base {base_folder} -- {lora_folder}",
63-
shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
64-
65-
# Write Modelfile for granite-rag:8b for Ollama
66-
model_file = "Modelfile"
67-
with open(model_file, "w") as modelfile:
68-
modelfile.write(f"""\
69-
FROM {granite3_base_model}
70-
ADAPTER {lora_gguf}
71-
""")
72-
73-
# Ideally we should use ollama.create() but there is no option to pass a modelfile
74-
subprocess.run(f"ollama create {granite3_rag_model} -f Modelfile",
75-
shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
76-
77-
# Rm dependencies
78-
os.remove(model_file)
79-
if os.path.isdir(llama_cpp_dir_name):
80-
shutil.rmtree(llama_cpp_dir_name)
81-
82-
result = f"Model {granite3_rag_model} not found on Ollama. Finished building {granite3_rag_model}"
83-
84-
- role: system
85-
contribute: [context]
86-
text:
87-
data:
88-
instruction: |
89-
Respond to the user's latest question based solely on the information provided
90-
in the documents. Ensure that your response is strictly aligned with the facts
91-
in the provided documents. If the information needed to answer the question is
92-
not available in the documents, inform the user that the question cannot be
93-
answered based on the available data. Make sure that your response follows
94-
the attributes mentioned in the 'meta' field.
95-
documents:
96-
- doc_id: 1
97-
text: |
98-
Audrey Faith McGraw (born September 21, 1967) is an American singer
99-
and record producer. She is one of the most successful country artists
100-
of all time, having sold more than 40 million albums worldwide. Hill is
101-
married to American singer Tim McGraw, with whom she has recorded several duets.
102-
Hill's first two albums, Take Me as I Am (1993) and It Matters to Me (1995),
103-
were major successes and placed a combined three number ones on Billboard's
104-
country charts. Hill's debut album was Take Me as I Am (1993); sales were strong,
105-
buoyed by the chart success of "Wild One". Hill became the first female country
106-
singer in 30 years to hold Billboard's number one position for four consecutive
107-
weeks when "Wild One" managed the feat in 1994. Her version of "Piece of My Heart",
108-
also went to the top of the country charts in 1994. The album sold a total of
109-
3 million copies. Other singles from the album include "Take Me as I Am". The recording
110-
of Faith's second album was delayed by surgery to repair a ruptured blood vessel on
111-
her vocal cords. It Matters to Me finally appeared in 1995 and was another
112-
success, with the title track becoming her third number-one country single.
113-
Several other top 10 singles followed, and more than 3 million copies of the
114-
album were sold. The fifth single from the album, "I Can't Do That Anymore",
115-
was written by country music artist Alan Jackson. Other singles from the album
116-
include "You Can't Lose Me", "Someone Else's Dream", and "Let's Go to Vegas".
117-
During this period, Hill appeared on the acclaimed PBS music program Austin City Limits.
118-
In spring 1996, Hill began the Spontaneous Combustion Tour with country singer Tim McGraw.
119-
At that time, Hill had recently become engaged to her former producer, Scott Hendricks,
120-
and McGraw had recently broken an engagement. McGraw and Hill were quickly
121-
attracted to each other and began an affair. After discovering that Hill was
122-
pregnant with their first child, the couple married on October 6, 1996. The
123-
couple have three daughters together: Gracie Katherine (born 1997), Maggie Elizabeth (born 1998)
124-
and Audrey Caroline (born 2001). Since their marriage, Hill and McGraw have endeavored
125-
never to be apart for more than three consecutive days. After the release of It Matters to Me,
126-
Hill took a three-year break from recording to give herself a rest from four years of touring
127-
and to begin a family with McGraw. During her break, she joined forces with her husband
128-
for their first duet, "It's Your Love". The song stayed at number one for six weeks,
129-
and won awards from both the Academy of Country Music and the Country Music Association.
130-
Hill has remarked that sometimes when they perform the song together,
131-
"it [doesn't] feel like anybody else was really watching."
132-
meta:
133-
hallucination_tags: true
134-
citations: true
135-
136-
# User Query
137-
# (This query produces a hallucination "low" with citation)
138-
- Did Faith Hill take a break from recording after releasing her second album, It Matters to Me?
139-
# (This query produces a hallucination "unanswerable" with no citation)
140-
# - Is the Academy of Country Music in Brooklyn, New York?
141-
# (This query produces a hallucination "high" with a citation)
142-
# - Where was Faith Hill born?
143-
144-
# Base model granite3-dense:8b setup with Granite RAG LoRA (Low-Rank Adaption) on ollama.
145-
- defs:
146-
# Store the results of making the LLM invocation in a JSON variable named 'out'
147-
out:
148-
model: ollama/granite3-rag:8b
149-
parameters:
150-
temperature: 0
151-
parser: json
152-
- |
153-
154-
155-
The answer is: ${ out[0].sentence }
156-
- match: ${out[0].meta.hallucination_level}
157-
with:
158-
- case: "high"
159-
then: Totally hallucinating, sorry!
160-
- case: "low"
161-
if: ${ out[0].meta.citation }
162-
then: |
163-
I am not hallucinating, promise!
164-
The citation is: ${ out[0].meta.citation.snippet }
165-
- then: Not sure if I am hallucinating...
16+
[ -d "llama.cpp/.git" ] || git clone https://github.com/ggerganov/llama.cpp.git llama.cpp
17+
# git -C "llama.cpp" pull || git clone https://github.com/ggerganov/llama.cpp.git "llama.cpp"
18+
# [ ! -d 'llama.cpp' ] && git clone https://github.com/ggerganov/llama.cpp.git
19+
# mkdir -p llama.cpp && [[ ! -d "llama.cpp" ]] && git clone --depth 1 https://github.com/ggerganov/llama.cpp.git llama.cpp
20+
# [[ ! -d "llama.cpp" ]] && git clone --depth 1 https://github.com/ggerganov/llama.cpp.git
21+
22+
# # Builds granite3-rag:8b
23+
# - contribute: []
24+
# lang: python
25+
# code: |
26+
# import os
27+
# import shutil
28+
# import subprocess
29+
# import huggingface_hub
30+
# import ollama
31+
32+
# granite3_rag_model = 'granite3-rag:8b'
33+
34+
# # Check if rag model exists
35+
# found = False
36+
# result = ""
37+
# local_ollama_models = ollama.list().models
38+
# for model in local_ollama_models:
39+
# if model.model == granite3_rag_model:
40+
# found = True
41+
# result = f"Model {granite3_rag_model} found in Ollama, skipping build"
42+
# break
43+
44+
# if not found:
45+
# # Install dependencies
46+
# # Define the repository and directory
47+
# llama_cpp_repo = "https://github.com/ggerganov/llama.cpp.git"
48+
# llama_cpp_dir_name = "llama.cpp"
49+
50+
# # Clone the repository if it doesn't already exist
51+
# if not os.path.isdir(llama_cpp_dir_name):
52+
# subprocess.run(["git", "clone", "--depth", "1", llama_cpp_repo], check=True)
53+
54+
# # Install the required packages
55+
# subprocess.run([
56+
# "pip", "install",
57+
# "git+https://github.com/ibm-granite-community/utils.git",
58+
# "huggingface_hub",
59+
# "langchain_community",
60+
# "langchain_ollama",
61+
# "langchain-milvus",
62+
# "docling",
63+
# "-r", f"{llama_cpp_dir_name}/requirements.txt"
64+
# ], check=True)
65+
66+
# # Ollama pull base model
67+
# granite3_base_model = 'granite3-dense:8b'
68+
# ollama.pull(granite3_base_model)
69+
70+
# # HG: download lora adapter
71+
# lora_folder = huggingface_hub.snapshot_download(repo_id="ibm-granite/granite-rag-3.0-8b-lora")
72+
# base_folder = huggingface_hub.snapshot_download(repo_id="ibm-granite/granite-3.0-8b-instruct", allow_patterns="*.json")
73+
# lora_gguf = "granite-rag-3.0-8b-lora-fp16.gguf"
74+
# subprocess.run(f"python3 llama.cpp/convert_lora_to_gguf.py --outtype f16 --outfile {lora_gguf} --base {base_folder} -- {lora_folder}",
75+
# shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
76+
77+
# # Write Modelfile for granite-rag:8b for Ollama
78+
# model_file = "Modelfile"
79+
# with open(model_file, "w") as modelfile:
80+
# modelfile.write(f"""\
81+
# FROM {granite3_base_model}
82+
# ADAPTER {lora_gguf}
83+
# """)
84+
85+
# # Ideally we should use ollama.create() but there is no option to pass a modelfile
86+
# subprocess.run(f"ollama create {granite3_rag_model} -f Modelfile",
87+
# shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
88+
89+
# # Rm dependencies
90+
# os.remove(model_file)
91+
# if os.path.isdir(llama_cpp_dir_name):
92+
# shutil.rmtree(llama_cpp_dir_name)
93+
94+
# result = f"Model {granite3_rag_model} not found on Ollama. Finished building {granite3_rag_model}"
95+
96+
# - role: system
97+
# contribute: [context]
98+
# text:
99+
# data:
100+
# instruction: |
101+
# Respond to the user's latest question based solely on the information provided
102+
# in the documents. Ensure that your response is strictly aligned with the facts
103+
# in the provided documents. If the information needed to answer the question is
104+
# not available in the documents, inform the user that the question cannot be
105+
# answered based on the available data. Make sure that your response follows
106+
# the attributes mentioned in the 'meta' field.
107+
# documents:
108+
# - doc_id: 1
109+
# text: |
110+
# Audrey Faith McGraw (born September 21, 1967) is an American singer
111+
# and record producer. She is one of the most successful country artists
112+
# of all time, having sold more than 40 million albums worldwide. Hill is
113+
# married to American singer Tim McGraw, with whom she has recorded several duets.
114+
# Hill's first two albums, Take Me as I Am (1993) and It Matters to Me (1995),
115+
# were major successes and placed a combined three number ones on Billboard's
116+
# country charts. Hill's debut album was Take Me as I Am (1993); sales were strong,
117+
# buoyed by the chart success of "Wild One". Hill became the first female country
118+
# singer in 30 years to hold Billboard's number one position for four consecutive
119+
# weeks when "Wild One" managed the feat in 1994. Her version of "Piece of My Heart",
120+
# also went to the top of the country charts in 1994. The album sold a total of
121+
# 3 million copies. Other singles from the album include "Take Me as I Am". The recording
122+
# of Faith's second album was delayed by surgery to repair a ruptured blood vessel on
123+
# her vocal cords. It Matters to Me finally appeared in 1995 and was another
124+
# success, with the title track becoming her third number-one country single.
125+
# Several other top 10 singles followed, and more than 3 million copies of the
126+
# album were sold. The fifth single from the album, "I Can't Do That Anymore",
127+
# was written by country music artist Alan Jackson. Other singles from the album
128+
# include "You Can't Lose Me", "Someone Else's Dream", and "Let's Go to Vegas".
129+
# During this period, Hill appeared on the acclaimed PBS music program Austin City Limits.
130+
# In spring 1996, Hill began the Spontaneous Combustion Tour with country singer Tim McGraw.
131+
# At that time, Hill had recently become engaged to her former producer, Scott Hendricks,
132+
# and McGraw had recently broken an engagement. McGraw and Hill were quickly
133+
# attracted to each other and began an affair. After discovering that Hill was
134+
# pregnant with their first child, the couple married on October 6, 1996. The
135+
# couple have three daughters together: Gracie Katherine (born 1997), Maggie Elizabeth (born 1998)
136+
# and Audrey Caroline (born 2001). Since their marriage, Hill and McGraw have endeavored
137+
# never to be apart for more than three consecutive days. After the release of It Matters to Me,
138+
# Hill took a three-year break from recording to give herself a rest from four years of touring
139+
# and to begin a family with McGraw. During her break, she joined forces with her husband
140+
# for their first duet, "It's Your Love". The song stayed at number one for six weeks,
141+
# and won awards from both the Academy of Country Music and the Country Music Association.
142+
# Hill has remarked that sometimes when they perform the song together,
143+
# "it [doesn't] feel like anybody else was really watching."
144+
# meta:
145+
# hallucination_tags: true
146+
# citations: true
147+
148+
# # User Query
149+
# # (This query produces a hallucination "low" with citation)
150+
# - Did Faith Hill take a break from recording after releasing her second album, It Matters to Me?
151+
# # (This query produces a hallucination "unanswerable" with no citation)
152+
# # - Is the Academy of Country Music in Brooklyn, New York?
153+
# # (This query produces a hallucination "high" with a citation)
154+
# # - Where was Faith Hill born?
155+
156+
# # Base model granite3-dense:8b setup with Granite RAG LoRA (Low-Rank Adaption) on ollama.
157+
# - defs:
158+
# # Store the results of making the LLM invocation in a JSON variable named 'out'
159+
# out:
160+
# model: ollama/granite3-rag:8b
161+
# parameters:
162+
# temperature: 0
163+
# parser: json
164+
# - |
165+
166+
167+
# The answer is: ${ out[0].sentence }
168+
# - match: ${out[0].meta.hallucination_level}
169+
# with:
170+
# - case: "high"
171+
# then: Totally hallucinating, sorry!
172+
# - case: "low"
173+
# if: ${ out[0].meta.citation }
174+
# then: |
175+
# I am not hallucinating, promise!
176+
# The citation is: ${ out[0].meta.citation.snippet }
177+
# - then: Not sure if I am hallucinating...

0 commit comments

Comments
 (0)