Skip to content

Commit 250ad7d

Browse files
MervinPraisonMervinPraison
authored
Merge pull request #622 from DhivyaBharathy-web/add-groq-Llama3-agent
Add Groq Llama3-8b-8192 Agent notebook
2 parents dc45059 + 9834260 commit 250ad7d

File tree

1 file changed

+245
-0
lines changed

1 file changed

+245
-0
lines changed

‎examples/cookbooks/Groq_LPU_Powered_AI_Assistant.ipynb‎

Lines changed: 245 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,245 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"id": "a9343007",
6+
"metadata": {
7+
"id": "a9343007"
8+
},
9+
"source": [
10+
"# Groq Llama3-8b-8192 Agent\n",
11+
"This agent uses Groq's `llama3-8b-8192` model to answer questions about Groq technology.\n",
12+
"It demonstrates a structured approach with YAML configs, prompt templates, and result handling."
13+
]
14+
},
15+
{
16+
"cell_type": "markdown",
17+
"source": [
18+
"[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/DhivyaBharathy-web/PraisonAI/blob/main/examples/cookbooks/Groq_LPU_Powered_AI_Assistant.ipynb)\n"
19+
],
20+
"metadata": {
21+
"id": "mBWaooyypsyD"
22+
},
23+
"id": "mBWaooyypsyD"
24+
},
25+
{
26+
"cell_type": "markdown",
27+
"id": "b36ff766",
28+
"metadata": {
29+
"id": "b36ff766"
30+
},
31+
"source": [
32+
"## Dependencies\n",
33+
"We'll install the required Groq Python SDK.\n"
34+
]
35+
},
36+
{
37+
"cell_type": "code",
38+
"execution_count": 1,
39+
"id": "518e7a6a",
40+
"metadata": {
41+
"colab": {
42+
"base_uri": "https://localhost:8080/"
43+
},
44+
"id": "518e7a6a",
45+
"outputId": "98149592-19d0-472a-c31b-aae763fc8b07"
46+
},
47+
"outputs": [
48+
{
49+
"output_type": "stream",
50+
"name": "stdout",
51+
"text": [
52+
"\u001b[?25l \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/129.6 kB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K \u001b[91m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[91m╸\u001b[0m\u001b[90m━━\u001b[0m \u001b[32m122.9/129.6 kB\u001b[0m \u001b[31m5.3 MB/s\u001b[0m eta \u001b[36m0:00:01\u001b[0m\r\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m129.6/129.6 kB\u001b[0m \u001b[31m2.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
53+
"\u001b[?25h"
54+
]
55+
}
56+
],
57+
"source": [
58+
"!pip install --quiet groq pyyaml"
59+
]
60+
},
61+
{
62+
"cell_type": "markdown",
63+
"id": "d8569a5f",
64+
"metadata": {
65+
"id": "d8569a5f"
66+
},
67+
"source": [
68+
"## Tools Setup\n",
69+
"Initialize Groq client and helper functions for prompt generation.\n"
70+
]
71+
},
72+
{
73+
"cell_type": "code",
74+
"execution_count": 2,
75+
"id": "b59b0d09",
76+
"metadata": {
77+
"id": "b59b0d09"
78+
},
79+
"outputs": [],
80+
"source": [
81+
"import os\n",
82+
"import yaml\n",
83+
"from groq import Groq\n",
84+
"\n",
85+
"# Set API key (replace with your own or use environment variables)\n",
86+
"os.environ['GROQ_API_KEY'] = 'enter your key'\n",
87+
"\n",
88+
"# Initialize Groq client\n",
89+
"client = Groq()\n",
90+
"\n",
91+
"def run_groq_chat(prompt_messages, model='llama3-8b-8192'):\n",
92+
" response = client.chat.completions.create(\n",
93+
" model=model,\n",
94+
" messages=prompt_messages\n",
95+
" )\n",
96+
" return response.choices[0].message.content\n"
97+
]
98+
},
99+
{
100+
"cell_type": "markdown",
101+
"id": "9302e8e6",
102+
"metadata": {
103+
"id": "9302e8e6"
104+
},
105+
"source": [
106+
"## YAML Configuration\n",
107+
"Define model and prompt templates via YAML.\n"
108+
]
109+
},
110+
{
111+
"cell_type": "code",
112+
"execution_count": 3,
113+
"id": "238b8f7f",
114+
"metadata": {
115+
"id": "238b8f7f"
116+
},
117+
"outputs": [],
118+
"source": [
119+
"yaml_config = '''\n",
120+
"model: llama3-8b-8192\n",
121+
"prompt_template: |\n",
122+
" You are an expert AI assistant knowledgeable about Groq's technology.\n",
123+
" Provide a detailed answer to the user's question.\n",
124+
" Then summarize the key points briefly.\n",
125+
"\n",
126+
" User question: \"{user_question}\"\n",
127+
"'''\n",
128+
"\n",
129+
"config = yaml.safe_load(yaml_config)\n"
130+
]
131+
},
132+
{
133+
"cell_type": "markdown",
134+
"id": "6d2ff6b1",
135+
"metadata": {
136+
"id": "6d2ff6b1"
137+
},
138+
"source": [
139+
"## Prompt Construction\n",
140+
"Fill in the prompt template with user input.\n"
141+
]
142+
},
143+
{
144+
"cell_type": "code",
145+
"execution_count": 4,
146+
"id": "2d42e686",
147+
"metadata": {
148+
"id": "2d42e686"
149+
},
150+
"outputs": [],
151+
"source": [
152+
"def build_prompt(user_question):\n",
153+
" prompt_text = config['prompt_template'].format(user_question=user_question)\n",
154+
" messages = [\n",
155+
" {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n",
156+
" {\"role\": \"user\", \"content\": prompt_text}\n",
157+
" ]\n",
158+
" return messages\n"
159+
]
160+
},
161+
{
162+
"cell_type": "markdown",
163+
"id": "3c9448d3",
164+
"metadata": {
165+
"id": "3c9448d3"
166+
},
167+
"source": [
168+
"## Main Logic\n",
169+
"Run the agent on an example question.\n"
170+
]
171+
},
172+
{
173+
"cell_type": "code",
174+
"execution_count": 5,
175+
"id": "c59933e4",
176+
"metadata": {
177+
"colab": {
178+
"base_uri": "https://localhost:8080/"
179+
},
180+
"id": "c59933e4",
181+
"outputId": "bad10728-fd20-4e0d-fded-c29fd987a9b7"
182+
},
183+
"outputs": [
184+
{
185+
"output_type": "stream",
186+
"name": "stdout",
187+
"text": [
188+
"Agent response:\n",
189+
"Groq's Low-Precision Unified (LPU) technology is a proprietary architecture designed to accelerate artificial intelligence (AI) and machine learning (ML) workloads. LPU-powered models, also known as Groq Models, have several advantages over traditional Graphics Processing Units (GPUs) in specific applications:\n",
190+
"\n",
191+
"1. **Improved energy efficiency**: LPU is optimized for low power consumption, making it suitable for edge, mobile, and embedded devices where power constraints are common. This is particularly important for applications that require long-lived deployments, such as autonomous vehicles or IoT sensors.\n",
192+
"2. **Enhanced accuracy**: LPU's customized precision and data type selection enable better representational precision for numeric computations, resulting in improved model accuracy. This is particularly beneficial for tasks that require high-fidelity calculations, such as medical imaging or natural language processing.\n",
193+
"3. **Simplified software development**: LPU's unified architecture simplifies the development process for AI/ML developers. Groq Models provide a consistent programming model across different inference scenarios, allowing for easier model deployment and optimization.\n",
194+
"4. **Increased throughput**: LPU's optimized arithmetic units and pipelined architecture enable higher Throughput per Watt (TPW) compared to traditional GPUs. This translates to faster processing times and higher compute density.\n",
195+
"5. **Flexibility and scalability**: LPU-powered models can be deployed across various hardware platforms, from small, low-power devices to large data center clusters. This flexibility allows developers to choose the optimal deployment scenario for their specific use case.\n",
196+
"6. **Customization and specialization**: LPU's architecture can be customized for specific workloads, allowing for optimized performance and power consumption. This customization potential enables developers to create highly specialized AI/ML hardware that matches their specific requirements.\n",
197+
"\n",
198+
"In summary, Groq's LPU-powered models offer significant advantages over traditional GPUs in terms of energy efficiency, accuracy, software development simplicity, throughput, flexibility, and customization.\n",
199+
"\n",
200+
"Key points:\n",
201+
"\n",
202+
"* Improved energy efficiency and suitability for edge, mobile, and embedded devices\n",
203+
"* Enhanced accuracy for high-fidelity calculations\n",
204+
"* Simplified software development with a unified programming model\n",
205+
"* Increased throughput and compute density\n",
206+
"* Flexibility and scalability across various hardware platforms\n",
207+
"* Customization potential for specific workloads and applications\n"
208+
]
209+
}
210+
],
211+
"source": [
212+
"user_question = \"What are the advantages of Groq's LPU-powered models compared to traditional GPUs?\"\n",
213+
"prompt_messages = build_prompt(user_question)\n",
214+
"agent_response = run_groq_chat(prompt_messages, model=config['model'])\n",
215+
"print(\"Agent response:\")\n",
216+
"print(agent_response)\n"
217+
]
218+
},
219+
{
220+
"cell_type": "markdown",
221+
"id": "304a52dd",
222+
"metadata": {
223+
"id": "304a52dd"
224+
},
225+
"source": [
226+
"## Output\n",
227+
"The agent provides a detailed answer followed by a brief summary.\n"
228+
]
229+
}
230+
],
231+
"metadata": {
232+
"colab": {
233+
"provenance": []
234+
},
235+
"language_info": {
236+
"name": "python"
237+
},
238+
"kernelspec": {
239+
"name": "python3",
240+
"display_name": "Python 3"
241+
}
242+
},
243+
"nbformat": 4,
244+
"nbformat_minor": 5
245+
}

0 commit comments

Comments
 (0)