Skip to content

Commit cd57b34

Browse files
Updated Basic_Evaluation.ipynb to use google-genai (#642)
* Updated module google-generativeai to use google-genai * Refactored imports * Modified how configs are being assigned to models * corrected indentation for better readability * add 2.5 flash preview to model id param --------- Co-authored-by: Nikita Namjoshi <[email protected]>
1 parent 0ec044d commit cd57b34

File tree

1 file changed

+49
-34
lines changed

1 file changed

+49
-34
lines changed

examples/prompting/Basic_Evaluation.ipynb

Lines changed: 49 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@
1111
},
1212
{
1313
"cell_type": "code",
14-
"execution_count": null,
14+
"execution_count": 30,
1515
"metadata": {
1616
"cellView": "form",
1717
"id": "2K6KP3hUl7_l"
@@ -66,25 +66,25 @@
6666
},
6767
{
6868
"cell_type": "code",
69-
"execution_count": null,
69+
"execution_count": 31,
7070
"metadata": {
7171
"id": "Ne-3gnXqR0hI"
7272
},
7373
"outputs": [],
7474
"source": [
75-
"%pip install -U -q \"google-generativeai>=0.7.2\""
75+
"%pip install -U -q \"google-genai>=1.7.0\""
7676
]
7777
},
7878
{
7979
"cell_type": "code",
80-
"execution_count": null,
80+
"execution_count": 32,
8181
"metadata": {
8282
"id": "EconMHePQHGw"
8383
},
8484
"outputs": [],
8585
"source": [
86-
"import google.generativeai as genai\n",
87-
"\n",
86+
"from google import genai\n",
87+
"from google.genai import types\n",
8888
"from IPython.display import Markdown"
8989
]
9090
},
@@ -101,7 +101,7 @@
101101
},
102102
{
103103
"cell_type": "code",
104-
"execution_count": null,
104+
"execution_count": 33,
105105
"metadata": {
106106
"id": "v-JZzORUpVR2"
107107
},
@@ -110,7 +110,7 @@
110110
"from google.colab import userdata\n",
111111
"GOOGLE_API_KEY=userdata.get('GOOGLE_API_KEY')\n",
112112
"\n",
113-
"genai.configure(api_key=GOOGLE_API_KEY)"
113+
"client = genai.Client(api_key=GOOGLE_API_KEY)"
114114
]
115115
},
116116
{
@@ -121,77 +121,92 @@
121121
"source": [
122122
"## Example\n",
123123
"\n",
124-
"Start by defining some system instructions for this problem. For demonstration purposes, the use case involves prompting the model to write an essay with some mistakes. Remember that for generation tasks like writing an essay, you can change the temperature of the model to get more creative answers. Here, you can use `\"temperature\": 0` for predictability."
124+
"Start by defining some system instructions for this problem. For demonstration purposes, the use case involves prompting the model to write an essay with some mistakes. Remember that for generation tasks like writing an essay, you can change the temperature of the model to get more creative answers."
125125
]
126126
},
127127
{
128128
"cell_type": "code",
129-
"execution_count": null,
129+
"execution_count": 34,
130130
"metadata": {
131131
"id": "zNw1Hele0Bne"
132132
},
133133
"outputs": [
134134
{
135135
"data": {
136-
"text/markdown": "## The Power of Words: Why Reading is the Best Thing Ever\n\nReading is a super important thing. It's like, the best thing ever. Everyone should read more, because it's good for you. Like, really good. Reading can make you smarter, more knowledgeable, and even a better person. It's like a magic potion that can transform your brain.\n\nFirstly, reading helps you learn new things. When you read, you're basically absorbing information like a sponge. You can learn about history, science, art, and even how to cook a mean lasagna. It's like having a personal tutor in your pocket, except it doesn't cost anything.\n\nSecondly, reading can make you a better writer. By reading, you learn how to use language effectively. You can see how different authors use words to create different effects. It's like learning from the masters. Plus, reading can help you expand your vocabulary, which is super important for sounding smart.\n\nThirdly, reading can help you relax and de-stress. When you're reading a good book, you can escape from the real world and get lost in another one. It's like taking a vacation without leaving your couch. Reading can also help you sleep better, because it can calm your mind and help you unwind.\n\nIn conclusion, reading is awesome. It's like a superpower that can make you smarter, more knowledgeable, and a better person. So, next time you're feeling bored or stressed, pick up a book and give it a try. You won't regret it. \n",
136+
"text/markdown": "Reading is like, a really good thing. It’s beneficial, you know? Like, a lot. That's what I'm gonna talk about. So, like, my thesis is that reading is really important for a bunch of reasons that I will now tell you.\n\nFirst off, reading makes you smart. I read this somewhere, I forget where. Probably on the internet. But, like, think about it. When you read, you learn new words. And new words are good because you can use them to impress people. The more words you know, the smarter you seem. Its like a secret code to being smart, I think. My English teacher is always saying, “use a thesaurus!” So, like, I do. Sometimes. Its useful. Plus, reading exposes you to new ideas, and new ideas are good for…stuff.\n\nSecondly, reading books is way better than watching TV. TV rots your brain. Everyone knows that. I mean, have you ever met someone who watches TV all day and is, like, a genius? No, right? Exactly. But books, books are different. Books make you think, even if the stuff you read doesn't make much sense. I read this book last week, something with like, a red cover, and it didn't make any sense. But I know it was making me smarter, because I kept having to go back and try to understand it. Therefore, books good, TV bad.\n\nFinally, reading can help you with writing. The more you read, the more you pick up on how other people write, their, like, style and stuff. So when it comes time for you to write something, like this essay even, it will be easier. It's like you’re absorbing their talent through your eyeballs. I think it's osmosis or something. And also, it's because you learn all these new words, that I mentioned before. I always learn a whole bunch of new words whenever I am reading. Its just natural, you know.\n\nIn conclusion, reading is really, really good. It makes you smart, its better than watching TV, and it helps you with writing. So everyone should read more. Seriously, go read a book. I did, and look at me, writing this essay! Like, I am super benefiting from reading right now. You should to. Also, I hope I get a good grade on this essay.\n",
137137
"text/plain": [
138138
"<IPython.core.display.Markdown object>"
139139
]
140140
},
141-
"execution_count": 8,
141+
"execution_count": 34,
142142
"metadata": {},
143143
"output_type": "execute_result"
144144
}
145145
],
146146
"source": [
147-
"student_system_prompt = \"\"\"You're a college student. Your job is to write an essay riddled with common mistakes and a few major ones.\n",
148-
"The essay should have mistakes regarding clarity, grammar, argumentation, and vocabulary.\n",
149-
"Ensure your essay includes a clear thesis statement. You should write only an essay, so do not include any notes.\"\"\"\n",
147+
"student_system_prompt = \"\"\"\n",
148+
" You're a college student. Your job is to write an essay riddled with common mistakes and a few major ones.\n",
149+
" The essay should have mistakes regarding clarity, grammar, argumentation, and vocabulary.\n",
150+
" Ensure your essay includes a clear thesis statement. You should write only an essay, so do not include any notes.\n",
151+
"\"\"\"\n",
150152
"\n",
151-
"student_model = genai.GenerativeModel(model_name='gemini-2.0-flash', generation_config={\"temperature\": 0},\n",
152-
" system_instruction=student_system_prompt)\n",
153+
"MODEL_ID=\"gemini-2.0-flash\" # @param [\"gemini-2.5-flash-preview-04-17\",\"gemini-2.0-flash-lite\",\"gemini-2.0-flash\",\"gemini-2.5-pro-exp-03-25\"] {\"allow-input\":true, isTemplate: true}\n",
153154
"\n",
154-
"essay = student_model.generate_content(\"Write an essay about benefits of reading.\").text\n",
155+
"student_response = client.models.generate_content(\n",
156+
" model=MODEL_ID,\n",
157+
" contents='Write an essay about the benefits of reading.',\n",
158+
" config=types.GenerateContentConfig(\n",
159+
" system_instruction=student_system_prompt\n",
160+
" ),\n",
161+
")\n",
162+
"\n",
163+
"essay = student_response.text\n",
155164
"Markdown(essay)"
156165
]
157166
},
158167
{
159168
"cell_type": "code",
160-
"execution_count": null,
169+
"execution_count": 35,
161170
"metadata": {
162171
"id": "mzp8ZFXf0-OT"
163172
},
164173
"outputs": [
165174
{
166175
"data": {
167-
"text/markdown": "## Evaluation:\n\n**Thesis Statement:** 2/5 - The essay states that reading is \"the best thing ever\" but doesn't offer a specific argument or claim. It needs a more focused thesis statement.\n\n**Clarity and Precision of Language:** 2/5 - The language is informal and uses slang (\"super important,\" \"like,\" \"really good,\" \"super important\"). It lacks the precision and formality expected in academic writing.\n\n**Grammar and Punctuation:** 2/5 - There are several grammatical errors, including comma splices and run-on sentences. The essay also lacks proper punctuation, particularly with the use of commas.\n\n**Argumentation:** 3/5 - The essay presents three benefits of reading, but the arguments are not fully developed. They rely on generalizations and lack specific examples or evidence.\n\n## Corrected Version:\n\n**The Power of Words: The Transformative Benefits of Reading**\n\nReading is a fundamental human activity with profound benefits for individuals and society. It is not merely a pastime but a powerful tool that can enhance cognitive abilities, broaden perspectives, and foster empathy. This essay will explore three key ways in which reading transforms individuals: by expanding knowledge, improving communication skills, and promoting emotional well-being.\n\nFirstly, reading serves as a gateway to knowledge and understanding. Through books, we can explore diverse cultures, historical events, scientific discoveries, and philosophical ideas. By immersing ourselves in different worlds and perspectives, we expand our intellectual horizons and develop a deeper understanding of the complexities of the world around us. For example, reading historical accounts can provide valuable insights into the past, while scientific texts can deepen our understanding of the natural world.\n\nSecondly, reading enhances communication skills. By engaging with different writing styles and literary techniques, readers develop a greater appreciation for the nuances of language. They learn how to use words effectively to convey ideas, emotions, and arguments. Moreover, reading exposes individuals to a wider vocabulary, enriching their communication and enabling them to express themselves with greater clarity and precision.\n\nThirdly, reading promotes emotional well-being. Engaging with a good book can provide a much-needed escape from the stresses of daily life. It allows individuals to immerse themselves in fictional worlds, connect with relatable characters, and experience a range of emotions. This process can be therapeutic, fostering empathy, reducing stress, and promoting relaxation. Furthermore, reading can inspire creativity and imagination, enriching our inner lives and fostering a sense of wonder.\n\nIn conclusion, reading is not simply a leisure activity but a transformative experience that enriches our lives in countless ways. By expanding our knowledge, improving our communication skills, and promoting emotional well-being, reading empowers us to become more informed, articulate, and compassionate individuals. \n\n**Changes Made:**\n\n* **Thesis Statement:** The thesis statement is now more specific and focused, outlining the three key benefits of reading that will be discussed in the essay.\n* **Clarity and Precision of Language:** The language is more formal and precise, avoiding slang and informal expressions.\n* **Grammar and Punctuation:** The essay has been corrected for grammatical errors, including comma splices and run-on sentences. Proper punctuation has been implemented throughout.\n* **Argumentation:** The arguments are more developed, providing specific examples and evidence to support the claims. The essay now uses a more academic tone and structure. \n",
176+
"text/markdown": "Okay, here's a breakdown of your essay, along with a corrected version.\n\n**Evaluation:**\n\n* **Thesis Statement (2/5):** The thesis is present but lacks sophistication and clarity. It's too conversational and doesn't precisely outline the specific arguments.\n* **Clarity and Precision of Language (2/5):** The language is overly informal, repetitive (\"like,\" \"you know\"), and lacks precision. The writing could benefit from more specific examples and a more academic tone.\n* **Grammar and Punctuation (3/5):** There are noticeable errors in grammar and punctuation, including incorrect use of \"its\" vs. \"it's,\" run-on sentences, and comma splices.\n* **Argumentation (2/5):** The arguments are simplistic and lack strong evidence. The connections between reading and intelligence, writing ability, etc., are not fully developed or supported with credible sources.\n\n**Corrected Essay:**\n\nReading offers multifaceted benefits that extend beyond mere entertainment. Its importance is underscored by its capacity to enhance cognitive abilities, provide a superior alternative to passive screen time, and cultivate effective writing skills.\n\nFirstly, reading significantly contributes to intellectual development. Exposure to a wide range of texts expands vocabulary and enhances comprehension. As one encounters new words and concepts within various contexts, they are integrated into one's cognitive framework, facilitating more nuanced and articulate communication. As such, reading becomes a powerful tool for self-improvement and intellectual growth. Furthermore, reading exposes individuals to diverse perspectives and ideas, which stimulates critical thinking and broadens one's understanding of the world.\n\nSecondly, engaging with literature offers a more enriching experience compared to passive forms of media consumption like television. While television often presents information in a simplified and visually driven format, reading requires active engagement and interpretation. This active engagement fosters concentration, analytical skills, and imagination, leading to deeper cognitive processing. Moreover, the ability to select books allows for a tailored and purposeful learning experience, unlike the often random and unfiltered nature of television programming.\n\nFinally, reading plays a crucial role in the development of writing proficiency. By immersing themselves in the works of skilled authors, readers subconsciously absorb stylistic elements, narrative structures, and effective rhetorical techniques. This implicit learning process enhances one's ability to articulate thoughts clearly, construct compelling arguments, and employ language effectively. Furthermore, the increased vocabulary acquired through reading directly translates into more precise and varied writing.\n\nIn conclusion, reading offers a wealth of cognitive, intellectual, and creative advantages. Its capacity to enhance intellect, surpass the limitations of passive media, and foster effective writing skills underscores its importance in personal and academic development. Therefore, individuals are encouraged to embrace reading as a means of lifelong learning and intellectual enrichment.\n\n**Explanation of Changes:**\n\n* **Thesis Statement:** The original thesis was very informal. The corrected version is more formal and specific, clearly stating the three main benefits.\n* **Language:** All instances of \"like,\" \"you know,\" and other informal expressions were removed. The language was made more precise and academic.\n* **Grammar and Punctuation:** Grammatical errors were corrected (e.g., \"its\" changed to \"it's\"). Run-on sentences were split, and commas were added or removed for clarity.\n* **Argumentation:** The arguments were strengthened by providing more specific explanations of how reading leads to intelligence, better writing, etc. The vague references to \"reading it somewhere\" were removed. While still lacking direct citations, the corrected version presents more logical reasoning.\n* **Structure:** The essay was structured with clear topic sentences for each paragraph and a more formal concluding paragraph.\n* **Tone:** The overall tone was shifted from conversational to academic.\n\n",
168177
"text/plain": [
169178
"<IPython.core.display.Markdown object>"
170179
]
171180
},
172-
"execution_count": 9,
181+
"execution_count": 35,
173182
"metadata": {},
174183
"output_type": "execute_result"
175184
}
176185
],
177186
"source": [
178-
"teacher_system_prompt = f\"\"\"\n",
179-
"As a teacher, you are tasked with grading students' essays.\n",
180-
"Please follow these instructions for evaluation:\n",
187+
"teacher_system_prompt = \"\"\"\n",
188+
" As a teacher, you are tasked with grading students' essays.\n",
189+
" Please follow these instructions for evaluation:\n",
181190
"\n",
182-
"1. Evaluate the essay on a scale of 1-5 based on the following criteria:\n",
183-
"- Thesis statement,\n",
184-
"- Clarity and precision of language,\n",
185-
"- Grammar and punctuation,\n",
186-
"- Argumentation\n",
191+
" 1. Evaluate the essay on a scale of 1-5 based on the following criteria:\n",
192+
" - Thesis statement,\n",
193+
" - Clarity and precision of language,\n",
194+
" - Grammar and punctuation,\n",
195+
" - Argumentation\n",
187196
"\n",
188-
"2. Write a corrected version of the essay, addressing any identified issues\n",
189-
"in the original submission. Point what changes were made.\n",
197+
" 2. Write a corrected version of the essay, addressing any identified issues\n",
198+
" in the original submission. Point out what changes were made.\n",
190199
"\"\"\"\n",
191-
"teacher_model = genai.GenerativeModel(model_name='gemini-2.0-flash', generation_config={\"temperature\": 0},\n",
192-
" system_instruction=teacher_system_prompt)\n",
200+
"teacher_response = client.models.generate_content(\n",
201+
" model=MODEL_ID,\n",
202+
" contents=essay,\n",
203+
" config=types.GenerateContentConfig(\n",
204+
" system_instruction=teacher_system_prompt\n",
205+
" ),\n",
206+
")\n",
207+
"\n",
193208
"\n",
194-
"Markdown(teacher_model.generate_content(essay).text)"
209+
"Markdown(teacher_response.text)"
195210
]
196211
},
197212
{

0 commit comments

Comments
 (0)