|
| 1 | +--- |
| 2 | +title: 'How to generate reproducible output with Azure OpenAI Service' |
| 3 | +titleSuffix: Azure OpenAI |
| 4 | +description: Learn how to generate reproducible output (preview) with Azure OpenAI Service |
| 5 | +services: cognitive-services |
| 6 | +manager: nitinme |
| 7 | +ms.service: azure-ai-openai |
| 8 | +ms.topic: how-to |
| 9 | +ms.date: 11/17/2023 |
| 10 | +author: mrbullwinkle |
| 11 | +ms.author: mbullwin |
| 12 | +recommendations: false |
| 13 | +keywords: |
| 14 | + |
| 15 | +--- |
| 16 | + |
| 17 | +# Learn how to use reproducible output (preview) |
| 18 | + |
| 19 | +By default if you ask an Azure OpenAI Chat Completion model the same question multiple times you are likely to get a different response. The responses are therefore considered to be non-deterministic. Reproducible output is a new preview feature that allows you to selectively change the default behavior towards producing more deterministic outputs. |
| 20 | + |
| 21 | +## Reproducible output support |
| 22 | + |
| 23 | +Reproducible output is only currently supported with the following: |
| 24 | + |
| 25 | +### Supported models |
| 26 | + |
| 27 | +- `gpt-4-1106-preview` |
| 28 | +- `gpt-35-turbo-1106` |
| 29 | + |
| 30 | +### API Version |
| 31 | + |
| 32 | +- `2023-12-01-preview` |
| 33 | + |
| 34 | +## Example |
| 35 | + |
| 36 | +First we'll generate three responses to the same question to demonstrate the variability that is common to Chat Completion responses even when other parameters are the same: |
| 37 | + |
| 38 | +```python |
| 39 | +import os |
| 40 | +from openai import AzureOpenAI |
| 41 | + |
| 42 | +client = AzureOpenAI( |
| 43 | + azure_endpoint = os.getenv("AZURE_OPENAI_ENDPOINT"), |
| 44 | + api_key=os.getenv("AZURE_OPENAI_KEY"), |
| 45 | + api_version="2023-12-01-preview" |
| 46 | +) |
| 47 | + |
| 48 | +for i in range(3): |
| 49 | + print(f'Story Version {i + 1}\n---') |
| 50 | + |
| 51 | + response = client.chat.completions.create( |
| 52 | + model="gpt-4-1106-preview", # Model = should match the deployment name you chose for your 1106-preview model deployment |
| 53 | + #seed=42, |
| 54 | + temperature=0.7, |
| 55 | + max_tokens =200, |
| 56 | + messages=[ |
| 57 | + {"role": "system", "content": "You are a helpful assistant."}, |
| 58 | + {"role": "user", "content": "Tell me a story about how the universe began?"} |
| 59 | + ] |
| 60 | + ) |
| 61 | + |
| 62 | + print(response.choices[0].message.content) |
| 63 | + print("---\n") |
| 64 | + |
| 65 | + del response |
| 66 | +``` |
| 67 | + |
| 68 | +### Output |
| 69 | + |
| 70 | +```output |
| 71 | +Story Version 1 |
| 72 | +--- |
| 73 | +In the beginning, there was nothingness, a vast expanse of empty space, a blank canvas waiting to be painted with the wonders of existence. Then, approximately 13.8 billion years ago, something extraordinary happened, an event that would mark the birth of the universe – the Big Bang. |
| 74 | +
|
| 75 | +The Big Bang was not an explosion in the conventional sense but rather an expansion, an incredibly rapid stretching of space that took place everywhere in the universe at once. In just a fraction of a second, the universe grew from smaller than a single atom to an incomprehensibly large expanse. |
| 76 | +
|
| 77 | +In these first moments, the universe was unimaginably hot and dense, filled with a seething soup of subatomic particles and radiant energy. As the universe expanded, it began to cool, allowing the first particles to form. Protons and neutrons came together to create the first simple atomic nuclei in a process known as nucleosynthesis. |
| 78 | +
|
| 79 | +For hundreds of thousands of years, the universe continued to cool and expand |
| 80 | +--- |
| 81 | +
|
| 82 | +Story Version 2 |
| 83 | +--- |
| 84 | +Once upon a time, in the vast expanse of nothingness, there was a moment that would come to define everything. This moment, a tiny fraction of a second that would be forever known as the Big Bang, marked the birth of the universe as we know it. |
| 85 | +
|
| 86 | +Before this moment, there was no space, no time, just an infinitesimally small point of pure energy, a singularity where all the laws of physics as we understand them did not apply. Then, suddenly, this singular point began to expand at an incredible rate. In a cosmic symphony of creation, matter, energy, space, and time all burst forth into existence. |
| 87 | +
|
| 88 | +The universe was a hot, dense soup of particles, a place of unimaginable heat and pressure. It was in this crucible of creation that the simplest elements were formed. Hydrogen and helium, the building blocks of the cosmos, came into being. |
| 89 | +
|
| 90 | +As the universe continued to expand and cool, these primordial elements began to co |
| 91 | +--- |
| 92 | +
|
| 93 | +Story Version 3 |
| 94 | +--- |
| 95 | +Once upon a time, in the vast expanse of nothingness, there was a singularity, an infinitely small and infinitely dense point where all the mass and energy of what would become the universe were concentrated. This singularity was like a tightly wound cosmic spring holding within it the potential of everything that would ever exist. |
| 96 | +
|
| 97 | +Then, approximately 13.8 billion years ago, something extraordinary happened. This singularity began to expand in an event we now call the Big Bang. In just a fraction of a second, the universe grew exponentially during a period known as cosmic inflation. It was like a symphony's first resounding chord, setting the stage for a cosmic performance that would unfold over billions of years. |
| 98 | +
|
| 99 | +As the universe expanded and cooled, the fundamental forces of nature that we know today – gravity, electromagnetism, and the strong and weak nuclear forces – began to take shape. Particles of matter were created and began to clump together under the force of gravity, forming the first atoms |
| 100 | +--- |
| 101 | +``` |
| 102 | + |
| 103 | +Notice that while each story might have similar elements and some verbatim repetition the longer the response goes on the more they tend to diverge. |
| 104 | + |
| 105 | +Now we'll run the same code as before but this time uncomment the line for the parameter that says `seed=42` |
| 106 | + |
| 107 | +```python |
| 108 | +import os |
| 109 | +from openai import AzureOpenAI |
| 110 | + |
| 111 | +client = AzureOpenAI( |
| 112 | + azure_endpoint = os.getenv("AZURE_OPENAI_ENDPOINT"), |
| 113 | + api_key=os.getenv("AZURE_OPENAI_KEY"), |
| 114 | + api_version="2023-12-01-preview" |
| 115 | +) |
| 116 | + |
| 117 | +for i in range(3): |
| 118 | + print(f'Story Version {i + 1}\n---') |
| 119 | + |
| 120 | + response = client.chat.completions.create( |
| 121 | + model="gpt-4-1106-preview", # Model = should match the deployment name you chose for your 1106-preview model deployment |
| 122 | + seed=42, |
| 123 | + temperature=0.7, |
| 124 | + max_tokens =200, |
| 125 | + messages=[ |
| 126 | + {"role": "system", "content": "You are a helpful assistant."}, |
| 127 | + {"role": "user", "content": "Tell me a story about how the universe began?"} |
| 128 | + ] |
| 129 | + ) |
| 130 | + |
| 131 | + print(response.choices[0].message.content) |
| 132 | + print("---\n") |
| 133 | + |
| 134 | + del response |
| 135 | +``` |
| 136 | + |
| 137 | +### Output |
| 138 | + |
| 139 | +``` |
| 140 | +Story Version 1 |
| 141 | +--- |
| 142 | +In the beginning, there was nothing but a vast emptiness, a void without form or substance. Then, from this nothingness, a singular event occurred that would change the course of existence forever—The Big Bang. |
| 143 | +
|
| 144 | +Around 13.8 billion years ago, an infinitely hot and dense point, no larger than a single atom, began to expand at an inconceivable speed. This was the birth of our universe, a moment where time and space came into being. As this primordial fireball grew, it cooled, and the fundamental forces that govern the cosmos—gravity, electromagnetism, and the strong and weak nuclear forces—began to take shape. |
| 145 | +
|
| 146 | +Matter coalesced into the simplest elements, hydrogen and helium, which later formed vast clouds in the expanding universe. These clouds, driven by the force of gravity, began to collapse in on themselves, creating the first stars. The stars were crucibles of nuclear fusion, forging heavier elements like carbon, nitrogen, and oxygen |
| 147 | +--- |
| 148 | +
|
| 149 | +Story Version 2 |
| 150 | +--- |
| 151 | +In the beginning, there was nothing but a vast emptiness, a void without form or substance. Then, from this nothingness, a singular event occurred that would change the course of existence forever—The Big Bang. |
| 152 | +
|
| 153 | +Around 13.8 billion years ago, an infinitely hot and dense point, no larger than a single atom, began to expand at an inconceivable speed. This was the birth of our universe, a moment where time and space came into being. As this primordial fireball grew, it cooled, and the fundamental forces that govern the cosmos—gravity, electromagnetism, and the strong and weak nuclear forces—began to take shape. |
| 154 | +
|
| 155 | +Matter coalesced into the simplest elements, hydrogen and helium, which later formed vast clouds in the expanding universe. These clouds, driven by the force of gravity, began to collapse in on themselves, creating the first stars. The stars were crucibles of nuclear fusion, forging heavier elements like carbon, nitrogen, and oxygen |
| 156 | +--- |
| 157 | +
|
| 158 | +Story Version 3 |
| 159 | +--- |
| 160 | +In the beginning, there was nothing but a vast emptiness, a void without form or substance. Then, from this nothingness, a singular event occurred that would change the course of existence forever—The Big Bang. |
| 161 | +
|
| 162 | +Around 13.8 billion years ago, an infinitely hot and dense point, no larger than a single atom, began to expand at an inconceivable speed. This was the birth of our universe, a moment where time and space came into being. As this primordial fireball grew, it cooled, and the fundamental forces that govern the cosmos—gravity, electromagnetism, and the strong and weak nuclear forces—began to take shape. |
| 163 | +
|
| 164 | +Matter coalesced into the simplest elements, hydrogen and helium, which later formed vast clouds in the expanding universe. These clouds, driven by the force of gravity, began to collapse in on themselves, creating the first stars. The stars were crucibles of nuclear fusion, forging heavier elements like carbon, nitrogen, and oxygen |
| 165 | +--- |
| 166 | +``` |
| 167 | + |
| 168 | +By using the same `seed` parameter of 42 for each of our three requests we're able to produce much more consistent (in this case identical) results. |
| 169 | + |
| 170 | +## Parameter details |
| 171 | + |
| 172 | +`seed` is an optional parameter, which can be set to an integer or null. |
| 173 | + |
| 174 | +This feature is in Preview. If specified, our system will make a best effort to sample deterministically, such that repeated requests with the same seed and parameters should return the same result. Determinism isn't guaranteed, and you should refer to the `system_fingerprint` response parameter to monitor changes in the backend. |
| 175 | + |
| 176 | +`system_fingerprint` is a string and is part of the chat completion object. |
| 177 | + |
| 178 | +This fingerprint represents the backend configuration that the model runs with. |
| 179 | + |
| 180 | +It can be used with the seed request parameter to understand when backend changes have been made that might affect determinism. |
| 181 | + |
| 182 | +To view the full chat completion object with `system_fingerprint`, you could add ` print(response.model_dump_json(indent=2))` to the previous code next to the existing print statement. This change results in the following additional information being part of the output: |
| 183 | + |
| 184 | +### Output |
| 185 | + |
| 186 | +```JSON |
| 187 | +{ |
| 188 | + "id": "chatcmpl-8LmLRatZxp8wsx07KGLKQF0b8Zez3", |
| 189 | + "choices": [ |
| 190 | + { |
| 191 | + "finish_reason": "length", |
| 192 | + "index": 0, |
| 193 | + "message": { |
| 194 | + "content": "In the beginning, there was nothing but a vast emptiness, a void without form or substance. Then, from this nothingness, a singular event occurred that would change the course of existence forever—The Big Bang.\n\nAround 13.8 billion years ago, an infinitely hot and dense point, no larger than a single atom, began to expand at an inconceivable speed. This was the birth of our universe, a moment where time and space came into being. As this primordial fireball grew, it cooled, and the fundamental forces that govern the cosmos—gravity, electromagnetism, and the strong and weak nuclear forces—began to take shape.\n\nMatter coalesced into the simplest elements, hydrogen and helium, which later formed vast clouds in the expanding universe. These clouds, driven by the force of gravity, began to collapse in on themselves, creating the first stars. The stars were crucibles of nuclear fusion, forging heavier elements like carbon, nitrogen, and oxygen", |
| 195 | + "role": "assistant", |
| 196 | + "function_call": null, |
| 197 | + "tool_calls": null |
| 198 | + }, |
| 199 | + "content_filter_results": { |
| 200 | + "hate": { |
| 201 | + "filtered": false, |
| 202 | + "severity": "safe" |
| 203 | + }, |
| 204 | + "self_harm": { |
| 205 | + "filtered": false, |
| 206 | + "severity": "safe" |
| 207 | + }, |
| 208 | + "sexual": { |
| 209 | + "filtered": false, |
| 210 | + "severity": "safe" |
| 211 | + }, |
| 212 | + "violence": { |
| 213 | + "filtered": false, |
| 214 | + "severity": "safe" |
| 215 | + } |
| 216 | + } |
| 217 | + } |
| 218 | + ], |
| 219 | + "created": 1700201417, |
| 220 | + "model": "gpt-4", |
| 221 | + "object": "chat.completion", |
| 222 | + "system_fingerprint": "fp_50a4261de5", |
| 223 | + "usage": { |
| 224 | + "completion_tokens": 200, |
| 225 | + "prompt_tokens": 27, |
| 226 | + "total_tokens": 227 |
| 227 | + }, |
| 228 | + "prompt_filter_results": [ |
| 229 | + { |
| 230 | + "prompt_index": 0, |
| 231 | + "content_filter_results": { |
| 232 | + "hate": { |
| 233 | + "filtered": false, |
| 234 | + "severity": "safe" |
| 235 | + }, |
| 236 | + "self_harm": { |
| 237 | + "filtered": false, |
| 238 | + "severity": "safe" |
| 239 | + }, |
| 240 | + "sexual": { |
| 241 | + "filtered": false, |
| 242 | + "severity": "safe" |
| 243 | + }, |
| 244 | + "violence": { |
| 245 | + "filtered": false, |
| 246 | + "severity": "safe" |
| 247 | + } |
| 248 | + } |
| 249 | + } |
| 250 | + ] |
| 251 | +} |
| 252 | +``` |
| 253 | + |
| 254 | +## Additional considerations |
| 255 | + |
| 256 | +When you want to use reproducible outputs, you need to set the `seed` to the same integer across chat completions calls. You should also match any other parameters like `temperature`, `max_tokens`, etc. |
0 commit comments