Why do InvokePromptAsync and InvokePromptStreamingAsync response with different results?

[tdashworth]

I was experimenting with Kernel Example for OpenAI function calling today and noticed the different responses from Azure OpenAI (Gen-4). I set the temperature to 0 and configured logging but I couldn't understand why.

trce: Program.<>c[0]
      Created KernelFunction 'GetCurrentUtcTime' for '<<Main>$>b__0_1'
======== Example 1: Use automated function calling with a non-streaming prompt ========
trce: Program.<>c[0]
      Created KernelFunction 'Get_Weather_For_City' for '<<Main>$>b__0_2'
trce: Microsoft.SemanticKernel.KernelPromptTemplate[0]
      Extracting blocks from template: Given the current time of day and weather, what is the likely color of the sky in Boston?
info: funce85673d5581a411e9e2b4a08184f881a[0]
      Function funce85673d5581a411e9e2b4a08184f881a invoking.
trce: funce85673d5581a411e9e2b4a08184f881a[0]
      Function arguments: {}
trce: Microsoft.SemanticKernel.KernelFunctionFactory[0]
      Rendered prompt: Given the current time of day and weather, what is the likely color of the sky in Boston?
info: Microsoft.SemanticKernel.Connectors.OpenAI.AzureOpenAIChatCompletionService[0]
      Prompt tokens: 101. Completion tokens: 80. Total tokens: 181.
dbug: Microsoft.SemanticKernel.Connectors.OpenAI.AzureOpenAIChatCompletionService[0]
      Tool requests: 2
trce: Microsoft.SemanticKernel.Connectors.OpenAI.AzureOpenAIChatCompletionService[0]
      Function call requests: HelperFunctions-GetCurrentUtcTime({}), HelperFunctions-Get_Weather_For_City({"cityName": "Boston"})
info: GetCurrentUtcTime[0]
      Function GetCurrentUtcTime invoking.
trce: GetCurrentUtcTime[0]
      Function arguments: {}
info: GetCurrentUtcTime[0]
      Function GetCurrentUtcTime succeeded.
trce: GetCurrentUtcTime[0]
      Function result: Sun, 25 Feb 2024 00:00:00 GMT
info: GetCurrentUtcTime[0]
      Function completed. Duration: 0.0035811s
info: Get_Weather_For_City[0]
      Function Get_Weather_For_City invoking.
trce: Get_Weather_For_City[0]
      Function arguments: {"cityName":"Boston"}
info: Get_Weather_For_City[0]
      Function Get_Weather_For_City succeeded.
trce: Get_Weather_For_City[0]
      Function result: 61 and rainy
info: Get_Weather_For_City[0]
      Function completed. Duration: 0.0019595s
info: Microsoft.SemanticKernel.Connectors.OpenAI.AzureOpenAIChatCompletionService[0]
      Prompt tokens: 185. Completion tokens: 173. Total tokens: 358.
info: Microsoft.SemanticKernel.KernelFunctionFactory[0]
      Prompt tokens: 185. Completion tokens: 173.
info: funce85673d5581a411e9e2b4a08184f881a[0]
      Function funce85673d5581a411e9e2b4a08184f881a succeeded.
trce: funce85673d5581a411e9e2b4a08184f881a[0]
      Function result: {"ToolCalls":[],"Role":{"Label":"assistant"},"Content":"The current time in UTC is Sun, 25 Feb 2024 00:00:00 GMT. Boston is typically 4 or 5 hours behind UTC, depending on daylight saving time. This would make it either 7:00 PM or 8:00 PM in Boston. Given that it is evening, the sky is likely transitioning from daylight to dusk, which means the sky could be various shades of orange, pink, or purple, gradually darkening to blue and then black as night falls.\n\nHowever, the weather in Boston is currently rainy with a temperature of 61 degrees Fahrenheit. Rain clouds are typically dark gray, so the sky is likely to be overcast with dark gray clouds, obscuring the colors of the sunset. The combination of the evening time and rainy weather suggests that the sky in Boston is likely a dark gray color.","Items":null,"ModelId":"gpt-4","Metadata":{"Id":"chatcmpl-8w92YKePf14Q4clYaXybQ8geHYieb","Created":"2024-02-25T13:41:06+00:00","PromptFilterResults":[{"PromptIndex":0,"ContentFilterResults":{"Sexual":{"Severity":{},"Filtered":false},"Violence":{"Severity":{},"Filtered":false},"Hate":{"Severity":{},"Filtered":false},"SelfHarm":{"Severity":{},"Filtered":false},"Profanity":null,"CustomBlocklists":[],"Error":null,"Jailbreak":null}}],"SystemFingerprint":"fp_8abb16fa4e","Usage":{"CompletionTokens":173,"PromptTokens":185,"TotalTokens":358}}}
info: funce85673d5581a411e9e2b4a08184f881a[0]
      Function completed. Duration: 9.8961126s
The current time in UTC is Sun, 25 Feb 2024 00:00:00 GMT. Boston is typically 4 or 5 hours behind UTC, depending on daylight saving time. This would make it either 7:00 PM or 8:00 PM in Boston. Given that it is evening, the sky is likely transitioning from daylight to dusk, which means the sky could be various shades of orange, pink, or purple, gradually darkening to blue and then black as night falls.

However, the weather in Boston is currently rainy with a temperature of 61 degrees Fahrenheit. Rain clouds are typically dark gray, so the sky is likely to be overcast with dark gray clouds, obscuring the colors of the sunset. The combination of the evening time and rainy weather suggests that the sky in Boston is likely a dark gray color.

======== Example 2: Use automated function calling with a streaming prompt ========
trce: Microsoft.SemanticKernel.KernelPromptTemplate[0]
      Extracting blocks from template: Given the current time of day and weather, what is the likely color of the sky in Boston?
info: funca8ba5df836fc490a93a420c9a74a3361[0]
      Function funca8ba5df836fc490a93a420c9a74a3361 streaming.
trce: funca8ba5df836fc490a93a420c9a74a3361[0]
      Function arguments: {}
trce: Microsoft.SemanticKernel.KernelFunctionFactory[0]
      Rendered prompt: Given the current time of day and weather, what is the likely color of the sky in Boston?
trce: Microsoft.SemanticKernel.Connectors.OpenAI.AzureOpenAIChatCompletionService[0]
      Function call requests: HelperFunctions-GetCurrentUtcTime({}), HelperFunctions-Get_Weather_For_City({"cityName": "Boston"})
info: GetCurrentUtcTime[0]
      Function GetCurrentUtcTime invoking.
trce: GetCurrentUtcTime[0]
      Function arguments: {}
info: GetCurrentUtcTime[0]
      Function GetCurrentUtcTime succeeded.
trce: GetCurrentUtcTime[0]
      Function result: Sun, 25 Feb 2024 00:00:00 GMT
info: GetCurrentUtcTime[0]
      Function completed. Duration: 0.0003581s
info: Get_Weather_For_City[0]
      Function Get_Weather_For_City invoking.
trce: Get_Weather_For_City[0]
      Function arguments: {"cityName":"Boston"}
info: Get_Weather_For_City[0]
      Function Get_Weather_For_City succeeded.
trce: Get_Weather_For_City[0]
      Function result: 61 and rainy
info: Get_Weather_For_City[0]
      Function completed. Duration: 0.0001897s
The current time in UTC is Sun, 25 Feb 2024 00:00:00 GMT. To determine the local time in Boston, we need to consider the time zone difference. Boston is typically in the Eastern Time Zone, which is UTC-5 hours during standard time and UTC-4 hours during daylight saving time. Since daylight saving time in the United States begins on the second Sunday in March, Boston would still be observing standard time on February 25. Therefore, the local time in Boston would be 7:00 PM on Sat, 24 Feb 2024.

Given that it is 7:00 PM, the sun has likely set or is setting, and the sky would be transitioning from daylight to twilight, which can range in color from shades of orange and pink to darker blues and eventually black as night falls.

The weather in Boston is reported to be 61°F and rainy. Rain clouds are typically dark gray, so the sky is likely to be overcast with these dark clouds, obscuring any colors that might be present during sunset. The combination of the time of day and the rainy weather suggests that the sky in Boston is likely to be dark and cloudy, with little to no visible color from the sunset.info: funca8ba5df836fc490a93a420c9a74a3361[0]
      Function streaming completed. Duration: 19.9567572s.

======== Example 3: Use manual function calling with a non-streaming prompt ========
info: Microsoft.SemanticKernel.Connectors.OpenAI.AzureOpenAIChatCompletionService[0]
      Prompt tokens: 101. Completion tokens: 80. Total tokens: 181.
info: GetCurrentUtcTime[0]
      Function GetCurrentUtcTime invoking.
trce: GetCurrentUtcTime[0]
      Function arguments: {}
info: GetCurrentUtcTime[0]
      Function GetCurrentUtcTime succeeded.
trce: GetCurrentUtcTime[0]
      Function result: Sun, 25 Feb 2024 00:00:00 GMT
info: GetCurrentUtcTime[0]
      Function completed. Duration: 3.86E-05s
info: Get_Weather_For_City[0]
      Function Get_Weather_For_City invoking.
trce: Get_Weather_For_City[0]
      Function arguments: {"cityName":"Boston"}
info: Get_Weather_For_City[0]
      Function Get_Weather_For_City succeeded.
trce: Get_Weather_For_City[0]
      Function result: 61 and rainy
info: Get_Weather_For_City[0]
      Function completed. Duration: 3.39E-05s
info: Microsoft.SemanticKernel.Connectors.OpenAI.AzureOpenAIChatCompletionService[0]
      Prompt tokens: 185. Completion tokens: 133. Total tokens: 318.
Given that the current UTC time is 00:00 on February 25, 2024, and the weather in Boston is 61°F and rainy, the likely color of the sky in Boston would be gray or overcast due to the rain. However, since Boston is in the Eastern Time Zone (UTC-5), the local time would be 19:00 (7:00 PM) on February 24, 2024. At this time, if it's winter and considering the weather conditions, the sky would likely be dark or dusky, transitioning into night, with cloud cover further obscuring any remaining light due to the rain.

[sophialagerkranspandey]

From chatting with the team, the temperature would need to be set to an absolute of zero.