[BUG] Intermittent System.Text.Json.JsonReaderException: The input does not contain any JSON tokens. Expected the input to start with a valid JSON token, when isFinalBlock is true.

[ambitiousrahul]

Library name and version

Azure.AI.OpenAI 2.1.0

Describe the bug

Getting following error intermittently from Azure OpenAI

System.Text.Json.JsonReaderException: The input does not contain any JSON tokens. Expected the input to start with a valid JSON token, when isFinalBlock is true. LineNumber: 0 | BytePositionInLine: 0.
   at System.Text.Json.ThrowHelper.ThrowJsonReaderException(Utf8JsonReader& json, ExceptionResource resource, Byte nextByte, ReadOnlySpan`1 bytes)
   at System.Text.Json.Utf8JsonReader.Read()
   at System.Text.Json.JsonDocument.Parse(ReadOnlySpan`1 utf8JsonSpan, JsonReaderOptions readerOptions, MetadataDb& database, StackRowStack& stack)
   at System.Text.Json.JsonDocument.Parse(ReadOnlyMemory`1 utf8Json, JsonReaderOptions readerOptions, Byte[] extraRentedArrayPoolBytes, PooledByteBufferWriter extraPooledByteBufferWriter)
   at OpenAI.Chat.ChatCompletion.FromResponse(PipelineResponse response)
   at OpenAI.Chat.ChatClient.CompleteChatAsync(IEnumerable`1 messages, ChatCompletionOptions options, CancellationToken cancellationToken)

with below code

var chatClient = detailsToUse.OpenAIClient.GetChatClient(detailsToUse.LoadBalanceItem.DeploymentName);

var response = await chatClient.CompleteChatAsync(messages, chatCompletionOptions, cancellationToken);

Expected behavior

I was always getting the response even when the generated text was empty. For any errors on response, ClientResultException is raised

Actual behavior

I am getting System.Text.Json.JsonReaderException error and so response object is null, and also no useful information retrieved.

Reproduction Steps

Azure OpenAI sdk: NuGet\Install-Package Azure.AI.OpenAI -Version 2.1.0

create messages in IEnumerable<ChatMessage> format.

[
        {
            "role": "user",
            "content": "You are a prompt writer. Your job is to write a LLM prompt to get me best career advice for software engineeer to grow exponencially"
        }
    ]

with that message, invoke the LLM request
request should be successful or ClientResultException should be raised.
however, getting System.Text.Json.JsonReaderException intermittently

Environment

No response

[github-actions]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @jpalvarezl @trrwilson.