[BUG] RuntimeException when using BlobContainerClient

[s17b2-voroneckij]

Describe the bug
I am trying to list and download blobs from a public blob storage. However, I am getting strange errors.
 
Exception or Stack Trace
Exception in thread "main" java.lang.RuntimeException: Unexpected header value. Expected response to echo x-ms-client-request-id: f64bd375-2558-4625-8626-7c647eae58c5. Got value b734e8e5-a1dc-4eb1-8829-82f97253e89f. at com.azure.storage.common.policy.ResponseValidationPolicyBuilder.lambda$addOptionalEcho$0(ResponseValidationPolicyBuilder.java:48) at com.azure.storage.common.policy.ResponseValidationPolicyBuilder$ResponseValidationPolicy.lambda$process$0(ResponseValidationPolicyBuilder.java:79) at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) at com.azure.storage.common.policy.ResponseValidationPolicyBuilder$ResponseValidationPolicy.lambda$process$1(ResponseValidationPolicyBuilder.java:79) at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:113) at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1839) at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:151) at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:79) at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1839) at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:151) at reactor.core.publisher.Operators$MonoInnerProducerBase.complete(Operators.java:2666) at reactor.core.publisher.MonoSingle$SingleSubscriber.onComplete(MonoSingle.java:180) at reactor.core.publisher.MonoFlatMapMany$FlatMapManyInner.onComplete(MonoFlatMapMany.java:260) at reactor.core.publisher.FluxContextWrite$ContextWriteSubscriber.onComplete(FluxContextWrite.java:126) at reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2402) at reactor.core.publisher.FluxContextWrite$ContextWriteSubscriber.request(FluxContextWrite.java:136) at reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onSubscribeInner(MonoFlatMapMany.java:150) at reactor.core.publisher.MonoFlatMapMany$FlatMapManyInner.onSubscribe(MonoFlatMapMany.java:245) at reactor.core.publisher.FluxContextWrite$ContextWriteSubscriber.onSubscribe(FluxContextWrite.java:101) at reactor.core.publisher.FluxJust.subscribe(FluxJust.java:68) at reactor.core.publisher.Flux.subscribe(Flux.java:8642) at reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onNext(MonoFlatMapMany.java:195) at reactor.core.publisher.SerializedSubscriber.onNext(SerializedSubscriber.java:99) at reactor.core.publisher.FluxRetryWhen$RetryWhenMainSubscriber.onNext(FluxRetryWhen.java:174) at reactor.core.publisher.MonoCreate$DefaultMonoSink.success(MonoCreate.java:172) at reactor.netty.http.client.HttpClientConnect$HttpIOHandlerObserver.onStateChange(HttpClientConnect.java:435) at reactor.netty.ReactorNetty$CompositeConnectionObserver.onStateChange(ReactorNetty.java:707) at reactor.netty.resources.DefaultPooledConnectionProvider$DisposableAcquire.onStateChange(DefaultPooledConnectionProvider.java:193) at reactor.netty.resources.DefaultPooledConnectionProvider$PooledConnection.onStateChange(DefaultPooledConnectionProvider.java:454) at reactor.netty.http.client.HttpClientOperations.onInboundNext(HttpClientOperations.java:703) at reactor.netty.channel.ChannelOperationsHandler.channelRead(ChannelOperationsHandler.java:114) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at com.azure.core.http.netty.implementation.AzureSdkHandler.channelRead(AzureSdkHandler.java:222) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:333) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:454) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290) at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1475) at io.netty.handler.ssl.SslHandler.decodeNonJdkCompatible(SslHandler.java:1349) at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1389) at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:529) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:468) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Thread.java:833) Suppressed: java.lang.Exception: #block terminated with an error at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:100) at reactor.core.publisher.Mono.block(Mono.java:1742) at com.azure.storage.common.implementation.StorageImplUtils.blockWithOptionalTimeout(StorageImplUtils.java:147) at com.azure.storage.blob.specialized.BlobClientBase.downloadContent(BlobClientBase.java:875) at org.example.Main.main(Main.java:14)

To Reproduce
Steps to reproduce the behavior:
Run the code provided below. It may succeed, but also fails quite often with the exception above

Code Snippet
`
package org.example;

import com.azure.storage.blob.BlobContainerClientBuilder;
import com.azure.storage.blob.models.ListBlobsOptions;

import java.time.Duration;

public class Main {
public static void main(String[] argv) {
var client = new BlobContainerClientBuilder()
.endpoint("https://archive.data.elexon.co.uk/iris-archive")
.buildClient();

    System.out.println(client.getBlobClient("data/PN/PN_202312081930_06190.json").downloadContent().toString());
    client.listBlobs(
            new ListBlobsOptions()
                    .setPrefix("data/PN/PN_20240130"), Duration.ofSeconds(20)
    ).forEach(blob -> {
        System.out.println(blob.getName());
    });
}

}
`

Expected behavior
No exceptions; blobs listed and downloaded

Screenshots
If applicable, add screenshots to help explain your problem.

Setup (please complete the following information):

• OS: Windows
• IDE: IntelliJ
• Library/Libraries: azure-storage-blob:12.25.1
• Java version: 17
• App Server/Environment:
• Frameworks: none
• Build system: maven

If you suspect a dependency version mismatch (e.g. you see NoClassDefFoundError, NoSuchMethodError or similar), please check out Troubleshoot dependency version conflict article first. If it doesn't provide solution for the problem, please provide:

• verbose dependency tree (mvn dependency:tree -Dverbose)
• exception message, full stack trace, and any available logs

Additional context
Add any other context about the problem here.

Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

• [* ] Bug Description Added
• [*] Repro Steps Added
• [*] Setup information Added

[github-actions]

@ibrahimrabab @ibrandes @seanmcc-msft

[github-actions]

Thank you for your feedback. Tagging and routing to the team member best able to assist.

[ibrahimrabab]

Hi @s17b2-voroneckij
Can you please try to reproduce the issue and provide us with the latest x-ms-client-request-id so that we can appropriately investigate with the service team? Thanks for your cooperation!

[github-actions]

Hi @s17b2-voroneckij. Thank you for opening this issue and giving us the opportunity to assist. To help our team better understand your issue and the details of your scenario please provide a response to the question asked above or the information requested above. This will help us more accurately address your issue.

[s17b2-voroneckij]

@ibrahimrabab, I have already done that multiple times. Please see here: #38794