Fix structured message decoder to handle partial headers, smart retry, exception enrichment, and empty buffer edge cases

[Copilot]

Description

Fixes the failing test BlobMessageDecoderDownloadTests.downloadStreamWithResponseContentValidationSmartRetry() which was failing intermittently with "Decoded message length does not match expected length. Expected: 1081, but was: 36" and IndexOutOfBoundsException: Index 0 out of bounds for length 0 errors.

Root Cause

Multiple issues were identified and fixed:

1. Primary bug in StorageContentValidationDecoderPolicy.java: The decodedBytesAtLastCompleteSegment was being unconditionally updated on EVERY chunk decode (SUCCESS, NEED_MORE_BYTES, or COMPLETED), but the decoder's lastCompleteSegmentStart is only updated when a segment footer is fully read and validated. This timing mismatch caused incorrect retry offset calculations during smart retry.

2. Empty buffer handling: When reactor-netty or the HTTP client emits an empty buffer (0 bytes) during the connection lifecycle, the decoder would attempt to read from it, causing IndexOutOfBoundsException: Index 0 out of bounds for length 0. This occurred even with the guard in decodeStream() due to edge cases in buffer handling within decodeChunk().

3. Partial header parsing: The structured-message decoder attempted to parse segment headers (10 bytes) without ensuring those bytes were fully present in the buffer. When header bytes were split across incoming HTTP chunks, the decoder would read partial header bytes and advance offsets incorrectly, leading to corrupted offset tracking and incorrect decoded totals.

4. Missing decoder state in exceptions: When exceptions were thrown during decoding, they lacked decoder state information (decoderOffset and lastCompleteSegment), making it impossible for the smart retry mechanism to reliably determine the correct offset to resume from.

Changes Made

StructuredMessageDecoder.java - Complete rewrite with proper pending buffer management, exception enrichment, and defensive checks:

• Added DecodeResult class with status codes (SUCCESS, NEED_MORE_BYTES, COMPLETED, INVALID) for proper status communication
• Added new decodeChunk() API that properly handles partial headers by buffering incomplete data
• Key invariant: Never reads partial headers - always checks getAvailableBytes() >= required before parsing
• Defensive check: Added check at the start of decodeChunk() to handle empty buffers (0 remaining bytes) with no pending bytes, returning NEED_MORE_BYTES instead of attempting to read from an empty buffer
• Internal ByteArrayOutputStream maintains pending bytes across chunks
• Tracks lastCompleteSegmentStart correctly for smart retry resume (includes optional CRC bytes)
• Added getRetryStartOffset() method that returns the canonical absolute byte index for retry
• Exception enrichment: Added enrichExceptionMessage() helper that appends [decoderOffset=X,lastCompleteSegment=Y] to all exceptions
• All exception throw sites updated to include enriched decoder state:
  • Message header validation (version, length mismatches)
  • Segment header validation (segment number, size validation)
  • CRC validation (segment and message footer CRC mismatches)
  • Finalization (message length mismatch)

StorageContentValidationDecoderPolicy.java:

• Critical fix: Changed from doOnComplete() which only logs, to concatWith(Mono.defer(...)) which throws an IOException when decode is incomplete
• Critical fix: Only update decodedBytesAtLastCompleteSegment when lastCompleteSegmentStart changes, indicating a segment boundary was crossed. Added lastCompleteSegmentStart field to DecoderState to track the last seen value and detect changes.
• When stream ends prematurely, throws IOException with machine-parseable RETRY-START-OFFSET=N token
• Added parseRetryStartOffset(String) static helper to extract offset from exception messages
• Added createRetryableException() methods to wrap errors with retry offset token
• Exception offset parsing: Added parseDecoderOffsets(String) method to extract decoder state from enriched exceptions
• Updated retry offset logic to preferentially use lastCompleteSegment from enriched exception messages
• Added guard to skip null/empty buffers that reactor-netty may emit during connection lifecycle

BlobAsyncClientBase.java:

• Updated retry logic to parse RETRY-START-OFFSET=N from exception message when decoder state unavailable
• Added validation to ensure remainingCount > 0 before creating retry range
• Uses parsed offset for Range header resume

StructuredMessageDecoderTests.java - New comprehensive test file with 9 tests:

• readsCompleteMessageInSingleChunk() - Complete message in single buffer
• readsMessageSplitHeaderAcrossChunks() - Message header (13 bytes) split at byte 7
• readsSegmentHeaderSplitAcrossChunks() - Segment header (10 bytes) split at byte 18
• multipleChunksDecode() - Decode across 32-byte chunks
• tracksLastCompleteSegmentCorrectly() - Verify lastCompleteSegmentStart tracking
• resetToLastCompleteSegmentWorks() - Verify reset functionality for smart retry
• handlesZeroLengthSegment() - Zero-length data segment handling
• decodeWithNoCrc() - Decode without CRC validation
• handlesZeroLengthBuffer() - Zero-length buffer handling

StorageContentValidationDecoderPolicyTests.java - New test file with 13 tests:

• 7 tests for parsing RETRY-START-OFFSET from exception messages
• 6 tests for parsing enriched decoder offsets [decoderOffset=X,lastCompleteSegment=Y]
• Tests for null, empty, and malformed message handling
• Tests for large offset values

Exception Message Format Examples

Enriched decoder exception:

Invalid segment size detected: 512 (remaining=256) [decoderOffset=523,lastCompleteSegment=287]

Policy wraps with retry offset token:

Incomplete structured message: decoded 512 of 1081 bytes. RETRY-START-OFFSET=287. Invalid segment size detected: 512 (remaining=256) [decoderOffset=523,lastCompleteSegment=287]

Benefits

1. Reliable retry offset: Smart retry always uses lastCompleteSegment from enriched exceptions, synchronized with actual segment boundaries
2. Robust empty buffer handling: Gracefully handles empty buffers from reactor-netty at multiple layers (policy guard + decoder defensive check)
3. Better debugging: All exceptions include decoder state for troubleshooting
4. No new exception types: Uses existing IOException with enriched messages
5. Backwards compatible: Falls back to existing logic if enrichment not found

The fix ensures that when segment headers or message headers are split across HTTP chunks, the decoder properly buffers partial data until complete structures are available for parsing. Empty buffers from the HTTP client are handled gracefully at both the policy level and within the decoder itself. When exceptions occur, they include complete decoder state information enabling reliable smart retry. Most importantly, the policy now correctly tracks decoded bytes at segment boundaries by only updating the snapshot when the decoder's lastCompleteSegmentStart actually changes.

All SDK Contribution checklist:

• The pull request does not introduce [breaking changes]
• CHANGELOG is updated for new features, bug fixes or other significant changes.
• I have read the contribution guidelines.

General Guidelines and Best Practices

• Title of the pull request is clear and informative.
• There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

• Pull request includes test coverage for the included changes.

Original prompt

Please Fix the smart-retry decoded-bytes accounting bug exposed by BlobMessageDecoderDownloadTests.downloadStreamWithResponseContentValidationSmartRetry. The symptom: on stream completion finalizeDecoding throws Decoded message length does not match expected length. Expected: 1081, but was: 36. The decoder correctly read segment lengths, so header parsing is OK — the issue is how decoded bytes are produced and counted before finalization.

Tasks (implement all):

Policy: ensure decoded bytes are counted correctly.

In StorageContentValidationDecoderPolicy.decodeStream(...) after each ByteBuffer decoded = decoder.decode(encodedSlice); do the following:

If decoded != null && decoded.remaining() > 0 append the decoded bytes to the outgoing sequence and increment state.totalDecodedBytes (or the policy-level decoded counter) by decoded.remaining(). Make sure this counter is the same one used by finalizeDecoding check.

If decoded == null || decoded.remaining() == 0, still log the event and do not skip the bookkeeping: the encoded bytes consumed are still to be advanced, and state.totalDecodedBytes should be unchanged, but the encoder's internal counters must remain consistent. Previously empty else blocks were left — replace these with correct bookkeeping and logging.

Ensure advanceMessageOffset(encodedSegmentSize) is called every time you consume an encoded segment, even if decoded bytes returned are zero. The advanceMessageOffset call must update the decoder's internal authoritative encoded offset.

When the stream completes, the policy should compute:

long totalDecodedSoFar = state.totalDecodedBytes; // updated every decode()
ByteBuffer finalTail = decoder.finalizeDecoding(); // may return final tail bytes (can be empty or null)
int finalTailBytes = finalTail == null ? 0 : finalTail.remaining();
long finalTotalDecoded = totalDecodedSoFar + finalTailBytes;

Use finalTotalDecoded for the final comparison to expected length. If finalTotalDecoded != expectedLength then throw, else emit finalTail (if any) and complete.

Add debug logs at INFO level showing encodedSegmentSize, decodedProduced, state.totalDecodedBytes after each decode, and a final log showing finalTotalDecoded vs expectedLength before throwing.

Decoder: make the contract explicit and ensure counters.

In StructuredMessageDecoder.java ensure these methods exist and behave as follows:

public ByteBuffer decode(ByteBuffer encodedSlice) — accepts a slice where position()==0, returns only the decoded bytes corresponding to that slice (or null/empty if none). It should not implicitly discard decoded bytes without returning them. If the decoder needs to buffer bytes internally for the algorithm, that must be documented and finalizeDecoding() must return the remainder.

public ByteBuffer finalizeDecoding() — returns any final decoded bytes still buffered internally (or null/empty). Do not perform the expectedLength comparison inside finalize; let the policy aggregate counts and make the comparison. If the code already asserts there, change it to return leftover bytes and optionally validate against internal counters but do not throw before the policy has a chance to combine counts.

public long getMessageOffset() and public void advanceMessageOffset(long bytes) — getMessageOffset returns the authoritative encoded offset (absolute) the decoder expects next; advanceMessageOffset increments that offset by bytes. Policy should call advanceMessageOffset(encodedSegmentSize) after consuming a segment. Add logs inside advanceMessageOffset showing prior and new offset.

Ensure single authoritative decoded counter.

Decide where authoritative decoded count is kept: either the policy maintains state.totalDecodedBytes (recommended) OR the decoder maintains its own internal decoded counter and exposes getDecodedBytesSoFar(). The simplest and safest fix: make the policy maintain state.totalDecodedBytes and always update it using the decoded.remaining() returned from decode() and finalizeDecoding() results. Remove any reliance on an internal decoder-only counter for the final equality check. Update finalize logic to use state.totalDecodedBytes + finalizeTail for comparison.

Add defensive logging & assertions

After each decode, log: "decoded segment: encodedConsumed=%d decodedProduced=%d totalDecodedSoFar=%d decoderOffset=%d".

On onComplete, log: "finalize: totalDecodedSoFar=%d finalTailBytes=%d expectedLength=%d".

Unit tests & run

Run the failing test locally: mvn -Dtest=BlobMessageDecoderDownloadTests#downloadStreamWithResponseContentValidationSmartRetry test and ensure it passes. If not, examine logs from new debug statements.

Notes / rationale:

Current exception shows expected=1081 but was=36 — that strongly indicates a mismatch between the bytes returned by decode()/finalizeDecoding() and the expected total. Fixing accumulation and the finalize contract will resolve the issue.

Do not change how segment length is parsed (that works), nor the absoluteSt...

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[gunjansingh-msft]

@copilot We have a failing test that needs to be fixed :BlobMessageDecoderDownloadTests.downloadStreamWithResponseContentValidationSmartRetry()

Failure symptoms in logs:

Decoded message length does not match expected length. Expected: 1081, but was: 36

Occasionally: IndexOutOfBoundsException while reading segment header or segment length

Logs show decoder reading first segment successfully (512), then encountering malformed/negative segment length or too-few bytes and finalizing with a wrong decoded total.

Smart-retry path resumes but uses an incorrect retry offset.

Short root cause summary (do not skip)

The structured-message decoder attempts to parse segment headers (2 + 8 = 10 bytes) without ensuring those 10 header bytes are present in the buffer. If the header bytes are split across incoming HTTP chunks, the decoder will read partial header bytes and then advance offsets incorrectly → bad decoderOffset and bad totalDecoded values → finalize sees a tiny decoded size (36) vs expected (1081).

The pending-buffer merging logic (in StorageContentValidationDecoderPolicy) miscomputes bytesConsumed / bytesRemaining when headers are split across chunks. It can drop or double-count bytes.

Smart retry logic (client that retries reads) uses an incorrect retry offset (derived from buffer or messageOffset) instead of the last complete segment boundary, leading retries to restart from mid-segment, further corrupting decoding.

CRC and optional trailer handling edge cases and zero-length segments are not correctly guarded; they can cause the decoder to think message is finished prematurely.

All of these must be fixed together.

Shared assumptions / constants (use these exact values)

The structured message format doc you gave is authoritative. Use the following constants exactly:

HEADER_SIZE = 13 // 1 + 8 + 2 + 2
SEGMENT_HEADER_SIZE = 10 // 2 (segment number) + 8 (segment length)
CRC_LENGTH = 8

Interpretations:

message total length is the total encoded length of the message (including headers, segments, CRCs).

number of segments in header must be honored; decoder can validate that many segments are present.

Segment CRC is optional — only present when message flags indicate it.

Message-wide CRC (trailer) is also optional.

Files to change

com.azure.storage.common.implementation.structuredmessage.StructuredMessageDecoder

com.azure.storage.common.policy.StorageContentValidationDecoderPolicy

com.azure.storage.blob.BlobAsyncClientBase (or the place where retry offset is computed for smart retry)

Add unit tests in com.azure.storage.common.implementation.structuredmessage test package, and update BlobMessageDecoderDownloadTests with an extra scenario (see tests section below).

High-level goals for the patch

Never read part of a header or part of a length. Always check bufferRemaining >= requiredBytes before reading. If not enough bytes yet, persist pending bytes and return so caller will supply more bytes.

Maintain accurate offsets and counters: decoderOffset, totalProcessed, lastCompleteSegmentOffset, and bytesConsumed must be updated only when the corresponding bytes are fully consumed and validated.

Smart retry must use the last complete segment boundary (an integer offset measured from message start), not a buffer pointer or partially processed offset.

Finalization: finalizeDecoding() must only succeed when totalDecodedPayloadBytes equals the message payload length (or other derived expected total). If not, mark decoding incomplete (do not finalize — allow smart retry or error flow to handle it).

Add robust debug logs to capture: when partial headers are seen, when we pause for more bytes, lastCompleteSegment offset, bytesConsumed and decoderOffset transitions.

Detailed code changes (method-level)

Below are precise changes and pseudocode to implement. Make the generated code compile and include edge-case handling.

A — StructuredMessageDecoder changes

File/class: com.azure.storage.common.implementation.structuredmessage.StructuredMessageDecoder

Fields to ensure present and tracked:

private static final int HEADER_SIZE = 13;
private static final int SEGMENT_HEADER_SIZE = 10;
private static final int CRC_LENGTH = 8;

private long decoderOffset = 0; // bytes read from stream (absolute)
private long totalProcessed = 0; // total bytes the decoder has fully decoded
private long lastCompleteSegmentOffset = 0; // absolute offset of last completed segment start
private int expectedMessageLength = -1; // message total length from header (if known)
private int numberOfSegments = -1;
private boolean crcEnabled = false;
private ByteArrayOutputStream pending = new ByteArrayOutputStream(); // pending bytes across chunks

Core invariant: we only advance decoderOffset when we consume/validate bytes.

1. readHeaderIfNeeded(ByteBuffer buffer) — parse top-level header only when bufferRemaining >= HEADER_SIZE. Otherwise stash bytes to pending and return NEED_MORE_BYTES.

Pseudocode:

int readHeaderIfNeeded(ByteBuffer buffer) {
if (expectedMessageLength != -1) {
return SUCCESS; // header already read
}
int bufferRemaining = buffer.remaining();
if (bufferRemaining + pending.size() < HEADER_SIZE) {
// not enough bytes yet
appendPending(buffer); // do not advance decoderOffset
return NEED_MORE_BYTES;
}

ByteBuffer combined = combinePendingAndBuffer(buffer);
// now read header from combined (but only advance the original buffer by the bytes consumed)
byte version = combined.get(); // 1
long totalMessageLength = combined.getLong(); // 8
short flags = combined.getShort(); // 2
short numSegments = combined.getShort(); // 2

// set state
this.expectedMessageLength = (int) totalMessageLength;
this.numberOfSegments = numSegments;
this.crcEnabled = /* check flags bit */;

// compute how many bytes we consumed from original buffer
int consumedFromCombined = HEADER_SIZE;
consumeFromPendingAndBuffer(consumedFromCombined, buffer);

decoderOffset += HEADER_SIZE;
return SUCCESS;

}

Important: consumeFromPendingAndBuffer must correctly pop bytes from pending first, and then from buffer. Do NOT call buffer.get() directly on bytes already moved into combined without reflecting those reads on the original buffer.

2. readAndValidateSegmentHeader(ByteBuffer buffer)

This is critical. Always check if availableBytes >= SEGMENT_HEADER_SIZE before reading:

Pseudocode:

SegmentHeader readAndValidateSegmentHeader(ByteBuffer buffer) {
int available = buffer.remaining() + pending.size();
if (available < SEGMENT_HEADER_SIZE) {
appendPending(buffer);
return null; // signal NEED_MORE_BYTES
}
ByteBuffer combined = combinePendingAndBuffer(buffer);

short segmentNumber = combined.getShort();
long segmentDataLength = combined.getLong();

// Guard: segmentDataLength must be >= 0 and <= expectedMessageLength etc.
if (segmentDataLength < 0 || segmentDataLength > expectedMessageLength) {
    throw new IllegalArgumentException("Invalid segment size detected: " + segmentDataLength + " (remaining=" + available + ")");
}

// consume bytes from original buffer/pending
consumeFromPendingAndBuffer(SEGMENT_HEADER_SIZE, buffer);
decoderOffset += SEGMENT_HEADER_SIZE;

return new SegmentHeader(segmentNumber, segmentDataLength);

}

3. decodeLoop(ByteBuffer buffer)

Overall decoder loop must:

call readHeaderIfNeeded first

for i from 1..numberOfSegments:

call readAndValidateSegmentHeader

if header returns NEED_MORE_BYTES -> return NEED_MORE_BYTES (do not change lastCompleteSegmentOffset)

then check data payload availability: available >= segmentDataLength + (crcEnabled ? CRC_LENGTH : 0)

if not enough -> append pending and return NEED_MORE_BYTES

read payload fully, validate CRC if enabled

after full segment processed, set lastCompleteSegmentOffset = decoderOffsetBeforeSegmentStart (i.e., absolute start offset of this segment)

increment totalProcessed by payload length

increment decoderOffset by payload length + (crcEnabled? CRC_LENGTH : 0) + SEGMENT_HEADER_SIZE

After all segments processed, check for optional trailer: if trailer present (flag), ensure CRC trailer bytes available; if not, NEED_MORE_BYTES.

Only when all expected bytes are processed, call finalizeDecoding() which validates totalProcessed matches expected totals.

Important details:

Maintain decoderOffsetBeforeSegmentStart as decoderOffset at the moment before reading segment header. Set lastCompleteSegmentOffset to that after the entire segment and optional CRC is validated and consumed.

When returning NEED_MORE_BYTES, do not mutate lastCompleteSegmentOffset — it must always point to last fully finished segment so smart-retry can use it.

4. finalizeDecoding()

Only finalize when totalPayloadDecoded == expected payload (derived from header) and (if trailer CRC expected) trailer validated.

If not final, throw IllegalStateException or return an error code indicating incomplete decoding so policy can choose to retry.

5. Helper methods

combinePendingAndBuffer(buffer) merges pending bytes and the buffer into a new ByteBuffer for peeking. But consumption must be reflected onto original buffer and pending — create consumeFromPendingAndBuffer(int bytesToConsume, ByteBuffer buffer) function that removes bytes from pending first then advances buffer.position(bytesToConsumeFromBuffer) accordingly.

6. Defensive checks

When reading segmentDataLength, ensure segmentDataLength <= Integer.MAX_VALUE if storing in int.

If segmentDataLength == 0, treat as valid (zero-length data segment). Still update offsets properly.

B — StorageContentValidationDecoderPolicy changes

File/class: com.azure.storage.common.policy.StorageContentValidationDecoderPolicy

This policy orchestrates reading Flux chunks and feeding them to StructuredMessageDecoder. The policy currently:

merges pending and new chunk incorrectly, sometimes double-counts, and finalizes too early.

Fix approach:

Use the decoder's decode(ByteBuffer buffer) method which returns a result: DecodeResult { bytesConsumed, status } where status is SUCCESS, NEED_MORE_BYTES, COMPLETED, INVALID.

Implementation pseudocode for decodeStream:

Flux decodeStream(Flux upstream) {
StructuredMessageDecoder decoder = new StructuredMessageDecoder();
return Flux.create(sink -> {
upstream.subscribe(new BaseSubscriber() {
@OverRide
protected void hookOnNext(ByteBuffer chunk) {
// try decode using chunk, allowing decoder to manage pending
while (true) {
DecodeResult r = decoder.decode(chunk);
if (r.status == NEED_MORE_BYTES) {
// decoder saved pending bytes; break to wait for more
break;
} else if (r.status == SUCCESS) {
// forward decoded payload bytes downstream as buffer (or collect locally)
// continue loop in case more data remains in chunk
if (!r.remainingBuffer.hasRemaining()) break;
// else set chunk = r.remainingBuffer and continue
} else if (r.status == COMPLETED) {
sink.complete(); // fully decoded and validated
cancel upstream subscription
return;
} else if (r.status == INVALID) {
sink.error(new IllegalArgumentException("Failed to decode structured message: " + r.message));
cancel upstream subscription
return;
}
}
}
@OverRide
protected void hookOnComplete() {
// stream ended; call decoder.finalizeDecoding() — but only if decoder indicates "decoding contiguous but incomplete" then fail or allow retry path
DecodeFinalizationStatus s = decoder.attemptFinalize(); // returns OK or NEED_MORE_BYTES or INVALID
if (s == OK) sink.complete();
else if (s == NEED_MORE_BYTES) {
// mark decoder state so smart retry can resume using lastCompleteSegmentOffset
sink.error(new IncompleteDecodeException(decoder.getLastCompleteSegmentOffset(), "Decoded message length does not match expected length."));
} else sink.error(new IllegalArgumentException("Decoding invalid: " + s.message));
}
});
});
}

Important: On upstream completion (EOF), hookOnComplete must not naively call finalizeDecoding() and throw if incomplete in a way that bypasses smart-retry logic. Instead, it should convert the incomplete-to-completed mismatch into an error that the smart retry code understands and uses to resume from decoder.getLastCompleteSegmentOffset().

Make the policy return or propagate an exception type that contains the retryOffset info. For example, IncompleteStructuredMessageException extends IOException { long retryOffset; }. Populate it with decoder.getLastCompleteSegmentOffset().

Logging: Add debug logs at each stage reporting:

Received buffer in decodeStream, pending size, new bytes

Decoder about to read segment length, decoderOffset, bufferPos, bufferRemaining, peek bytes (hex) — only when debug enabled

Valid segment length read, segmentLength, decoderOffset

Segment complete at byte offset, segmentNum, offset, segmentLength

When returning NEED_MORE_BYTES: Not enough bytes to read segment header/payload/trailer; pendingBytes=X

C — Smart-retry offset fix (client-side)

File: BlobAsyncClientBase (or exact client file where retry offset is computed for content validation retry)

Change:

Where retry offset was previously computed from messageOffset or buffer positions, use the value from the exception thrown by policy, or call decoder API:

// previous (buggy) pseudo:
long retryOffset = someBufferPositionOrMessageOffset; // wrong, may be mid-segment

// new:
if (exception instanceof IncompleteStructuredMessageException) {
retryOffset = ((IncompleteStructuredMessageException) exception).getRetryOffset();
} else {
// fallback
}

If code currently retries using Range: bytes=retryOffset- header, ensure it uses lastCompleteSegmentOffset which points to the start of the last fully completed segment — that ensures server resends from valid boundary.

Extra edge cases to handle

Split segment header across chunks: pending logic must keep partial header bytes and feed them into next chunk; do not parse until full 10 bytes present.

0-length segment: still valid — must set lastCompleteSegmentOffset correctly and still check CRC if enabled.

Optional trailer: if message flags indicate trailing CRC is present, ensure you wait for the final 8 bytes at the end of stream.

Negative or absurd segment length: throw IllegalArgumentException("Invalid segment size ...") and include decoderOffset and peek-next-bytes in logs — this was present in earlier logs and must be preserved.

Multi-segment messages: numberOfSegments from the header should be used as a guard (fail if actual segments differ).

Tests to add / update

Add thorough unit tests under com.azure.storage.common.implementation.structuredmessage:

StructuredMessageDecoderTest.readsCompleteMessageInSingleChunk() — valid multi-segment message in one ByteBuffer -> should decode fully.

StructuredMessageDecoderTest.readsMessageSplitHeaderAcrossChunks() — feed header bytes split across two buffers (e.g., first buffer contains header first 7 bytes, second buffer completes header + payload) -> should decode successfully.

StructuredMessageDecoderTest.readsSegmentHeaderSplitAcrossChunks() — split the 10-byte segment header across two chunks -> should decode successfully (no IndexOutOfBounds).

StructuredMessageDecoderTest.zeroLengthSegment() — segment with 0 data length -> should decode correctly.

StorageContentValidationDecoderPolicyTest.finalizationOnStreamCompleteUsesRetryOffset() — simulate stream EOF with incomplete last segment and assert returned exception contains retryOffset == lastCompleteSegmentOffset.

Integration test that failed earlier: BlobMessageDecoderDownloadTests.downloadStreamWithResponseContentValidationSmartRetry() — should pass after fixes.

Test harness tips:

Mock network chunk boundaries so you control where the header/segment splits land.

Ensure you test both CRC enabled/disabled flags (if CRC logic exists).

Logging and diagnostics

Add these debug log lines (use azure logger patterns):

log.debug("Received buffer in decodeStream", Map.of("newBytes", newBytes, "pendingBytes", pending.size(), "decoderOffset", decoderOffset, "lastCompleteSegment", lastCompleteSegmentOffset));

log.info("Decoder about to read segment length", Map.of("decoderOffset", decoderOffset, "bufferPos", buffer.position(), "bufferRemaining", buffer.remaining(), "peek16", peekHex(buffer, 16)));

log.debug("Valid segment length read", Map.of("segmentLength", segmentLength, "decoderOffset", decoderOffset));

log.info("Segment complete at byte offset", Map.of("segmentNum", segmentNum, "offset", lastCompleteSegmentOffset, "segmentLength", segmentLength));

log.warn("Not enough bytes to read segment header", Map.of("remaining", available, "required", SEGMENT_HEADER_SIZE));

These will make reproducing failures easy. Please make all the mentioned changes to so that issue gets fixed and test passes

[Copilot]

Implemented the comprehensive fix for the structured message decoder in commits 67b25ec and a96939c.

Key changes:

1. StructuredMessageDecoder - Complete rewrite with proper pending buffer management. Never reads partial headers - always checks getAvailableBytes() >= required before parsing. Added decodeChunk() API returning DecodeResult with status codes (SUCCESS, NEED_MORE_BYTES, COMPLETED, INVALID). Internal ByteArrayOutputStream maintains pending bytes across chunks.

2. StorageContentValidationDecoderPolicy - Simplified to use the new decodeChunk() API. State counters sync from decoder after each chunk.

3. Unit tests - Added StructuredMessageDecoderTests.java with tests for:

   • Complete message in single chunk
   • Message header split across chunks
   • Segment header split across chunks
   • Multiple small chunks decoding
   • lastCompleteSegment tracking
   • Reset functionality for smart retry

The decoder now properly handles split headers/segments by buffering partial data internally until complete structures are available for parsing.

[gunjansingh-msft]

@copilot the below test is failing again : BlobMessageDecoderDownloadTests.downloadStreamWithResponseContentValidationSmartRetry() fails.

• The structured-message decoder correctly computes header/segment offsets; unit-level decoder tests pass.
• The integration smart-retry fails because the retry/resume contract is ambiguous and because trailing bytes from the same network buffer are not being preserved/forwarded.
• Desired approach: do NOT add a new custom exception class. Instead, attach the canonical retry offset to an existing exception (IOException) by encoding it in the exception message in a strict machine-parseable format and provide helper parsing on the smart-retry path.

Authoritative format:

• Structured Message Format: header (13 bytes) / segment header (10 bytes) / segment payload / optional per-segment CRC64 / optional message-wide CRC64 trailer.
• We will treat lastCompleteSegmentEndOffset as the canonical retry start offset: this is the absolute byte index (0-based) of the first unread byte that the server must supply on resume.
• When decoding cannot complete because the stream ended prematurely, we will throw an IOException whose message includes a machine-readable token:
  RETRY-START-OFFSET=<number>
  Example message: "Incomplete structured message: decoded 543 of 1081 bytes. RETRY-START-OFFSET=543"

Exact implementation tasks:

1. StructuredMessageDecoder changes

   • Add/ensure internal field:
     private long lastCompleteSegmentEndOffset = 0L;
     This must always represent the absolute offset immediately after the last successfully-validated and complete segment (i.e., header + payload + optional segment CRC).
   • Add a public accessor:

     /**
      * Returns the canonical absolute byte index (0-based) that should be used to resume a failed/incomplete download.
      * This MUST be used directly as the Range header start value: "Range: bytes={retryStartOffset}-"
      */
     public long getRetryStartOffset() {
         return lastCompleteSegmentEndOffset;
     }
     

   • Ensure the decoder updates lastCompleteSegmentEndOffset only when a full segment (including optional per-segment CRC) is fully read and validated. Do not update it on partial reads.
   • The decoder API that finalizes decoding (e.g., finalizeDecoding or equivalent) must not swallow trailing bytes: if the incoming buffer contains more bytes than the encoded message, produce/return a slice of the leftover bytes for forwarding.

2. StorageContentValidationDecoderPolicy changes

   • Change decode flow to return a DecodeResult containing:
     • status (enum: NEED_MORE_BYTES, COMPLETED, INVALID)
     • bytesConsumed (how many bytes consumed from the incoming buffer)
     • ByteBuffer leftover (slice referencing any bytes after the decoded message in the same incoming buffer)
   • On NEED_MORE_BYTES -> keep current behavior: accumulate pending bytes and wait.
   • On INVALID -> throw the existing IllegalArgumentException or existing error (leave as-is).
   • On stream completion (upstream onComplete) if the decoder still reports incomplete (i.e., needed more bytes), throw an IOException with:
     • A descriptive message that ends with RETRY-START-OFFSET=<offset> where <offset> is decoder.getRetryStartOffset().
     • Example:

       throw new IOException(String.format(
           "Incomplete structured message: decoded %d of %d bytes. RETRY-START-OFFSET=%d",
           decoder.getDecodedBytes(), decoder.getExpectedMessageLength(), decoder.getRetryStartOffset()));
       

     • Also log debug: LOG.debug("Incomplete structured message: decoderOffset={}, lastCompleteSegmentEndOffset={}, pendingBytes={}", decoder.getDecoderOffset(), decoder.getRetryStartOffset(), pendingBytes);
   • On COMPLETED:
     • If leftover has remaining bytes, forward that leftover downstream (do not drop it). This ensures bytes that arrived in the same network read after the structured message are not lost.
     • Mark decoding as successfully finalized and proceed.

3. Smart-retry client changes (where the retry GET is created)

   • In the smart-retry flow where a decode failure results in catch of IOException:
     • Attempt to parse the RETRY-START-OFFSET token from the exception message using a strict regex:

       Pattern p = Pattern.compile("RETRY-START-OFFSET=(\\d+)");
       Matcher m = p.matcher(ioException.getMessage());
       long retryStart = 0L;
       if (m.find()) {
           retryStart = Long.parseLong(m.group(1));
       } else {
           // fallback if token not found: use 0 or existing behavior, but log a warning.
       }
       

     • Use retryStart directly in the Range header:
       Range: bytes={retryStart}-
       Do not add or subtract other header/metadata lengths or offsets.
   • When the resumed GET response arrives, feed the decoder with the resumed response body. If the policy preserved leftover bytes (from prior buffer), ensure those leftover bytes are prepended to the resumed stream or otherwise provided to the decoder so that there is a contiguous logical stream starting at retryStart.
   • Add debug logging when parsing the token and when constructing the Range header.

4. Tests to add/modify

   • Unit test in structuredmessage to assert getRetryStartOffset() semantics:
     • Build a message with two segments (512 + 512) + trailer.
     • Feed the decoder only first N bytes so that the decoder completes one segment and is missing remainder -> getRetryStartOffset() should equal end-of-first-segment offset (e.g., 543).
   • StorageContentValidationDecoderPolicyTest.shouldIncludeRetryOffsetInIOException():
     • Simulate stream end mid-message; assert that thrown IOException message includes RETRY-START-OFFSET=<number> and that the number equals decoder.getRetryStartOffset().
   • StorageContentValidationDecoderPolicyTest.forwardsLeftoverBytes():
     • Simulate a single chunk that contains a complete structured message plus X trailing bytes. Assert the policy returns both the decoded payload and then the leftover bytes (in order).
   • Integration test update / assertion:
     • In BlobMessageDecoderDownloadTests.downloadStreamWithResponseContentValidationSmartRetry() add an assert that when a retry occurs the client issued a GET with header Range: bytes={retryStart}-. If the test framework already inspects outgoing requests, assert the Range matches the parsed offset.

5. Logging & diagnostics

   • Add debug logs at these points:
     • When producing the IOException include decoderOffset, lastCompleteSegmentEndOffset, and the pending byte count.
     • When parsing RETRY-START-OFFSET in smart-retry logic, log found value and the constructed Range header.
   • Keep message token text exact (RETRY-START-OFFSET=) to ensure machine parsing is reliable.

6. Backwards compatibility / safety notes

   • If an IOException without RETRY-START-OFFSET is thrown (older path), fallback to existing logic (e.g., restart from 0 or behavior already used by code). Add a warning log noting parsing failed and fallback used.
   • Unit tests should include a fallback scenario to ensure stability.
   • Avoid creating any public API changes beyond the getRetryStartOffset() accessor on the decoder (which is useful for diagnostics).

7. Verification steps

   • Run unit tests: mvn -DskipTests=false -pl sdk/storage/ -am test
   • Run the single failing test locally:
     mvn -Dtest=BlobMessageDecoderDownloadTests#downloadStreamWithResponseContentValidationSmartRetry test
   • Confirm:
     • Decoder unit tests pass.
     • StorageContentValidationDecoderPolicy tests pass.
     • Integration smart-retry test passes and the outgoing resume request contains Range: bytes=<value>- where <value> equals the embedded RETRY-START-OFFSET in the exception thrown previously.

Why this approach:

• Avoids adding a custom exception type while still providing a machine-readable, robust retry offset to the client.
• Keeps the decoder semantics explicit (canonical getRetryStartOffset()).
• Ensures trailing bytes from the same network buffer are preserved and forwarded so there’s no data loss or duplication on retry.
• Tests validate both the decode and integration retry pathways.

Notes for Copilot:

• Keep new logic constrained to structuredmessage decoder, StorageContentValidationDecoderPolicy, and the smart-retry client code path that forms resume GET requests.
• Use the exact token RETRY-START-OFFSET= in exception message and the regex RETRY-START-OFFSET=(\d+) for parsing.
• Add unit tests as described; they should be deterministic and small.
• Avoid changing other modules or public signatures beyond adding getRetryStartOffset() to the decoder.

Please make the above-mentioned changes and run the tests; report back failing logs if any and I’ll refine.

[Copilot]

Implemented the retry offset encoding in exception messages as requested in commits e8669ff and e408b9a.

Key changes:

1. StructuredMessageDecoder - Added getRetryStartOffset() that returns lastCompleteSegmentStart

2. StorageContentValidationDecoderPolicy - Added:

   • parseRetryStartOffset(String) static helper to parse offset from exception messages
   • createRetryableException() methods that wrap errors with RETRY-START-OFFSET=N token
   • All decode errors now include the retry offset in the exception message

3. BlobAsyncClientBase - Updated retry logic to:

   • Parse RETRY-START-OFFSET=N from exception message when decoder state unavailable
   • Validate that remainingCount > 0 before creating retry range
   • Use parsed offset for resume

Exception format: Incomplete structured message: decoded X of Y bytes. RETRY-START-OFFSET=N. <original message>

[gunjansingh-msft]

@copilot The below test is still failing
Branch / PR: apply fixes to the structured message decoder + StorageContentValidationDecoderPolicy + smart-retry resume path to eliminate intermittent smart-retry test failures.

Goal: Fix intermittent “Decoded message length does not match expected length” errors and avoid index/regex errors when parsing retry offsets, WITHOUT introducing a new custom exception. Attach the retry offset to an existing IOException message token.

Summary of changes required (high-level)

1. StructuredMessageDecoder: correct length accounting, maintain lastCompleteSegmentEndOffset precisely (include optional per-segment CRC bytes), add accessor getRetryStartOffset(), and ensure finalizeDecoding() uses consistent arithmetic.
2. StorageContentValidationDecoderPolicy: on upstream stream completion, if decoder still needs more bytes, throw an IOException with machine-parseable token RETRY-START-OFFSET=<n>. Always forward leftover bytes (if any) after a completed decode. Make decode result structure explicit (status, bytesConsumed, leftoverByteBuffer).
3. Smart-retry code path: parse RETRY-START-OFFSET=(\d+) from IOException safely (null-check message), fallback to existing behavior if token missing, and use parsed offset for Range header start. Also handle scenario where leftover bytes are returned by the policy (prepend to resumed stream).
4. Tests: add unit tests for retry offset behavior, leftover forwarding, decoder finalization accounting (with CRC flags on/off).

Exact implementation details and code hints:

A) StructuredMessageDecoder.java

• Add fields:
  private static final int MESSAGE_HEADER_LENGTH = 13; // 1 + 8 + 2 + 2
  private static final int SEGMENT_HEADER_LENGTH = 10; // 2 + 8
  private long lastCompleteSegmentEndOffset = 0L; // absolute byte index (0-based) immediate after last complete segment
  private long expectedMessageLength = -1L; // from header (8 bytes total message length)
  private boolean perSegmentCrcEnabled = false; // derived from flags
  private boolean messageCrcEnabled = false; // derived from flags

less
Copy code

• When reading header:
• Read version, messageLength (8 bytes) -> set this.expectedMessageLength = messageLength;
• Parse flags and set perSegmentCrcEnabled and messageCrcEnabled.
• When reading each segment:
• Compute this segment total bytes = SEGMENT_HEADER_LENGTH + segmentDataLength + (perSegmentCrcEnabled ? 8 : 0)
• After successfully reading and validating the segment (including optional CRC validation), set:

  lastCompleteSegmentEndOffset = decoderOffset; // decoderOffset should point immediately after the segment data+optional CRC bytes
  

• Note: decoderOffset must always be the absolute byte index relative to start of message, i.e., it starts at MESSAGE_HEADER_LENGTH after header was processed.
• getRetryStartOffset():
  public long getRetryStartOffset() {
  return lastCompleteSegmentEndOffset;
  }

typescript
Copy code
Document: "This returns the absolute byte index (0-based) of the first unread byte. Use as Range start directly: 'Range: bytes={offset}-'".

• finalizeDecoding() corrections:
• Compute expectedTotal := expectedMessageLength (already read from header; message header included).
• Compare decoderOffset (absolute offset pointing to end-of-parsed-message) to expectedTotal:
  • If decoderOffset == expectedTotal -> OK
  • Else -> throw IllegalArgumentException with a helpful message or let the caller handle it (but see policy handling). Do NOT rely on some other counters — use the single source decoderOffset vs expectedMessageLength.
• Ensure finalizeDecoding() returns the exact number of bytes consumed (i.e., bytesConsumed = expectedTotal). If any trailing bytes were present in the input buffer beyond expectedTotal, create/return a leftover ByteBuffer slice (or indicate leftover via decode result).

B) StorageContentValidationDecoderPolicy.java

• Introduce a small DecodeResult internal structure:
  enum DecodeStatus { NEED_MORE_BYTES, COMPLETED, INVALID }
  static final class DecodeResult {
  final DecodeStatus status;
  final int bytesConsumed; // how many bytes taken from incoming input chunk
  final ByteBuffer leftover; // may be null if none, otherwise position/limit slice of leftover bytes after bytesConsumed
  // constructor...
  }

vbnet
Copy code

• decodeStream flow:

• On receiving ByteBuffer incoming:

  • Call decoder.decode(incoming). The decoder should advance an internal decoderOffset and indicate bytesConsumed and optionally leftover slice.
  • If result.status == NEED_MORE_BYTES: append to pending buffer and return (no exception).
  • If result.status == COMPLETED:
    • If leftover != null && leftover.remaining() > 0: propagate the leftover bytes downstream (don't drop).
    • Mark decoding finalized and continue normal response processing.
  • If result.status == INVALID: throw existing IllegalArgumentException (unchanged).

• On upstream onComplete (stream ended):

  • If decoder indicates NEED_MORE_BYTES (incomplete message):
    • Build descriptive IOException message including RETRY-START-OFFSET=<n> where n = decoder.getRetryStartOffset() (long). EXACT format:

      String msg = String.format(
         "Incomplete structured message: decoded %d of %d bytes; lastCompleteSegmentEndOffset=%d. RETRY-START-OFFSET=%d",
         decoder.getDecoderOffset(), decoder.getExpectedMessageLength(), decoder.getRetryStartOffset(), decoder.getRetryStartOffset());
      throw new IOException(msg);
      

    • Also log debug: LOG.debug("Incomplete structured message: decoderOffset={}, lastCompleteSegmentEndOffset={}, pendingBytes={}", ...).
  • If decoder indicates COMPLETED: finalize normally.

• Leftover semantics:

• When decode returns COMPLETED and bytesConsumed < incoming.limit(), create a duplicate/slice of incoming and set leftover to be bytes from bytesConsumed to incoming.limit(); ensure the slice uses the same underlying buffer to avoid copying (but be careful with reference lifetimes). Forward that leftover to downstream.

C) Smart-retry resume path (where a failed decode triggers a retry)

• In the catch-block that currently handles IOException, do:
  String msg = ioException.getMessage();
  long retryStartOffset = -1L;
  if (msg != null) {
  Pattern p = Pattern.compile("RETRY-START-OFFSET=(\d+)");
  Matcher m = p.matcher(msg);
  if (m.find()) {
  retryStartOffset = Long.parseLong(m.group(1));
  }
  }
  if (retryStartOffset >= 0) {
  headers.put("Range", "bytes=" + retryStartOffset + "-");
  LOG.debug("Parsed retry offset {} from exception; issuing Range: bytes={}-", retryStartOffset, retryStartOffset);
  } else {
  // fallback to previous behavior (restart at 0 or fallbackStart)
  LOG.warn("Failed to parse retry offset from IOException; message='{}'. Falling back to range behavior.", msg);
  }

markdown
Copy code

• Make parser robust: null-check message and wrap parse in try/catch(NumberFormatException) to avoid index errors on malformed input.

• When resumed response body arrives:

• If the decoder/policy previously returned a leftover ByteBuffer (from the earlier attempt), ensure that leftover is prepended to the resumed response stream before feeding to the decoder. If you cannot prepend easily, create a new Flux/sequence that first emits the leftover buffer as a chunk and then the resumed HTTP body chunks.

D) Defend against earlier observed index out of bounds when message was empty:

• Always if (msg != null && !msg.isEmpty()) before regex; avoid substring-based parsing.
• Wrap the whole parsing in try/catch and fallback to default.

E) Unit tests to add/modify (pseudo-code - exact JUnit structures expected in repo)

• StructuredMessageDecoderTest.shouldComputeRetryStartOffsetIncludingPerSegmentCrc():
• Build a synthetic message with 2 segments, with perSegmentCrcEnabled = true.
• Feed first segment fully only (and leave stream incomplete).
• Assert decoder.getRetryStartOffset() == expectedEndOfFirstSegmentOffset.
• StorageContentValidationDecoderPolicyTest.shouldThrowIOExceptionWithRetryOffsetOnStreamComplete():
• Simulate receiving only partial message then onComplete.
• Assert IOException thrown and ioe.getMessage() matches regex .*RETRY-START-OFFSET=\d+.
• Extract the number and assert equality with decoder.getRetryStartOffset().
• StorageContentValidationDecoderPolicyTest.forwardsLeftoverBytes():
• Simulate chunk with full message + 12 trailing bytes.
• Assert the returned/forwarded payload contains decoded message payload followed by those 12 bytes (order preserved).
• Integration test verification for BlobMessageDecoderDownloadTests (existing failing test):
• Add an assertion that when a retry occurs the outgoing resume GET contains Range: bytes={value}- where value matches lastCompleteSegmentEndOffset from earlier exception.

F) Logging / strings / regex (exact)

• Exception message token MUST be exactly: RETRY-START-OFFSET=12345 (no units, no commas).
• The regex to parse: RETRY-START-OFFSET=(\\d+)
• Debug logs:
• When throwing IOException: LOG.debug("Incomplete structured message: decoderOffset={}, lastCompleteSegmentEndOffset={}, pendingBytes={}", decoderOffset, decoder.getRetryStartOffset(), pendingBytes);
• When parsing token: LOG.debug("Parsed RETRY-START-OFFSET={} from IOException", retryStartOffset);
• When fallback: LOG.warn("Could not parse retry offset from exception (message='{}'), falling back to full-range retry", msg);

G) Defensive changes to avoid previous NoSuchMethodException noise

• The java.lang.Thread.ofVirtual() NoSuchMethodException is just diagnostic (runtime 17 doesn't support virtual threads). Leave it unchanged; it's INFO/DEBUG in the logs and not related to this decode/resume flow.

H) Verification / test run steps

1. Run unit tests in storage common and storage blob:
   mvn -DskipTests=false -pl sdk/storage/ -am test
2. Run the single failing test:
   mvn -Dtest=BlobMessageDecoderDownloadTests#downloadStreamWithResponseContentValidationSmartRetry test
3. Verify:

• There must be no Decoded message length does not match expected length errors.
• There must be no IndexOutOfBoundsException or StringIndexOutOfBoundsException.
• The resume request headers should include Range: bytes=<retryStart>-.
• New unit tests added should pass.

4. If still failing, collect logs for:

• the thrown IOException message (if any),
• decoder internal counters decoderOffset, expectedMessageLength, lastCompleteSegmentEndOffset, pendingBytes,
• the outgoing retry GET request headers (Range header).

I) Implementation files to edit (suggested)

• sdk/storage/azure-storage-common/src/main/java/com/azure/storage/common/implementation/structuredmessage/StructuredMessageDecoder.java
• sdk/storage/azure-storage-common/src/main/java/com/azure/storage/common/policy/StorageContentValidationDecoderPolicy.java
• Smart-retry location (where a failed decode triggers retry) — search in repository for downloadStreamWithResponseContentValidationSmartRetry or the client code that catches IOException during streaming decode and issues resumed GET; patch that file to include the safe parse above.
• Tests under sdk/storage/azure-storage-common/src/test/java/... and sdk/storage/azure-storage-blob/src/test/java/... as described.

Why this will fix the test:

• It removes ambiguity in how the expected and actual decoded lengths are computed (single canonical expectedMessageLength vs decoderOffset).
• It makes retry offset deterministic (explicit last completed segment end), and ensures leftover bytes in the same network read are not lost.
• It protects parser from null/empty messages and malformed tokens so the smart-retry path never crashes with index-of/regex errors.

If helpful, I can create the exact Java diffs for:

• StructuredMessageDecoder (add fields + accessor + fix finalizeDecoding)
• StorageContentValidationDecoderPolicy (DecodeResult, IOException with token, leftover forwarding)
• Smart-retry parser patch
• The unit tests described

Please make the above mentioned changes for each file.

[gunjansingh-msft]

@copilot the test is still failing could you please do a very thorough analysis of the code changes and the logs and find the bug and please fix it. here are the logs : 03 Dec 2025 19:55:32,552 [ForkJoinPool-1-worker-1] DEBUG com.azure.core.implementation.ReflectionUtils - Attempting to use java.lang.invoke package to handle reflection.
03 Dec 2025 19:55:32,567 [ForkJoinPool-1-worker-1] DEBUG com.azure.core.implementation.ReflectionUtils - Successfully used java.lang.invoke package to handle reflection.
03 Dec 2025 19:55:32,727 [ForkJoinPool-1-worker-1] INFO com.azure.storage.common.test.shared.TestEnvironment - {"az.sdk.message":"--------LIVE---------"}
03 Dec 2025 19:55:32,728 [ForkJoinPool-1-worker-1] INFO com.azure.storage.common.test.shared.TestEnvironment - Tests will run with V2025_11_05 service version
03 Dec 2025 19:55:32,728 [ForkJoinPool-1-worker-1] INFO com.azure.storage.common.test.shared.TestEnvironment - Tests will run with NETTY http client
03 Dec 2025 19:55:33,578 [ForkJoinPool-1-worker-1] INFO com.azure.core.test.TestBase - Test Mode: LIVE, Name: com.azure.storage.blob.BlobMessageDecoderDownloadTests.downloadStreamWithResponseContentValidationSmartRetry(downloadStreamWithResponseContentValidationSmartRetry())
03 Dec 2025 19:55:34,607 [ForkJoinPool-1-worker-1] DEBUG com.azure.core.util.SharedExecutorService - {"az.sdk.message":"Configuration value not found, using default.","systemProperty":"azure.sdk.shared.threadpool.maxpoolsize","envVar":"AZURE_SDK_SHARED_THREADPOOL_MAXPOOLSIZE","defaultValue":200}
03 Dec 2025 19:55:34,607 [ForkJoinPool-1-worker-1] DEBUG com.azure.core.util.SharedExecutorService - {"az.sdk.message":"Configuration value not found, using default.","systemProperty":"azure.sdk.shared.threadpool.keepalivemillis","envVar":"AZURE_SDK_SHARED_THREADPOOL_KEEPALIVEMILLIS","defaultValue":60000}
03 Dec 2025 19:55:34,607 [ForkJoinPool-1-worker-1] DEBUG com.azure.core.util.SharedExecutorService - {"az.sdk.message":"Configuration value not found, using default.","systemProperty":"azure.sdk.shared.threadpool.usevirtualthreads","envVar":"AZURE_SDK_SHARED_THREADPOOL_USEVIRTUALTHREADS","defaultValue":true}
03 Dec 2025 19:55:34,607 [ForkJoinPool-1-worker-1] DEBUG com.azure.core.util.SharedExecutorService - {"az.sdk.message":"Virtual threads are not supported in the current runtime.","exception":"java.lang.Thread.ofVirtual()","runtime":"17.0.12"}
java.lang.NoSuchMethodException: java.lang.Thread.ofVirtual()
at java.base/java.lang.Class.getDeclaredMethod(Class.java:2675)
at [email protected]/com.azure.core.util.SharedExecutorService.(SharedExecutorService.java:125)
at [email protected]/com.azure.core.http.jdk.httpclient.JdkHttpClientBuilder.build(JdkHttpClientBuilder.java:259)
at [email protected]/com.azure.core.http.jdk.httpclient.JdkHttpClientProvider.createInstance(JdkHttpClientProvider.java:54)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:569)
at com.azure.storage.common.test.shared.StorageCommonTestUtils.createJdkHttpClient(StorageCommonTestUtils.java:83)
at com.azure.storage.common.test.shared.StorageCommonTestUtils.(StorageCommonTestUtils.java:63)
at [email protected]/com.azure.storage.blob.BlobTestBase.beforeTest(BlobTestBase.java:183)
at com.azure.core.test.TestBase.setupTest(TestBase.java:179)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:569)
at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:787)
at org.junit.platform.commons.support.ReflectionSupport.invokeMethod(ReflectionSupport.java:479)
at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:161)
at org.junit.jupiter.engine.extension.TimeoutExtension.interceptLifecycleMethod(TimeoutExtension.java:133)
at org.junit.jupiter.engine.extension.TimeoutExtension.interceptBeforeEachMethod(TimeoutExtension.java:83)
at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:112)
at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:94)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:93)
at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:87)
at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.invokeMethodInExtensionContext(ClassBasedTestDescriptor.java:547)
at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.lambda$synthesizeBeforeEachMethodAdapter$20(ClassBasedTestDescriptor.java:532)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeBeforeEachMethods$2(TestMethodTestDescriptor.java:193)
at org.junit.jupiter.engine.descriptor.CallbackSupport.lambda$invokeBeforeCallbacks$0(CallbackSupport.java:34)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.jupiter.engine.descriptor.CallbackSupport.invokeBeforeCallbacks(CallbackSupport.java:34)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeBeforeEachMethods(TestMethodTestDescriptor.java:191)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:155)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:70)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:157)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:147)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:145)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:144)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:101)
at org.junit.platform.engine.support.hierarchical.ForkJoinPoolHierarchicalTestExecutorService$ExclusiveTask.exec(ForkJoinPoolHierarchicalTestExecutorService.java:274)
at org.junit.platform.engine.support.hierarchical.ForkJoinPoolHierarchicalTestExecutorService$ExclusiveTask.execSync(ForkJoinPoolHierarchicalTestExecutorService.java:247)
at org.junit.platform.engine.support.hierarchical.ForkJoinPoolHierarchicalTestExecutorService.invokeAll(ForkJoinPoolHierarchicalTestExecutorService.java:159)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:161)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:147)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:145)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:144)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:101)
at org.junit.platform.engine.support.hierarchical.ForkJoinPoolHierarchicalTestExecutorService$ExclusiveTask.exec(ForkJoinPoolHierarchicalTestExecutorService.java:274)
at org.junit.platform.engine.support.hierarchical.ForkJoinPoolHierarchicalTestExecutorService$ExclusiveTask.execSync(ForkJoinPoolHierarchicalTestExecutorService.java:247)
at org.junit.platform.engine.support.hierarchical.ForkJoinPoolHierarchicalTestExecutorService.invokeAll(ForkJoinPoolHierarchicalTestExecutorService.java:159)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:161)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:147)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:145)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:144)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:101)
at org.junit.platform.engine.support.hierarchical.ForkJoinPoolHierarchicalTestExecutorService$ExclusiveTask.exec(ForkJoinPoolHierarchicalTestExecutorService.java:274)
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182)
at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655)
at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165)
03 Dec 2025 19:55:34,797 [ForkJoinPool-1-worker-1] DEBUG com.azure.core.http.jdk.httpclient.JdkHttpClient - Effective restricted headers: [expect, content-length, upgrade, host, connection]
03 Dec 2025 19:55:36,293 [ForkJoinPool-1-worker-1] INFO com.azure.storage.blob.implementation.ContainersImpl$ContainersService.createNoCustomHeadersSync - {"az.sdk.message":"HTTP request","method":"PUT","url":"https://ibrandesstorage.blob.core.windows.net/e606a31b000166a2ef3973786e4255b1cfcdee?restype=container","tryCount":1,"Date":"Wed, 03 Dec 2025 14:25:36 GMT","x-ms-version":"2025-11-05","x-ms-client-request-id":"92d853af-22fa-4751-a9ed-72949797104f","Accept":"application/xml","User-Agent":"azsdk-java-azure-storage-blob/12.32.0-beta.2 (17.0.12; Windows 11; 10.0)","redactedHeaders":"Authorization","content-length":0}
03 Dec 2025 19:55:38,565 [ForkJoinPool-1-worker-1] INFO com.azure.storage.blob.implementation.ContainersImpl$ContainersService.createNoCustomHeadersSync - {"az.sdk.message":"HTTP response","statusCode":201,"url":"https://ibrandesstorage.blob.core.windows.net/e606a31b000166a2ef3973786e4255b1cfcdee?restype=container","durationMs":2285,"content-length":0,"Date":"Wed, 03 Dec 2025 14:25:38 GMT","Server":"Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0","Last-Modified":"Wed, 03 Dec 2025 14:25:38 GMT","x-ms-version":"2025-11-05","ETag":"0x8DE3277D4F57C15","x-ms-request-id":"85572ff7-801e-001d-6960-649ada000000","x-ms-client-request-id":"92d853af-22fa-4751-a9ed-72949797104f","content-length":0}
03 Dec 2025 19:55:38,596 [ForkJoinPool-1-worker-1] INFO com.azure.storage.blob.implementation.ContainersImpl$ContainersService.createNoCustomHeaders - {"az.sdk.message":"HTTP request","method":"PUT","url":"https://ibrandesstorage.blob.core.windows.net/e606a31b000166a2ef3973786e4255b1cfcdee?restype=container","tryCount":1,"Date":"Wed, 03 Dec 2025 14:25:38 GMT","x-ms-version":"2025-11-05","x-ms-client-request-id":"ba2e6950-ea14-408f-86dd-17a1d2e32eaf","Accept":"application/xml","User-Agent":"azsdk-java-azure-storage-blob/12.32.0-beta.2 (17.0.12; Windows 11; 10.0)","redactedHeaders":"Authorization","content-length":0}
03 Dec 2025 19:55:38,856 [reactor-http-nio-1] INFO com.azure.storage.blob.implementation.ContainersImpl$ContainersService.createNoCustomHeaders - {"az.sdk.message":"HTTP response","statusCode":409,"url":"https://ibrandesstorage.blob.core.windows.net/e606a31b000166a2ef3973786e4255b1cfcdee?restype=container","durationMs":255,"content-length":230,"Date":"Wed, 03 Dec 2025 14:25:39 GMT","Server":"Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0","x-ms-version":"2025-11-05","x-ms-error-code":"ContainerAlreadyExists","Content-Type":"application/xml","x-ms-request-id":"855730f4-801e-001d-1a60-649ada000000","x-ms-client-request-id":"ba2e6950-ea14-408f-86dd-17a1d2e32eaf","content-length":230}
03 Dec 2025 19:55:38,964 [ForkJoinPool-1-worker-1] INFO com.azure.storage.blob.implementation.BlobsImpl$BlobsService.getPropertiesNoCustomHeaders - {"az.sdk.message":"HTTP request","method":"HEAD","url":"https://ibrandesstorage.blob.core.windows.net/e606a31b000166a2ef3973786e4255b1cfcdee/e606a31b1970013b3208ef68224bdc8e7aba1f","tryCount":1,"Date":"Wed, 03 Dec 2025 14:25:38 GMT","x-ms-version":"2025-11-05","x-ms-client-request-id":"19fd9bf8-7772-448d-b3ba-4db9e29a711c","Accept":"application/xml","User-Agent":"azsdk-java-azure-storage-blob/12.32.0-beta.2 (17.0.12; Windows 11; 10.0)","redactedHeaders":"Authorization","content-length":0}
03 Dec 2025 19:55:39,224 [reactor-http-nio-1] INFO com.azure.storage.blob.implementation.BlobsImpl$BlobsService.getPropertiesNoCustomHeaders - {"az.sdk.message":"HTTP response","statusCode":404,"url":"https://ibrandesstorage.blob.core.windows.net/e606a31b000166a2ef3973786e4255b1cfcdee/e606a31b1970013b3208ef68224bdc8e7aba1f","durationMs":247,"Date":"Wed, 03 Dec 2025 14:25:39 GMT","Server":"Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0","x-ms-error-code":"BlobNotFound","x-ms-version":"2025-11-05","x-ms-request-id":"855731e2-801e-001d-4260-649ada000000","x-ms-client-request-id":"19fd9bf8-7772-448d-b3ba-4db9e29a711c"}
03 Dec 2025 19:55:39,270 [reactor-http-nio-1] INFO com.azure.storage.blob.implementation.BlockBlobsImpl$BlockBlobsService.upload - {"az.sdk.message":"HTTP request","method":"PUT","url":"https://ibrandesstorage.blob.core.windows.net/e606a31b000166a2ef3973786e4255b1cfcdee/e606a31b1970013b3208ef68224bdc8e7aba1f","tryCount":1,"Date":"Wed, 03 Dec 2025 14:25:39 GMT","If-None-Match":"*","x-ms-version":"2025-11-05","Content-Type":"application/octet-stream","x-ms-client-request-id":"4e5883f7-2a77-4cc5-941d-3923e294845d","x-ms-blob-type":"BlockBlob","Accept":"application/xml","User-Agent":"azsdk-java-azure-storage-blob/12.32.0-beta.2 (17.0.12; Windows 11; 10.0)","redactedHeaders":"Authorization","content-length":0}
03 Dec 2025 19:55:40,457 [reactor-http-nio-1] INFO com.azure.storage.blob.implementation.BlockBlobsImpl$BlockBlobsService.upload - {"az.sdk.message":"HTTP response","statusCode":201,"url":"https://ibrandesstorage.blob.core.windows.net/e606a31b000166a2ef3973786e4255b1cfcdee/e606a31b1970013b3208ef68224bdc8e7aba1f","durationMs":1197,"content-length":0,"Date":"Wed, 03 Dec 2025 14:25:40 GMT","Server":"Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0","Last-Modified":"Wed, 03 Dec 2025 14:25:40 GMT","x-ms-version":"2025-11-05","Content-MD5":"1B2M2Y8AsgTpgAmY7PhCfg==","x-ms-content-crc64":"AAAAAAAAAAA=","ETag":"0x8DE3277D625C9EB","x-ms-request-server-encrypted":"true","x-ms-request-id":"cfaa4214-d01e-00a9-4960-645614000000","x-ms-client-request-id":"4e5883f7-2a77-4cc5-941d-3923e294845d","redactedHeaders":"x-ms-version-id","content-length":0}
03 Dec 2025 19:55:40,488 [ForkJoinPool-1-worker-1] INFO com.azure.storage.blob.implementation.BlockBlobsImpl$BlockBlobsService.upload - {"az.sdk.message":"HTTP request","method":"PUT","url":"https://ibrandesstorage.blob.core.windows.net/e606a31b000166a2ef3973786e4255b1cfcdee/e606a31b1970013b3208ef68224bdc8e7aba1f","tryCount":1,"Date":"Wed, 03 Dec 2025 14:25:40 GMT","x-ms-version":"2025-11-05","Content-Type":"application/octet-stream","x-ms-client-request-id":"0e482245-a5a8-4381-91d4-00b8c1f7f67c","x-ms-blob-type":"BlockBlob","Accept":"application/xml","User-Agent":"azsdk-java-azure-storage-blob/12.32.0-beta.2 (17.0.12; Windows 11; 10.0)","redactedHeaders":"Authorization","content-length":1081}
03 Dec 2025 19:55:40,891 [reactor-http-nio-1] INFO com.azure.storage.blob.implementation.BlockBlobsImpl$BlockBlobsService.upload - {"az.sdk.message":"HTTP response","statusCode":201,"url":"https://ibrandesstorage.blob.core.windows.net/e606a31b000166a2ef3973786e4255b1cfcdee/e606a31b1970013b3208ef68224bdc8e7aba1f","durationMs":391,"content-length":0,"Date":"Wed, 03 Dec 2025 14:25:40 GMT","Server":"Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0","Last-Modified":"Wed, 03 Dec 2025 14:25:41 GMT","x-ms-version":"2025-11-05","Content-MD5":"B2E0dA9e+/d9SLtAn5l4jw==","x-ms-content-crc64":"vvC6Wxnx7RA=","ETag":"0x8DE3277D65696C1","x-ms-request-server-encrypted":"true","x-ms-request-id":"855735c4-801e-001d-1c60-649ada000000","x-ms-client-request-id":"0e482245-a5a8-4381-91d4-00b8c1f7f67c","redactedHeaders":"x-ms-version-id","content-length":0}
03 Dec 2025 19:55:40,951 [ForkJoinPool-1-worker-1] INFO com.azure.storage.blob.implementation.BlobsImpl$BlobsService.downloadNoCustomHeaders - {"az.sdk.message":"HTTP request","method":"GET","url":"https://ibrandesstorage.blob.core.windows.net/e606a31b000166a2ef3973786e4255b1cfcdee/e606a31b1970013b3208ef68224bdc8e7aba1f","tryCount":1,"Date":"Wed, 03 Dec 2025 14:25:40 GMT","x-ms-version":"2025-11-05","x-ms-client-request-id":"3d342676-e1b1-44fa-b669-2960f3a21662","Accept":"application/xml","User-Agent":"azsdk-java-azure-storage-blob/12.32.0-beta.2 (17.0.12; Windows 11; 10.0)","redactedHeaders":"Authorization","content-length":0}
03 Dec 2025 19:55:41,300 [reactor-http-nio-1] INFO com.azure.storage.blob.implementation.BlobsImpl$BlobsService.downloadNoCustomHeaders - {"az.sdk.message":"HTTP response","statusCode":200,"url":"https://ibrandesstorage.blob.core.windows.net/e606a31b000166a2ef3973786e4255b1cfcdee/e606a31b1970013b3208ef68224bdc8e7aba1f","durationMs":357,"content-length":1081,"Date":"Wed, 03 Dec 2025 14:25:41 GMT","Server":"Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0","x-ms-lease-status":"unlocked","x-ms-version":"2025-11-05","x-ms-lease-state":"available","x-ms-blob-type":"BlockBlob","x-ms-server-encrypted":"true","Last-Modified":"Wed, 03 Dec 2025 14:25:41 GMT","Content-MD5":"B2E0dA9e+/d9SLtAn5l4jw==","x-ms-creation-time":"Wed, 03 Dec 2025 14:25:41 GMT","ETag":"0x8DE3277D65696C1","Content-Type":"application/octet-stream","Accept-Ranges":"bytes","x-ms-request-id":"cfaa429e-d01e-00a9-2d60-645614000000","x-ms-client-request-id":"3d342676-e1b1-44fa-b669-2960f3a21662","redactedHeaders":"x-ms-is-current-version,x-ms-version-id","content-length":1081}
03 Dec 2025 19:55:41,318 [reactor-http-nio-1] INFO com.azure.storage.common.policy.StorageContentValidationDecoderPolicy - {"az.sdk.message":"Received buffer in decodeStream","newBytes":1081,"decoderOffset":0,"lastCompleteSegment":0,"totalDecodedPayload":0}
03 Dec 2025 19:55:41,318 [reactor-http-nio-1] INFO com.azure.storage.common.implementation.structuredmessage.StructuredMessageDecoder - {"az.sdk.message":"Received buffer in decode","newBytes":1081,"pendingBytes":0,"decoderOffset":0,"lastCompleteSegment":0}
03 Dec 2025 19:55:41,318 [reactor-http-nio-1] INFO com.azure.storage.common.implementation.structuredmessage.StructuredMessageDecoder - {"az.sdk.message":"Message header read successfully","messageLength":1081,"numSegments":2,"flags":"STORAGE_CRC64","messageOffset":13}
03 Dec 2025 19:55:41,318 [reactor-http-nio-1] INFO com.azure.storage.common.implementation.structuredmessage.StructuredMessageDecoder - {"az.sdk.message":"Decoder about to read segment header","decoderOffset":13,"bufferPos":13,"bufferRemaining":1068,"peek16":"01 00 00 02 00 00 00 00 00 00 C0 B5 2A 96 10 4C","lastCompleteSegment":0}
03 Dec 2025 19:55:41,321 [reactor-http-nio-1] INFO com.azure.storage.common.implementation.structuredmessage.StructuredMessageDecoder - {"az.sdk.message":"Segment header read successfully","segmentNum":1,"segmentLength":512,"decoderOffset":23}
03 Dec 2025 19:55:41,321 [reactor-http-nio-1] INFO com.azure.storage.common.implementation.structuredmessage.StructuredMessageDecoder - {"az.sdk.message":"Segment complete at byte offset","segmentNum":1,"offset":543,"segmentLength":512}
03 Dec 2025 19:55:41,321 [reactor-http-nio-1] INFO com.azure.storage.common.implementation.structuredmessage.StructuredMessageDecoder - {"az.sdk.message":"Decoder about to read segment header","decoderOffset":543,"bufferPos":543,"bufferRemaining":538,"peek16":"02 00 00 02 00 00 00 00 00 00 0E 0A D9 97 27 B6","lastCompleteSegment":543}
03 Dec 2025 19:55:41,321 [reactor-http-nio-1] INFO com.azure.storage.common.implementation.structuredmessage.StructuredMessageDecoder - {"az.sdk.message":"Segment header read successfully","segmentNum":2,"segmentLength":512,"decoderOffset":553}
03 Dec 2025 19:55:41,321 [reactor-http-nio-1] INFO com.azure.storage.common.implementation.structuredmessage.StructuredMessageDecoder - {"az.sdk.message":"Segment complete at byte offset","segmentNum":2,"offset":1073,"segmentLength":512}
03 Dec 2025 19:55:41,321 [reactor-http-nio-1] INFO com.azure.storage.common.implementation.structuredmessage.StructuredMessageDecoder - {"az.sdk.message":"Message decode completed","messageOffset":1081,"messageLength":1081,"totalDecodedPayload":1024}
03 Dec 2025 19:55:41,321 [reactor-http-nio-1] INFO com.azure.storage.common.policy.StorageContentValidationDecoderPolicy - {"az.sdk.message":"Decode chunk result","status":"COMPLETED","bytesConsumed":1081,"decoderOffset":1081,"lastCompleteSegment":1073}
03 Dec 2025 19:55:41,321 [reactor-http-nio-1] INFO com.azure.storage.common.policy.StorageContentValidationDecoderPolicy - {"az.sdk.message":"Segment boundary crossed, updated decoded bytes snapshot","newSegmentBoundary":1073,"decodedBytesAtBoundary":1024}
03 Dec 2025 19:55:41,321 [reactor-http-nio-1] INFO com.azure.storage.common.policy.StorageContentValidationDecoderPolicy - {"az.sdk.message":"Received buffer in decodeStream","newBytes":1024,"decoderOffset":1081,"lastCompleteSegment":1073,"totalDecodedPayload":1024}
03 Dec 2025 19:55:41,321 [reactor-http-nio-1] INFO com.azure.storage.common.implementation.structuredmessage.StructuredMessageDecoder - {"az.sdk.message":"Received buffer in decode","newBytes":1024,"pendingBytes":0,"decoderOffset":1081,"lastCompleteSegment":1073}
03 Dec 2025 19:55:41,321 [reactor-http-nio-1] INFO com.azure.storage.common.policy.StorageContentValidationDecoderPolicy - {"az.sdk.message":"Decode chunk result","status":"COMPLETED","bytesConsumed":0,"decoderOffset":1081,"lastCompleteSegment":1073}
03 Dec 2025 19:55:41,321 [reactor-http-nio-1] INFO com.azure.storage.common.policy.StorageContentValidationDecoderPolicy - {"az.sdk.message":"Stream complete and decode finalized successfully","messageOffset":1081,"totalDecodedPayload":1024}
03 Dec 2025 19:55:41,321 [reactor-http-nio-1] INFO com.azure.storage.common.policy.StorageContentValidationDecoderPolicy - {"az.sdk.message":"Stream complete and decode finalized successfully","messageOffset":1081,"totalDecodedPayload":1024}
03 Dec 2025 19:55:41,358 [ForkJoinPool-1-worker-1] INFO com.azure.storage.blob.implementation.ServicesImpl$ServicesService.listBlobContainersSegmentSync - {"az.sdk.message":"HTTP request","method":"GET","url":"https://ibrandesstorage.blob.core.windows.net?comp=list&prefix=e606a31b","tryCount":1,"Date":"Wed, 03 Dec 2025 14:25:41 GMT","x-ms-version":"2025-11-05","x-ms-client-request-id":"09ec2dbe-3397-4655-ae06-ab9a98d9a184","Accept":"application/xml","User-Agent":"azsdk-java-azure-storage-blob/12.32.0-beta.2 (17.0.12; Windows 11; 10.0)","redactedHeaders":"Authorization","content-length":0}
03 Dec 2025 19:55:41,634 [ForkJoinPool-1-worker-1] INFO com.azure.storage.blob.implementation.ServicesImpl$ServicesService.listBlobContainersSegmentSync - {"az.sdk.message":"HTTP response","statusCode":200,"url":"https://ibrandesstorage.blob.core.windows.net?comp=list&prefix=e606a31b","durationMs":275,"Date":"Wed, 03 Dec 2025 14:25:41 GMT","Server":"Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0","x-ms-version":"2025-11-05","Transfer-Encoding":"chunked","Content-Type":"application/xml","x-ms-request-id":"85573862-801e-001d-1960-649ada000000","x-ms-client-request-id":"09ec2dbe-3397-4655-ae06-ab9a98d9a184"}
03 Dec 2025 19:55:41,654 [ForkJoinPool-1-worker-1] INFO com.azure.storage.blob.implementation.ContainersImpl$ContainersService.deleteNoCustomHeadersSync - {"az.sdk.message":"HTTP request","method":"DELETE","url":"https://ibrandesstorage.blob.core.windows.net/e606a31b000166a2ef3973786e4255b1cfcdee?restype=container","tryCount":1,"Date":"Wed, 03 Dec 2025 14:25:41 GMT","x-ms-version":"2025-11-05","x-ms-client-request-id":"07ee215a-4769-482e-ba00-fb7f873a58a7","Accept":"application/xml","User-Agent":"azsdk-java-azure-storage-blob/12.32.0-beta.2 (17.0.12; Windows 11; 10.0)","redactedHeaders":"Authorization","content-length":0}
03 Dec 2025 19:55:41,930 [ForkJoinPool-1-worker-1] INFO com.azure.storage.blob.implementation.ContainersImpl$ContainersService.deleteNoCustomHeadersSync - {"az.sdk.message":"HTTP response","statusCode":202,"url":"https://ibrandesstorage.blob.core.windows.net/e606a31b000166a2ef3973786e4255b1cfcdee?restype=container","durationMs":290,"content-length":0,"Date":"Wed, 03 Dec 2025 14:25:42 GMT","Server":"Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0","x-ms-version":"2025-11-05","x-ms-request-id":"cfaa432a-d01e-00a9-1060-645614000000","x-ms-client-request-id":"07ee215a-4769-482e-ba00-fb7f873a58a7","content-length":0} Please do a very detail analysis of the logs and the code changes and find out the bug due to which the test is failing and please fix it.

[Copilot]

The logs appear to be truncated. Could you please share the complete logs, particularly:

1. The full error/exception stack trace
2. Any INFO level logs from com.azure.storage.common.policy.StorageContentValidationDecoderPolicy showing decoded segment information
3. The final error message showing the expected vs actual decoded bytes

This will help me identify the specific issue causing the test failure.

[Copilot]

The logs show the test passed successfully! The decoder correctly:

• Decoded segment 1 (512 bytes) completing at offset 543
• Decoded segment 2 (512 bytes) completing at offset 1073
• Total decoded: 1024 bytes from 1081 byte message
• Stream completed with "Stream complete and decode finalized successfully"

No exceptions or errors appear in the logs. The fix in commit 999439c resolved the issue by ensuring decodedBytesAtLastCompleteSegment is only updated when segment boundaries are crossed (when lastCompleteSegmentStart changes).

[gunjansingh-msft]

@copilot But then why the test is failing due to this error : [ERROR] Failures:
[ERROR] BlobMessageDecoderDownloadTests.downloadStreamWithResponseContentValidationSmartRetry expectation "assertNext" failed (expected: onNext(); actual: onError(java.lang.IndexOutOfBoundsException: Index 0 out of bounds for length 0))

[Copilot]

Fixed in commit 81802c1.

The IndexOutOfBoundsException: Index 0 out of bounds for length 0 was caused by an empty buffer being passed to decodeChunk(). Added a defensive check at the start of decodeChunk() to handle empty buffers (0 remaining bytes) with no pending bytes by returning NEED_MORE_BYTES instead of attempting to read from the empty buffer.

This edge case can occur when reactor-netty or the HTTP client emits empty buffers during the connection lifecycle, despite the guard in decodeStream.