Add Example of computing content length for Spring WebClient multipart/form-data for upload file in chunks #33628
Description
Hi, I've noticed that multipart/form-data does not include a content length when the request is created with multipart/form-data.
I come across to this issue "Spring WebClient multipart/form-data does not include a content length" . Provided solution working fine and inside ExchangeFilterFunction, content-length is well calculated in case of uploading file in a single chunk using any Third Part API.
But in case of uploading file in multiple chunks (for large size) functionality started failing and I am receiving below exception response from Third Party API
{The length of the file input stream does not match the provided length. Expected: X, Actual (approximate): 'Y' "}
Could someone help me to provide the solution how to calculate the content for each chunk while uploading file in multiple chunks for large file.
Below is code snippet for reference
public Mono<Object> publishFileUsingChunks(Mono<DataBuffer> dataBufferMono, String generatedFileName, String accessToken) {
this.webClientBuild(); // initialize webclient.builder
Mono<InputStream> inputStreamMono = dataBufferMono.map(dataBuffer -> dataBuffer.asInputStream(true));
log.debug("dataBufferMono to inputStreamMono ... and the given filename :: "+ generatedFileName);
return inputStreamMono.zipWhen(
inputStream -> {
try(BufferedInputStream in = new BufferedInputStream(inputStream)) {
int chunkSize= 40485760; //40MB (approx)
byte[] bytes = new byte[chunkSize];
int length = 0;
Mono<String> fileUploadResponseMono = null;
while((length = in.read(bytes)) != -1){
log.debug("upload in chunks has been started and length:: "+ length);
byte[] inpBufChunk = ArrayUtils.subarray(bytes, 0, length);
Mono<String> resp = importFileInChunks(accessToken, inpBufChunk, generatedFileName);
if(fileUploadResponseMono !=null){
fileUploadResponseMono = fileUploadResponseMono.zipWhen(
fileUploadResponse -> resp, (fileUploadResp, response) -> response);
}else {
fileUploadResponseMono = resp;
}
}
return fileUploadResponseMono;
} catch (IOException e) {
e.printStackTrace();
throw new CMSWriteServiceException(e.getMessage());
}
},
(inputStream, uploadResponse)-> uploadResponse
);
}
private Mono<String> importFileInChunks(String accessToken, byte[] inpBufChunk, String generatedFileName) {
String uri=RELATIVE_URL;
MultipartBodyBuilder builder = new MultipartBodyBuilder();
builder.part("file", inpBufChunk);
return webClient.post().uri(uriBuilder -> uriBuilder.path(uri)
.build())
.header(HttpHeaders.AUTHORIZATION, "Bearer " +accessToken)
.header(HttpHeaders.CONTENT_TYPE, MediaType.MULTIPART_FORM_DATA_VALUE)
.bodyValue(builder.build())
.retrieve()
.onStatus(HttpStatus::isError, response ->
response.bodyToMono(String.class).flatMap(error ->
Mono.error(new Exception(" Error: --->"+ error)))
)
.bodyToMono(String.class)
.doOnSuccess( response ->
log.debug("uploaded multipart successfully:: {}", response)
)
.doOnError(WebClientResponseException.class, error ->
log.error("Error Message :: {} Error Response :: {}",error.getMessage(), error.getResponseBodyAsString())
);
}
private void webClientBuild() {
this.webClient = WebClient.builder()
.codecs(clientCodecConfigurer -> clientCodecConfigurer.defaultCodecs()
.maxInMemorySize(500000000))
.baseUrl(BASE_URL)
.filter(new MultipartExchangeFilterFunction())
.defaultHeader(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE)
.defaultHeader(HttpHeaders.ACCEPT, MediaType.APPLICATION_JSON_VALUE)
.defaultHeader(HttpHeaders.ACCEPT_ENCODING, "gzip, deflate, br")
.defaultHeader("KeepAlive", "true")
.build();
}