Response body is truncated in script

[ajithvl]

Hi,

I am trying to process a response body and send it back using the onResponse script. Here is my code

async function onResponse(context, url, request, response) {
  var url = "http://localhost:8080/proxyman/process";
  console.log("Executing POST " + url + " with " + response.body.length + " bytes")
  var output = await $http.post(url, {
    body: response.rawBody,
  });
  
  response = output
  return response;
}

The response.rawBody and response.body objects are getting truncated and its only sending partial text to my localhost endpoint.

This is the output I see on Script console in proxyman

[00:05:02.719]: onResponse() is executed with Flow ID = 24950
Executing POST http://localhost:8080/proxyman/process with 1210 bytes
00:05:02.729 onResponse() is executed!

The content length is much larger in my case but its always truncated to 1210 bytes. Is there a way to fetch the entire response body?
I suspect if its happening because of "content-encoding": "gzip" or "Transfer-Encoding": "chunked"

These are my headers

Request:

{
  "Host": "somehost.com",
  "Content-Type": "application/json",
  "Accept": "*/*",
  "Accept-Language": "en-US,en;q=0.9",
  "Accept-Encoding": "gzip, deflate, br",
  "User-Agent": "SomeAgent",
  "Connection": "close",
  "Content-Length": "34903"
}

Response:

{
  "date": "Thu, 10 Apr 2025 07:10:41 GMT",
  "content-type": "application/json",
  "cache-control": "no-store, no-cache, must-revalidate, post-check=0, pre-check=0",
  "content-encoding": "gzip",
  "expires": "Tue, 2 Jan 2001 08:00:00 GMT",
  "last-modified": "Thu, 10 Apr 2025 07:10:41 UTC",
  "via": "1.1 google",
  "Connection": "close",
  "Transfer-Encoding": "chunked"
}

Any help or guidance would be greatly appreciated.

Thanks

[NghiaTranUIT]

@ajithvl can you try to add this

console.log("response.rawBody = ", response.rawBody);

I'd like to see if rawBody is truncated or a entire body?

[ajithvl]

Thanks for the quick reply. Both rawBody and body has the same value and length of 1210. Actual response length should be 90664. Please let me know if there is anything else I could provide to troubleshoot this?

[ajithvl]

I was able to finally figure out the issue. The original request has Content-Type header as application/json because of which the proxyman script is serializing it as JSON but actually the request body is binary format.

In order to work around this i am doing the following

onRequest

request.headers["Content-Type"] = "application/octet-stream"

[NghiaTranUIT]

nice, thanks for the update 👍