Health check request should be chunked

[paulb-elastic]

Similar to #1808, this issue describes the need to support chunked encoding for the health check in of the Elastic Agent with the Fleet Server, to support situations with Synthetics Private Locations where the policy size is too big and the health check can fail, which results in the Agent appearing to be offline (even though it is still online and getting updates).

Refer to the knowledge base article where this is referred to.

As a result of elastic/fleet-server#5069 it’s possible to see body too large when these problems occur.

Note that whilst server.limits.checkin_limit.max_body_byte_size can be set to -1 or something larger than the default, it should be possible for Elastic Agent running as Synthetics Private Locations to check in their health status without this manual intervention.

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[ebeahan]

@paulb-elastic discussed this a bit with @jlind23. I've already filed elastic/fleet-server#5549 (at the request of @lucabelluccini) to increase the checkin body size limit defaults in Fleet Server.

Our suggestion is to approach this in two phases:

• First, increase the default setting server.limits.checkin_limit.max_body_byte_size to a default to better accommodate Synthetic Private Location use cases. We'd hope to address 90+% of user needs to avoid manual intervention.
• Second, in the medium term implement chuck encoding as suggested here.