Proposal: Use a global HttpClient instance for performance optimization

[RecursionTaoist]

Since 0.9.0, each HTTP request creates a new HttpClient instance. According to JDK documentation, HttpClient is designed to be immutable and thread-safe, and should be reused throughout the application's lifetime. Initializing a new HttpClient per request can introduce significant overhead.

Proposal: Refactor the internal implementation to use a single global (or session-scoped) HttpClient instance, reused for all requests and redirects unless special configurations (e.g. proxy, SSL context) require otherwise.

Please consider this optimization.

[lihaoyi]

How much optimization can we expect this to provide?

[RecursionTaoist]

From what I recall, when running the same set of unit tests, the v0.9 implementation using JDK HttpClient was about 10x slower than the JDK8 Http(s)URLConnection implementation (or at least noticeably much slower).

It seems like the Http(s)URLConnection implementation should represent the optimization ceiling for v0.9.

Since Http(s)URLConnection isn't marked as enabled, could we consider continuing to use it?

[lihaoyi]

@RecursionTaoist HttpURLConnection has problems with doing HEAD requests and others, which is why we changed. You can continue to use older versions of requests-scala if you want to keep using it.

It would be great if you could run some benchmarks for shared HttpClient v.s. per-request HttpClient v.s. HttpURLConnection to see what kind of performance differences there are. If there are significant differences, I would be happy to move to a shared HttpClient model