Add an option to disable eager initialization of curl

[bjorn3]

Initializing curl in turn initializes OpenSSL, which takes up a non-trivial amount of time (~30%) for rustup which rustc even though it is wasted effort when rustup doesn't need to download anything after all. Lazily initializing curl when it actually needs to download something would avoid paying the initialization penalty when not necessary.

[sagebind]

See this issue for why the eager init constructor exists, for context: #333. The TL;DR is that it exists to help ensure that safety requirements of curl and OpenSSL are upheld so as to not invalidate our safe API wrapper. That safety requirement is essentially this:

• If you will use curl at all during the lifetime of the program, then curl::init() must be called before any calls to pthread_create have ever been made by any code within the same process.

Knowing that this is an incredibly difficult requirement to fulfill, the eager init ensures that this requirement is met safely and by default for most programs.

I could see a crate feature to enable or disable this might be reasonable. Features are additive so normally I would say that this would have to be an opt-in feature that is enabled by default, but the eager init is very much a cross-cutting concern that affects the entire binary. Maybe in practice an unsafe-disable-eager-init feature would work acceptably here.

However, for rustup's use-case, I'd have to ask if this actually solves rustup's problem. If rustup disables the eager init, can rustup meet this safety requirement? If it can't easily do so, then you'd need to call curl::init() manually early in your main() anyway, and thus you don't save any startup time.

To put it differently, to take advantage of such an opt-out feature, you must know early in your main() whether you will or will not use curl at all for the remainder of your process lifetime with 100% certainty in order to decide whether or not you will conditionally invoke curl::init(), before any threads have spawned.

[TimNN]

The TL;DR is that it exists to help ensure that safety requirements of curl and OpenSSL are upheld so as to not invalidate our safe API wrapper.

That actually seems to have changed since #333: https://curl.se/libcurl/c/curl_global_init.html now mentions this requirement only if the "threadsafe" flag is not set. (My understanding is that this flag should be enabled in most situations).

[sagebind]

It would appear that threadsafe was added in 7.84.0, so this would not apply when dynamically linking to a version of libcurl older than that. It does appear that curl_version_info can be used to check whether the currently-linked libcurl is threadsafe or not, but unfortunately, the docs say this about this function:

This function can alter the returned static data as long as curl_global_init has not been called. It is therefore not thread-safe before libcurl initialization occurs.

[TimNN]

Would something along these lines work?

• Where the global init is currently being done, check the version info
• If the version info indicates thread safe, defer initialization and do it lazily when needed.
• If the version info indicates not thread safe, eagerly perform the global initialization.