Compile time and load time performance of jlls

[SimonDanisch]

I was assuming that JLLs and JLLWrappers must be incredibly well optimized considering there's almost no Julia package which doesn't load some JLLs.
Is that actually the case, or has there never been time/resources for it?
Context: we just saw that adding a very small JLL increased Makie's load time between 0.2-0.5s which seems huge.
I have a few thoughts I want to discuss, first of all this:

julia> @time using NativeFileDialog_jll
  0.103745 seconds (349.88 k allocations: 17.341 MiB, 4.40% gc time, 8.28% compilation time)

It stays with the compilation overhead, regardless if this was loaded before or not - this must be avoidable, no?
Also, doing everything in macros, doesn't that mean we copy and paste code and compile it separately while it could be compiled one time in JLLWrappers if it was a function?
I feel like there should only be a few functions with String/Pointer arguments, which could get compiled one time for each Julia version.
I guess we need macros for defining the different functions - do we know if just generating them directly as source code in the JLL could be faster?
Also, would something like product(:ffmpeg) instead of defining a new function for each product help here?
Considering that there have been breaking releases for JLLs, should we think about optimizing the interface for compile time and loading performance, or is that out of the question?
Another thing is, do we really need an __init__ function or could any setup be done lazily on the first call to e.g. ffmpeg()?
I'm also surprised that init also calls a bunch of macros, which seems avoidable? I'm loading ~50 JLLs, and pretty much only call into ~5% of them.
I admit I only had a very superficial look at the whole situation, since I don't have much time right now, so please excuse me if this has been discussed a million times or the questions are easily answerable if I'd dig a little deeper.
I would love to make some time to optimize this a bit, since Makie uses quite a few JLLs, but only if everyone involved thinks it's a good idea :D
To be fair, using @time_imports gives quite another picture (note this is after running this multiple times in a new session):

julia> @time_imports using GLMakie

Worst JLL offenders:

Package	Total Time	__init__ Time	Compilation %
FreeType2_jll	8.8 ms	8.6 ms	96.12%
OpenBLASConsistentFPCSR_jll	1.4 ms	1.2 ms
FFMPEG_jll	1.3 ms	1.0 ms
Xorg_libxcb_jll	1.1 ms	0.7 ms
Pango_jll	1.1 ms	0.8 ms
Libglvnd_jll	0.9 ms	0.6 ms
Glib_jll	0.9 ms	0.7 ms
SuiteSparse_jll	0.8 ms	0.5 ms
Xorg_libxkbfile_jll	0.8 ms	0.3 ms
FFTW_jll	0.7 ms	0.5 ms
EarCut_jll	0.5 ms	0.2 ms
Bzip2_jll	0.5 ms	0.2 ms
FriBidi_jll	0.5 ms	0.2 ms
GettextRuntime_jll	0.5 ms	0.1 ms
Fontconfig_jll	0.5 ms	0.1 ms
Cairo_jll	0.5 ms	0.3 ms
Wayland_jll	0.5 ms	0.2 ms

[giordano]

a very small JLL

NativeFileDialog_jll depends on GTK, which is one the largest stacks we have, it brings 70 other JLLs behind, so calling it "very small" is very misleading.

[SimonDanisch]

Oh, I was working of the description:

A tiny, neat C library that portably invokes native file open, folder select and save dialogs.

This indicated to me, that they go the extra mile to call into OS apis, without heavy dependencies

Guess I should have looked at the actual Project.toml.

I'm still not sure if it should recompile on every using?

[SimonDanisch]

So, 0.103745s is relatively close to 70*1ms. That still leaves the question if that can't be loaded more quickly, especially if GTK isn't even used on e.g. windows.

[giordano]

On macOS and Windows the GTK stack shouldn't be loaded at all: https://github.com/JuliaPackaging/Yggdrasil/blob/c58c5abe64e177bc6ed6aa775aff1fefbfe85517/N/NativeFileDialog/build_tarballs.jl#L42. It's only relevant on Linux and FreeBSD.