Documentation around dear_bindings in a .dll / .so / .dylib for plugin-aware C software

[ludolpif]

Hi,
I came more with questions than solutions initially, but I hope this can became a .md in this repository about how-to use dear_bindings in some non-trivial use cases.

To be transparent, I currently work on a mock up of a OBS like live streaming software, exploring alternative software designs here : https://ludolpif.fr/llss. I try a "plain C" mock up with this stack : SDL3 + SDL_gpu + (ImGui + dear_bindings for C) + FLECS + plugin-system based on SDL_loadso.h (which is dlopen() and for windows LoadLibrary()). Dynamically loaded plugins should be able to draw additional ImGui Window and widgets in the main app UI.
So I am particularly interested to get this use case nice and neat but document here other (hopefully simpler) use cases seems a positive thing to try to do, if they turn to have some dear_bindings specificity.

Various affirmations from original project :

• "No specific build process is required" [1]
• "It is recommended that you follow those steps and not attempt to build Dear ImGui as a static or shared library!" [2]
• "Note that Dear ImGui is a very call-heavy API, so building as a shared library is not ideal because of increased function calling overhead" [2]
• "If your application already uses the API you pulled the backends for, things should compile and link already." [2]
• DLL users : [3]
  • Heaps and globals are not shared across DLL boundaries!
  • You will need to call SetCurrentContext() + SetAllocatorFunctions() for each static/DLL boundary you are calling from.
  • Same applies for hot-reloading mechanisms that are reliant on reloading DLL (note that many hot-reloading mechanisms work without DLL).
  • Using Dear ImGui via a shared library is not recommended, because of function call overhead and because we don't guarantee backward nor forward ABI compatibility.

[1] https://github.com/ocornut/imgui?tab=readme-ov-file#usage
[2] https://github.com/ocornut/imgui/wiki/Getting-Started#compilinglinking
[3] https://github.com/ocornut/imgui/blob/v1.92.5/imgui.cpp#L1362

Note that I am trying to dig first the "dynamic loading" topic (which is dlopen() like) and not "dynamic linking" which is what the OS do when execve() a dynamically linked program) and in ImGui case is not recommended and except in a plugin-system based software, you could always skip it as far as I know.

I recently found "may be good" entry points around "dynamic loading"

• https://maskray.me/blog/2021-05-16-elf-interposition-and-bsymbolic
• https://blog.k3170makan.com/2018/11/introduction-to-elf-format-part-vii.html

Now : I am far from knowing everything is said there. But I hope learn. I am a linux sysadmin that know some of low-level stuff but not ELF details nor DLL model principles.

In the plugin-system based software case, few questions arose :

• the "ABI" question : is a plugin compiled last year will be loadable in the newest release of the software ?
  • ImGui don't give any warranty, but the software may choose to upgrade ImGui only when it is raising ABI version for other reasons.
  • Plugin will use basic widgets, integrating frequently bleeding edge release of ImGui will probably never required.
• the "where put the code" question :
  • ImGui + dear_binding code reside in the hosting software only ?
    • so it have to expose every dear_bindings C API functions as dynamic symbols to the plugins.
  • ImGui + dear_binding code reside in the plugins too ?
    • it may to greatly reduce dynamic symbol quantities (ImGui data structures only ?)
    • it may help to have near-maximal performance
    • but from a compiler to another (plugins authors aren't main software authors), bitfield layout or other problems could arise ?
• the "how I really do compile" question :
  • in GNU/Linux and gcc toolchain world
  • in Windows Visual Studio world
  • (in MacOS X world but I didn't have any so I may never try myself)
• the "don't try that" question :
  • what is already known to be a dead path to follow / explore ?

Hoping that some people have clues, ideas, think-out-the-box perspectives around here !
Thanks for all the fish,

[ShironekoBen]

Hmm... that's a very wide topic, to be honest. My instinctive responses would be along the lines of:

the "ABI" question : is a plugin compiled last year will be loadable in the newest release of the software ?

I suspect that's going to be very hard to guarantee unless you either control the whole API surface (i.e. by wrapping ImGui and any other third-party libraries), or as you say are prepared to sit on a specific version until you are prepared to break compatibility. There's a whole class of changes that are effectively benign in terms of source compatibility (for example, adding a field to a struct) which will immediately break binary compatibility in pretty horrible ways.

You might have some luck with defining a "safe" subset of the API and only exposing that to plugins, but I suspect the moment you start looking at anything remotely complex you're going to end up either having to wrap everything or lock your versions.

If it was me I'm pretty sure I'd go down the route of "upgrading ImGui (or almost any other third party component exposed to the plugin API) is always considered a hard compatibility break" and refuse to even load plugins that were compiled with a different version - the potential for subtle memory corruption issues causing end-user bug reports that are nigh-on impossible to track down is just too high.

the "where put the code" question

ImGui in the host is the obvious answer here, as that is definitely something that can be made to work (albeit with the ABI concerns previously discussed!). I don't personally think the performance issues are worth getting too worried about - they're definitely real, but at the same time you'd need to be doing a lot of UI before the overhead of a bit of dynamic linking thunk magic becomes something that really starts to hurt, I think. Given your other constraints, it doesn't feel like it would be worth putting a lot of weight on it as a design consideration, especially since at this stage it's not even clear if it would ever be a problem (and if it did become one, the chances are high you could do some sort of relatively straightforward workaround like hoisting a few high-overhead functions into your plugin API layer rather than needing a full redesign to fix it).

That said, ImGui in the plugin itself is also definitely doable, I think - the easy way would be to simply expose a basic triangle drawing API to plugins, and the write a backend that lets ImGui sit on top of that. That would be pretty elegant way to handle it - you wouldn't have any ABI issues (well, aside from those in your own API, of course), and plugins could use whatever version of ImGui they wanted to, even mix-and-matching within the same app. The downside - and it's a very big one - is that those ImGui instances wouldn't really know anything about each other, and so if your design requires that multiple plugins present windows at once you'll need to do a lot of behind-the-scenes hackery to get stuff like input focus and window ordering to behave itself. That's almost certainly a very painful thing to sort out, I suspect. But if you're more in the "only one plugin has control of the screen at a time" space then it would be a very viable option.

A half-way house where some ImGui stuff lives in the host and some in the plugin.... sounds like a nightmare, to be honest. There might be a clever way to do it, but off the top of my head I can't think of one that isn't going to basically take every version compatibility problem and DLL memory allocator issue and so on and make them twenty times worse. :-(

the "how I really do compile" question

That's really a question of what you're trying to achieve, I think. ImGui is very compiler and build-system agnostic so I think whatever works best for the rest of your code should be fine. From what I've heard so far your biggest constraint here may be that you'll likely need to pick something that is easy for plugin authors to work with, as allowing mix-and-matching compilers/toolchains opens up another can of worms with compatibility - similarly compiler options will need to be carefully considered as some have a big impact on binary compatibility (LTCG, for example). (See also: the hilarity that is standard library binary compatibility between compilers/versions)

the "don't try that" question

I don't really know here - almost everything I've worked on personally that has plugins or similar went for source-level compatibility (i.e. the assumption is that you recompile plugins for every new version of the host) as that just makes life a million times easier... but those were all cases where "please recompile everything" had a reasonably minimal overhead and so it was an easy decision to make.

Anyway, that's a brain-dump of what occurred to me whilst reading the above - don't take any of it as any kind of absolute truth because there are almost certainly things I'm forgetting/never knew in the first place/haven't thought of, but hopefully it's of some kind of use!