LLEXT: deferred logging still broken in multiprocessor configurations

[lyakh]

Describe the bug

This is a race between a context, destroying an LLEXT and the logging thread. Previously #77289 this race has been fixed for cases, when both contexts are running on the same CPU. However that fix doesn't work when the contexts run on different CPUs.

To Reproduce

This has been reproduced with SOF when using the "nocodec" topology, which utilises multiple DSP cores.

Expected behavior

A proper synchronisation mechanism should reliably suspend LLEXT destruction until logging is flushed.

Impact

Blocks SOF LLEXT use in multi-core configurations

[teburd]

I tried a few things... none of them were very helpful sadly.

I add a void log_flush(void) function to log_core.c with the attempts below

k_thread_suspend(&log_process_thread);
while(log_process() == true) {}
k_thread_resume(&log_process_thread);

Or

k_sem_give(&log_process_sem);
k_thread_suspend(&log_process_thread);
k_thread_resume(&log_process_thread);

Or even with an explicit semaphore denoting the thread is done processing...

k_thread_suspend(&log_process_thread);
k_sem_reset(&log_process_done);
k_sem_give(&log_process_thread);
k_thread_resume(&log_process_thread);
k_sem_take(&log_process_done, K_FOREVER);

All of these attempts failed to actually flush the log and get the expected unload behavior we want.

[teburd]

The failure turned out to be a weird mis-build, with __syscall the symbol for log_flush existed so the build would work sort of, but then result in very strange behavior as log_core.c wasn't always built (with z_impl_log_flush).

Sorting this out so its only called when the symbol exists now, and removing __syscall sorted this all out though does require some ifdefry.

[pillo79]

with __syscall the symbol for log_flush existed so the build would work sort of

Isn't this a side effect of the whole "weak NULL _impl definitions for syscalls export" thing? I'm surprised it wasn't mentioned earlier (or I hadn't heard of before!). Building with CONFIG_LLEXT will happily link any declared syscall with NULL, if they aren't actually defined... 💣.

Would be nice to have an actual link error on weak uses outside of syscall_export_llext.c, but achieving that requires serious linker wizardry AFAICS.

[teburd]

with __syscall the symbol for log_flush existed so the build would work sort of

Isn't this a side effect of the whole "weak NULL _impl definitions for syscalls export" thing? I'm surprised it wasn't mentioned earlier (or I hadn't heard of before!). Building with CONFIG_LLEXT will happily link any declared syscall with NULL, if they aren't actually defined... 💣.

Would be nice to have an actual link error on weak uses outside of syscall_export_llext.c, but achieving that requires serious linker wizardry AFAICS.

I wasted some time on this for sure, was not immediately obvious this was the problem.

[teburd]

This should be fixed with #81360