Replies: 14 comments
-
|
I don't believe this has been implemented. |
Beta Was this translation helpful? Give feedback.
-
|
@morrone it has not. the current framework does not retain all the information needed to do this reliably, and i'm pretty sure that the way the apis are and the way remote references are managed, it would be very difficult to prove what a correct shutdown sequence is. But given that caveat, i've actually implemented (externally) a shutdown sequence heuristic that for file-based configurations has not failed me yet. It is implemented in ldms-static-test.sh and consists, overall, of generating the inverse of the setup input script. It has proved extremely useful in testing with valgrind for 'real' leaks. |
Beta Was this translation helpful? Give feedback.
-
|
Well, how about we constrain the ticket to sampler plugins. Things are pretty straight forward there, correct? We just need to call the term() function for each sampler. |
Beta Was this translation helpful? Give feedback.
-
|
Even with sampler plugins, we need the reverse-of-startup sequence for minimum generation of goofy errors/warnings. It's fairly typical to start something like a job_id sampler first (and with a slightly negative offset) and then all the other plugins that would like to depend on jobid being present to avoid log whinging. afaik, plugins to not save a 'time-started' field by which we could sort them for shut down. |
Beta Was this translation helpful? Give feedback.
-
|
I think there was at a point some notion of being able to ask a daemon to dump its configuration history, but that's an even harder problem than "show me your current configuration". I don't know where either is in the current work plans. Where I've found the 'tear-down by reversing config history' most useful is in fact on the storage daemons. |
Beta Was this translation helpful? Give feedback.
-
|
I don't have any interest in a daemon dumping its configuration history. A daemon has a table somewhere of every sampler plugin and that plugin's associated function pointers. At exit time, the daemon should walk that table and call term() on each sampler. That is all that I'm asking for. It is a really basic things that most daemons do. |
Beta Was this translation helpful? Give feedback.
-
|
We (volunteers?) could iterate over ldmsd_config.c:plugin_list checking for entries that are SAMPLER type, call stop in each, then repeat that calling term on each. This process must be fault tolerant (term has 'busy' as a reply). More significantly, sampler plugins are able to be publishers and subscribers, so it's conceivable that arbitrary stop/term may hang in event handler conflicts. |
Beta Was this translation helpful? Give feedback.
-
|
@morrone, @baallan please reopen as necessary. But this is a bag of worms in my opinion. Reopen if you would like to go fishing. |
Beta Was this translation helpful? Give feedback.
-
|
If it is a bag of worms, then should plugins even bother to have term() functions?? I think this should remain open on the books, because this is just standard code shutdown behavior that is missing in ldmsd. |
Beta Was this translation helpful? Give feedback.
-
|
@morrone, I agree. Moved to 4.5.1 |
Beta Was this translation helpful? Give feedback.
-
|
In a future plan, where we first document and make fully sane/thread-well-defined the lifecycle of each plugin type, I expect the proper approach will be for each plugin to have a stop function (or at least for the plugin-driving loops to have a stop state for each plugin (one that also block further stream deliveries)) and then for all the plugins to be terminated. This would let us quiesce an ldmsd instance and reconfigure it with a new set of plugins without killing the instance. Part of this may involve loss of messages or set updates, but so would restarting the daemon entirely. |
Beta Was this translation helpful? Give feedback.
-
|
Why? If you are blowing away all the state what is the point? Let the
kernel do it.
…On Tue, Oct 22, 2024, 5:33 PM Benjamin Allan ***@***.***> wrote:
In a future plan, where we first document and make fully
sane/thread-well-defined the lifecycle of each plugin type, I expect the
proper approach will be for each plugin to have a stop function (or at
least for the plugin-driving loops to have a stop state for each plugin
(one that also block further stream deliveries)) and then for all the
plugins to be terminated. This would let us quiesce an ldmsd instance and
reconfigure it with a new set of plugins without killing the instance. Part
of this may involve loss of messages or set updates, but so would
restarting the daemon entirely.
—
Reply to this email directly, view it on GitHub
<#51 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABVTPXF5R4LLPIS4PSBBEP3Z43OEFAVCNFSM6AAAAABQNMBLLOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMZQGUYDKMZYGQ>
.
You are receiving this because you modified the open/close state.Message
ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
Well, that's not terribly helpful for a daemon (or plugin) that involves a lot of handshaking with stream or web protocols or databases. |
Beta Was this translation helpful? Give feedback.
-
|
Transferred to a discussion to facilitate planning for a proposed fix. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
The ldmsd should stop and call term() for plugins at exit time. Right now (at least in v3) exit happens without plugin cleanup unless the user explicitly issues stop and term commands.
Beta Was this translation helpful? Give feedback.
All reactions