Skip to content

Conversation

@alecholmes
Copy link
Contributor

@alecholmes alecholmes commented Jul 9, 2025

This refactors the Go plugin glue code with a few goals in mind:

  1. Correctness, especially clearer behavior around state transitions, acquiring locks, and cleaner shutdown. No special case behavior exposed for testing.
  2. Readability, especially around control flow and concurrency.
  3. Testability without using or mutating global state, and also more explicit lifecycle tests. No more test-only locks like theInputLock or test-only branching around multiInstance.

This cleanly separates the stateless CGO boundary functions in cshared.go, such as FLBPluginInit, from the stateful implementations. The C interface functions and their behavior are unchanged.

The changes are largely:

  1. Immutable plugin metadata is encapsulated in the pluginMetadata struct.
  2. The plugin instance, which changes over time on multiple threads, is encapsulated in the pluginInstance struct.

The new TestInputCallbackLifecycle gives a sense of how this comes together. This test notably relies on no global state.

You can view this PR as a precursor to supporting multiple instances of the same plugin in fluent-bit.

@alecholmes
Copy link
Contributor Author

alecholmes commented Jul 9, 2025

This change is part of the following stack:

Change managed by git-spice.

@alecholmes alecholmes force-pushed the alec/2025-07-08-encapsulate-plugin-instance branch from d9607df to beb6f18 Compare July 10, 2025 16:20
// also, the interfaces for input and output plugins.
package plugin

import "C"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is coming in from C.size_t use here:

func (p *pluginInstance) callback(data *unsafe.Pointer, csize *C.size_t) int {

If there's some reason we want to contain CGO to cshared.go, that's doable.

"github.com/calyptia/plugin/metric"
)

var (
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All the globals are in this var block.

@alecholmes alecholmes force-pushed the alec/2025-07-08-encapsulate-plugin-instance branch 2 times, most recently from 4e14c14 to f02efa2 Compare July 10, 2025 16:54
@alecholmes alecholmes changed the title Encapsulation for Go plugin instances [sc-00000] Encapsulation for Go plugin instances [sc-141627] Jul 10, 2025
@alecholmes alecholmes force-pushed the alec/2025-07-08-encapsulate-plugin-instance branch from d641376 to 5add7d9 Compare July 10, 2025 23:43
@alecholmes alecholmes force-pushed the alec/2025-07-08-encapsulate-plugin-instance branch from cbb2a94 to 06cf1dc Compare July 11, 2025 16:47
return input.FLB_ERROR
}

func cleanup() int {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Global cleanup logic moved into FLBPluginExit, which was previously calling this.

Copy link
Contributor

@stoksc stoksc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this looks really good, nice. all questions are just for my own learning.

initWG.Add(1)
defer initWG.Done()
func FLBPluginInit(ptr unsafe.Pointer) (respCode int) {
currInstanceMu.Lock()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just so i'm following, currInstanceMu replaces initWG? i'm guessing the ordering it looked like it was enforcing was already assured, just needed mutual exclusion?

if runCancel != nil {
runCancel()
runCancel = nil
currInstanceMu.Lock()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why did the previous code do a trylock and just bail if it didn't work..? would it have left the plugin unpaused?

p.mu.Lock()
defer p.mu.Unlock()

if p.state != instanceStateInitialized {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice that paused can just be initialized

if p.state == instanceStatePreExit {
return nil
}
if p.state != instanceStateRunnable {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i notice this can newly cause FLBPluginOutputPreExit to panic

how concerned are you that there are paths through the state machine that don't make sense / arent really supposed to happen but do

p.runCancel()

// Wait for any running callback/flush to finish before closing the message channel
p.runningWG.Wait()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

noticed this used to not block to. i imagine this is on purpose, but any concern about unintended fallout?

@stoksc
Copy link
Contributor

stoksc commented Jul 11, 2025

obviously it is still in draft, but approved in concept and that once tests are passing think it is great

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants