Skip to content

Comments

fix(inputs.prometheus): Add thread safety and proper cleanup for shared informer factories#18377

Merged
mstrandboge merged 3 commits intoinfluxdata:masterfrom
skartikey:inputs_prometheus_thread_safety
Feb 23, 2026
Merged

fix(inputs.prometheus): Add thread safety and proper cleanup for shared informer factories#18377
mstrandboge merged 3 commits intoinfluxdata:masterfrom
skartikey:inputs_prometheus_thread_safety

Conversation

@skartikey
Copy link
Contributor

Summary

Follow-up to #18304 which added delete(informerfactory, p.PodNamespace) in Stop() to clean up shared informer factories. That fix addressed the resource leak but left three issues:

  • Race condition: The informerfactory map is package-level shared state accessed concurrently by multiple plugin instances during Start()/Stop() without synchronization, this is undefined behavior in Go and can panic at runtime

  • Goroutine leak: Deleting from the map doesn't stop the watcher goroutines spawned by SharedInformerFactory.Start(), calling Shutdown() is required to terminate them

  • Premature deletion: If two instances share a factory (same namespace), the first to stop deletes it from under the second

Checklist

Related issues

related #18304

…ed informer factories

The shared informer factory map is accessed concurrently by multiple
plugin instances during Start/Stop, causing race conditions. The prior
cleanup (delete from map) also leaked goroutines by not calling
Shutdown() and could prematurely remove a factory still in use by
another instance.

Add a mutex to protect all access to the shared informer factory map,
a reference counter to track active users per namespace, and an explicit
Shutdown() call when the last user releases a factory.
@telegraf-tiger telegraf-tiger bot added area/prometheus fix pr to fix corresponding bug plugin/input 1. Request for new input plugins 2. Issues/PRs that are related to input plugins labels Feb 19, 2026
Copy link
Member

@srebhan srebhan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @skartikey! One question...

Copy link
Member

@srebhan srebhan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One more formatting issue...

@telegraf-tiger
Copy link
Contributor

Copy link
Member

@srebhan srebhan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @skartikey!

@srebhan srebhan added the ready for final review This pull request has been reviewed and/or tested by multiple users and is ready for a final review. label Feb 23, 2026
@srebhan srebhan assigned mstrandboge and unassigned srebhan Feb 23, 2026
@mstrandboge mstrandboge merged commit a1916c3 into influxdata:master Feb 23, 2026
26 of 27 checks passed
@github-actions github-actions bot added this to the v1.37.3 milestone Feb 23, 2026
mstrandboge pushed a commit that referenced this pull request Feb 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/prometheus fix pr to fix corresponding bug plugin/input 1. Request for new input plugins 2. Issues/PRs that are related to input plugins ready for final review This pull request has been reviewed and/or tested by multiple users and is ready for a final review.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants