Skip to content

Fix potential connection storm #398

@wenhaocs

Description

@wenhaocs

This affects 5.2.0.

NebulaPool creates multiple clients. When strictly_server_healthy is set to false, each client creates a connection to every server. Consider a scenario where one Graphd instance goes down. When calling get_client, if test_on_borrow is set to true and ping_before_execute is set to false, the client will send a ping to one host in a round-robin manner. If the ping happens to be sent to the failed host, the ping will fail, and the client will be evicted, causing all of that client’s connections to all hosts to be closed. Since the ping target is effectively random, it is possible that all clients are evicted because they ping the failed server. When the number of clients in the pool drops below a threshold, the pool will recreate clients. However, the newly created clients may again be closed due to ping failures. This entire process can lead to a connection storm.

Metadata

Metadata

Assignees

No one assigned

    Labels

    affects/nonePR/issue: this bug affects none version.severity/noneSeverity of bugtype/bugType: something is unexpected

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions