Skip to content

[HZ-5309] Improve Asyncio Client's Resiliency #766

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
yuce merged 6 commits into from
Jan 13, 2026
Merged

[HZ-5309] Improve Asyncio Client's Resiliency #766

yuce merged 6 commits into from
Jan 13, 2026
+32 −5

Conversation

@yuce
Copy link
Contributor

@yuce yuce commented Jan 7, 2026 •
edited
Loading

Asyncio client exited early during connection problems.
For example, in the following situation:

  1. Hazelcast cluster has more than one member. Members are able to access each other with their private IPs, but they are accessible by the client only by their public IPs.
  2. The client connects to one of the members using its public IP, without enabling client public address discovery (use_public_ip=True).

In that case, the client must route all invocations to the member it could connect.
This PR fixes that for the asyncio client, by retrying to connect instead of giving up with TargetDisconnectedError.

@codecov-commenter
Copy link

codecov-commenter commented Jan 7, 2026 •
edited
Loading

Codecov Report

❌ Patch coverage is 76.00000% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 94.58%. Comparing base (e06c30d) to head (353ff24).

Files with missing lines Patch % Lines
hazelcast/internal/asyncio_reactor.py 78.94% 4 Missing ⚠️
hazelcast/internal/asyncio_connection.py 66.66% 2 Missing ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##           master     #766      +/-   ##
==========================================
- Coverage   94.62%   94.58%   -0.04%     
==========================================
  Files         393      393              
  Lines       25056    25077      +21     
==========================================
+ Hits        23708    23720      +12     
- Misses       1348     1357       +9

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@yuce yuce assigned ihsandemir and unassigned ihsandemir Jan 9, 2026
@yuce yuce requested review from emreyigit and ihsandemir January 9, 2026 08:01
@yuce yuce changed the title Unreachable members fix Improve Asyncio Client's Resiliency Jan 9, 2026
@yuce yuce changed the title Improve Asyncio Client's Resiliency [HZ-5309] Improve Asyncio Client's Resiliency Jan 9, 2026
ihsandemir
ihsandemir reviewed Jan 9, 2026
View reviewed changes
Copy link
Contributor

@ihsandemir ihsandemir left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I understand from the changes you added a task for reconnect if it fails to connect. But where is the code that will route the invocations to random member connection available? I assumed this would be the fix for the PR. What will we do to test and verify the change?

@yuce
Copy link
Contributor Author

yuce commented Jan 9, 2026

@ihsandemir The invocation logic handles the routing.
If an invocation is directed to a partition, and if there is no active connection to the member that owns the partition, the invocation is sent to a random member.

hazelcast-python-client/hazelcast/internal/asyncio_invocation.py

Line 197 in e06c30d

if invocation.partition_id != -1:

I've manually tested that the scenario works as expected.
Automated testing requires simulating the described scenario, which is to be done after the release.

ihsandemir
ihsandemir reviewed Jan 12, 2026
View reviewed changes
@yuce yuce merged commit 7a78d6a into hazelcast:master Jan 13, 2026
11 checks passed
@yuce yuce deleted the unreachable-member-fix branch January 13, 2026 11:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ihsandemir ihsandemir ihsandemir approved these changes

@emreyigit emreyigit emreyigit approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@yuce @codecov-commenter @ihsandemir @emreyigit