Skip to content

CLI: new health check that detects QQs without an elected leader #13433

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 25 commits into from
Closed

CLI: new health check that detects QQs without an elected leader #13433

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
c26edbe
Implement rabbitmq-queues leader_health_check command for quorum queues
Ayanda-D Jun 27, 2024
6cc03b0
Tests for rabbitmq-queues leader_health_check command
Ayanda-D Jun 27, 2024
76d66a1
Ensure calling ParentPID in leader health check execution and
Ayanda-D Jun 28, 2024
857e2a7
Extend core leader health check tests and update badrpc error handlin…
Ayanda-D Jun 28, 2024
6cf9339
Refactor leader_health_check command validators and ignore vhost arg
Ayanda-D Jun 28, 2024
96b8bce
Update leader_health_check_command description and banner
Ayanda-D Jul 3, 2024
239a69b
Improve output formatting for healthy leaders and support
Ayanda-D Jul 4, 2024
48ba3e1
Support global flag to run leader health check for
Ayanda-D Jul 11, 2024
7873737
Return immediately for leader health checks on empty vhosts
Ayanda-D Jul 18, 2024
b7dec89
Rename leader health check timeout refs
Ayanda-D Jul 18, 2024
c7da4d5
Update banner message for global leader health check
Ayanda-D Aug 2, 2024
1736845
QQ leader-health-check: check_process_limit_safety before spawning le…
Ayanda-D Nov 11, 2024
1084179
Log leader health check result in broker logs (if any leaderless queues)
Ayanda-D Jan 10, 2025
68739a6
Ensure check_passed result for leader health internal calls)
Ayanda-D Jan 13, 2025
5f5e992
Extend CLI format output to process check_passed payload
Ayanda-D Jan 23, 2025
ebffd7d
Format leader healthcheck result log and function exports
Ayanda-D Feb 21, 2025
663fc98
Change leader_health_check command scope from queues to diagnostics
Ayanda-D Feb 26, 2025
df82f12
Update (c) line year
Ayanda-D Feb 26, 2025
b2acbae
Rename command to check_for_quorum_queues_without_an_elected_leader
Ayanda-D Feb 26, 2025
7a8e166
Use rabbit_db_queue for qq leader health check lookups
Ayanda-D Feb 26, 2025
9bdb81f
Update tests: quorum_queue_SUITE and rabbit_db_queue_SUITE
Ayanda-D Feb 26, 2025
6158568
Fix typo (cli test module)
Ayanda-D Feb 27, 2025
ea07938
Small refactor - simpler final leader health check result return on f…
Ayanda-D Feb 28, 2025
a45aa81
Clear dialyzer warning & fix type spec
Ayanda-D Mar 3, 2025
bb43c0b
Ignore result without strict match to avoid diayzer warning
Ayanda-D Mar 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion deps/rabbit/src/amqqueue.erl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -572,7 +572,7 @@ to_printable(#amqqueue{name = QName = #resource{name = Name},
<<"virtual_host">> => VHost,
<<"type">> => Type}.

-spec to_printable(rabbit_types:r(), atom() | binary()) -> #{binary() => any()}.
-spec to_printable(rabbit_types:r(queue), atom() | binary()) -> #{binary() => any()}.
to_printable(QName = #resource{name = Name, virtual_host = VHost}, Type) ->
_ = rabbit_queue_type:discover(Type),
#{<<"readable_name">> => rabbit_data_coercion:to_binary(rabbit_misc:rs(QName)),
Expand Down
13 changes: 7 additions & 6 deletions deps/rabbit/src/rabbit_quorum_queue.erl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2186,12 +2186,13 @@ leader_health_check(QueueNameOrRegEx, VHost, ProcessLimitThreshold) ->

run_leader_health_check(ClusterName, QResource, HealthCheckRef, From) ->
Leader = ra_leaderboard:lookup_leader(ClusterName),
case ra_server_proc:ping(Leader, ?LEADER_HEALTH_CHECK_TIMEOUT) of
{pong,leader} ->
From ! {ok, HealthCheckRef, QResource};
_ ->
From ! {error, HealthCheckRef, QResource}
end,
{_, _, _} =
Copy link
Collaborator

@michaelklishin michaelklishin Mar 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds like _ = … then would do?

Copy link
Contributor Author

@Ayanda-D Ayanda-D Mar 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, dialyzer didnt complain for both. switching to _ if thats preferred 👍

case ra_server_proc:ping(Leader, ?LEADER_HEALTH_CHECK_TIMEOUT) of
{pong,leader} ->
From ! {ok, HealthCheckRef, QResource};
_ ->
From ! {error, HealthCheckRef, QResource}
end,
ok.

wait_for_leader_health_checks(_Ref, 0, UnhealthyAcc) -> UnhealthyAcc;
Expand Down
Loading