Need help to find root cause of `badmatch,{error,noproc}` #11440

laurentthiebaudezm · 2024-06-12T15:57:03Z

laurentthiebaudezm
Jun 12, 2024

Hi,

We use rabbit-mq (docker image - 3.12.10) in our test suite. During the setup, multiple services connect to the rabbitmq and randomly there is a failure, which results to shutdown rabbitmq.

This happens about 1/20 times. We cannot understand the root cause from the logs, where should we look at?

Here is the crash logs

�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0> ** Generic server <0.1191.0> terminating�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0> ** Last message in was {'$gen_cast',�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>                            {method,�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>                                {'basic.consume',0,�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>                                    <<"my_queue">>,�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>                                    <<>>,false,false,false,false,[]},�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>                                none,noflow}}�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0> ** When Server state == {ch,�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>                          {conf,running,rabbit_framing_amqp_0_9_1,2,�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>                           <0.1177.0>,<0.1189.0>,<0.1177.0>,�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>                           <<"127.0.0.1:53252 -> 127.0.0.1:5672">>,undefined,�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>                           {user,<<"guest">>,�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>                            [administrator],�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>                            [{rabbit_auth_backend_internal,�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>                              #Fun<rabbit_auth_backend_internal.3.20918515>}]},�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>                           <<"/">>,�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>                           <<"my_queue">>,�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>                           <0.1178.0>,�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>                           [{<<"exchange_exchange_bindings">>,bool,true},�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>                            {<<"connection.blocked">>,bool,true},�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>                            {<<"authentication_failure_close">>,bool,true},�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>                            {<<"basic.nack">>,bool,true},�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>                            {<<"publisher_confirms">>,bool,true},�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>                            {<<"consumer_cancel_notify">>,bool,true}],�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>                           none,1,134217728,1800000,#{},1000000000,false},�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>                          {lstate,<0.1190.0>,false},�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>                          none,1,�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>                          {0,[],[]},�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>                          {state,#{},erlang},�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>                          #{},#{},�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>                          {state,fine,5000,#Ref<0.1787854094.725876746.210238>},�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>                          false,1,�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>                          {rabbit_confirms,undefined,#{}},�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>                          [],[],none,flow,[],�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>                          {rabbit_queue_type,#{}},�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>                          #Ref<0.1787854094.725876746.210234>,false}�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0> ** Reason for termination ==�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0> ** {{badmatch,{error,noproc}},�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>     [{rabbit_quorum_queue,consume,3,�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>                           [{file,"rabbit_quorum_queue.erl"},{line,796}]},�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>      {rabbit_queue_type,consume,3,[{file,"rabbit_queue_type.erl"},{line,392}]},�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>      {rabbit_channel,'-basic_consume/8-fun-0-',10,�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>                      [{file,"rabbit_channel.erl"},{line,1746}]},�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>      {rabbit_misc,with_exit_handler,2,[{file,"rabbit_misc.erl"},{line,511}]},�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>      {rabbit_channel,basic_consume,8,�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>                      [{file,"rabbit_channel.erl"},{line,1743}]},�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>      {rabbit_channel,handle_method,3,�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>                      [{file,"rabbit_channel.erl"},{line,1422}]},�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>      {rabbit_channel,handle_cast,2,[{file,"rabbit_channel.erl"},{line,633}]},�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0>      {gen_server2,handle_msg,2,[{file,"gen_server2.erl"},{line,1056}]}]}�[0m
�[38;5;160m2024-06-07 02:28:25.473126+00:00 [error] <0.1191.0> �[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>   crasher:�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>     initial call: rabbit_channel:init/1�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>     pid: <0.1191.0>�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>     registered_name: []�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>     exception exit: {{badmatch,{error,noproc}},�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                      [{rabbit_quorum_queue,consume,3,�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                           [{file,"rabbit_quorum_queue.erl"},{line,796}]},�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                       {rabbit_queue_type,consume,3,�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                           [{file,"rabbit_queue_type.erl"},{line,392}]},�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                       {rabbit_channel,'-basic_consume/8-fun-0-',10,�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                           [{file,"rabbit_channel.erl"},{line,1746}]},�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                       {rabbit_misc,with_exit_handler,2,�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                           [{file,"rabbit_misc.erl"},{line,511}]},�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                       {rabbit_channel,basic_consume,8,�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                           [{file,"rabbit_channel.erl"},{line,1743}]},�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                       {rabbit_channel,handle_method,3,�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                           [{file,"rabbit_channel.erl"},{line,1422}]},�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                       {rabbit_channel,handle_cast,2,�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                           [{file,"rabbit_channel.erl"},{line,633}]},�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                       {gen_server2,handle_msg,2,�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                           [{file,"gen_server2.erl"},{line,1056}]}]}�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>       in function  gen_server2:terminate/3 (gen_server2.erl, line 1172)�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>     ancestors: [<0.1188.0>,<0.1181.0>,<0.1176.0>,<0.1175.0>,<0.707.0>,�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                   <0.706.0>,<0.705.0>,<0.703.0>,<0.702.0>,rabbit_sup,�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                   <0.230.0>]�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>     message_queue_len: 1�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>     messages: [{'$gen_cast',�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                       {command,�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                           {'basic.consume_ok',�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                               <<"amq.ctag-M87s7t4oCVQYqviuQY_nlg">>}}}]�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>     links: [<0.1188.0>]�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>     dictionary: [{topic_permission_cache,�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                       [{{resource,<<"/">>,topic,<<"sitecc_external">>},�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                         #{routing_key =>�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                               <<"#.acceptance_test.#.#.go_to_mission_state_changed">>,�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                           variable_map =>�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                               #{<<"username">> => <<"guest">>,�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                                 <<"vhost">> => <<"/">>}},�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                         read}]},�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                   {guid_secure,�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                       {{0,rabbit@9db935373b6a,�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                         #Ref<0.1787854094.725876746.210400>},�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                        0}},�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                   {permission_cache,�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                       [{{resource,<<"/">>,queue,�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                             <<"my_queue">>},�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                         #{},read},�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                        {{resource,<<"/">>,exchange,<<"sitecc_external">>},�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                         #{},read},�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                        {{resource,<<"/">>,queue,�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                             <<"my_queue">>},�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                         #{},write},�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                        {{resource,<<"/">>,exchange,<<"sitecc_external">>},�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                         #{},configure},�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                        {{resource,<<"/">>,queue,�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                             <<"my_queue">>},�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                         #{},configure}]},�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                   {rand_seed,�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                       {#{jump => #Fun<rand.3.34006561>,�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                          max => 288230376151711743,�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                          next => #Fun<rand.5.34006561>,type => exsplus},�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                        [175994982168808641|256309637859926001]}},�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                   {channel_operation_timeout,15000},�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                   {permission_cache_can_expire,false},�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                   {process_name,�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                       {rabbit_channel,�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                           {<<"127.0.0.1:53252 -> 127.0.0.1:5672">>,2}}},�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>                   {{xtype_to_module,topic},rabbit_exchange_type_topic}]�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>     trap_exit: true�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>     status: running�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>     heap_size: 17731�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>     stack_size: 28�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>     reductions: 43814�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0>   neighbours:�[0m
�[38;5;160m2024-06-07 02:28:25.473978+00:00 [error] <0.1191.0> �[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>     supervisor: {<0.1188.0>,rabbit_channel_sup}�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>     errorContext: child_terminated�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>     reason: {{badmatch,{error,noproc}},�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>              [{rabbit_quorum_queue,consume,3,�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>                                    [{file,"rabbit_quorum_queue.erl"},�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>                                     {line,796}]},�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>               {rabbit_queue_type,consume,3,�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>                                  [{file,"rabbit_queue_type.erl"},{line,392}]},�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>               {rabbit_channel,'-basic_consume/8-fun-0-',10,�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>                               [{file,"rabbit_channel.erl"},{line,1746}]},�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>               {rabbit_misc,with_exit_handler,2,�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>                            [{file,"rabbit_misc.erl"},{line,511}]},�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>               {rabbit_channel,basic_consume,8,�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>                               [{file,"rabbit_channel.erl"},{line,1743}]},�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>               {rabbit_channel,handle_method,3,�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>                               [{file,"rabbit_channel.erl"},{line,1422}]},�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>               {rabbit_channel,handle_cast,2,�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>                               [{file,"rabbit_channel.erl"},{line,633}]},�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>               {gen_server2,handle_msg,2,�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>                            [{file,"gen_server2.erl"},{line,1056}]}]}�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>     offender: [{pid,<0.1191.0>},�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>                {id,channel},�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>                {mfargs,�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>                    {rabbit_channel,start_link,�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>                        [2,<0.1177.0>,<0.1189.0>,<0.1177.0>,�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>                         <<"127.0.0.1:53252 -> 127.0.0.1:5672">>,�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>                         rabbit_framing_amqp_0_9_1,�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>                         {user,<<"guest">>,�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>                             [administrator],�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>                             [{rabbit_auth_backend_internal,�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>                                  #Fun<rabbit_auth_backend_internal.3.20918515>}]},�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>                         <<"/">>,�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>                         [{<<"exchange_exchange_bindings">>,bool,true},�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>                          {<<"connection.blocked">>,bool,true},�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>                          {<<"authentication_failure_close">>,bool,true},�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>                          {<<"basic.nack">>,bool,true},�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>                          {<<"publisher_confirms">>,bool,true},�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>                          {<<"consumer_cancel_notify">>,bool,true}],�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>                         <0.1178.0>,<0.1190.0>]}},�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>                {restart_type,transient},�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>                {significant,true},�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>                {shutdown,70000},�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0>                {child_type,worker}]�[0m
�[38;5;160m2024-06-07 02:28:25.475294+00:00 [error] <0.1188.0> �[0m
�[38;5;160m2024-06-07 02:28:25.475864+00:00 [error] <0.1188.0>     supervisor: {<0.1188.0>,rabbit_channel_sup}�[0m
�[38;5;160m2024-06-07 02:28:25.475864+00:00 [error] <0.1188.0>     errorContext: shutdown�[0m
�[38;5;160m2024-06-07 02:28:25.475864+00:00 [error] <0.1188.0>     reason: reached_max_restart_intensity�[0m
�[38;5;160m2024-06-07 02:28:25.475864+00:00 [error] <0.1188.0>     offender: [{pid,<0.1191.0>},�[0m
�[38;5;160m2024-06-07 02:28:25.475864+00:00 [error] <0.1188.0>                {id,channel},�[0m
�[38;5;160m2024-06-07 02:28:25.475864+00:00 [error] <0.1188.0>                {mfargs,�[0m
�[38;5;160m2024-06-07 02:28:25.475864+00:00 [error] <0.1188.0>                    {rabbit_channel,start_link,�[0m
�[38;5;160m2024-06-07 02:28:25.475864+00:00 [error] <0.1188.0>                        [2,<0.1177.0>,<0.1189.0>,<0.1177.0>,�[0m
�[38;5;160m2024-06-07 02:28:25.475864+00:00 [error] <0.1188.0>                         <<"127.0.0.1:53252 -> 127.0.0.1:5672">>,�[0m
�[38;5;160m2024-06-07 02:28:25.475864+00:00 [error] <0.1188.0>                         rabbit_framing_amqp_0_9_1,�[0m
�[38;5;160m2024-06-07 02:28:25.475864+00:00 [error] <0.1188.0>                         {user,<<"guest">>,�[0m
�[38;5;160m2024-06-07 02:28:25.475864+00:00 [error] <0.1188.0>                             [administrator],�[0m
�[38;5;160m2024-06-07 02:28:25.475864+00:00 [error] <0.1188.0>                             [{rabbit_auth_backend_internal,�[0m
�[38;5;160m2024-06-07 02:28:25.475864+00:00 [error] <0.1188.0>                                  #Fun<rabbit_auth_backend_internal.3.20918515>}]},�[0m
�[38;5;160m2024-06-07 02:28:25.475864+00:00 [error] <0.1188.0>                         <<"/">>,�[0m
�[38;5;160m2024-06-07 02:28:25.475864+00:00 [error] <0.1188.0>                         [{<<"exchange_exchange_bindings">>,bool,true},�[0m
�[38;5;160m2024-06-07 02:28:25.475864+00:00 [error] <0.1188.0>                          {<<"connection.blocked">>,bool,true},�[0m
�[38;5;160m2024-06-07 02:28:25.475864+00:00 [error] <0.1188.0>                          {<<"authentication_failure_close">>,bool,true},�[0m
�[38;5;160m2024-06-07 02:28:25.475864+00:00 [error] <0.1188.0>                          {<<"basic.nack">>,bool,true},�[0m
�[38;5;160m2024-06-07 02:28:25.475864+00:00 [error] <0.1188.0>                          {<<"publisher_confirms">>,bool,true},�[0m
�[38;5;160m2024-06-07 02:28:25.475864+00:00 [error] <0.1188.0>                          {<<"consumer_cancel_notify">>,bool,true}],�[0m
�[38;5;160m2024-06-07 02:28:25.475864+00:00 [error] <0.1188.0>                         <0.1178.0>,<0.1190.0>]}},�[0m
�[38;5;160m2024-06-07 02:28:25.475864+00:00 [error] <0.1188.0>                {restart_type,transient},�[0m
�[38;5;160m2024-06-07 02:28:25.475864+00:00 [error] <0.1188.0>                {significant,true},�[0m
�[38;5;160m2024-06-07 02:28:25.475864+00:00 [error] <0.1188.0>                {shutdown,70000},�[0m
�[38;5;160m2024-06-07 02:28:25.475864+00:00 [error] <0.1188.0>                {child_type,worker}]�[0m
�[38;5;160m2024-06-07 02:28:25.475864+00:00 [error] <0.1188.0> �[0m
�[38;5;160m2024-06-07 02:28:25.475984+00:00 [error] <0.1177.0> Error on AMQP connection <0.1177.0> (127.0.0.1:53252 -> 127.0.0.1:5672 - 9db935373b6a-my-consumer, vhost: '/', user: 'guest', state: running), channel 2:�[0m
�[38;5;160m2024-06-07 02:28:25.475984+00:00 [error] <0.1177.0>  {{badmatch,{error,noproc}},�[0m
�[38;5;160m2024-06-07 02:28:25.475984+00:00 [error] <0.1177.0>  [{rabbit_quorum_queue,consume,3,�[0m
�[38;5;160m2024-06-07 02:28:25.475984+00:00 [error] <0.1177.0>                        [{file,"rabbit_quorum_queue.erl"},{line,796}]},�[0m
�[38;5;160m2024-06-07 02:28:25.475984+00:00 [error] <0.1177.0>   {rabbit_queue_type,consume,3,[{file,"rabbit_queue_type.erl"},{line,392}]},�[0m
�[38;5;160m2024-06-07 02:28:25.475984+00:00 [error] <0.1177.0>   {rabbit_channel,'-basic_consume/8-fun-0-',10,�[0m
�[38;5;160m2024-06-07 02:28:25.475984+00:00 [error] <0.1177.0>                   [{file,"rabbit_channel.erl"},{line,1746}]},�[0m
�[38;5;160m2024-06-07 02:28:25.475984+00:00 [error] <0.1177.0>   {rabbit_misc,with_exit_handler,2,[{file,"rabbit_misc.erl"},{line,511}]},�[0m
�[38;5;160m2024-06-07 02:28:25.475984+00:00 [error] <0.1177.0>   {rabbit_channel,basic_consume,8,[{file,"rabbit_channel.erl"},{line,1743}]},�[0m
�[38;5;160m2024-06-07 02:28:25.475984+00:00 [error] <0.1177.0>   {rabbit_channel,handle_method,3,[{file,"rabbit_channel.erl"},{line,1422}]},�[0m
�[38;5;160m2024-06-07 02:28:25.475984+00:00 [error] <0.1177.0>   {rabbit_channel,handle_cast,2,[{file,"rabbit_channel.erl"},{line,633}]},�[0m
�[38;5;160m2024-06-07 02:28:25.475984+00:00 [error] <0.1177.0>   {gen_server2,handle_msg,2,[{file,"gen_server2.erl"},{line,1056}]}]}�[0m

Thanks.

laurentthiebaudezm · 2024-06-20T08:26:29Z

laurentthiebaudezm
Jun 20, 2024
Author

We upgraded to the latest version 3.13.3 and we still have the issue. Error message seems to be the same:
rabbitmq-log-11440.txt

0 replies

lukebakken · 2024-06-20T16:07:01Z

lukebakken
Jun 20, 2024
Maintainer

@laurentthiebaudezm please familiarize yourself with the ability to upload files to GitHub comments. It's much easier to deal with when compared to pasting a large amount of text into the comment box. Thank you.

0 replies

michaelklishin · 2024-06-24T09:48:42Z

michaelklishin
Jun 24, 2024
Maintainer

2024-06-20 07:15:36.738135+00:00 [error] <0.978.0> ** {{badmatch,{error,noproc}},
2024-06-20 07:15:36.738135+00:00 [error] <0.978.0>     [{rabbit_quorum_queue,consume,3,
2024-06-20 07:15:36.738135+00:00 [error] <0.978.0>                           [{file,"rabbit_quorum_queue.erl"},{line,863}]},
2024-06-20 07:15:36.738135+00:00 [error] <0.978.0>      {rabbit_queue_type,consume,3,[{file,"rabbit_queue_type.erl"},{line,430}]},
2024-06-20 07:15:36.738135+00:00 [error] <0.978.0>      {rabbit_channel,'-basic_consume/8-fun-0-',10,
2024-06-20 07:15:36.738135+00:00 [error] <0.978.0>                      [{file,"rabbit_channel.erl"},{line,1769}]},

The QQ in question does not have an elected leader, so it cannot be consumed from. We cannot suggest why since you haven't included any log parts before this exception.

There is an upgrading strategy that can easily leave a QQ without any known members and thus an electable leader. The queue might have lost its quorum.

Otherwise leader election issues are fairly rare to come by for quorum queues.

2 replies

laurentthiebaudezm Jun 25, 2024
Author

rabbit.log

Hi thanks for your reply! I joined the full log if that can help to understand the root cause?

To explain a bit more our use case, we use the dockerized version of rabbitmq (a single container started locally, without any docker compose); the crash is happening during some integration tests, when a microservice tries to connect to rabbitmq and listen to a queue.

Could it be that we connect too soon, and the quorum is not totally up yet? If that's a possible cause, maybe we should ensure at first that rabbit or the queue is up?

Thanks.

kjnilsson Jun 27, 2024
Maintainer

yes in this case the queue isn't fully started by the time you try to consume. Is your app declaring and consuming in parallel?

Could you re-run your test with RabbitMQ debug logging enabled and share the log for that?

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Need help to find root cause of `badmatch,{error,noproc}` #11440

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Replies: 3 comments 2 replies

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Need help to find root cause of badmatch,{error,noproc} #11440

Uh oh!

Uh oh!

laurentthiebaudezm Jun 12, 2024

Replies: 3 comments · 2 replies

Uh oh!

Uh oh!

laurentthiebaudezm Jun 20, 2024 Author

Uh oh!

lukebakken Jun 20, 2024 Maintainer

Uh oh!

michaelklishin Jun 24, 2024 Maintainer

Uh oh!

laurentthiebaudezm Jun 25, 2024 Author

Uh oh!

kjnilsson Jun 27, 2024 Maintainer

Need help to find root cause of `badmatch,{error,noproc}` #11440

laurentthiebaudezm
Jun 12, 2024

Replies: 3 comments 2 replies

laurentthiebaudezm
Jun 20, 2024
Author

lukebakken
Jun 20, 2024
Maintainer

michaelklishin
Jun 24, 2024
Maintainer

laurentthiebaudezm Jun 25, 2024
Author

kjnilsson Jun 27, 2024
Maintainer