CODE: 2 DESC: [REJECTREQUEST]system busy, start flow control for a while BROKER

[quattro-abc]

Hello everyone, when I used RocketMQ, the log file output the following exception.Please help me. How can I handle the exception?

org.springframework.messaging.MessagingException: CODE: 2 DESC: [REJECTREQUEST]system busy, start flow control for a while BROKER: 10.11.2.12:30911 For more information, please visit the url, http://rocketmq.apache.org/docs/faq/; nested exception is org.apache.rocketmq.client.exception.MQBrokerException: CODE: 2 DESC: [REJECTREQUEST]system busy, start flow control for a while BROKER: 10.11.2.12:30911 For more information, please visit the url, http://rocketmq.apache.org/docs/faq/ org.springframework.messaging.MessagingException: CODE: 2 DESC: [REJECTREQUEST]system busy, start flow control for a while BROKER: 10.11.2.12:30911 For more information, please visit the url, http://rocketmq.apache.org/docs/faq/; nested exception is org.apache.rocketmq.client.exception.MQBrokerException: CODE: 2 DESC: [REJECTREQUEST]system busy, start flow control for a while BROKER: 10.11.2.12:30911 For more information, please visit the url, http://rocketmq.apache.org/docs/faq/ at org.apache.rocketmq.spring.core.RocketMQTemplate.syncSend(RocketMQTemplate.java:563) at org.apache.rocketmq.spring.core.RocketMQTemplate.syncSend(RocketMQTemplate.java:484) at org.apache.rocketmq.spring.core.RocketMQTemplate.syncSend(RocketMQTemplate.java:472) at org.apache.rocketmq.spring.core.RocketMQTemplate.doSend(RocketMQTemplate.java:886) at org.apache.rocketmq.spring.core.RocketMQTemplate.doSend(RocketMQTemplate.java:57) at org.springframework.messaging.core.AbstractMessageSendingTemplate.send(AbstractMessageSendingTemplate.java:108) at org.springframework.messaging.core.AbstractMessageSendingTemplate.convertAndSend(AbstractMessageSendingTemplate.java:150) at org.springframework.messaging.core.AbstractMessageSendingTemplate.convertAndSend(AbstractMessageSendingTemplate.java:128) at org.springframework.messaging.core.AbstractMessageSendingTemplate.convertAndSend(AbstractMessageSendingTemplate.java:121)

[ShadowySpirits]

I noticed that you are getting an exception while trying to send messages. This could be due to high disk pressure. See the following code for more information.

rocketmq/broker/src/main/java/org/apache/rocketmq/broker/processor/SendMessageProcessor.java

Lines 121 to 132 in c17baf1

	@Override
	public boolean rejectRequest() {
	if (!this.brokerController.getBrokerConfig().isEnableSlaveActingMaster() && this.brokerController.getMessageStoreConfig().getBrokerRole() == BrokerRole.SLAVE) {
	return true;
	}
	if (this.brokerController.getMessageStore().isOSPageCacheBusy() || this.brokerController.getMessageStore().isTransientStorePoolDeficient()) {
	return true;
	}
	return false;
	}

[quattro-abc]

Thank you very much.

[yijieyu]

@quattro-abc How to solve it

this is conf

brokerClusterName = Cluster
brokerName = rocket01
brokerId = -1
brokerRole = ASYNC_MASTER
#flushDiskType=ASYNC_FLUSH
deleteWhen = 04
fileReservedTime = 48
enableControllerMode = true
controllerAddr = 10.203.1.115:9878;10.203.0.175:9878;10.203.1.50:9878
namesrvAddr = 10.203.1.115:9876;10.203.0.175:9876;10.203.1.50:9876
allAckInSyncStateSet=true
listenPort=30911
storePathRootDir=/data/rmqstore/rocket02/
storePathCommitLog=/data/rmqstore/rocket02/commitlog
sendMessageThreadPoolNums=4
useReentrantLockWhenPutMessage=true
isEnableSlaveActingMaster=true

autoCreateTopicEnable=false
autoCreateSubscriptionGroup=false

[quattro-abc]

@quattro-abc How to solve it

this is conf

brokerClusterName = Cluster brokerName = rocket01 brokerId = -1 brokerRole = ASYNC_MASTER #flushDiskType=ASYNC_FLUSH deleteWhen = 04 fileReservedTime = 48 enableControllerMode = true controllerAddr = 10.203.1.115:9878;10.203.0.175:9878;10.203.1.50:9878 namesrvAddr = 10.203.1.115:9876;10.203.0.175:9876;10.203.1.50:9876 allAckInSyncStateSet=true listenPort=30911 storePathRootDir=/data/rmqstore/rocket02/ storePathCommitLog=/data/rmqstore/rocket02/commitlog sendMessageThreadPoolNums=4 useReentrantLockWhenPutMessage=true isEnableSlaveActingMaster=true

autoCreateTopicEnable=false autoCreateSubscriptionGroup=false

发生这个问题的原因大致两个方面
1:磁盘空间不足;
2:磁盘IO超负荷;
我的这个问题主要是磁盘IO瓶颈导致的.在业务高峰期,你可以观察下服务器的读写情况.

[yijieyu]

okkk

[zhishengzhang]

Scaling out nodes or upgrading disks to ESSD