批量拉取数量不对 #7596
-
|
我使用的是DefaultMQPushConsumer api，设置了pullBatchSize和consumeMessageBatchMaxSize都为512，同时修改了broker相关参数,maxTransferCountOnMessageInMemory,maxTransferBytesOnMessageInMemory,maxTransferBytesOnMessageInDisk,maxTransferCountOnMessageInDisk。为什么在消息堆积的时候，我拉取到的消息数还是很少？不是我期望的拉满呢？比如说拉到512条。 |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments 6 replies
-
|
看代码这个不仅和消息的条数有关系，和每条消息的大小也有关系； |
Beta Was this translation helpful? Give feedback.
-
|
maxTransferCountOnMessageInMemory,maxTransferBytesOnMessageInMemory,maxTransferBytesOnMessageInDisk,maxTransferCountOnMessageInDisk这些参数分别设置的值是多少？ |
Beta Was this translation helpful? Give feedback.
-
|
maxTransferCountOnMessageInMemory 8192 maxTransferCountOnMessageInDisk 1024 maxTransferBytesOnMessageInMemory 33554432 maxTransferBytesOnMessageInDisk 8388608 |
Beta Was this translation helpful? Give feedback.
-
|
我也看过代码，当读取的消息数或者对应的字节数，超过设置的上限时，就会提前返回。这个对应的bytes或者count，是针对全部topic一起的值吗？ 因为就字节数来说，是内存里面类似于有缓冲池的值总和吗？ |
Beta Was this translation helpful? Give feedback.
-
|
因为就设置来看，我的上限很高，如果是单个toipc，产生了堆积。那么一次能拉取的消息数应该很多才对，但是还是很少，不会达到上限。 |
Beta Was this translation helpful? Give feedback.
-
|
我也遇到同样的问题，后来解决没有啊？ |
Beta Was this translation helpful? Give feedback.
-
|
上面的设置是生效的，我发现部分机器会拉满数据，部分不可以，还在找原因。 |
Beta Was this translation helpful? Give feedback.
-
|
看了下源码，猜测还有一个可能性就是，consumeQueue索引生成的速度太慢了，导致消息不可见。可以改成并行生成试试。#5184 |
Beta Was this translation helpful? Give feedback.
看代码这个不仅和消息的条数有关系，和每条消息的大小也有关系；