Skip to content

Commit a68aa07

Browse files
committed
Add a caption in command examples of IoT workloads
Makes the goal of the command more explicit.
1 parent ccfd856 commit a68aa07

File tree

1 file changed

+92
-51
lines changed

1 file changed

+92
-51
lines changed

src/docs/asciidoc/usage-advanced.adoc

Lines changed: 92 additions & 51 deletions
Original file line numberDiff line numberDiff line change
@@ -149,9 +149,13 @@ set to 1 ms for 60 seconds then 1 second for 30 seconds:
149149
PertTest supports balancing the publishing and the consumption
150150
across a sequence of queues, e.g.:
151151

152-
bin/runjava com.rabbitmq.perf.PerfTest --queue-pattern 'perf-test-%d' \
153-
--queue-pattern-from 1 --queue-pattern-to 10 \
154-
--producers 100 --consumers 100
152+
.Using a sequence of queues
153+
[source,bash,indent=0]
154+
--------
155+
bin/runjava com.rabbitmq.perf.PerfTest --queue-pattern 'perf-test-%d' \
156+
--queue-pattern-from 1 --queue-pattern-to 10 \
157+
--producers 100 --consumers 100
158+
--------
155159

156160
The previous command would create the `perf-test-1`, `perf-test-2`, ...,
157161
`perf-test-10` queues and spreads the producers and consumers across them.
@@ -186,17 +190,25 @@ The following command line launches a first PerfTest process that
186190
creates 500 queues (from `perf-test-1` to `perf-test-500`).
187191
Each queue will have 3 consumers and 1 producer sending messages to it:
188192

189-
bin/runjava com.rabbitmq.perf.PerfTest --queue-pattern 'perf-test-%d' \
190-
--queue-pattern-from 1 --queue-pattern-to 500 \
191-
--producers 500 --consumers 1500
193+
.Creating a first set of 500 queues
194+
[source,bash,indent=0]
195+
--------
196+
bin/runjava com.rabbitmq.perf.PerfTest --queue-pattern 'perf-test-%d' \
197+
--queue-pattern-from 1 --queue-pattern-to 500 \
198+
--producers 500 --consumers 1500
199+
--------
192200

193201
Then the following command line launches a second PerfTest process
194202
that creates 500 queues (from `perf-test-501` to `perf-test-1000`).
195203
Each queue will have 3 consumers and 1 producer sending messages to it:
196204

197-
bin/runjava com.rabbitmq.perf.PerfTest --queue-pattern 'perf-test-%d' \
198-
--queue-pattern-from 501 --queue-pattern-to 1000 \
199-
--producers 500 --consumers 1500
205+
.Creating a second set of 500 queues
206+
[source,bash,indent=0]
207+
--------
208+
bin/runjava com.rabbitmq.perf.PerfTest --queue-pattern 'perf-test-%d' \
209+
--queue-pattern-from 501 --queue-pattern-to 1000 \
210+
--producers 500 --consumers 1500
211+
--------
200212

201213
Those 2 processes will simulate 1000 producers and 3000 consumers spread
202214
across 1000 queues.
@@ -206,9 +218,13 @@ A PerfTest process can exhaust its file descriptors limit and throw
206218
exceptions. A first way to avoid this is to reduce the number of Java threads
207219
PerfTest uses with the `--heartbeat-sender-threads` option:
208220

209-
bin/runjava com.rabbitmq.perf.PerfTest --queue-pattern 'perf-test-%d' \
210-
--queue-pattern-from 1 --queue-pattern-to 1000 \
211-
--producers 1000 --consumers 3000 --heartbeat-sender-threads 10
221+
.Using `--heartbeat-sender-threads` to reduce the number of threads
222+
[source,bash,indent=0]
223+
--------
224+
bin/runjava com.rabbitmq.perf.PerfTest --queue-pattern 'perf-test-%d' \
225+
--queue-pattern-from 1 --queue-pattern-to 1000 \
226+
--producers 1000 --consumers 3000 --heartbeat-sender-threads 10
227+
--------
212228

213229
By default, each producer and consumer connection uses a dedicated thread
214230
to send heartbeats to the broker, so this is 4000 threads for heartbeats
@@ -232,53 +248,70 @@ that exchange messages at a low and mostly constant rate, usually a message ever
232248
Simulating such workloads requires a different set of settings compared to
233249
the workloads that have higher throughput and a small number of clients. With the appropriate set of flags,
234250
PerfTest can simulate IoT workloads without requiring too many resources, especially threads.
251+
Let's explore these flags.
235252

236253
With an IoT workload, publishers usually don't publish many messages per second,
237254
but rather a message every fixed period of time. This can be achieved by using the `--publishing-interval`
238255
flag instead of the `--rate` one. For example:
239256

240-
bin/runjava com.rabbitmq.perf.PerfTest --publishing-interval 5
257+
.Using `--publishing-interval` for low-throughput workloads
258+
[source,bash,indent=0]
259+
--------
260+
bin/runjava com.rabbitmq.perf.PerfTest --publishing-interval 5
261+
--------
241262

242263
The command above makes the publisher publish a message every 5 seconds.
243264
To simulate a group of consumers, use the `--queue-pattern` flag to simulate many consumers across
244265
many queues:
245266

246-
bin/runjava com.rabbitmq.perf.PerfTest --queue-pattern 'perf-test-%d' \
247-
--queue-pattern-from 1 --queue-pattern-to 1000 \
248-
--producers 1000 --consumers 1000 \
249-
--heartbeat-sender-threads 10 \
250-
--publishing-interval 5
267+
.Simulating 2000 clients on 1000 queues
268+
[source,bash,indent=0]
269+
--------
270+
bin/runjava com.rabbitmq.perf.PerfTest --queue-pattern 'perf-test-%d' \
271+
--queue-pattern-from 1 --queue-pattern-to 1000 \
272+
--producers 1000 --consumers 1000 \
273+
--heartbeat-sender-threads 10 \
274+
--publishing-interval 5
275+
--------
251276

252277
To prevent publishers from publishing at roughly the same time and
253278
distribute the rate more evenly, use
254-
the `--producer-random-start-delay` option to add an random
279+
the `--producer-random-start-delay` option to add a random
255280
delay before the first published message:
256281

257-
bin/runjava com.rabbitmq.perf.PerfTest --queue-pattern 'perf-test-%d' \
258-
--queue-pattern-from 1 --queue-pattern-to 1000 \
259-
--producers 1000 --consumers 1000 \
260-
--heartbeat-sender-threads 10 \
261-
--publishing-interval 5 --producer-random-start-delay 120
282+
.Using `--producer-random-start-delay` to spread publishing in a random way
283+
[source,bash,indent=0]
284+
--------
285+
bin/runjava com.rabbitmq.perf.PerfTest --queue-pattern 'perf-test-%d' \
286+
--queue-pattern-from 1 --queue-pattern-to 1000 \
287+
--producers 1000 --consumers 1000 \
288+
--heartbeat-sender-threads 10 \
289+
--publishing-interval 5 --producer-random-start-delay 120
290+
--------
262291

263292
With the command above, each publisher will start with a random delay
264293
between 1 and 120 seconds.
265294

266-
When using `--publishing-interval`, PerfTest will use one thread
267-
for scheduling publishing for all 50 producers. So 1000 producers should keep 20 threads busy for
268-
the publishing scheduling. This ratio can be decreased or increased with the
269-
`--producer-scheduler-threads` options depending on the load and the target environment.
270-
Very few threads can be used for very slow publishers:
271-
272-
bin/runjava com.rabbitmq.perf.PerfTest --queue-pattern 'perf-test-%d' \
273-
--queue-pattern-from 1 --queue-pattern-to 1000 \
274-
--producers 1000 --consumers 1000 \
275-
--heartbeat-sender-threads 10 \
276-
--publishing-interval 60 --producer-random-start-delay 1800 \
277-
--producer-scheduler-threads 10
295+
When using `--publishing-interval`, PerfTest will use one thread for 100 operations per second.
296+
So 1,000 producers publishing at 1 message / second should keep 10 threads busy for
297+
the publishing scheduling.
298+
It is possible to set the number of threads used with the `--producer-scheduler-threads` options.
299+
Set your own value if the default value is not appropriate for some reasons:
300+
301+
.Using `--producer-scheduler-threads` to set the number of publishing threads
302+
[source,bash,indent=0]
303+
--------
304+
bin/runjava com.rabbitmq.perf.PerfTest --queue-pattern 'perf-test-%d' \
305+
--queue-pattern-from 1 --queue-pattern-to 1000 \
306+
--producers 1000 --consumers 1000 \
307+
--heartbeat-sender-threads 10 \
308+
--publishing-interval 60 --producer-random-start-delay 1800 \
309+
--producer-scheduler-threads 5
310+
--------
278311

279312
In the example above, 1000 publishers will publish every 60 seconds
280313
with a random start-up delay between 1 second and 30 minutes (1800 seconds). They
281-
will be scheduled by only 10 threads (instead of 20 by default). Such delay
314+
will be scheduled by only 5 threads. Such delay
282315
values are suitable for long running tests.
283316

284317
Another option can be useful when simulating many consumers with a moderate message rate:
@@ -288,13 +321,17 @@ would use a 1-thread thread pool, which is overkill considering consumers proces
288321
is fast and producers publish one message every second. We can set the number of thread pools
289322
to use with `--consumers-thread-pools` and they will be shared by the consumers:
290323

291-
bin/runjava com.rabbitmq.perf.PerfTest --queue-pattern 'perf-test-%d' \
292-
--queue-pattern-from 1 --queue-pattern-to 1000 \
293-
--producers 1000 --consumers 1000 \
294-
--heartbeat-sender-threads 10 \
295-
--publishing-interval 60 --producer-random-start-delay 1800 \
296-
--producer-scheduler-threads 10 \
297-
--consumers-thread-pools 10
324+
.Using `--consumers-thread-pools` to reduce the number of consumer threads
325+
[source,bash,indent=0]
326+
--------
327+
bin/runjava com.rabbitmq.perf.PerfTest --queue-pattern 'perf-test-%d' \
328+
--queue-pattern-from 1 --queue-pattern-to 1000 \
329+
--producers 1000 --consumers 1000 \
330+
--heartbeat-sender-threads 10 \
331+
--publishing-interval 60 --producer-random-start-delay 1800 \
332+
--producer-scheduler-threads 10 \
333+
--consumers-thread-pools 10
334+
--------
298335

299336
The previous example uses only 10 thread pools for all consumers instead of 1000 by default.
300337
These are 1-thread thread pools in this case, so this is 10 threads overall instead of 1000, another
@@ -309,13 +346,17 @@ In the slow publisher example above, a handful of threads should be enough
309346
to handle the I/O. That's what the
310347
`--nio-threads` flag is for:
311348

312-
bin/runjava com.rabbitmq.perf.PerfTest --queue-pattern 'perf-test-%d' \
313-
--queue-pattern-from 1 --queue-pattern-to 1000 \
314-
--producers 1000 --consumers 1000 \
315-
--heartbeat-sender-threads 10 \
316-
--publishing-interval 60 --producer-random-start-delay 1800 \
317-
--producer-scheduler-threads 10
318-
--nio-threads 10
349+
.Reducing the number of IO threads by enabling the NIO mode with `--nio-threads`
350+
[source,bash,indent=0]
351+
--------
352+
bin/runjava com.rabbitmq.perf.PerfTest --queue-pattern 'perf-test-%d' \
353+
--queue-pattern-from 1 --queue-pattern-to 1000 \
354+
--producers 1000 --consumers 1000 \
355+
--heartbeat-sender-threads 10 \
356+
--publishing-interval 60 --producer-random-start-delay 1800 \
357+
--producer-scheduler-threads 10 \
358+
--nio-threads 10
359+
--------
319360

320361
This way PerfTest will use 12 threads for I/O over all the connections.
321362
With the default blocking I/O mode, each producer (or consumer)

0 commit comments

Comments
 (0)