1
1
---
2
- title : shuffle sharding
2
+ title : Shuffle- sharding
3
3
id : shuffle-sharding
4
4
date : 2020-03-04
5
5
full_link :
@@ -18,28 +18,28 @@ We are often concerned with insulating different flows of requests
18
18
from each other, so that a high-intensity flow does not crowd out low-intensity flows.
19
19
A simple way to put requests into queues is to hash some
20
20
characteristics of the request, modulo the number of queues, to get
21
- the index of the queue to use. The hash function uses as input
22
- characteristics of the request that align with flows. For example, in
21
+ the index of the queue to use. The hash function uses as input
22
+ characteristics of the request that align with flows. For example, in
23
23
the Internet this is often the 5-tuple of source and destination
24
24
address, protocol, and source and destination port.
25
25
26
26
That simple hash-based scheme has the property that any high-intensity flow
27
27
will crowd out all the low-intensity flows that hash to the same queue.
28
28
Providing good insulation for a large number of flows requires a large
29
- number of queues, which is problematic. Shuffle sharding is a more
29
+ number of queues, which is problematic. Shuffle- sharding is a more
30
30
nimble technique that can do a better job of insulating the low-intensity
31
- flows from the high-intensity flows. The terminology of shuffle sharding uses
31
+ flows from the high-intensity flows. The terminology of shuffle- sharding uses
32
32
the metaphor of dealing a hand from a deck of cards; each queue is a
33
- metaphorical card. The shuffle sharding technique starts with hashing
33
+ metaphorical card. The shuffle- sharding technique starts with hashing
34
34
the flow-identifying characteristics of the request, to produce a hash
35
- value with dozens or more of bits. Then the hash value is used as a
35
+ value with dozens or more of bits. Then the hash value is used as a
36
36
source of entropy to shuffle the deck and deal a hand of cards
37
- (queues). All the dealt queues are examined, and the request is put
38
- into one of the examined queues with the shortest length. With a
37
+ (queues). All the dealt queues are examined, and the request is put
38
+ into one of the examined queues with the shortest length. With a
39
39
modest hand size, it does not cost much to examine all the dealt cards
40
40
and a given low-intensity flow has a good chance to dodge the effects of a
41
- given high-intensity flow. With a large hand size it is expensive to examine
41
+ given high-intensity flow. With a large hand size it is expensive to examine
42
42
the dealt queues and more difficult for the low-intensity flows to dodge the
43
- collective effects of a set of high-intensity flows. Thus, the hand size
43
+ collective effects of a set of high-intensity flows. Thus, the hand size
44
44
should be chosen judiciously.
45
45
0 commit comments