You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Dec 1, 2022. It is now read-only.
Copy file name to clipboardExpand all lines: README.md
+32-16Lines changed: 32 additions & 16 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,7 +6,12 @@
6
6
+ Flexible code over configuration style
7
7
+ Autoscale table and global secondary indexes
8
8
+ Autoscale multiple tables
9
+
+ Autoscale by fixed settings
10
+
+ Autoscale by provisioned capacity utilisation
11
+
+ Autoscale by throttled event metrics
12
+
+ Optimised for large spikes in usage and hotkey issues by incorporating throttled event metrics
9
13
+ Optimised performance using concurrent queries
14
+
+ RateLimitedDecrement as imposed by AWS
10
15
+ Statistics via 'measured'
11
16
+ AWS credential configuration via 'dotenv'
12
17
+ Optimised lambda package via 'webpack'
@@ -34,12 +39,12 @@ configuration. Please see the respective websites for advantages / reasons.
34
39
2. Clone your fork
35
40
3. Create a new file in the root folder called 'config.env.production'
36
41
4. Put your AWS credentials into the file in the following format, only if you want to run a local test (not needed for lambda)
37
-
42
+
38
43
```javascript
39
44
AWS_ACCESS_KEY_ID="###################"
40
45
AWS_SECRET_ACCESS_KEY="###############"
41
46
```
42
-
47
+
43
48
5. Update [Region.json](./src/configuration/Region.json) to match the region of your DynamoDB instance
44
49
6. Run 'npm install'
45
50
7. Run 'npm run build'
@@ -52,7 +57,7 @@ configuration. Please see the respective websites for advantages / reasons.
52
57
2. Create an AWS Policy and Role
53
58
1. Create a policy called 'DynamoDBLambdaAutoscale'
54
59
2. Use the following content to give access to dynamoDB, cloudwatch and lambda logging
55
-
60
+
56
61
```javascript
57
62
{
58
63
"Version": "2012-10-17",
@@ -73,7 +78,7 @@ configuration. Please see the respective websites for advantages / reasons.
73
78
]
74
79
}
75
80
```
76
-
81
+
77
82
3. Create a role called 'DynamoDBLambdaAutoscale'
78
83
4. Attach the newly created policy to the role
79
84
3. Create a AWS Lambda function
@@ -97,17 +102,17 @@ A breakdown of the configuration behaviour is as follows:
97
102
- Separate 'Read' and 'Write' capacity adjustment strategies
98
103
- Separate asymmetric 'Increment' and 'Decrement' capacity adjustment strategies
99
104
- Read/Write provisioned capacity increased
100
-
- when capacity utilisation > 90%
101
-
- by 3 units or to 110% of the current consumed capacity, which ever is the greater
102
-
- with hard min/max limits of 1 and 10 respectively
105
+
- when capacity utilisation > 75% or throttled events in the last minute > 25
106
+
- by 3 + (0.7 * throttled events) units or by 30% + (0.7 * throttled events) of provisioned value or to 130% of the current consumed capacity, which ever is the greater
107
+
- with hard min/max limits of 1 and 100 respectively
103
108
- Read/Write provisioned capacity decreased
104
109
- when capacity utilisation < 30% AND
105
110
- when at least 60 minutes have passed since the last increment AND
106
111
- when at least 60 minutes have passed since the last decrement AND
107
112
- when the adjustment will be at least 5 units AND
108
113
- when we are allowed to utilise 1 of our 4 AWS enforced decrements
109
114
- to the consumed throughput value
110
-
- with hard min/max limits of 1 and 10 respectively
115
+
- with hard min/max limits of 1 and 100 respectively
111
116
112
117
## Strategy Settings
113
118
@@ -117,6 +122,7 @@ both the Increment/Decrement. Using the options below many different strategies
117
122
- ReadCapacity.Max : (Optional) Define a maximum allowed capacity, otherwise unlimited
118
123
- ReadCapacity.Increment : (Optional) Defined an increment strategy
119
124
- ReadCapacity.Increment.When : (Required) Define when capacity should be incremented
125
+
- ReadCapacity.Increment.When.ThrottledEventsPerMinuteIsAbove : (Optional) Define a threshold at which throttled events trigger an increment
120
126
- ReadCapacity.Increment.When.UtilisationIsAbovePercent : (Optional) Define a percentage utilisation upper threshold at which capacity is subject to recalculation
121
127
- ReadCapacity.Increment.When.UtilisationIsBelowPercent : (Optional) Define a percentage utilisation lower threshold at which capacity is subject to recalculation, possible but non sensical for increments however.
122
128
- ReadCapacity.Increment.When.AfterLastIncrementMinutes : (Optional) Define a grace period based off the previous increment in which capacity adjustments should not occur
@@ -126,6 +132,7 @@ both the Increment/Decrement. Using the options below many different strategies
126
132
- ReadCapacity.Increment.By.ConsumedPercent : (Optional) Define a 'relative' percentage adjustment based on the current ConsumedCapacity
127
133
- ReadCapacity.Increment.By.ProvisionedPercent : (Optional) Define a 'relative' percentage adjustment based on the current ProvisionedCapacity
128
134
- ReadCapacity.Increment.By.Units : (Optional) Define a 'relative' unit adjustment
135
+
- ReadCapacity.Increment.By.ThrottledEventsWithMultiplier : (Optional) Define a 'multiple' of the throttled events in the last minute which are added to all other 'By' unit adjustments
129
136
- ReadCapacity.Increment.To : (Optional) Define an 'absolute' value to change the capacity to
130
137
- ReadCapacity.Increment.To.ConsumedPercent : (Optional) Define an 'absolute' percentage adjustment based on the current ConsumedCapacity
131
138
- ReadCapacity.Increment.To.ProvisionedPercent : (Optional) Define an 'absolute' percentage adjustment based on the current ProvisionedCapacity
@@ -136,16 +143,19 @@ A sample of the strategy setting json is...
136
143
{
137
144
"ReadCapacity": {
138
145
"Min":1,
139
-
"Max":10,
146
+
"Max":100,
140
147
"Increment": {
141
148
"When": {
142
-
"UtilisationIsAbovePercent":90
149
+
"UtilisationIsAbovePercent":75,
150
+
"ThrottledEventsPerMinuteIsAbove":25
143
151
},
144
152
"By": {
145
-
"Units":3
153
+
"Units":3,
154
+
"ProvisionedPercent":30,
155
+
"ThrottledEventsWithMultiplier":0.7
146
156
},
147
157
"To": {
148
-
"ConsumedPercent":110
158
+
"ConsumedPercent":130
149
159
}
150
160
},
151
161
"Decrement": {
@@ -162,16 +172,19 @@ A sample of the strategy setting json is...
162
172
},
163
173
"WriteCapacity": {
164
174
"Min":1,
165
-
"Max":10,
175
+
"Max":100,
166
176
"Increment": {
167
177
"When": {
168
-
"UtilisationIsAbovePercent":90
178
+
"UtilisationIsAbovePercent":75,
179
+
"ThrottledEventsPerMinuteIsAbove":25
169
180
},
170
181
"By": {
171
-
"Units":3
182
+
"Units":3,
183
+
"ProvisionedPercent":30,
184
+
"ThrottledEventsWithMultiplier":0.7
172
185
},
173
186
"To": {
174
-
"ConsumedPercent":110
187
+
"ConsumedPercent":130
175
188
}
176
189
},
177
190
"Decrement": {
@@ -226,6 +239,9 @@ class in a layered approach. The layers are as follows:
226
239
- [ProvisionerConfigurableBase.js](./src/provisioning/ProvisionerConfigurableBase.js) abstract base class which breaks out the 'getTableUpdateAsync' function into more manageable abstract methods
227
240
- [ProvisionerBase.js](./src/provisioning/ProvisionerBase.js) the root abstract base class which defines the minimum contract
228
241
242
+
## Throttled Events
243
+
Throttled events are now taken into account as part of the provisioning calculation. A multiple of the events can be added to the existing calculation so that both large spikes in usage and hot key issues are both dealt with.
244
+
229
245
## Rate Limited Decrement
230
246
231
247
AWS only allows 4 table decrements in a calendar day. To account for this we have included
0 commit comments