Skip to content
This repository was archived by the owner on Dec 1, 2022. It is now read-only.

Commit ed93caa

Browse files
committed
Update README.md
1 parent eb4de24 commit ed93caa

File tree

1 file changed

+88
-39
lines changed

1 file changed

+88
-39
lines changed

README.md

Lines changed: 88 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -1,25 +1,18 @@
1-
dynamodb-lambda-autoscale
2-
=========================
3-
Autoscale AWS DynamoDB using an AWS Lambda function
1+
# dynamodb-lambda-autoscale
2+
**Autoscale AWS DynamoDB using an AWS Lambda function
43

5-
DynamoDB does not currently have an official way of autoscaling the capacity throughput for reads and writes.
4+
+ 5 minute setup process
5+
+ Serverless design
6+
+ Flexible code over configuration style
7+
+ Autoscale table and global secondary indexes
8+
+ Autoscale multiple tables
9+
+ Optimised performance using concurrent queries
10+
+ Statistics via 'measured'
11+
+ AWS credential configuration via 'dotenv'
12+
+ Optimised lambda package via 'webpack'
13+
+ ES7 code
614

7-
Features
8-
--------
9-
10-
- 5 minute setup process
11-
- Serverless design
12-
- Flexible code over configuration style
13-
- Autoscale table and global secondary indexes
14-
- Autoscale multiple tables
15-
- Optimised performance using concurrent queries
16-
- Statistics via 'measured'
17-
- AWS credential configuration via 'dotenv'
18-
- Optimised lambda package via 'webpack'
19-
- ES7 code
20-
21-
Setup
22-
-----
15+
## Getting started
2316

2417
1. Build and package the code
2518
1. Fork the repo
@@ -34,6 +27,10 @@ Setup
3427
4. Run 'npm run build'
3528
5. Verify this has created a 'dist.zip' file
3629
6. Optionally, run a local test by running 'npm run start'
30+
31+
## Running on AWS Lambda
32+
33+
1. Follow the steps in 'Running locally'
3734
2. Create an AWS Policy and Role
3835
1. Create a policy called 'DynamoDBLambdaAutoscale'
3936
2. Use the following content to give access to dynamoDB, cloudwatch and lambda logging
@@ -70,27 +67,79 @@ Setup
7067
8. Set the Timeout to 5 seconds (higher possibly depending on the amount of tables you have)
7168
9. Once the function is created, attach a 'scheduled event' event source and make it run every minute
7269

73-
License
74-
-------
70+
## Configuration
71+
72+
The default setup of the configuration is to apply autoscaling to all tables,
73+
allowing for a no touch quick setup.
74+
75+
dynamodb-lambda-autoscale takes a different approach to autoscaling
76+
configuration compared to other community projects. Rather than making changes
77+
to a config file dynamodb-lambda-autoscale provides a function 'getTableUpdate'
78+
which must be implemented.
79+
80+
81+
```javascript
82+
{
83+
connection: {
84+
dynamoDB: { apiVersion: '2012-08-10', region: 'us-east-1' },
85+
cloudWatch: { apiVersion: '2010-08-01', region: 'us-east-1' }
86+
},
87+
getTableUpdate: (description, consumedCapacityDescription) => {
88+
// The 'description' parameter is JSON in the following format
89+
<http://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_DescribeTable.html#API_DescribeTable_ResponseSyntax>
90+
91+
// The 'consumedCapacityDescription' follows the same format but details
92+
// the consumed capacity found via the cloudwatch api
93+
94+
// The return type is either null for no update or JSON in the
95+
// following format
96+
<http://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_UpdateTable.html#API_UpdateTable_ResponseSyntax>
97+
}
98+
};
99+
```
100+
101+
The function is given the information such as the table name, current table
102+
provisioned throughput and the consumed throughput for the past minute.
103+
Table updates will only be sent to AWS if the values are different for the
104+
current, this approach follows the popular code first pattern used in React.
105+
106+
In most cases the default 'ConfigurableProvisioner' supplied will provide
107+
enough functionality out of box such that additional coding is not required.
108+
The default provisioner provides the following features.
109+
110+
- Separate 'Read' and 'Write' capacity adjustment
111+
- Separate 'Increment' and 'Decrement' capacity adjustment
112+
- Read/Write provisioned capacity increased
113+
- if capacity utilisation > 90%
114+
- by either 100% or 3 units, which ever is the greater
115+
- with hard min/max limits of 1 and 10 respectively
116+
- Read/Write provisioned capacity decreased
117+
- if capacity utilisation < 30% AND
118+
- if at least 60 minutes have passed since the last increment AND
119+
- if at least 60 minutes have passed since the last decrement AND
120+
- if the adjustment will be at least 3 units AND
121+
- if we are allowed to utilise 1 of our 4 AWS enforced decrements
122+
- to the consumed throughput value
123+
- with a hard min limit of 1
75124

76-
The MIT License (MIT)
125+
As AWS only allows 4 table decrements in a calendar day we have an intelligent
126+
algorithm which segments the remaining time to midnight by the amount of
127+
decrements we have left. This logic allows us to utilise each 4 decrements
128+
efficiently. The increments are unlimited so the algorithm follows a unique
129+
'sawtooth' profile, dropping the provisioned throughput all the way down to
130+
the consumed throughput rather than gradually. Please see
131+
[RateLimitedDecrement.js](./src/RateLimitedDecrement.js) for full
132+
implementation.
77133

78-
Copyright (c) 2016 for dynamodb-lambda-autoscale
134+
## Dependencies
79135

80-
Permission is hereby granted, free of charge, to any person obtaining a copy
81-
of this software and associated documentation files (the "Software"), to deal
82-
in the Software without restriction, including without limitation the rights
83-
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
84-
copies of the Software, and to permit persons to whom the Software is
85-
furnished to do so, subject to the following conditions:
136+
dynamodb-lambda-autoscale has the following main dependencies:
137+
+ aws-sdk - Access to AWS services
138+
+ winston - Logging
139+
+ dotenv - Environment variable configuration useful for lambda
140+
+ measured - Statistics gathering
86141

87-
The above copyright notice and this permission notice shall be included in all
88-
copies or substantial portions of the Software.
142+
## Licensing
89143

90-
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
91-
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
92-
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
93-
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
94-
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
95-
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
96-
SOFTWARE.
144+
The source code is licensed under the MIT license found in the
145+
[LICENSE](LICENSE) file in the root directory of this source tree.

0 commit comments

Comments
 (0)