You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: Management-Utilities/auto_set_fsxn_auto_grow/README.md
+31-23Lines changed: 31 additions & 23 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,7 +8,7 @@ CloudWatch event to trigger the Lambda function whenever a volume is created. Wi
8
8
combination it ensures that all volumes are effectively created with the auto size mode
9
9
set up the way you want for all volumes.
10
10
11
-
Note that a CloudWatch event is not created when a volume is create directly from the
11
+
Note that a CloudWatch event is not created when a volume is created directly from the
12
12
ONTAP side, either using the ONTAP CLI, System Manager, or REST API. So, it is assumed
13
13
if you are creating them that way, that you will set them with the auto size mode set
14
14
the way you want.
@@ -19,14 +19,17 @@ AWS allows a Lambda function to run within a VPC, it will not have access to the
19
19
even if normally from that subnet it is running in would. Therefore, you will have to set up
20
20
VPC endpoints for the AWS services that the Lambda function will need to communicate with.
21
21
This includes:
22
-
- FSx for ONTAP
22
+
- FSx
23
23
- AWS Secrets Manager
24
24
- DynamoDB if you are using it to store the secrets table
25
-
Instructions on how to set up these endpoints are provided below.
25
+
26
+
These endpoints are created for you if you use the CloudFormation template provided in this
27
+
repository. If you are setting up the Lambda function manually, you will have to create
28
+
these endpoints yourself.
26
29
27
30
The way this script authenticates to the FSx for ONTAP management endpoint is by using
28
31
the credentials stored in AWS Secrets Manager. Since it can manage multiple FSxN file
29
-
systems a table is used to specify which secret to use for each file system. This `scretsTable`
32
+
systems a table is used to specify which secret to use for each file system. This `secretsTable`
30
33
can either be stored in a DynamoDB table, or just hard coded in the source code of the
31
34
Lambda function. The schema for the `secretsTable` is as follows:
32
35
```json
@@ -48,21 +51,21 @@ Where the values associated with each key are as follows:
48
51
49
52
## Deployment
50
53
There are two ways to deploy this script. The first way to is use the CloudFormation
51
-
template provided in the cloudformation.yaml file. The second way is to following the
52
-
steps in the "Manual Setup" second below.
54
+
template provided in the cloudformation.yaml file. The second way is to follow the
55
+
steps in the "Manual Setup" section below.
53
56
54
57
### CloudFormation Deployment
55
58
Copy the `cloudformation.yaml` file to your local machine. Then, go to the CloudFormation
56
59
service in the AWS console, and click on "Create stack." Select the "Upload a template file"
57
-
option, and upload the `cloudformation.yaml` file. Click "Next."
60
+
option and upload the `cloudformation.yaml` file. Click "Next."
58
61
59
62
On the next page, give the stack a name. Note that this name is used as a suffix to most of the resources it creates
60
-
so you might want to keep is short, but meaningful. After the stack name you will need to fill in the following parameters:
63
+
so you might want to keep it short, but meaningful. After the stack name you will need to fill in the following parameters:
61
64
62
65
| Parameter Name | Description |
63
66
|:--------------|:------------|
64
67
| subNetIds| List the subnets that you want the Lambda function to run in. They must have connectivity to the FSxN file systems management endpoints. |
65
-
| vpcId | The VPC that contains the subnets. This is only used if you having this CloudFormation template create the AWS service VPC endpoints. |
68
+
| vpcId | The VPC that contains the subnets. This is only used if you are having this CloudFormation template create the AWS service VPC endpoints. |
66
69
| securityGroupIds | The security group that the Lambda function will use. This security group should allow access to the AWS service endpoints and the FSx for ONTAP management endpoint over TCP port 443. |
67
70
| dynamoDbSecretsTableName | The name of the DynamoDB table that contains the `secretsTable` described above. This value is optional, but if not set, the table commented out in the code will have to be updated to provide the needed information.|
68
71
| dynamoDbRegion| The region where the DynamoDB table is located. |
@@ -73,17 +76,22 @@ so you might want to keep is short, but meaningful. After the stack name you wil
73
76
| createFSxEndpoint| If set to `true` a FSx VPC endpoint will be created. |
74
77
| createDynamoDbEndpoint| If set to `true` a DynamoDB VPC endpoint will be created. |
75
78
| routeTableIds| Since the DynamoDB endpoint is a `Gateway` type, routing tables have to be updated to use it. Set this parameter to any route table IDs you want updated. |
76
-
| endpointSecurityGroupIds| The security group that the VPC endpoints will use. This security group should allow access to the AWS service the endpoint from the Lambda function over port 443. Since the Lambda function will have the security group specified above assigned to it, it can be used as a network `source` for this security group. |
79
+
| endpointSecurityGroupIds| The security group that the VPC endpoints will use. This security group should allow access to the AWS service the endpoints from the Lambda function over port 443. Since the Lambda function will have the security group specified above assigned to it, it can be used as a network `source` for this security group. |
77
80
| autoSizeMode| The auto size mode you want to set the volume to. Valid values are: `grow`, `grow_shrink`, and `off`. |
78
81
| growThresholdPrecentage| The percentage of the volume that must be used before a volume will grow. |
79
82
| maxGrowSizePercentage| The maximum size the volume can auto grow to expressed in terms of a percentage of the initial volume size. |
80
83
| shrinkThresholdPrecentage| The percentage of the volume that must be used before a volume will shrink. |
81
84
| minShrinkSizePercentage| The minimum size the volume can auto shrink to expressed in terms of a percentage of the initial volume size. |
82
-
| maxWaitTime| The maximum time, in seconds, the script will wait for the volume to be created before it will give up and exits. This can happen if a lot of volumes are created at the same time. |
85
+
| maxWaitTime| The maximum time, in seconds, that the script will wait for the volume to be created before it will give up and exit. This can happen if a lot of volumes are created at the same time. |
83
86
84
-
Once you have filled in these parameters, click `Next`. On the next page you must accept that this script can, and does, create roles. Click `Next`. Finally, on the last page, you can review the stack and click `Submit`.
87
+
Once you have filled in these parameters, click `Next`. On the next page you must accept that this
88
+
script can, and does, create roles. Click `Next`. Finally, on the last page, you can review the stack and click `Submit`.
85
89
86
-
After the stack is created and everything should be setup. To test, simply create a volume in the AWS console and check that auto size mode from the ONTAP CLI. If it isn't set, check the CloudWatch logs for the Lambda function to see what went wrong. You can quickly go to the correct Lambda function by clicking on the Resources tab with in the CloudFormation stack, and clicking on the link to the Lambda function.
90
+
After the stack has been created everything should be ready. To test, simply create a volume in the
91
+
AWS console and check from the ONTAP CLI that auto size mode appropriately. If it isn't set, check the CloudWatch
92
+
logs for the Lambda function to see what went wrong. You can quickly go to the correct Lambda
93
+
function by clicking on the Resources tab within the CloudFormation stack and clicking on the
94
+
link to the Lambda function.
87
95
88
96
### Manual Setup
89
97
If for some reason you can't run the CloudFormation template, here are the steps you can use to manually setup the service:
@@ -105,30 +113,30 @@ The Lambda function doesn't leverage that many AWS services, so only a few permi
105
113
| Allow:secretsmanager:GetSecretValue|\<ARNs_OF_SECRETS_WITHIN_SECRETS_MANAGER> | This is required so the Lambda function can get the credentials for the FSxN file system. |
106
114
| Allow:dynamodb:Scan|\<ARN_OF_DYNAMODB_TABLE> | This is optional, depending on if you put your `secretsTable` in a DynamoDB table. |
107
115
| Allow:fsx:DescribeFileSystems<BR>Allow:fsx:DescribeVolumes| * | You can't limit the scope of these APIs. They are required to get information regarding the file system and volumes. |
108
-
| Allow:ec2:CreateNetworkInterface<BR>Allow:ec2:DeleteNetworkInterface<BR>Allow:ec2:DescribeNetworkInterfaces| * | Since the Lambda function is going to run within your VPC, it has to be able to create a network interface to communicate with the FSxn file system endpoint. |
116
+
| Allow:ec2:CreateNetworkInterface<BR>Allow:ec2:DeleteNetworkInterface<BR>Allow:ec2:DescribeNetworkInterfaces| * | Since the Lambda function is going to run within your VPC, it has to be able to create a network interface to communicate with the FSxN file system endpoints. |
109
117
110
118
#### Create AWS Endpoints
111
119
Since the Lambda function will be configured to run within a VPC that can communicate with the FSxN
112
120
file systems, so it can issue API calls against them, there will need to be AWS endpoints so
113
-
the Lambda function can also access some of the AWS service. If you have a Transit Gateway setup
121
+
the Lambda function can also access some of the AWS services. If you have a Transit Gateway setup
114
122
that allows access to the Internet, you may not have to create these endpoints, otherwise, the
115
123
following endpoints will need to be created, and attached to the VPC and subnets that the Lambda
116
124
function will run in:
117
125
118
126
- FSx
119
127
- SecretsManager
120
-
- DynamoDB - You only need this one if you are going to store you `secrtsTable` in DynamoDB. It is recommended that this be a `Gateway` type endpoint, but if you set it up that way you will also have to update the routing tables to use it.
128
+
- DynamoDB - You only need this one if you are going to store your `secretsTable` in DynamoDB. It is recommended that this be a `Gateway` type endpoint. However, if you do that you will also have to update the routing tables associated with the subnets that the Lambda function is deployed on in order for the Lambda function to be able to use it.
121
129
122
130
#### Create the Lambda Function
123
131
Create a Lambda function with the following parameters:
124
132
125
133
- Authored from scratch.
126
-
-Uses the Python runtime.
134
+
-Use the Python runtime.
127
135
- Set the permissions to the role created above.
128
136
- Enable VPC. Found under the Advanced Settings.
129
137
- Attached to the VPC that can communicate with the FSxN file systems.
130
138
- Attached to the Subnets that can communicate with the FSxN file systems.
131
-
- Attached a security group that allows access from any IP within the two subnets over port 443.
139
+
- Attached to a security group that allows access from any IP within the two subnets over port 443.
132
140
133
141
After you create the function, you will be able to insert the code included with this
134
142
sample into the code box. Once you have inserted the code, modify the definitions
@@ -140,7 +148,7 @@ is a dictionary with the following keys:
140
148
- passwordKey - The name of the key in the secret that contains the password.
141
149
142
150
**NOTE:** Instead of defining the secretsTable in the script, you can define
143
-
dynamodbSecretsTableName and dynamodbRegion and the script will read in the
151
+
dynamoDbSecretsTableName and dynamoDbRegion and the script will read in the
144
152
secretsTable information from the specified DynamoDB table. The table should have
145
153
the same fields as the `secretsTable` defined above.
146
154
@@ -149,10 +157,10 @@ the same fields as the `secretsTable` defined above.
149
157
- grow - The volume will automatically grow when it reaches the grow threshold.
150
158
- grow_shrink - The volume will automatically grow, and shrink when it reaches the shrink threshold.
151
159
- off - The volume will not automatically grow or shrink.
152
-
- growThresholdPercentage - The percentage of the volume that must be used before the volume will grow.
153
-
- maxGrowSizePercentage - The maximum size the volume can auto grow to expressed in terms of a percentage of the volume size. The default is 200%.
154
-
- shrinkThresholdPercentage - The percentage of the volume that must be used before the volume will shrink.
155
-
- minShrinkSizePercentage - The minimum size the volume can auto shrink to expressed in terms of a percentage of the volume size. The default is 50%.
160
+
- growThresholdPercentage - The percentage of the volume that must be in use before the volume will grow.
161
+
- maxGrowSizePercentage - The maximum size the volume can auto grow to, expressed in terms of a percentage of the initial volume size.
162
+
- shrinkThresholdPercentage - The percentage of the volume that must be in use before the volume will shrink.
163
+
- minShrinkSizePercentage - The minimum size the volume can auto shrink to, expressed in terms of a percentage of the initial volume size.
156
164
- maxWaitTime - The maximum time, in seconds, the script will wait for the volume to be created before it will give up and exit.
157
165
158
166
**NOTE:** Do not delete the variables or set them to None or empty strings, as the script will not run properly if done so.
0 commit comments