Skip to content

Commit 5dee70e

Browse files
kcantrelkcantrel
committed
Added a CloudFormation template.
1 parent 39e4af0 commit 5dee70e

File tree

4 files changed

+850
-59
lines changed

4 files changed

+850
-59
lines changed

‎Management-Utilities/auto_set_fsxn_auto_grow/README.md‎

Lines changed: 112 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -2,57 +2,133 @@
22

33
## Introduction
44
This sample shows one way to mitigate the issue of not being able to set the auto size mode
5-
on an FSxN volume when creating it from the AWS console or API. It does this by providing
5+
on an FSx for ONTAP volume when creating it from the AWS console or API. It does this by providing
66
a Lambda function that will set the mode for you, and instructions on how to set up a
77
CloudWatch event to trigger the Lambda function whenever a volume is created. With this
88
combination it ensures that all volumes are effectively created with the auto size mode
99
set up the way you want for all volumes.
1010

11-
## Set Up
12-
There are just a few things you have to do to set this up:
13-
14-
### Create secrets in AWS Secrets Manager
11+
Note that a CloudWatch event is not created when a volume is create directly from the
12+
ONTAP side, either using the ONTAP CLI, System Manager, or REST API. So, it is assumed
13+
if you are creating them that way, that you will set them with the auto size mode set
14+
the way you want.
15+
16+
Note that since the Lambda function has to communicate with the FSx for ONTAP management
17+
endpoint, it has to run within a VPC that has that connectivity. Because of the way
18+
AWS allows a Lambda function to run within a VPC, it will not have access to the Internet
19+
even if normally from that subnet it is running in would. Therefore, you will have to set up
20+
VPC endpoints for the AWS services that the Lambda function will need to communicate with.
21+
This includes:
22+
- FSx for ONTAP
23+
- AWS Secrets Manager
24+
- DynamoDB if you are using it to store the secrets table
25+
Instructions on how to set up these endpoints are provided below.
26+
27+
The way this script authenticates to the FSx for ONTAP management endpoint is by using
28+
the credentials stored in AWS Secrets Manager. Since it can manage multiple FSxN file
29+
systems a table is used to specify which secret to use for each file system. This `scretsTable`
30+
can either be stored in a DynamoDB table, or just hard coded in the source code of the
31+
Lambda function. The schema for the `secretsTable` is as follows:
32+
```json
33+
[
34+
{"fsxId": "fs-XXXXXXXXXXXXXXXXX", "secretName": "fsxn-credentials", "usernameKey": "username", "passwordKey": "password"},
35+
{"fsxId": "fs-XXXXXXXXXXXXXXXXX", "secretName": "fsxn-credentials", "usernameKey": "username", "passwordKey": "password"},
36+
{"fsxId": "fs-XXXXXXXXXXXXXXXXX", "secretName": "fsxn-credentials", "usernameKey": "username", "passwordKey": "password"},
37+
{"fsxId": "fs-XXXXXXXXXXXXXXXXX", "secretName": "fsxn-credentials", "usernameKey": "username", "passwordKey": "password"}
38+
]
39+
```
40+
Where the values associated with each key are as follows:
41+
42+
| Key | Value | Example Value shown above|
43+
|:----|:------| :------------------------|
44+
| `fsxId` | The ID of the FSxN file system. | `fs-XXXXXXXXXXXXXXXXX` |
45+
| `secretName` | The name of the secret in Secrets Manager. | `fsxn-credentials` |
46+
| `usernameKey` | The key in the secret that contains the username. | `username` |
47+
| `passwordKey` | The key in the secret that contains the password. | `password` |
48+
49+
## Deployment
50+
There are two ways to deploy this script. The first way to is use the CloudFormation
51+
template provided in the cloudformation.yaml file. The second way is to following the
52+
steps in the "Manual Setup" second below.
53+
54+
### CloudFormation Deployment
55+
Copy the `cloudformation.yaml` file to your local machine. Then, go to the CloudFormation
56+
service in the AWS console, and click on "Create stack." Select the "Upload a template file"
57+
option, and upload the `cloudformation.yaml` file. Click "Next."
58+
59+
On the next page, give the stack a name. Note that this name is used as a suffix to most of the resources it creates
60+
so you might want to keep is short, but meaningful. After the stack name you will need to fill in the following parameters:
61+
62+
| Parameter Name | Description |
63+
|:--------------|:------------|
64+
| subNetIds| List the subnets that you want the Lambda function to run in. They must have connectivity to the FSxN file systems management endpoints. |
65+
| vpcId | The VPC that contains the subnets. This is only used if you having this CloudFormation template create the AWS service VPC endpoints. |
66+
| securityGroupIds | The security group that the Lambda function will use. This security group should allow access to the AWS service endpoints and the FSx for ONTAP management endpoint over TCP port 443. |
67+
| dynamoDbSecretsTableName | The name of the DynamoDB table that contains the `secretsTable` described above. This value is optional, but if not set, the table commented out in the code will have to be updated to provide the needed information.|
68+
| dynamoDbRegion| The region where the DynamoDB table is located. |
69+
| secretsManagerRegion| The region where the AWS Secrets Manager secrets are located. |
70+
| createWatchdogAlarm | If set to `true` a CloudWatch alarm will be created that will trigger if the Lambda function fails while trying to set the auto size mode on a volume. |
71+
| snsTopicArn| The ARN of the SNS topic that the CloudWatch alarm will send a message to if the Lambda function fails. |
72+
| createSecretManagerEndpoint| If set to `true` a Secrets Manager VPC endpoint will be created. |
73+
| createFSxEndpoint| If set to `true` a FSx VPC endpoint will be created. |
74+
| createDynamoDbEndpoint| If set to `true` a DynamoDB VPC endpoint will be created. |
75+
| routeTableIds| Since the DynamoDB endpoint is a `Gateway` type, routing tables have to be updated to use it. Set this parameter to any route table IDs you want updated. |
76+
| endpointSecurityGroupIds| The security group that the VPC endpoints will use. This security group should allow access to the AWS service the endpoint from the Lambda function over port 443. Since the Lambda function will have the security group specified above assigned to it, it can be used as a network `source` for this security group. |
77+
| autoSizeMode| The auto size mode you want to set the volume to. Valid values are: `grow`, `grow_shrink`, and `off`. |
78+
| growThresholdPrecentage| The percentage of the volume that must be used before a volume will grow. |
79+
| maxGrowSizePercentage| The maximum size the volume can auto grow to expressed in terms of a percentage of the initial volume size. |
80+
| shrinkThresholdPrecentage| The percentage of the volume that must be used before a volume will shrink. |
81+
| minShrinkSizePercentage| The minimum size the volume can auto shrink to expressed in terms of a percentage of the initial volume size. |
82+
| maxWaitTime| The maximum time, in seconds, the script will wait for the volume to be created before it will give up and exits. This can happen if a lot of volumes are created at the same time. |
83+
84+
Once you have filled in these parameters, click `Next`. On the next page you must accept that this script can, and does, create roles. Click `Next`. Finally, on the last page, you can review the stack and click `Submit`.
85+
86+
After the stack is created and everything should be setup. To test, simply create a volume in the AWS console and check that auto size mode from the ONTAP CLI. If it isn't set, check the CloudWatch logs for the Lambda function to see what went wrong. You can quickly go to the correct Lambda function by clicking on the Resources tab with in the CloudFormation stack, and clicking on the link to the Lambda function.
87+
88+
### Manual Setup
89+
If for some reason you can't run the CloudFormation template, here are the steps you can use to manually setup the service:
90+
91+
#### Create secrets in AWS Secrets Manager
1592
Create a secret in Secrets Manager for each of the FSxN file systems you want to manage with
1693
this script. Each secret should have two key value pairs. One that specifies the
1794
user account to use when issuing API calls, and the other that specifies the password for
1895
that account. Note that if you use the same username and password, it is okay
1996
to use the same secret for multiple file systems.
2097

21-
### Create a role for the Lambda function
98+
#### Create a role for the Lambda function
2299
The Lambda function doesn't leverage that many AWS services, so only a few permissions are required:
23100

24-
25101
| Permission | Minimal Scope | Notes
26102
|:------------------------|:----------------|:----------------|
27-
| Allow:logs:CreateLogGroup | arn:aws:logs:<LAMBDA_REGION>:<ACCOUNT_ID>:* | This is required so you can get logs from the Lambda function. |
28-
| Allow:logs:CreateLogStream<BR>Allow:logs:PutLogEvents | arn:aws:logs:<LAMBDA_REGION>:<ACCOUNT_ID>:/aws/lambda/<LAMBDA_FUNCTION_NAME>:* | This is required so you can get logs from the Lambda function. |
29-
| Allow:secretsmanager:GetSecretValue | <ARN_OF_SECRET_WITHIN_SECRETS_MANAGER> | This is required so the Lambda function can get the credentials for the FSxN file system. |
30-
| Allow:dynamodb:Scan | <ARN_OF_DYNAMODB_TABLE> | This is optional, depending on if you put your secretsTable in a DynamoDB. |
31-
| Allow:fsx:DescribeFileSystems<BR>Allow:fsx:DescribeVolumes | * | You can't limit these API. They are required to get information regarding the file system and volumes. |
32-
| Allow:ec2:CreateNetworkInterface<BR>Allow:ec2:DeleteNetworkInterface<BR>Allow:ec2:DescribeNetworkInterfaces | * | Since the Lambda function is going to run within your VPC, it has to be able to create a network interface to communicate with the FSxn file system API. |
33-
34-
### Create AWS Endpoints
35-
Since the Lambda function will be configured to run within the VPC that contains the FSxN
36-
file system, so it can issue API calls against it, there will need to be AWS endpoints so
37-
the Lambda function can access some of the AWS service. If you have a Transit Gateway setup
103+
| Allow:logs:CreateLogGroup | arn:aws:logs:\<LAMBDA_REGION>:\<ACCOUNT_ID>:* | This is required so you can get logs from the Lambda function. |
104+
| Allow:logs:CreateLogStream<BR>Allow:logs:PutLogEvents | arn:aws:logs:\<LAMBDA_REGION>:\<ACCOUNT_ID>:/aws/lambda/\<LAMBDA_FUNCTION_NAME>:* | This is required so you can get logs from the Lambda function. |
105+
| Allow:secretsmanager:GetSecretValue | \<ARNs_OF_SECRETS_WITHIN_SECRETS_MANAGER> | This is required so the Lambda function can get the credentials for the FSxN file system. |
106+
| Allow:dynamodb:Scan | \<ARN_OF_DYNAMODB_TABLE> | This is optional, depending on if you put your `secretsTable` in a DynamoDB table. |
107+
| Allow:fsx:DescribeFileSystems<BR>Allow:fsx:DescribeVolumes | * | You can't limit the scope of these APIs. They are required to get information regarding the file system and volumes. |
108+
| Allow:ec2:CreateNetworkInterface<BR>Allow:ec2:DeleteNetworkInterface<BR>Allow:ec2:DescribeNetworkInterfaces | * | Since the Lambda function is going to run within your VPC, it has to be able to create a network interface to communicate with the FSxn file system endpoint. |
109+
110+
#### Create AWS Endpoints
111+
Since the Lambda function will be configured to run within a VPC that can communicate with the FSxN
112+
file systems, so it can issue API calls against them, there will need to be AWS endpoints so
113+
the Lambda function can also access some of the AWS service. If you have a Transit Gateway setup
38114
that allows access to the Internet, you may not have to create these endpoints, otherwise, the
39-
following endpoints will need to be created, and attached to the VPC and subnets that the
40-
FSxN file system is attached to.
115+
following endpoints will need to be created, and attached to the VPC and subnets that the Lambda
116+
function will run in:
41117

42118
- FSx
43119
- SecretsManager
44-
- DynamoDB - You only need this one if you are going to store you secrtsTable in DynamoDB. It can be a Gateway endpoint.
120+
- DynamoDB - You only need this one if you are going to store you `secrtsTable` in DynamoDB. It is recommended that this be a `Gateway` type endpoint, but if you set it up that way you will also have to update the routing tables to use it.
45121

46-
### Create the Lambda Function
122+
#### Create the Lambda Function
47123
Create a Lambda function with the following parameters:
48124

49125
- Authored from scratch.
50126
- Uses the Python runtime.
51127
- Set the permissions to the role created above.
52128
- Enable VPC. Found under the Advanced Settings.
53-
- Attached to the VPC that contains the FSxN file system
54-
- Attached to the Subnets that contain the FSxN file system.
55-
- Attached a security group that allows access from any IP within the two subnets.
129+
- Attached to the VPC that can communicate with the FSxN file systems.
130+
- Attached to the Subnets that can communicate with the FSxN file systems.
131+
- Attached a security group that allows access from any IP within the two subnets over port 443.
56132

57133
After you create the function, you will be able to insert the code included with this
58134
sample into the code box. Once you have inserted the code, modify the definitions
@@ -66,38 +142,37 @@ is a dictionary with the following keys:
66142
**NOTE:** Instead of defining the secretsTable in the script, you can define
67143
dynamodbSecretsTableName and dynamodbRegion and the script will read in the
68144
secretsTable information from the specified DynamoDB table. The table should have
69-
the same fields as the secretsTable defined above.
145+
the same fields as the `secretsTable` defined above.
70146

71147
- secretsManagerRegion - Defines the region where your secrets are stored.
72148
- autoSizeMode - Defines the auto size mode you want to set the volume to. Valid values are:
73149
- grow - The volume will automatically grow when it reaches the grow threshold.
74-
- grow_shrink - The volume will automatically grow, and shrink when it reachs the shrink threshold.
150+
- grow_shrink - The volume will automatically grow, and shrink when it reaches the shrink threshold.
75151
- off - The volume will not automatically grow or shrink.
76152
- growThresholdPercentage - The percentage of the volume that must be used before the volume will grow.
77153
- maxGrowSizePercentage - The maximum size the volume can auto grow to expressed in terms of a percentage of the volume size. The default is 200%.
78154
- shrinkThresholdPercentage - The percentage of the volume that must be used before the volume will shrink.
79155
- minShrinkSizePercentage - The minimum size the volume can auto shrink to expressed in terms of a percentage of the volume size. The default is 50%.
80156
- maxWaitTime - The maximum time, in seconds, the script will wait for the volume to be created before it will give up and exit.
81157

82-
**NOTE:** Do not delete the variables or set them to None or empty
83-
strings, as the script will not run properly if done so.
158+
**NOTE:** Do not delete the variables or set them to None or empty strings, as the script will not run properly if done so.
84159

85160
Once you have updated the program, click on the "Deploy" button.
86161

87162
Next, click on the Configuration tab, then General and set the timeout to 2 minutes, or
88-
two times the number of seconds you set the maxWaitTime variable. Note that typically
163+
two times the number of seconds you set the `maxWaitTime` variable. Note that typically
89164
the program will not run this long, but if there are a lot of volumes being created at the
90165
same time, it may have to wait a while for the volume to get created on the ONTAP side before
91166
it can set the auto size mode.
92167

93-
### Create an Event Bridge Rule (a.k.a. CloudWatch Event) that will trigger when a FSx Volume is created
168+
#### Create an Event Bridge Rule (a.k.a. CloudWatch Event) that will trigger when a FSx Volume is created
94169
Once on the "Event Bridge" page, click on Rules on the left-hand side. From there click
95170
on Create Rule. Give the rule a name, and make sure to put the rule on the "Default" bus.
96171
Finally select "Rule with an event pattern" and click Next.
97172

98173
Select "other" as the event source, skip pass the "Sample Event" section, and click on
99-
"Custom pattern (JSON editor)" under the Creation Method. Paste the following in the
100-
Edit Event Pattern text box:
174+
"Custom pattern (JSON editor)" under the Creation Method paste the following in the
175+
`Edit Event Pattern` text box:
101176
```json
102177
{
103178
"detail-type": [
@@ -114,11 +189,14 @@ Edit Event Pattern text box:
114189
}
115190
```
116191

117-
Click Next. This next page will allow you to select the Lambda function you created above.
192+
Click `Next`. The next page will allow you to select the Lambda function you created above.
118193
Just take the defaults for the remaining pages and click on "Create Rule."
119194

120195
At this point every time a volume is created the Lambda function will be called, and it will
121196
attempt to set the auto size mode as specified via the variables at the top of the code.
197+
To confirm it is working, create a volume in the AWS console and check the auto size mode
198+
from the ONTAP CLI. If it isn't set, check the CloudWatch logs for the Lambda function to
199+
see what went wrong.
122200

123201
## Author Information
124202

0 commit comments

Comments
 (0)