Skip to content

Commit 5208d3a

Browse files
baileympearsonbaileympearson
authored
ci(NODE-6681): secrets manager for AWS auth tests (#4400)
1 parent 6528c8d commit 5208d3a

8 files changed

+365
-777
lines changed

‎.evergreen/config.in.yml

Lines changed: 92 additions & 268 deletions
Large diffs are not rendered by default.

‎.evergreen/config.yml

Lines changed: 172 additions & 471 deletions
Large diffs are not rendered by default.

‎.evergreen/generate_evergreen_tasks.js

Lines changed: 4 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -307,8 +307,7 @@ AWS_LAMBDA_HANDLER_TASKS.push({
307307
}),
308308
{ func: 'install dependencies' },
309309
{ func: 'bootstrap mongo-orchestration' },
310-
{ func: 'add aws auth variables to file' },
311-
{ func: 'setup aws env' },
310+
{ func: 'assume secrets manager rule' },
312311
{ func: 'run lambda handler example tests with aws auth' }
313312
]
314313
});
@@ -365,8 +364,7 @@ for (const VERSION of AWS_AUTH_VERSIONS) {
365364
}),
366365
{ func: 'install dependencies' },
367366
{ func: 'bootstrap mongo-orchestration' },
368-
{ func: 'add aws auth variables to file' },
369-
{ func: 'setup aws env' },
367+
{ func: 'assume secrets manager rule' },
370368
{ func: fn.func }
371369
]
372370
}));
@@ -385,9 +383,7 @@ for (const VERSION of AWS_AUTH_VERSIONS) {
385383
}),
386384
{ func: 'install dependencies' },
387385
{ func: 'bootstrap mongo-orchestration' },
388-
{ func: 'add aws auth variables to file' },
389-
{ func: 'setup aws env' },
390-
{ func: 'remove aws-credential-providers' },
386+
{ func: 'assume secrets manager rule' },
391387
{ func: fn.func }
392388
]
393389
}));
@@ -684,6 +680,7 @@ for (const version of ['5.0', 'rapid', 'latest']) {
684680
{ func: 'bootstrap mongo-orchestration' },
685681
{ func: 'bootstrap kms servers' },
686682
{ func: 'install mongodb-client-encryption' },
683+
{ func: 'assume secrets manager rule' },
687684
{ func: 'run custom csfle tests' }
688685
]
689686
});

‎.evergreen/prepare-mongodb-aws-ecs-auth.sh

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
#! /usr/bin/env bash
2+
3+
AUTH_AWS_DIR=${DRIVERS_TOOLS}/.evergreen/auth_aws
4+
ECS_SRC_DIR=$AUTH_AWS_DIR/src
5+
6+
bash $DRIVERS_TOOLS/.evergreen/auth_aws/setup-secrets.sh
7+
8+
# pack up project directory to ssh it to the container
9+
mkdir -p $ECS_SRC_DIR/.evergreen
10+
set -ex
11+
12+
# write test file
13+
echo "export MONGODB_AWS_SDK=$MONGODB_AWS_SDK" >>$PROJECT_DIRECTORY/.evergreen/run-mongodb-aws-ecs-test.sh
14+
echo "if [ $MONGODB_AWS_SDK = 'false' ]; then rm -rf ./node_modules/@aws-sdk/credential-providers; fi" >>$PROJECT_DIRECTORY/.evergreen/run-mongodb-aws-ecs-test.sh
15+
echo "npm run check:aws" >>$PROJECT_DIRECTORY/.evergreen/run-mongodb-aws-ecs-test.sh
16+
17+
# copy test file to AWS ecs test directory
18+
cp $PROJECT_DIRECTORY/.evergreen/run-mongodb-aws-ecs-test.sh $ECS_SRC_DIR/.evergreen/
19+
20+
cat $ECS_SRC_DIR/.evergreen/run-mongodb-aws-ecs-test.sh
21+
22+
# tar the file and drivers tools and do the same
23+
cd ..
24+
tar -czf src.tgz src drivers-tools
25+
mv src.tgz $ECS_SRC_DIR/src.tgz
26+
27+
export MONGODB_BINARIES="${MONGODB_BINARIES}"
28+
29+
export PROJECT_DIRECTORY=$ECS_SRC_DIR
30+
31+
bash $AUTH_AWS_DIR/aws_setup.sh ecs

‎.evergreen/run-lambda-aws-tests.sh

Lines changed: 4 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1,24 +1,13 @@
11
#!/bin/bash
22
# set -o xtrace # Write all commands first to stderr
3-
set -o errexit # Exit the script with error if any of the commands fail
4-
5-
MONGODB_URI=${MONGODB_URI:-}
6-
7-
# ensure no secrets are printed in log files
8-
set +x
9-
10-
# load node.js environment
11-
source $DRIVERS_TOOLS/.evergreen/init-node-and-npm-env.sh
3+
set -o errexit # Exit the script with error if any of the commands fail
124

135
# the default connection string, may be overridden by the environment script
146
export MONGODB_URI="mongodb://localhost:27017/aws"
157

16-
# load the script
17-
shopt -s expand_aliases # needed for `urlencode` alias
18-
[ -s "$PROJECT_DIRECTORY/prepare_mongodb_aws.sh" ] && source "$PROJECT_DIRECTORY/prepare_mongodb_aws.sh"
8+
source .evergreen/setup-mongodb-aws-auth-tests.sh
199

20-
# revert to show test output
21-
set -x
10+
# load node.js environment
11+
source $DRIVERS_TOOLS/.evergreen/init-node-and-npm-env.sh
2212

23-
npm install aws4
2413
npm run check:lambda:aws

‎.evergreen/run-mongodb-aws-test.sh

Lines changed: 2 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1,25 +1,12 @@
11
#!/bin/bash
22
# set -o xtrace # Write all commands first to stderr
3-
set -o errexit # Exit the script with error if any of the commands fail
3+
set -o errexit # Exit the script with error if any of the commands fail
44

55
MONGODB_URI=${MONGODB_URI:-}
66

7-
# ensure no secrets are printed in log files
8-
set +x
7+
source .evergreen/setup-mongodb-aws-auth-tests.sh
98

109
# load node.js environment
1110
source $DRIVERS_TOOLS/.evergreen/init-node-and-npm-env.sh
1211

13-
# the default connection string, may be overridden by the environment script
14-
export MONGODB_URI="mongodb://localhost:27017/aws?authMechanism=MONGODB-AWS"
15-
16-
# load the script
17-
shopt -s expand_aliases # needed for `urlencode` alias
18-
[ -s "$PROJECT_DIRECTORY/prepare_mongodb_aws.sh" ] && source "$PROJECT_DIRECTORY/prepare_mongodb_aws.sh"
19-
20-
# revert to show test output
21-
set -x
22-
23-
npm install aws4
24-
if [ $MONGODB_AWS_SDK = 'false' ]; then rm -rf ./node_modules/@aws-sdk/credential-providers; fi
2512
npm run check:aws

‎.evergreen/setup-mongodb-aws-auth-tests.sh

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
#!/bin/bash
2+
# set -o xtrace # Write all commands first to stderr
3+
set -o errexit # Exit the script with error if any of the commands fail
4+
5+
# ensure no secrets are printed in log files
6+
set +x
7+
8+
if [ -z ${MONGODB_URI+omitted} ]; then echo "MONGODB_URI is unset" && exit 1; fi
9+
if [ -z ${DRIVERS_TOOLS+omitted} ]; then echo "DRIVERS_TOOLS is unset" && exit 1; fi
10+
if [ -z ${AWS_CREDENTIAL_TYPE+omitted} ]; then echo "AWS_CREDENTIAL_TYPE is unset" && exit 1; fi
11+
if [ -z ${MONGODB_AWS_SDK+omitted} ]; then echo "MONGODB_AWS_SDK is unset" && exit 1; fi
12+
13+
bash $DRIVERS_TOOLS/.evergreen/auth_aws/setup-secrets.sh
14+
15+
BEFORE=$(pwd)
16+
17+
cd $DRIVERS_TOOLS/.evergreen/auth_aws
18+
19+
# Create a python virtual environment.
20+
. ./activate-authawsvenv.sh
21+
# Source the environment variables. Configure the environment and the server.
22+
. aws_setup.sh $AWS_CREDENTIAL_TYPE
23+
24+
cd $BEFORE
25+
26+
npm install --no-save aws4
27+
28+
if [ $MONGODB_AWS_SDK = 'false' ]; then rm -rf ./node_modules/@aws-sdk/credential-providers; fi
29+
30+
# revert to show test output
31+
set -x

‎test/readme.md

Lines changed: 29 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,7 @@ about the types of tests and how to run them.
3939
- [Search Indexes](#search-indexes)
4040
- [Deployed Lambda Tests](#deployed-lambda-tests)
4141
- [Kerberos Tests](#kerberos-tests)
42+
- [AWS Authentication tests](#aws-authentication-tests)
4243
- [TODO Special Env Sections](#todo-special-env-sections)
4344
- [Testing driver changes with mongosh](#testing-driver-changes-with-mongosh)
4445
- [Point mongosh to the driver](#point-mongosh-to-the-driver)
@@ -617,11 +618,38 @@ TODO(NODE-6698): Update deployed lambda test section.
617618
618619
### Kerberos Tests
619620
621+
You must be in an office or connected to the VPN to run these tests.
622+
620623
Run `.evergreen/run-kerberos-tests.sh`.
621624
625+
### AWS Authentication tests
626+
627+
> [!NOTE]
628+
> AWS ECS tests have a different set up process. Don't even bother running these locally, just pray to the CI gods that things work and you never have to touch these tests.
629+
630+
AWS tests require a cluster configured with MONGODB_AWS auth enabled. This is easy to set up using drivers-evergreen-tools
631+
by specifying the `aws-auth.json` orchestration file (this is what CI does).
632+
633+
1. Set up your cluster and export the URI of your cluster as MONGODB_URI.
634+
2. Choose your configuration and set the relevant environment variables.
635+
636+
Do you want the AWS SDK to be installed while running auth? If not, set MONGODB_AWS_SDK to false.
637+
638+
Choose your AWS authentication credential type and export the `AWS_CREDENTIAL_TYPE` type with the chosen value:
639+
640+
| AWS Credential Type | Explanation |
641+
| ------------------- | ----------------------------------------------------------------------------------------------- |
642+
| regular | The AWS credentials are present in the URI as username:password |
643+
| env-creds | AWS credentials are loaded into the environment as AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY |
644+
| assume-role | The machine assumes a particular authentication role, associated with the machine |
645+
| ec2 | The driver authenticates against a local endpoint (on an AWS ec2 instance) |
646+
| web-identity | Credentials are sourced from an AssumeRoleWithWebIdentity |
647+
| session-creds | Similar to env-creds, but the credentials are temporary and include a session token |
648+
649+
1. Run the `bash .evergreen/run-mongodb-aws-tests.sh`.
650+
622651
### TODO Special Env Sections
623652

624-
- AWS Authentication
625653
- TLS
626654
- Atlas Data Lake
627655
- LDAP

0 commit comments

Comments
 (0)