Skip to content

Commit 8110d3a

Browse files
grantgrant
authored
ci: add npm run conformance (#374)
Adds command to run conformance tests: - `npm run conformance` Very similar to the Go setup, with a few minor modifications. Conformance `package-lock.json` is ignored, because it's just pointing to `file:../../`. <details><summary>FULL LOGS</summary> <p> ``` npm run conformance > @google-cloud/[email protected] conformance > ./run_conformance_tests.sh Defaulting to latest client. Use './run_conformance_tests vX.X.X' to specify a specific release version. up to date, audited 3 packages in 866ms found 0 vulnerabilities ========== INSTALLING CLIENT@latest ========== Note: only works with Go 1.16+ by default, see run_conformance_tests.sh for more information. Done installing client@latest ========== HTTP CONFORMANCE TESTS ========== 2021/10/26 16:45:24 Validating for http... 2021/10/26 16:45:24 Framework server started. 2021/10/26 16:45:29 HTTP validation started... 2021/10/26 16:45:29 HTTP validation passed! 2021/10/26 16:45:29 Framework server shut down. Wrote logs to /var/folders/27/9q8h1w651z91hycvdf4qf0h0009rgz/T/serverlog_stdout.txt and /var/folders/27/9q8h1w651z91hycvdf4qf0h0009rgz/T/serverlog_stderr.txt. 2021/10/26 16:45:29 All validation passed! ========== BACKGROUND EVENT CONFORMANCE TESTS ========== 2021/10/26 16:45:29 Validating for legacyevent... 2021/10/26 16:45:29 Framework server started. 2021/10/26 16:45:34 Legacy event validation with legacy event requests... 2021/10/26 16:45:34 Events tried: - firebase-auth (PASSED) - firebase-db1 (PASSED) - firebase-db2 (PASSED) - firebase-db3 (PASSED) - firebase-db4 (PASSED) - firebase-db5 (PASSED) - firebase-db6 (PASSED) - firebase-db7 (PASSED) - firebase-db8 (PASSED) - firebase-dbdelete1 (PASSED) - firebase-dbdelete2 (PASSED) - firestore_complex (PASSED) - firestore_simple (PASSED) - legacy_pubsub (PASSED) - pubsub_binary (PASSED) - pubsub_text (PASSED) - storage (PASSED) 2021/10/26 16:45:34 Legacy event validation with CloudEvent requests... 2021/10/26 16:45:34 Events tried: - firebase-auth (PASSED) - firebase-db1 (PASSED) - firebase-db2 (PASSED) - firebase-db3 (PASSED) - firebase-db4 (PASSED) - firebase-db5 (PASSED) - firebase-db6 (PASSED) - firebase-db7 (PASSED) - firebase-db8 (PASSED) - firebase-dbdelete1 (PASSED) - firebase-dbdelete2 (PASSED) - firestore_complex (PASSED) - firestore_simple (PASSED) - legacy_pubsub (PASSED) - pubsub_binary (PASSED) - pubsub_text (PASSED) - storage (PASSED) 2021/10/26 16:45:34 Legacy event validation passed! 2021/10/26 16:45:34 Framework server shut down. Wrote logs to /var/folders/27/9q8h1w651z91hycvdf4qf0h0009rgz/T/serverlog_stdout.txt and /var/folders/27/9q8h1w651z91hycvdf4qf0h0009rgz/T/serverlog_stderr.txt. 2021/10/26 16:45:34 All validation passed! ========== CLOUDEVENT CONFORMANCE TESTS ========== 2021/10/26 16:45:34 Validating for cloudevent... 2021/10/26 16:45:34 Framework server started. 2021/10/26 16:45:39 CloudEvent validation with CloudEvent requests... 2021/10/26 16:45:39 Events tried: - firebase-auth (PASSED) - firebase-db1 (PASSED) - firebase-db2 (PASSED) - firebase-db3 (PASSED) - firebase-db4 (PASSED) - firebase-db5 (PASSED) - firebase-db6 (PASSED) - firebase-db7 (PASSED) - firebase-db8 (PASSED) - firebase-dbdelete1 (PASSED) - firebase-dbdelete2 (PASSED) - firestore_complex (PASSED) - firestore_simple (PASSED) - legacy_pubsub (PASSED) - pubsub_binary (PASSED) - pubsub_text (PASSED) - storage (PASSED) 2021/10/26 16:45:39 CloudEvent validation with legacy event requests... 2021/10/26 16:45:39 Events tried: - firebase-auth (PASSED) - firebase-db1 (PASSED) - firebase-db2 (PASSED) - firebase-db3 (PASSED) - firebase-db4 (PASSED) - firebase-db5 (PASSED) - firebase-db6 (PASSED) - firebase-db7 (PASSED) - firebase-db8 (PASSED) - firebase-dbdelete1 (PASSED) - firebase-dbdelete2 (PASSED) - firestore_complex (PASSED) - firestore_simple (PASSED) - legacy_pubsub (PASSED) - pubsub_binary (PASSED) - pubsub_text (PASSED) - storage (PASSED) 2021/10/26 16:45:39 CloudEvent validation passed! 2021/10/26 16:45:39 Framework server shut down. Wrote logs to /var/folders/27/9q8h1w651z91hycvdf4qf0h0009rgz/T/serverlog_stdout.txt and /var/folders/27/9q8h1w651z91hycvdf4qf0h0009rgz/T/serverlog_stderr.txt. 2021/10/26 16:45:39 All validation passed! ========== DECLARATIVE HTTP CONFORMANCE TESTS ========== 2021/10/26 16:45:39 Validating for http... 2021/10/26 16:45:39 Framework server started. 2021/10/26 16:45:44 HTTP validation started... 2021/10/26 16:45:44 HTTP validation passed! 2021/10/26 16:45:44 Framework server shut down. Wrote logs to /var/folders/27/9q8h1w651z91hycvdf4qf0h0009rgz/T/serverlog_stdout.txt and /var/folders/27/9q8h1w651z91hycvdf4qf0h0009rgz/T/serverlog_stderr.txt. 2021/10/26 16:45:44 All validation passed! ========== DECLARATIVE CLOUDEVENT CONFORMANCE TESTS ========== 2021/10/26 16:45:44 Validating for cloudevent... 2021/10/26 16:45:44 Framework server started. 2021/10/26 16:45:49 CloudEvent validation with CloudEvent requests... 2021/10/26 16:45:49 Events tried: - firebase-auth (PASSED) - firebase-db1 (PASSED) - firebase-db2 (PASSED) - firebase-db3 (PASSED) - firebase-db4 (PASSED) - firebase-db5 (PASSED) - firebase-db6 (PASSED) - firebase-db7 (PASSED) - firebase-db8 (PASSED) - firebase-dbdelete1 (PASSED) - firebase-dbdelete2 (PASSED) - firestore_complex (PASSED) - firestore_simple (PASSED) - legacy_pubsub (PASSED) - pubsub_binary (PASSED) - pubsub_text (PASSED) - storage (PASSED) 2021/10/26 16:45:49 CloudEvent validation with legacy event requests... 2021/10/26 16:45:49 Events tried: - firebase-auth (PASSED) - firebase-db1 (PASSED) - firebase-db2 (PASSED) - firebase-db3 (PASSED) - firebase-db4 (PASSED) - firebase-db5 (PASSED) - firebase-db6 (PASSED) - firebase-db7 (PASSED) - firebase-db8 (PASSED) - firebase-dbdelete1 (PASSED) - firebase-dbdelete2 (PASSED) - firestore_complex (PASSED) - firestore_simple (PASSED) - legacy_pubsub (PASSED) - pubsub_binary (PASSED) - pubsub_text (PASSED) - storage (PASSED) 2021/10/26 16:45:49 CloudEvent validation passed! 2021/10/26 16:45:49 Framework server shut down. Wrote logs to /var/folders/27/9q8h1w651z91hycvdf4qf0h0009rgz/T/serverlog_stdout.txt and /var/folders/27/9q8h1w651z91hycvdf4qf0h0009rgz/T/serverlog_stderr.txt. 2021/10/26 16:45:49 All validation passed! ``` </p> </details>
1 parent 78ba5c1 commit 8110d3a

File tree

4 files changed

+69
-1
lines changed

4 files changed

+69
-1
lines changed

‎.gitignore‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,4 +13,5 @@ yarn-error.log
1313
function_output.json
1414
serverlog_stderr.txt
1515
serverlog_stdout.txt
16-
temp
16+
temp
17+
test/conformance/package-lock.json

‎CONTRIBUTING.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,14 @@ To run an individual test, you can run a command such as the following:
3434
npm run test -- -g 'loading function'
3535
```
3636

37+
### Conformance Tests
38+
39+
To run the conformance tests, first install Go 1.16+, then run the tests:
40+
41+
```
42+
npm run conformance
43+
```
44+
3745
### Manual Testing
3846

3947
When developing a feature locally, you can install a local version of the Functions Framework

‎package.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@
1818
},
1919
"scripts": {
2020
"test": "mocha build/test --recursive",
21+
"conformance": "./run_conformance_tests.sh",
2122
"check": "gts check",
2223
"clean": "gts clean",
2324
"compile": "tsc -p .",

‎run_conformance_tests.sh‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
#!/bin/bash
2+
# Runs the conformance tests locally from https://github.com/GoogleCloudPlatform/functions-framework-conformance
3+
#
4+
# Servers may fail to shutdown between tests on error, leaving port 8080 bound.
5+
# You can see what's running on port 8080 by running `lsof -i :8080`. You can
6+
# run `kill -9 <PID>` to terminate a process.
7+
#
8+
# USAGE:
9+
# ./run_conformance_tests.sh [client_version]
10+
#
11+
# client_version (optional):
12+
# The version of the conformance tests client to use, formatted as "vX.X.X".
13+
# Defaults to the latest version of the repo, which may be ahead of the
14+
# latest release.
15+
16+
CLIENT_VERSION=$1
17+
if [ $CLIENT_VERSION ]; then
18+
CLIENT_VERSION="@$CLIENT_VERSION"
19+
else
20+
echo "Defaulting to latest client."
21+
echo "Use './run_conformance_tests vX.X.X' to specify a specific release version."
22+
CLIENT_VERSION="@latest"
23+
fi
24+
25+
function print_header() {
26+
echo
27+
echo "========== $1 =========="
28+
}
29+
30+
# Fail if any command fails
31+
set -e
32+
33+
# Navigate to the conformance tests and install the local FF.
34+
cd test/conformance;
35+
npm i;
36+
37+
print_header "INSTALLING CLIENT$CLIENT_VERSION"
38+
echo "Note: only works with Go 1.16+ by default, see run_conformance_tests.sh for more information."
39+
# Go install @version only works on go 1.16+, if using a lower Go version
40+
# replace command with:
41+
# go get github.com/GoogleCloudPlatform/functions-framework-conformance/client$CLIENT_VERSION && go install github.com/GoogleCloudPlatform/functions-framework-conformance/client
42+
go install github.com/GoogleCloudPlatform/functions-framework-conformance/client$CLIENT_VERSION
43+
echo "Done installing client$CLIENT_VERSION"
44+
45+
print_header "HTTP CONFORMANCE TESTS"
46+
client -buildpacks=false -type=http -cmd='npm start -- --target=writeHttp --signature-type=http' -start-delay 5 -validate-mapping=true
47+
48+
print_header "BACKGROUND EVENT CONFORMANCE TESTS"
49+
client -buildpacks=false -type=legacyevent -cmd='npm start -- --target=writeLegacyEvent --signature-type=event' -start-delay 5 -validate-mapping=true
50+
51+
print_header "CLOUDEVENT CONFORMANCE TESTS"
52+
client -buildpacks=false -type=cloudevent -cmd='npm start -- --target=writeCloudEvent --signature-type=cloudevent' -start-delay 5 -validate-mapping=true
53+
54+
print_header "DECLARATIVE HTTP CONFORMANCE TESTS"
55+
client -buildpacks=false -cmd='npm start -- --target=writeHttpDeclarative' -start-delay 5 -validate-mapping=true
56+
57+
print_header "DECLARATIVE CLOUDEVENT CONFORMANCE TESTS"
58+
client -buildpacks=false -type=cloudevent -cmd='npm start -- --target=writeCloudEventDeclarative' -start-delay 5 -validate-mapping=true

0 commit comments

Comments
 (0)