Merge pull request #61 from terraform-google-modules/feature/add-cloud-build

Run tests in Cloud Build

Author: morgante

CONTRIBUTING.md

@@ -0,0 +1,137 @@
+## File structure
+The project has the following folders and files:
+
+- /: root folder
+- /examples: examples for using this module
+- /test: Folders with files for testing the module (see Testing section on this file)
+- /main.tf: main file for this module, contains all the resources to create
+- /variables.tf: all the variables for the module
+- /output.tf: the outputs of the module
+- /README.md: this file
+
+## Testing and documentation generation
+
+### Requirements
+- [docker](https://docker.com/)
+
+### Integration testing
+##### Terraform integration tests
+
+The module's integration tests are designed to be run within a Docker
+container containing all the dependencies required for testing. The
+`docker_test_integration` make target wraps this behavior but requires the
+following configuration to execute properly:
+
+- Configure a service account with the roles documented above and export the JSON key to the `SERVICE_ACCOUNT_JSON` environment variable
+
+        export SERVICE_ACCOUNT_JSON=$(< /path/to/credentials.json)
+
+- Create `test/fixtures/shared/terraform.tfvars` and populate with the required Terraform input variables (see `test/fixtures/shared/terraform.tfvars.sample` for more information)
+
+Once those steps have been completed run `make docker_test_integration` from
+the root of the repository to execute the tests within the `project_id`
+provided. Infrastructure from `test/fixtures/*` will be provisioned,
+integration tests from `test/integration/*` will be executed, and the
+infrastructure will be deprovisioned to complete the process.
+
+### Autogeneration of documentation from .tf files
+Run
+```
+make generate_docs
+```
+
+### Lint testing
+
+Lint testing is also performed within a Docker container containing all the
+dependencies required for lint tests. Execute those tests by running `make
+docker_test_lint` from the root of the repository.
+
+Successful output looks similar to the following:
+
+```
+Checking for trailing whitespace
+Checking for missing newline at end of file
+Running shellcheck
+Checking file headers
+Running flake8
+Running terraform fmt
+terraform fmt -diff -check=true -write=false .
+terraform fmt -diff -check=true -write=false ./codelabs/simple
+terraform fmt -diff -check=true -write=false ./examples/delete_default_gateway_routes
+terraform fmt -diff -check=true -write=false ./examples/multi_vpc
+terraform fmt -diff -check=true -write=false ./examples/secondary_ranges
+terraform fmt -diff -check=true -write=false ./examples/simple_project
+terraform fmt -diff -check=true -write=false ./examples/simple_project_with_regional_network
+terraform fmt -diff -check=true -write=false ./examples/submodule_firewall
+terraform fmt -diff -check=true -write=false ./examples/submodule_svpc_access
+terraform fmt -diff -check=true -write=false ./modules/fabric-net-firewall
+terraform fmt -diff -check=true -write=false ./modules/fabric-net-svpc-access
+terraform fmt -diff -check=true -write=false ./test/fixtures/all_examples
+terraform fmt -diff -check=true -write=false ./test/fixtures/delete_default_gateway_routes
+terraform fmt -diff -check=true -write=false ./test/fixtures/multi_vpc
+terraform fmt -diff -check=true -write=false ./test/fixtures/secondary_ranges
+terraform fmt -diff -check=true -write=false ./test/fixtures/shared
+terraform fmt -diff -check=true -write=false ./test/fixtures/simple_project
+terraform fmt -diff -check=true -write=false ./test/fixtures/simple_project_with_regional_network
+terraform fmt -diff -check=true -write=false ./test/fixtures/simulated_ci_environment
+terraform fmt -diff -check=true -write=false ./test/fixtures/submodule_firewall
+Running terraform validate
+terraform_validate .
+Success! The configuration is valid.
+
+terraform_validate ./codelabs/simple
+Success! The configuration is valid.
+
+terraform_validate ./examples/delete_default_gateway_routes
+Success! The configuration is valid.
+
+terraform_validate ./examples/multi_vpc
+Success! The configuration is valid.
+
+terraform_validate ./examples/secondary_ranges
+Success! The configuration is valid.
+
+terraform_validate ./examples/simple_project
+Success! The configuration is valid.
+
+terraform_validate ./examples/simple_project_with_regional_network
+Success! The configuration is valid.
+
+terraform_validate ./examples/submodule_firewall
+Success! The configuration is valid.
+
+terraform_validate ./examples/submodule_svpc_access
+Success! The configuration is valid.
+
+terraform_validate ./modules/fabric-net-firewall
+Success! The configuration is valid.
+
+terraform_validate ./modules/fabric-net-svpc-access
+Success! The configuration is valid.
+
+terraform_validate ./test/fixtures/all_examples
+Success! The configuration is valid.
+
+terraform_validate ./test/fixtures/delete_default_gateway_routes
+Success! The configuration is valid.
+
+terraform_validate ./test/fixtures/multi_vpc
+Success! The configuration is valid.
+
+terraform_validate ./test/fixtures/secondary_ranges
+Success! The configuration is valid.
+
+terraform_validate ./test/fixtures/simple_project
+Success! The configuration is valid.
+
+terraform_validate ./test/fixtures/simple_project_with_regional_network
+Success! The configuration is valid.
+
+terraform_validate ./test/fixtures/simulated_ci_environment
+Success! The configuration is valid.
+
+terraform_validate ./test/fixtures/submodule_firewall
+Success! The configuration is valid.
+```
+
+[terraform-provider-google]: https://github.com/terraform-providers/terraform-provider-google

Makefile

@@ -15,7 +15,7 @@
 # Make will use bash instead of sh
 SHELL := /usr/bin/env bash
 
-DOCKER_TAG_VERSION_DEVELOPER_TOOLS := 0.0.1
+DOCKER_TAG_VERSION_DEVELOPER_TOOLS := 0.1.0
 DOCKER_IMAGE_DEVELOPER_TOOLS := cft/developer-tools
 REGISTRY_URL := gcr.io/cloud-foundation-cicd
 
@@ -28,6 +28,30 @@ docker_run:
 		$(REGISTRY_URL)/${DOCKER_IMAGE_DEVELOPER_TOOLS}:${DOCKER_TAG_VERSION_DEVELOPER_TOOLS} \
 		/bin/bash
 
+# Execute prepare tests within the docker container
+.PHONY: docker_test_prepare
+docker_test_prepare:
+	docker run --rm -it \
+		-e SERVICE_ACCOUNT_JSON \
+		-e TF_VAR_org_id \
+		-e TF_VAR_folder_id \
+		-e TF_VAR_billing_account \
+		-v $(CURDIR):/workspace \
+		$(REGISTRY_URL)/${DOCKER_IMAGE_DEVELOPER_TOOLS}:${DOCKER_TAG_VERSION_DEVELOPER_TOOLS} \
+		/usr/local/bin/execute_with_credentials.sh prepare_environment
+
+# Clean up test environment within the docker container
+.PHONY: docker_test_cleanup
+docker_test_prepare:
+	docker run --rm -it \
+		-e SERVICE_ACCOUNT_JSON \
+		-e TF_VAR_org_id \
+		-e TF_VAR_folder_id \
+		-e TF_VAR_billing_account \
+		-v $(CURDIR):/workspace \
+		$(REGISTRY_URL)/${DOCKER_IMAGE_DEVELOPER_TOOLS}:${DOCKER_TAG_VERSION_DEVELOPER_TOOLS} \
+		/usr/local/bin/execute_with_credentials.sh cleanup_environment
+
 # Execute integration tests within the docker container
 .PHONY: docker_test_integration
 docker_test_integration:

README.md

@@ -158,141 +158,3 @@ If you are going to manage a Shared VPC, you must have either:
 In order to operate with the Service Account you must activate the following API on the project where the Service Account was created:
 
 - Compute Engine API - compute.googleapis.com
-
-## File structure
-The project has the following folders and files:
-
-- /: root folder
-- /examples: examples for using this module
-- /test: Folders with files for testing the module (see Testing section on this file)
-- /main.tf: main file for this module, contains all the resources to create
-- /variables.tf: all the variables for the module
-- /output.tf: the outputs of the module
-- /README.md: this file
-
-## Testing and documentation generation
-
-### Requirements
-- [docker](https://docker.com/)
-
-### Integration testing
-##### Terraform integration tests
-
-The module's integration tests are designed to be run within a Docker
-container containing all the dependencies required for testing. The
-`docker_test_integration` make target wraps this behavior but requires the
-following configuration to execute properly:
-
-- Configure a service account with the roles documented above and export the JSON key to the `SERVICE_ACCOUNT_JSON` environment variable
-
-        export SERVICE_ACCOUNT_JSON=$(< /path/to/credentials.json)
-
-- Create `test/fixtures/shared/terraform.tfvars` and populate with the required Terraform input variables (see `test/fixtures/shared/terraform.tfvars.sample` for more information)
-
-Once those steps have been completed run `make docker_test_integration` from
-the root of the repository to execute the tests within the `project_id`
-provided. Infrastructure from `test/fixtures/*` will be provisioned,
-integration tests from `test/integration/*` will be executed, and the
-infrastructure will be deprovisioned to complete the process.
-
-### Autogeneration of documentation from .tf files
-Run
-```
-make generate_docs
-```
-
-### Lint testing
-
-Lint testing is also performed within a Docker container containing all the
-dependencies required for lint tests. Execute those tests by running `make
-docker_test_lint` from the root of the repository.
-
-Successful output looks similar to the following:
-
-```
-Checking for trailing whitespace
-Checking for missing newline at end of file
-Running shellcheck
-Checking file headers
-Running flake8
-Running terraform fmt
-terraform fmt -diff -check=true -write=false .
-terraform fmt -diff -check=true -write=false ./codelabs/simple
-terraform fmt -diff -check=true -write=false ./examples/delete_default_gateway_routes
-terraform fmt -diff -check=true -write=false ./examples/multi_vpc
-terraform fmt -diff -check=true -write=false ./examples/secondary_ranges
-terraform fmt -diff -check=true -write=false ./examples/simple_project
-terraform fmt -diff -check=true -write=false ./examples/simple_project_with_regional_network
-terraform fmt -diff -check=true -write=false ./examples/submodule_firewall
-terraform fmt -diff -check=true -write=false ./examples/submodule_svpc_access
-terraform fmt -diff -check=true -write=false ./modules/fabric-net-firewall
-terraform fmt -diff -check=true -write=false ./modules/fabric-net-svpc-access
-terraform fmt -diff -check=true -write=false ./test/fixtures/all_examples
-terraform fmt -diff -check=true -write=false ./test/fixtures/delete_default_gateway_routes
-terraform fmt -diff -check=true -write=false ./test/fixtures/multi_vpc
-terraform fmt -diff -check=true -write=false ./test/fixtures/secondary_ranges
-terraform fmt -diff -check=true -write=false ./test/fixtures/shared
-terraform fmt -diff -check=true -write=false ./test/fixtures/simple_project
-terraform fmt -diff -check=true -write=false ./test/fixtures/simple_project_with_regional_network
-terraform fmt -diff -check=true -write=false ./test/fixtures/simulated_ci_environment
-terraform fmt -diff -check=true -write=false ./test/fixtures/submodule_firewall
-Running terraform validate
-terraform_validate .
-Success! The configuration is valid.
-
-terraform_validate ./codelabs/simple
-Success! The configuration is valid.
-
-terraform_validate ./examples/delete_default_gateway_routes
-Success! The configuration is valid.
-
-terraform_validate ./examples/multi_vpc
-Success! The configuration is valid.
-
-terraform_validate ./examples/secondary_ranges
-Success! The configuration is valid.
-
-terraform_validate ./examples/simple_project
-Success! The configuration is valid.
-
-terraform_validate ./examples/simple_project_with_regional_network
-Success! The configuration is valid.
-
-terraform_validate ./examples/submodule_firewall
-Success! The configuration is valid.
-
-terraform_validate ./examples/submodule_svpc_access
-Success! The configuration is valid.
-
-terraform_validate ./modules/fabric-net-firewall
-Success! The configuration is valid.
-
-terraform_validate ./modules/fabric-net-svpc-access
-Success! The configuration is valid.
-
-terraform_validate ./test/fixtures/all_examples
-Success! The configuration is valid.
-
-terraform_validate ./test/fixtures/delete_default_gateway_routes
-Success! The configuration is valid.
-
-terraform_validate ./test/fixtures/multi_vpc
-Success! The configuration is valid.
-
-terraform_validate ./test/fixtures/secondary_ranges
-Success! The configuration is valid.
-
-terraform_validate ./test/fixtures/simple_project
-Success! The configuration is valid.
-
-terraform_validate ./test/fixtures/simple_project_with_regional_network
-Success! The configuration is valid.
-
-terraform_validate ./test/fixtures/simulated_ci_environment
-Success! The configuration is valid.
-
-terraform_validate ./test/fixtures/submodule_firewall
-Success! The configuration is valid.
-```
-
-[terraform-provider-google]: https://github.com/terraform-providers/terraform-provider-google

build/int.cloudbuild.yaml

@@ -0,0 +1,41 @@
+# Copyright 2018 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+timeout: 3600s
+steps:
+- id: prepare
+  name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
+  args: ['/bin/bash', '-c', 'source /usr/local/bin/task_helper_functions.sh && prepare_environment']
+  env:
+  - 'TF_VAR_org_id=$_ORG_ID'
+  - 'TF_VAR_folder_id=$_FOLDER_ID'
+  - 'TF_VAR_billing_account=$_BILLING_ACCOUNT'
+- id: create
+  name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
+  args: ['/bin/bash', '-c', 'source /usr/local/bin/task_helper_functions.sh && kitchen_do create']
+- id: converge
+  name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
+  args: ['/bin/bash', '-c', 'source /usr/local/bin/task_helper_functions.sh && kitchen_do converge']
+- id: verify
+  name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
+  args: ['/bin/bash', '-c', 'source /usr/local/bin/task_helper_functions.sh && kitchen_do verify']
+- id: destroy
+  name: 'gcr.io/cloud-foundation-cicd/$_DOCKER_IMAGE_DEVELOPER_TOOLS:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
+  args: ['/bin/bash', '-c', 'source /usr/local/bin/task_helper_functions.sh && kitchen_do destroy']
+tags:
+- 'ci'
+- 'integration'
+substitutions:
+  _DOCKER_IMAGE_DEVELOPER_TOOLS: 'cft/developer-tools'
+  _DOCKER_TAG_VERSION_DEVELOPER_TOOLS: '0.1.0'

build/lint.cloudbuild.yaml

@@ -0,0 +1,24 @@
+# Copyright 2018 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+steps:
+- name: 'gcr.io/cloud-foundation-cicd/cft/developer-tools:$_DOCKER_TAG_VERSION_DEVELOPER_TOOLS'
+  id: 'lint'
+  args: ['/usr/local/bin/test_lint.sh']
+tags:
+- 'ci'
+- 'lint'
+substitutions:
+  _DOCKER_IMAGE_DEVELOPER_TOOLS: 'cft/developer-tools'
+  _DOCKER_TAG_VERSION_DEVELOPER_TOOLS: '0.1.0'

test/.gitignore

@@ -0,0 +1 @@
+source.sh