Skip to content

Commit 4090499

Browse files
feat: define smt examples for pubsub topics and subscriptions (#14627) (#1072)
[upstream:8c5be926dde6d0cdb47717b96fee75df3b9a72f0] Signed-off-by: Modular Magician <[email protected]>
1 parent 9ccf460 commit 4090499

File tree

16 files changed

+532
-0
lines changed

16 files changed

+532
-0
lines changed
Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
# This file has some scaffolding to make sure that names are unique and that
2+
# a region and zone are selected when you try to create your Terraform resources.
3+
4+
locals {
5+
name_suffix = "${random_pet.suffix.id}"
6+
}
7+
8+
resource "random_pet" "suffix" {
9+
length = 2
10+
}
11+
12+
provider "google" {
13+
region = "us-central1"
14+
zone = "us-central1-c"
15+
}
Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
resource "google_pubsub_topic" "example" {
2+
name = "example-topic-${local.name_suffix}"
3+
}
4+
5+
locals {
6+
smts = [
7+
{
8+
function_name = "redactSSN"
9+
code = <<EOF
10+
function redactSSN(message, metadata) {
11+
const data = JSON.parse(message.data);
12+
delete data['ssn'];
13+
message.data = JSON.stringify(data);
14+
return message;
15+
}
16+
EOF
17+
},
18+
{
19+
function_name = "otherFunc",
20+
code = <<EOF
21+
function otherFunc(message, metadata) {
22+
return null;
23+
}
24+
EOF
25+
},
26+
{
27+
function_name = "someSMTWeDisabled",
28+
code = "..."
29+
disabled = true
30+
}
31+
]
32+
}
33+
34+
resource "google_pubsub_subscription" "example" {
35+
name = "example-subscription-${local.name_suffix}"
36+
topic = google_pubsub_topic.example.id
37+
38+
dynamic "message_transforms" {
39+
for_each = local.smts
40+
41+
content {
42+
disabled = lookup(message_transforms.value, "disabled", null)
43+
javascript_udf {
44+
function_name = message_transforms.value.function_name
45+
code = message_transforms.value.code
46+
}
47+
}
48+
}
49+
}
Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
===
2+
3+
These examples use real resources that will be billed to the
4+
Google Cloud Platform project you use - so make sure that you
5+
run "terraform destroy" before quitting!
6+
7+
===
Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
# Pubsub Subscription Multiple Smts - Terraform
2+
3+
## Setup
4+
5+
<walkthrough-author name="[email protected]" analyticsId="UA-125550242-1" tutorialName="pubsub_subscription_multiple_smts" repositoryUrl="https://github.com/terraform-google-modules/docs-examples"></walkthrough-author>
6+
7+
Welcome to Terraform in Google Cloud Shell! We need you to let us know what project you'd like to use with Terraform.
8+
9+
<walkthrough-project-billing-setup></walkthrough-project-billing-setup>
10+
11+
Terraform provisions real GCP resources, so anything you create in this session will be billed against this project.
12+
13+
## Terraforming!
14+
15+
Let's use {{project-id}} with Terraform! Click the Cloud Shell icon below to copy the command
16+
to your shell, and then run it from the shell by pressing Enter/Return. Terraform will pick up
17+
the project name from the environment variable.
18+
19+
```bash
20+
export GOOGLE_CLOUD_PROJECT={{project-id}}
21+
```
22+
23+
After that, let's get Terraform started. Run the following to pull in the providers.
24+
25+
```bash
26+
terraform init
27+
```
28+
29+
With the providers downloaded and a project set, you're ready to use Terraform. Go ahead!
30+
31+
```bash
32+
terraform apply
33+
```
34+
35+
Terraform will show you what it plans to do, and prompt you to accept. Type "yes" to accept the plan.
36+
37+
```bash
38+
yes
39+
```
40+
41+
42+
## Post-Apply
43+
44+
### Editing your config
45+
46+
Now you've provisioned your resources in GCP! If you run a "plan", you should see no changes needed.
47+
48+
```bash
49+
terraform plan
50+
```
51+
52+
So let's make a change! Try editing a number, or appending a value to the name in the editor. Then,
53+
run a 'plan' again.
54+
55+
```bash
56+
terraform plan
57+
```
58+
59+
Afterwards you can run an apply, which implicitly does a plan and shows you the intended changes
60+
at the 'yes' prompt.
61+
62+
```bash
63+
terraform apply
64+
```
65+
66+
```bash
67+
yes
68+
```
69+
70+
## Cleanup
71+
72+
Run the following to remove the resources Terraform provisioned:
73+
74+
```bash
75+
terraform destroy
76+
```
77+
```bash
78+
yes
79+
```
Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
# This file has some scaffolding to make sure that names are unique and that
2+
# a region and zone are selected when you try to create your Terraform resources.
3+
4+
locals {
5+
name_suffix = "${random_pet.suffix.id}"
6+
}
7+
8+
resource "random_pet" "suffix" {
9+
length = 2
10+
}
11+
12+
provider "google" {
13+
region = "us-central1"
14+
zone = "us-central1-c"
15+
}
Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
resource "google_pubsub_topic" "example" {
2+
name = "example-topic-${local.name_suffix}"
3+
}
4+
5+
resource "google_pubsub_subscription" "example" {
6+
name = "example-subscription-${local.name_suffix}"
7+
topic = google_pubsub_topic.example.id
8+
9+
message_transforms {
10+
javascript_udf {
11+
function_name = "isYearEven"
12+
code = <<EOF
13+
function isYearEven(message, metadata) {
14+
const data = JSON.parse(message.data);
15+
return message.year %2 === 0;
16+
}
17+
EOF
18+
}
19+
}
20+
}
Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
===
2+
3+
These examples use real resources that will be billed to the
4+
Google Cloud Platform project you use - so make sure that you
5+
run "terraform destroy" before quitting!
6+
7+
===
Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
# Pubsub Subscription Single Smt - Terraform
2+
3+
## Setup
4+
5+
<walkthrough-author name="[email protected]" analyticsId="UA-125550242-1" tutorialName="pubsub_subscription_single_smt" repositoryUrl="https://github.com/terraform-google-modules/docs-examples"></walkthrough-author>
6+
7+
Welcome to Terraform in Google Cloud Shell! We need you to let us know what project you'd like to use with Terraform.
8+
9+
<walkthrough-project-billing-setup></walkthrough-project-billing-setup>
10+
11+
Terraform provisions real GCP resources, so anything you create in this session will be billed against this project.
12+
13+
## Terraforming!
14+
15+
Let's use {{project-id}} with Terraform! Click the Cloud Shell icon below to copy the command
16+
to your shell, and then run it from the shell by pressing Enter/Return. Terraform will pick up
17+
the project name from the environment variable.
18+
19+
```bash
20+
export GOOGLE_CLOUD_PROJECT={{project-id}}
21+
```
22+
23+
After that, let's get Terraform started. Run the following to pull in the providers.
24+
25+
```bash
26+
terraform init
27+
```
28+
29+
With the providers downloaded and a project set, you're ready to use Terraform. Go ahead!
30+
31+
```bash
32+
terraform apply
33+
```
34+
35+
Terraform will show you what it plans to do, and prompt you to accept. Type "yes" to accept the plan.
36+
37+
```bash
38+
yes
39+
```
40+
41+
42+
## Post-Apply
43+
44+
### Editing your config
45+
46+
Now you've provisioned your resources in GCP! If you run a "plan", you should see no changes needed.
47+
48+
```bash
49+
terraform plan
50+
```
51+
52+
So let's make a change! Try editing a number, or appending a value to the name in the editor. Then,
53+
run a 'plan' again.
54+
55+
```bash
56+
terraform plan
57+
```
58+
59+
Afterwards you can run an apply, which implicitly does a plan and shows you the intended changes
60+
at the 'yes' prompt.
61+
62+
```bash
63+
terraform apply
64+
```
65+
66+
```bash
67+
yes
68+
```
69+
70+
## Cleanup
71+
72+
Run the following to remove the resources Terraform provisioned:
73+
74+
```bash
75+
terraform destroy
76+
```
77+
```bash
78+
yes
79+
```
Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
# This file has some scaffolding to make sure that names are unique and that
2+
# a region and zone are selected when you try to create your Terraform resources.
3+
4+
locals {
5+
name_suffix = "${random_pet.suffix.id}"
6+
}
7+
8+
resource "random_pet" "suffix" {
9+
length = 2
10+
}
11+
12+
provider "google" {
13+
region = "us-central1"
14+
zone = "us-central1-c"
15+
}

pubsub_topic_multiple_smts/main.tf

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
locals {
2+
smts = [
3+
{
4+
function_name = "redactSSN"
5+
code = <<EOF
6+
function redactSSN(message, metadata) {
7+
const data = JSON.parse(message.data);
8+
delete data['ssn'];
9+
message.data = JSON.stringify(data);
10+
return message;
11+
}
12+
EOF
13+
},
14+
{
15+
function_name = "otherFunc",
16+
code = <<EOF
17+
function otherFunc(message, metadata) {
18+
return null;
19+
}
20+
EOF
21+
},
22+
{
23+
function_name = "someSMTWeDisabled",
24+
code = "..."
25+
disabled = true
26+
}
27+
]
28+
}
29+
30+
resource "google_pubsub_topic" "example" {
31+
name = "example-topic-${local.name_suffix}"
32+
33+
dynamic "message_transforms" {
34+
for_each = local.smts
35+
36+
content {
37+
disabled = lookup(message_transforms.value, "disabled", null)
38+
javascript_udf {
39+
function_name = message_transforms.value.function_name
40+
code = message_transforms.value.code
41+
}
42+
}
43+
}
44+
}

0 commit comments

Comments
 (0)