Skip to content

Commit 18ea97b

Browse files
authored
Remove unsupported interface and regenerate the docs. (#3245)
1 parent 2c55616 commit 18ea97b

File tree

2 files changed

+4
-40
lines changed

2 files changed

+4
-40
lines changed

v2/pubsub-to-redis/README_Cloud_PubSub_to_Redis.md

Lines changed: 2 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -6,10 +6,6 @@ Pub/Sub subscription and writes the message payload to Redis. The most common us
66
case of this template is to export logs to Redis Enterprise for advanced
77
search-based log analysis in real time.
88

9-
Before writing to Redis, you can apply a JavaScript user-defined function to the
10-
message payload. Any messages that experience processing failures are forwarded
11-
to a Pub/Sub unprocessed topic for further troubleshooting and reprocessing.
12-
139
For added security, enable an SSL connection when setting up your database
1410
endpoint connection.
1511

@@ -37,20 +33,7 @@ on [Metadata Annotations](https://github.com/GoogleCloudPlatform/DataflowTemplat
3733
* **redisSinkType**: The Redis sink. Supported values are `STRING_SINK, HASH_SINK, STREAMS_SINK, and LOGGING_SINK`. For example, `STRING_SINK`. Defaults to: STRING_SINK.
3834
* **connectionTimeout**: The Redis connection timeout in milliseconds. For example, `2000`. Defaults to: 2000.
3935
* **ttl**: The key expiration time in seconds. The `ttl` default for `HASH_SINK` is -1, which means it never expires.
40-
* **javascriptTextTransformGcsPath**: The Cloud Storage URI of the .js file that defines the JavaScript user-defined function (UDF) to use. For example, `gs://my-bucket/my-udfs/my_file.js`.
41-
* **javascriptTextTransformFunctionName**: The name of the JavaScript user-defined function (UDF) to use. For example, if your JavaScript function code is `myTransform(inJson) { /*...do stuff...*/ }`, then the function name is `myTransform`. For sample JavaScript UDFs, see UDF Examples (https://github.com/GoogleCloudPlatform/DataflowTemplates#udf-examples).
42-
* **javascriptTextTransformReloadIntervalMinutes**: Specifies how frequently to reload the UDF, in minutes. If the value is greater than 0, Dataflow periodically checks the UDF file in Cloud Storage, and reloads the UDF if the file is modified. This parameter allows you to update the UDF while the pipeline is running, without needing to restart the job. If the value is `0`, UDF reloading is disabled. The default value is `0`.
43-
44-
45-
## User-Defined functions (UDFs)
46-
47-
The Pub/Sub to Redis Template supports User-Defined functions (UDFs).
48-
UDFs allow you to customize functionality by providing a JavaScript function
49-
without having to maintain or build the entire template code.
5036

51-
Check [Create user-defined functions for Dataflow templates](https://cloud.google.com/dataflow/docs/guides/templates/create-template-udf)
52-
and [Using UDFs](https://github.com/GoogleCloudPlatform/DataflowTemplates#using-udfs)
53-
for more information about how to create and test those functions.
5437

5538

5639
## Getting Started
@@ -152,9 +135,6 @@ export SSL_ENABLED=false
152135
export REDIS_SINK_TYPE=STRING_SINK
153136
export CONNECTION_TIMEOUT=2000
154137
export TTL=-1
155-
export JAVASCRIPT_TEXT_TRANSFORM_GCS_PATH=<javascriptTextTransformGcsPath>
156-
export JAVASCRIPT_TEXT_TRANSFORM_FUNCTION_NAME=<javascriptTextTransformFunctionName>
157-
export JAVASCRIPT_TEXT_TRANSFORM_RELOAD_INTERVAL_MINUTES=0
158138

159139
gcloud dataflow flex-template run "cloud-pubsub-to-redis-job" \
160140
--project "$PROJECT" \
@@ -167,10 +147,7 @@ gcloud dataflow flex-template run "cloud-pubsub-to-redis-job" \
167147
--parameters "sslEnabled=$SSL_ENABLED" \
168148
--parameters "redisSinkType=$REDIS_SINK_TYPE" \
169149
--parameters "connectionTimeout=$CONNECTION_TIMEOUT" \
170-
--parameters "ttl=$TTL" \
171-
--parameters "javascriptTextTransformGcsPath=$JAVASCRIPT_TEXT_TRANSFORM_GCS_PATH" \
172-
--parameters "javascriptTextTransformFunctionName=$JAVASCRIPT_TEXT_TRANSFORM_FUNCTION_NAME" \
173-
--parameters "javascriptTextTransformReloadIntervalMinutes=$JAVASCRIPT_TEXT_TRANSFORM_RELOAD_INTERVAL_MINUTES"
150+
--parameters "ttl=$TTL"
174151
```
175152

176153
For more information about the command, please check:
@@ -199,9 +176,6 @@ export SSL_ENABLED=false
199176
export REDIS_SINK_TYPE=STRING_SINK
200177
export CONNECTION_TIMEOUT=2000
201178
export TTL=-1
202-
export JAVASCRIPT_TEXT_TRANSFORM_GCS_PATH=<javascriptTextTransformGcsPath>
203-
export JAVASCRIPT_TEXT_TRANSFORM_FUNCTION_NAME=<javascriptTextTransformFunctionName>
204-
export JAVASCRIPT_TEXT_TRANSFORM_RELOAD_INTERVAL_MINUTES=0
205179

206180
mvn clean package -PtemplatesRun \
207181
-DskipTests \
@@ -210,7 +184,7 @@ mvn clean package -PtemplatesRun \
210184
-Dregion="$REGION" \
211185
-DjobName="cloud-pubsub-to-redis-job" \
212186
-DtemplateName="Cloud_PubSub_to_Redis" \
213-
-Dparameters="inputSubscription=$INPUT_SUBSCRIPTION,redisHost=$REDIS_HOST,redisPort=$REDIS_PORT,redisPassword=$REDIS_PASSWORD,sslEnabled=$SSL_ENABLED,redisSinkType=$REDIS_SINK_TYPE,connectionTimeout=$CONNECTION_TIMEOUT,ttl=$TTL,javascriptTextTransformGcsPath=$JAVASCRIPT_TEXT_TRANSFORM_GCS_PATH,javascriptTextTransformFunctionName=$JAVASCRIPT_TEXT_TRANSFORM_FUNCTION_NAME,javascriptTextTransformReloadIntervalMinutes=$JAVASCRIPT_TEXT_TRANSFORM_RELOAD_INTERVAL_MINUTES" \
187+
-Dparameters="inputSubscription=$INPUT_SUBSCRIPTION,redisHost=$REDIS_HOST,redisPort=$REDIS_PORT,redisPassword=$REDIS_PASSWORD,sslEnabled=$SSL_ENABLED,redisSinkType=$REDIS_SINK_TYPE,connectionTimeout=$CONNECTION_TIMEOUT,ttl=$TTL" \
214188
-f v2/pubsub-to-redis
215189
```
216190

@@ -263,9 +237,6 @@ resource "google_dataflow_flex_template_job" "cloud_pubsub_to_redis" {
263237
# redisSinkType = "STRING_SINK"
264238
# connectionTimeout = "2000"
265239
# ttl = "-1"
266-
# javascriptTextTransformGcsPath = "<javascriptTextTransformGcsPath>"
267-
# javascriptTextTransformFunctionName = "<javascriptTextTransformFunctionName>"
268-
# javascriptTextTransformReloadIntervalMinutes = "0"
269240
}
270241
}
271242
```

v2/pubsub-to-redis/src/main/java/com/google/cloud/teleport/v2/templates/PubSubToRedis.java

Lines changed: 2 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,6 @@
2727
import com.google.cloud.teleport.v2.common.UncaughtExceptionLogger;
2828
import com.google.cloud.teleport.v2.templates.io.RedisHashIO;
2929
import com.google.cloud.teleport.v2.templates.transforms.MessageTransformation;
30-
import com.google.cloud.teleport.v2.transforms.JavascriptTextTransformer;
3130
import java.util.Map;
3231
import org.apache.beam.sdk.Pipeline;
3332
import org.apache.beam.sdk.PipelineResult;
@@ -94,9 +93,6 @@
9493
"The Pub/Sub to Redis template is a streaming pipeline that reads messages from a Pub/Sub subscription and "
9594
+ "writes the message payload to Redis. The most common use case of this template is to export logs to Redis "
9695
+ "Enterprise for advanced search-based log analysis in real time.",
97-
"Before writing to Redis, you can apply a JavaScript user-defined function to the message payload. Any "
98-
+ "messages that experience processing failures are forwarded to a Pub/Sub unprocessed topic for further "
99-
+ "troubleshooting and reprocessing.",
10096
"For added security, enable an SSL connection when setting up your database endpoint connection."
10197
},
10298
optionsClass = PubSubToRedis.PubSubToRedisOptions.class,
@@ -106,7 +102,6 @@
106102
"https://cloud.google.com/dataflow/docs/guides/templates/provided/pubsub-to-redis",
107103
requirements = {
108104
"The source Pub/Sub subscription must exist prior to running the pipeline.",
109-
"The Pub/Sub unprocessed topic must exist prior to running the pipeline.",
110105
"The Redis database endpoint must be accessible from the Dataflow workers' subnetwork.",
111106
},
112107
preview = true,
@@ -126,11 +121,9 @@ public class PubSubToRedis {
126121
* The {@link PubSubToRedisOptions} class provides the custom execution options passed by the
127122
* executor at the command-line.
128123
*
129-
* <p>Inherits standard configuration options, options from {@link
130-
* JavascriptTextTransformer.JavascriptTextTransformerOptions}.
124+
* <p>Inherits standard configuration options.
131125
*/
132-
public interface PubSubToRedisOptions
133-
extends JavascriptTextTransformer.JavascriptTextTransformerOptions, PipelineOptions {
126+
public interface PubSubToRedisOptions extends PipelineOptions {
134127
@TemplateParameter.PubsubSubscription(
135128
order = 1,
136129
groupName = "Source",

0 commit comments

Comments
 (0)