Skip to content

Commit 2dcf62f

Browse files
authored
Retry DLQ Docs refactoring (#1253)
* Retry DLQ refactoring * minor
1 parent f3614aa commit 2dcf62f

File tree

1 file changed

+13
-12
lines changed

1 file changed

+13
-12
lines changed

docs/troubleshoot/minimal.md

Lines changed: 13 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -129,28 +129,29 @@ These job parameters can be taken from the original job.
129129
This will reprocess the records marked as ‘severe' error records from the DLQ.
130130
Before running the Dataflow job, check if the main Dataflow job has non-zero retryable error count. In case there are referential error records - check that the dependent table data is populated completely from the source database.
131131

132+
The following parameters can be taken from the regular forward migration Dataflow job:
133+
region
134+
instanceId
135+
databaseId
136+
sessionFilePath
137+
deadLetterQueueDirectory
138+
132139
Sample command to run the Dataflow job in retryDLQ mode is
133140

134141
```sh
135142
gcloud dataflow flex-template run <jobname> \
136143
--region=<the region where the dataflow job must run> \
137144
--template-file-gcs-location=gs://dataflow-templates/latest/flex/Cloud_Datastream_to_Spanner \
138145
--additional-experiments=use_runner_v2 \
139-
--parameters gcsPubSubSubscription=<pubsub subscription being used in a gcs notification policy>,streamName=<Datastream name>, \
146+
--parameters datastreamSourceType="mysql", \
140147
instanceId=<Spanner Instance Id>,databaseId=<Spanner Database Id>,sessionFilePath=<GCS path to session file>, \
141-
dlqGcsPubSubSubscription=<pubsub subscription being used in a dlq gcs notification policy>, \
142148
deadLetterQueueDirectory=<GCS path to the DLQ>,runMode=retryDLQ
143149
```
144150

145-
The following parameters can be taken from the regular forward migration Dataflow job:
151+
#### Checking if all DLQ entries are applied
152+
153+
To check if all DLQ entries have been applied to spanner, you could count the DLQ files in GCS and wait for it to go to 0.
146154

147155
```sh
148-
region
149-
gcsPubSubSubscription
150-
streamName
151-
instanceId
152-
databaseId
153-
sessionFilePath
154-
deadLetterQueueDirectory
155-
dlqGcsPubSubSubscription
156-
```
156+
gcloud storage ls <GCS path to the DLQ>/severe/**.json | wc -l
157+
```

0 commit comments

Comments
 (0)