Skip to content

Commit 11f558c

Browse files
Add numbering and fix links
1 parent 4546a71 commit 11f558c

File tree

1 file changed

+18
-18
lines changed
  • content/integrate/redis-data-integration/data-pipelines/prepare-dbs

1 file changed

+18
-18
lines changed

content/integrate/redis-data-integration/data-pipelines/prepare-dbs/spanner.md

Lines changed: 18 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -16,16 +16,20 @@ type: integration
1616
weight: 2
1717
---
1818

19-
Google Cloud Spanner requires specific configuration to enable change data capture (CDC) with RDI.
20-
RDI operates in two phases with Spanner: snapshot (initial sync) and streaming. During the snapshot
21-
phase, RDI uses the JDBC driver to connect directly to Spanner and read the current state of the
22-
database. In the streaming phase, RDI uses Spanner's Change Streams to capture changes related to
19+
Google Cloud Spanner requires specific configuration to enable change data capture (CDC) with RDI.
20+
RDI operates in two phases with Spanner: snapshot (initial sync) and streaming. During the snapshot
21+
phase, RDI uses the JDBC driver to connect directly to Spanner and read the current state of the
22+
database. In the streaming phase, RDI uses Spanner's Change Streams to capture changes related to
2323
the monitored schemas and tables.
2424

25-
You must have the necessary privileges to manage the database schema and create service accounts
25+
{{< note >}}
26+
Spanner is only supported with RDI deployed on Kubernetes/Helm. RDI VM mode does not support Spanner as a source database.
27+
{{< /note >}}
28+
29+
You must have the necessary privileges to manage the database schema and create service accounts
2630
with the appropriate permissions, so that RDI can access the Spanner database.
2731

28-
## Prepare for snapshot
32+
## 1. Prepare for snapshot
2933

3034
During the snapshot phase, RDI executes multiple transactions to capture data at an exact point
3135
in time that remains consistent across all queries. This is achieved using a Spanner feature called
@@ -37,7 +41,7 @@ which is set to 1 hour by default. Depending on the database tier, the volume of
3741
ingested into RDI, and the load on the database, this setting may need to be increased. You can
3842
update it using [this method](https://cloud.google.com/spanner/docs/use-pitr#set-period).
3943

40-
## Prepare for streaming
44+
## 2. Prepare for streaming
4145

4246
To enable streaming, you must create a change stream in Spanner at the database level. Use the
4347
option `value_capture_type = 'NEW_ROW_AND_OLD_VALUES'` to capture both the previous and updated
@@ -57,7 +61,7 @@ CREATE CHANGE STREAM change_stream_table1_and_table2
5761
Refer to the [official documentation](https://cloud.google.com/spanner/docs/change-streams/manage#googlesql)
5862
for more details, including additional configuration options and dialect-specific syntax.
5963

60-
## Create a service account
64+
## 3. Create a service account
6165

6266
To allow RDI to access the Spanner instance, you'll need to create a service account with the
6367
appropriate permissions. This service account will then be provided to RDI as a secret for
@@ -108,11 +112,11 @@ gcloud iam service-accounts keys create ~/spanner-reader-account.json \
108112
--project=YOUR_PROJECT_ID
109113
```
110114

111-
## Set up secrets for Kubernetes deployment
115+
## 4. Set up secrets for Kubernetes deployment
112116

113117
Before deploying the RDI pipeline, you need to configure the necessary secrets for both the source
114118
and target databases. Instructions for setting up the target database secrets are available in the
115-
[RDI deployment guide](/integrate/redis-data-integration/data-pipelines/deploy#set-secrets-for-k8shelm-deployment-using-kubectl-command).
119+
[RDI deployment guide]({{< relref "/integrate/redis-data-integration/data-pipelines/deploy#set-secrets-for-k8shelm-deployment-using-kubectl-command" >}}).
116120

117121
In addition to the target database secrets, you'll also need to create a Spanner-specific secret
118122
named `source-db-credentials`. This secret should contain the service account key file generated
@@ -127,7 +131,7 @@ kubectl create secret generic source-db-credentials --namespace=rdi \
127131
Be sure to adjust the file path (`~/spanner-reader-account.json`) if your service account key is
128132
stored elsewhere.
129133

130-
## Configure RDI for Spanner
134+
## 5. Configure RDI for Spanner
131135

132136
When configuring your RDI pipeline for Spanner, use the following example configuration in your
133137
`config.yaml` file:
@@ -184,7 +188,7 @@ processors:
184188
Make sure to replace the relevant connection details with your own for both the Spanner and target
185189
Redis databases.
186190
187-
## Additional Kubernetes configuration
191+
## 6. Additional Kubernetes configuration
188192
189193
In your `rdi-values.yaml` file for Kubernetes deployment, make sure to configure the `dataPlane`
190194
section like this:
@@ -202,10 +206,6 @@ operator:
202206
- hostname # Replace with your desired ingress hostname
203207
```
204208

205-
## Next steps
206-
207-
After completing the Spanner preparation steps, you can proceed with:
209+
## 7. Configuration is complete
208210

209-
1. [Installing RDI on Kubernetes](/integrate/redis-data-integration/installation/install-k8s)
210-
2. [Deploying your RDI pipeline](/integrate/redis-data-integration/data-pipelines/deploy)
211-
3. [Using Redis Insight to manage your RDI pipeline](/develop/tools/insight/rdi-connector)
211+
Once you have followed the steps above, your Google Spanner database is ready for RDI to use.

0 commit comments

Comments
 (0)