@@ -16,16 +16,20 @@ type: integration
1616weight : 2
1717---
1818
19- Google Cloud Spanner requires specific configuration to enable change data capture (CDC) with RDI.
20- RDI operates in two phases with Spanner: snapshot (initial sync) and streaming. During the snapshot
21- phase, RDI uses the JDBC driver to connect directly to Spanner and read the current state of the
22- database. In the streaming phase, RDI uses Spanner's Change Streams to capture changes related to
19+ Google Cloud Spanner requires specific configuration to enable change data capture (CDC) with RDI.
20+ RDI operates in two phases with Spanner: snapshot (initial sync) and streaming. During the snapshot
21+ phase, RDI uses the JDBC driver to connect directly to Spanner and read the current state of the
22+ database. In the streaming phase, RDI uses Spanner's Change Streams to capture changes related to
2323the monitored schemas and tables.
2424
25- You must have the necessary privileges to manage the database schema and create service accounts
25+ {{< note >}}
26+ Spanner is only supported with RDI deployed on Kubernetes/Helm. RDI VM mode does not support Spanner as a source database.
27+ {{< /note >}}
28+
29+ You must have the necessary privileges to manage the database schema and create service accounts
2630with the appropriate permissions, so that RDI can access the Spanner database.
2731
28- ## Prepare for snapshot
32+ ## 1. Prepare for snapshot
2933
3034During the snapshot phase, RDI executes multiple transactions to capture data at an exact point
3135in time that remains consistent across all queries. This is achieved using a Spanner feature called
@@ -37,7 +41,7 @@ which is set to 1 hour by default. Depending on the database tier, the volume of
3741ingested into RDI, and the load on the database, this setting may need to be increased. You can
3842update it using [ this method] ( https://cloud.google.com/spanner/docs/use-pitr#set-period ) .
3943
40- ## Prepare for streaming
44+ ## 2. Prepare for streaming
4145
4246To enable streaming, you must create a change stream in Spanner at the database level. Use the
4347option ` value_capture_type = 'NEW_ROW_AND_OLD_VALUES' ` to capture both the previous and updated
@@ -57,7 +61,7 @@ CREATE CHANGE STREAM change_stream_table1_and_table2
5761Refer to the [ official documentation] ( https://cloud.google.com/spanner/docs/change-streams/manage#googlesql )
5862for more details, including additional configuration options and dialect-specific syntax.
5963
60- ## Create a service account
64+ ## 3. Create a service account
6165
6266To allow RDI to access the Spanner instance, you'll need to create a service account with the
6367appropriate permissions. This service account will then be provided to RDI as a secret for
@@ -108,11 +112,11 @@ gcloud iam service-accounts keys create ~/spanner-reader-account.json \
108112 --project=YOUR_PROJECT_ID
109113```
110114
111- ## Set up secrets for Kubernetes deployment
115+ ## 4. Set up secrets for Kubernetes deployment
112116
113117Before deploying the RDI pipeline, you need to configure the necessary secrets for both the source
114118and target databases. Instructions for setting up the target database secrets are available in the
115- [ RDI deployment guide] ( /integrate/redis-data-integration/data-pipelines/deploy#set-secrets-for-k8shelm-deployment-using-kubectl-command ) .
119+ [ RDI deployment guide] ({{< relref " /integrate/redis-data-integration/data-pipelines/deploy#set-secrets-for-k8shelm-deployment-using-kubectl-command" >}} ).
116120
117121In addition to the target database secrets, you'll also need to create a Spanner-specific secret
118122named ` source-db-credentials ` . This secret should contain the service account key file generated
@@ -127,7 +131,7 @@ kubectl create secret generic source-db-credentials --namespace=rdi \
127131Be sure to adjust the file path (` ~/spanner-reader-account.json ` ) if your service account key is
128132stored elsewhere.
129133
130- ## Configure RDI for Spanner
134+ ## 5. Configure RDI for Spanner
131135
132136When configuring your RDI pipeline for Spanner, use the following example configuration in your
133137` config.yaml ` file:
@@ -184,7 +188,7 @@ processors:
184188Make sure to replace the relevant connection details with your own for both the Spanner and target
185189Redis databases.
186190
187- ## Additional Kubernetes configuration
191+ ## 6. Additional Kubernetes configuration
188192
189193In your ` rdi-values.yaml` file for Kubernetes deployment, make sure to configure the `dataPlane`
190194section like this :
@@ -202,10 +206,6 @@ operator:
202206 - hostname # Replace with your desired ingress hostname
203207` ` `
204208
205- # # Next steps
206-
207- After completing the Spanner preparation steps, you can proceed with :
209+ # # 7. Configuration is complete
208210
209- 1. [Installing RDI on Kubernetes](/integrate/redis-data-integration/installation/install-k8s)
210- 2. [Deploying your RDI pipeline](/integrate/redis-data-integration/data-pipelines/deploy)
211- 3. [Using Redis Insight to manage your RDI pipeline](/develop/tools/insight/rdi-connector)
211+ Once you have followed the steps above, your Google Spanner database is ready for RDI to use.
0 commit comments