Merge pull request #2089 from redis/DOC-5648

RDI Cloud: Update user journey

Author: cmilesb

content/embeds/rc-rdi-secrets-encryption-permissions.md

@@ -0,0 +1,5 @@
+* **Encryption key**: Select the [encryption key](#create-encryption-key) you created earlier.
+
+* **Resource permissions**: Add the following permissions to your secret to allow the Redis data pipeline to access your secret. Replace `<AWS ACCOUNT ID>` with the AWS account ID for the Redis Cloud cluster that you saved earlier.
+
+{{< embed-md "rc-rdi-secrets-permissions.md" >}}

content/embeds/rc-rdi-secrets-permissions.md

@@ -0,0 +1,17 @@
+```json
+{
+    "Version" : "2012-10-17",
+    "Statement" : [ {
+        "Sid" : "RedisDataIntegrationRoleAccess",
+        "Effect" : "Allow",
+        "Principal" : "*",
+        "Action" : [ "secretsmanager:GetSecretValue", "secretsmanager:DescribeSecret" ],
+        "Resource" : "*",
+        "Condition" : {
+            "StringLike" : {
+                "aws:PrincipalArn" : "arn:aws:iam::<AWS ACCOUNT ID>:role/redis-data-pipeline-secrets-role"
+            }
+        }
+    } ]
+}
+```

content/operate/rc/databases/rdi/define.md

@@ -30,7 +30,12 @@ After you have [prepared your source database]({{<relref "/operate/rc/databases/
     - **Database**: Your database's name, or the root database *(PostgreSQL, Oracle only)*, or a comma-separated list of one or more databases you want to connect to *(SQL Server only)*
     - **Database Server ID**: Unique ID for the replication client. Enter a number that is not used by any existing replication clients *(mySQL and mariaDB only)*
     - **PDB**: Name of the Oracle pluggable database *(Oracle only)*
-1. Enter the ARN of your [database credentials secret]({{< relref "/operate/rc/databases/rdi/setup#share-source-database-credentials" >}}) in the **Source database secrets ARN** field.
+1. Enter the ARN of your [database credentials secret]({{< relref "/operate/rc/databases/rdi/setup#create-database-credentials-secrets" >}}) in the **Source database secrets ARN** field.
+1. If your database requires TLS, select **Use TLS**. Enter the ARN of your [CA certificate secret]({{< relref "/operate/rc/databases/rdi/setup#create-database-credentials-secrets" >}}) in the **CA Cert Secret ARN** field.
+    {{<image filename="images/rc/rdi/rdi-define-tls.png" alt="The Source database connectivity section, with Use TLS selected and the CA Cert Secret ARN field." >}}
+1. If your database requires mTLS, select **Use mTLS**. Enter the ARN of your [Client certificate secret]({{< relref "/operate/rc/databases/rdi/setup#create-database-credentials-secrets" >}}) in the **Client Certificate Secret ARN** field and the ARN of your [Client key secret]({{< relref "/operate/rc/databases/rdi/setup#create-database-credentials-secrets" >}}) in the **Client Key Secret ARN** field.
+    {{<image filename="images/rc/rdi/rdi-define-mtls.png" alt="The Source database connectivity section, with Use TLS selected and the Client Certificate Secret ARN and Client Key Secret ARN fields." >}}
+1. If your database requires mTLS with a client key passphrase, enter the ARN of your [Client key passphrase secret]({{< relref "/operate/rc/databases/rdi/setup#create-database-credentials-secrets" >}}) in the **Please add a secret ARN for the password to use with the secret store** field.
 1. Select **Start pipeline setup**.
     {{<image filename="images/rc/rdi/rdi-start-pipeline-setup.png" alt="The start pipeline setup button." width=200px >}}
 1. Redis Cloud will attempt to connect to PrivateLink. If your PrivateLink does not allow automatic acceptance of incoming connections, accept the incoming connection on AWS PrivateLink to proceed. See [Accept or Reject PrivateLink connection requests](https://docs.aws.amazon.com/vpc/latest/privatelink/configure-endpoint-service.html#accept-reject-connection-requests).
@@ -53,21 +58,29 @@ After your pipeline is provisioned, you will be able to define your pipeline. Yo
 
 ### Configure a new pipeline
 
-1. In the [Redis Cloud console](https://cloud.redis.io/), go to your target database and select the **Data Pipeline** tab. If your pipeline is already provisioned, select **Complete setup** to go to the **Select data** section.
+1. In the [Redis Cloud console](https://cloud.redis.io/), go to your target database and select the **Data Pipeline** tab. If your pipeline is already provisioned, select **Complete setup** to go to the **Data modeling** section.
     {{<image filename="images/rc/rdi/rdi-complete-setup.png" alt="The complete setup button." width=200px >}}
-1. Select the Schema and Tables you want to migrate to the target database from the **Source data selection** list. 
-    {{<image filename="images/rc/rdi/rdi-select-source-data.png" alt="The select source data section. " width=75% >}}
+1. Select the Schema and Tables you want to migrate to the target database from the list. 
+    {{<image filename="images/rc/rdi/rdi-select-source-data.png" alt="The data modeling section. " width=75% >}}
+
+    Select **Manage Columns** to choose which columns you want to import.
+
+    {{<image filename="images/rc/rdi/rdi-manage-columns.png" alt="The manage columns button." width=150px >}}
 
     You can select any number of columns from a table.
 
-    {{<image filename="images/rc/rdi/rdi-select-columns.png" alt="The select source data section. A table is expanded with a few columns selected." width=75% >}}
+    {{<image filename="images/rc/rdi/rdi-select-columns.png" alt="The manage columns screen, with a few columns selected from one table" width=75% >}}
 
-    If any tables are missing a unique constraint, the **Missing unique constraint** list will appear. Select the columns that define a unique constraint for those tables from the list.
+    If any tables are missing a unique constraint, a warning will appear in the **Data modeling** section. Select **Manage columns** to select the columns that define a unique constraint for those tables.
 
     {{<image filename="images/rc/rdi/rdi-missing-unique-constraint.png" alt="The missing unique constraint list." width=75% >}}
 
     {{<image filename="images/rc/rdi/rdi-select-constraints.png" alt="The missing unique constraint list with columns selected." width=75% >}}
 
+    Select **Save** to save your column changes and go back to schema selection.
+
+    {{<image filename="images/rc/button-save.png" alt="The save button." width=100px >}}
+
     Select **Add schema** to add more database schemas.
 
     {{<image filename="images/rc/rdi/rdi-add-schema.png" alt="The add schema button." width=150px >}}
@@ -80,19 +93,21 @@ After your pipeline is provisioned, you will be able to define your pipeline. Yo
 
      {{<image filename="images/rc/rdi/rdi-continue-button.png" alt="The continue button." width=150px >}}
 
-1. In the **Pipeline definition** section, select the Redis data type to write keys to the target. You can choose **Hash** or **JSON** if the target database supports JSON. 
+1. Select the Redis data type to write keys to the target. You can choose **Hash** or **JSON** if the target database supports JSON. 
     {{<image filename="images/rc/rdi/rdi-configure-new-pipeline.png" alt="The pipeline definition screen." width=75% >}}
 
     You can also supply one or more [transformation job files]({{< relref "/integrate/redis-data-integration/data-pipelines/transform-examples" >}}) that specify how you want to transform the captured data before writing it to the target. Select **Upload jobs** to upload your job files.
 
     {{<image filename="images/rc/rdi/rdi-transformation-jobs.png" alt="The transformation jobs section. Select Upload jobs to upload transformation jobs." >}}
 
+    When you upload job files, Redis Cloud will validate the job files to check for errors. 
+
     Select **Continue**.
     {{<image filename="images/rc/rdi/rdi-continue-button.png" alt="The continue button." width=150px >}}
 
-1. Review the tables you selected in the **Summary**. If everything looks correct, select **Start ingest** to start ingesting data from your source database.
+1. Review the tables you selected in the **Review and deploy** section. If everything looks correct, select **Confirm & Deploy** to start ingesting data from your source database.
 
-    {{<image filename="images/rc/rdi/rdi-start-ingest.png" alt="The start ingest button." width=175px >}}
+    {{<image filename="images/rc/rdi/rdi-confirm-deploy.png" alt="The Confirm & Deploy button." width=175px >}}
 
 At this point, the data pipeline will ingest data from the source database to your target Redis database. This process will take time, especially if you have a lot of records in your source database. 
 

content/operate/rc/databases/rdi/setup.md

@@ -198,7 +198,7 @@ You need to share your source database credentials and certificates in an Amazon
 
 To do this, you need to:
 1. [Create an encryption key](#create-encryption-key) using AWS Key Management Service with the right permissions.
-1. [Create a secret](#create-database-credentials-secret) containing the source database credentials encrypted using that key.
+1. [Create secrets](#create-database-credentials-secrets) containing the source database credentials encrypted using that key.
 
 ### Create encryption key
 
@@ -216,44 +216,90 @@ In the [AWS Management Console](https://console.aws.amazon.com/), use the **Serv
 
 Review the key policy and key settings, and then select **Finish** to create the key.
 
-### Create database credentials secret
+### Create database credentials secrets
+
+To let Redis Cloud access your source database, you need to create AWS secrets for the source database's credentials and certificates. 
+
+The required secrets depend on your source database's security configuration. The following table shows the required secrets for each configuration:
+
+| Security configuration | Required secrets |
+| :-- | :-- |
+| Username and password only | <ul><li>Credentials secret (username and password)</li></ul> |
+| TLS connection | <ul><li>Credentials secret (username and password)</li><li>CA Certificate secret (server certificate)</li></ul> |
+| mTLS connection | <ul><li>Credentials secret (username and password)</li><li>CA Certificate secret (server certificate)</li><li>Client certificate secret</li><li>Client key secret</li></ul> |
+| mTLS connection with client key passphrase | <ul><li>Credentials secret (username and password)</li><li>CA Certificate secret (server certificate)</li><li>Client certificate secret</li><li>Client key secret</li><li>Client key passphrase secret</li></ul> |
+
+Select a tab to learn how to create the required secret.
+
+{{< multitabs id="rdi-cloud-secrets"
+      tab1="Credentials secret"
+      tab2="CA Certificate secret"
+      tab3="Client certificate secret"
+      tab4="Client key secret"
+      tab5="Client key passphrase secret" >}}
 
 In the [AWS Management Console](https://console.aws.amazon.com/), use the **Services** menu to locate and select **Security, Identity, and Compliance** > **Secrets Manager**. [Create a secret](https://docs.aws.amazon.com/secretsmanager/latest/userguide/create_secret.html) of type **Other type of secret** with the following settings:
 
 - **Key/value pairs**: Enter the following key/value pairs.
 
     - `username`: Database username
     - `password`: Database password
-    - `trust_certificate`: Server certificate in PEM format *(TLS only)*
-    - `client_public_key`: [X.509 client certificate](https://en.wikipedia.org/wiki/X.509) or chain in PEM format *(mTLS only)*
-    - `client_private_key`: Key for the client certificate or chain in PEM format *(mTLS only)*
-    - `client_private_key_passphrase`: Passphrase or password for the client certificate or chain in PEM format *(mTLS only)*
-
-    {{<note>}}
-If your source database has TLS or mTLS enabled, we recommend that you enter the `trust_certificate`, `client_public_key`, and `client_private_key` into the secret editor using the **Key/Value** input method instead of the **JSON** input method. Pasting directly into the JSON editor may cause an error. 
-    {{</note>}}
-
-- **Encryption key**: Select the [encryption key](#create-encryption-key) you created earlier.
-
-- **Resource permissions**: Add the following permissions to your secret to allow the Redis data pipeline to access your secret. Replace `<AWS ACCOUNT ID>` with the AWS account ID for the Redis Cloud cluster that you saved earlier.
-
-    ```json
-    {
-        "Version" : "2012-10-17",
-        "Statement" : [ {
-            "Sid" : "RedisDataIntegrationRoleAccess",
-            "Effect" : "Allow",
-            "Principal" : "*",
-            "Action" : [ "secretsmanager:GetSecretValue", "secretsmanager:DescribeSecret" ],
-            "Resource" : "*",
-            "Condition" : {
-                "StringLike" : {
-                    "aws:PrincipalArn" : "arn:aws:iam::<AWS ACCOUNT ID>:role/redis-data-pipeline-secrets-role"
-                }
-            }
-        } ]
-    }
-    ```
+
+{{< embed-md "rc-rdi-secrets-encryption-permissions.md" >}}
+
+--tab-sep--
+
+In the [AWS Management Console](https://console.aws.amazon.com/), use the **Services** menu to locate and select **Security, Identity, and Compliance** > **Secrets Manager**. [Create a secret](https://docs.aws.amazon.com/secretsmanager/latest/userguide/create_secret.html) of type **Other type of secret** with the following settings:
+
+- **Key/value pairs**: Select **Plaintext** and enter the server certificate.
+
+{{< embed-md "rc-rdi-secrets-encryption-permissions.md" >}}
+
+--tab-sep--
+
+In the [AWS Management Console](https://console.aws.amazon.com/), use the **Services** menu to locate and select **Security, Identity, and Compliance** > **Secrets Manager**. [Create a secret](https://docs.aws.amazon.com/secretsmanager/latest/userguide/create_secret.html) of type **Other type of secret** with the following settings:
+
+- **Key/value pairs**: Select **Plaintext** and enter the client certificate.
+
+{{< embed-md "rc-rdi-secrets-encryption-permissions.md" >}}
+
+--tab-sep--
+
+Use the [AWS CLI create-secret command](https://docs.aws.amazon.com/cli/latest/reference/secretsmanager/create-secret.html) or the [AWS CreateSecret API endpoint](https://docs.aws.amazon.com/secretsmanager/latest/apireference/API_CreateSecret.html) to create a binary secret containing the client key.
+
+For example, using the AWS CLI, run the following command:
+
+```sh
+aws secretsmanager create-secret \
+    --name <secret-name> \
+    --secret-binary fileb://<path-to-client-key> \
+    --kms-key-id <encryption-key-arn> 
+```
+
+Where:
+- `<secret-name>` - Name of the secret
+- `<path-to-client-key>` - Path to the client key file
+- `<encryption-key-arn>` - ARN of the [encryption key](#create-encryption-key) you created earlier
+
+After you create the secret, you need to add permissions to allow the data pipeline to access it. 
+
+In the [AWS Management Console](https://console.aws.amazon.com/), use the **Services** menu to locate and select **Security, Identity, and Compliance** > **Secrets Manager**. Select the private key secret you just created and then select **Edit permissions**. 
+
+Add the following permissions to your secret:
+
+{{< embed-md "rc-rdi-secrets-permissions.md" >}}
+
+Replace `<AWS ACCOUNT ID>` with the AWS account ID for the Redis Cloud cluster that you saved earlier.
+
+--tab-sep--
+
+In the [AWS Management Console](https://console.aws.amazon.com/), use the **Services** menu to locate and select **Security, Identity, and Compliance** > **Secrets Manager**. [Create a secret](https://docs.aws.amazon.com/secretsmanager/latest/userguide/create_secret.html) of type **Other type of secret** with the following settings:
+
+- **Key/value pairs**: Select **Plaintext** and enter the client key passphrase.
+
+{{< embed-md "rc-rdi-secrets-encryption-permissions.md" >}}
+
+{{< /multitabs >}}
 
 After you store this secret, you can view and copy the [Amazon Resource Name (ARN)](https://docs.aws.amazon.com/secretsmanager/latest/userguide/reference_iam-permissions.html#iam-resources) of your secret on the secret details page. 
 

content/operate/rc/databases/rdi/view-edit.md

@@ -49,20 +49,27 @@ To change the data you want to ingest from the data pipeline:
 
     {{<image filename="images/rc/rdi/rdi-edit-button.png" alt="The edit pipeline button." width=100px >}}
 
-1. Select the schema and tables you want to migrate to the target database from the **Source data selection** list. 
+1. Select the Schema and Tables you want to migrate to the target database from the list. 
+    {{<image filename="images/rc/rdi/rdi-select-source-data.png" alt="The data modeling section. " width=75% >}}
 
-    {{<image filename="images/rc/rdi/rdi-select-source-data.png" alt="The select source data section. " width=75% >}}
+    Select **Manage Columns** to choose which columns you want to import.
+
+    {{<image filename="images/rc/rdi/rdi-manage-columns.png" alt="The manage columns button." width=150px >}}
 
     You can select any number of columns from a table.
 
-    {{<image filename="images/rc/rdi/rdi-select-columns.png" alt="The select source data section. A table is expanded with a few columns selected." width=75% >}}
+    {{<image filename="images/rc/rdi/rdi-select-columns.png" alt="The manage columns screen, with a few columns selected from one table" width=75% >}}
 
-    If any tables are missing a unique constraint, the **Missing unique constraint** list will appear. Select the columns that define a unique constraint for those tables from the list.
+    If any tables are missing a unique constraint, a warning will appear in the **Data modeling** section. Select **Manage columns** to select the columns that define a unique constraint for those tables.
 
     {{<image filename="images/rc/rdi/rdi-missing-unique-constraint.png" alt="The missing unique constraint list." width=75% >}}
 
     {{<image filename="images/rc/rdi/rdi-select-constraints.png" alt="The missing unique constraint list with columns selected." width=75% >}}
 
+    Select **Save** to save your column changes and go back to schema selection.
+
+    {{<image filename="images/rc/button-save.png" alt="The save button." width=100px >}}
+
     Select **Add schema** to add more database schemas.
 
     {{<image filename="images/rc/rdi/rdi-add-schema.png" alt="The add schema button." width=150px >}}
@@ -74,16 +81,20 @@ To change the data you want to ingest from the data pipeline:
     After you've selected the schemas and tables you want to sync, select **Continue**.
 
      {{<image filename="images/rc/rdi/rdi-continue-button.png" alt="The continue button." width=150px >}}
+    
+1. Select the Redis data type to write keys to the target. You can choose **Hash** or **JSON** if the target database supports JSON. 
+    {{<image filename="images/rc/rdi/rdi-configure-new-pipeline.png" alt="The pipeline definition screen." width=75% >}}
 
-1. In the **Pipeline definition** section, select the Redis data type to write keys to the target. You can choose **Hash** or **JSON** if the target database supports JSON. 
+    You can also supply one or more [transformation job files]({{< relref "/integrate/redis-data-integration/data-pipelines/transform-examples" >}}) that specify how you want to transform the captured data before writing it to the target. Select **Upload jobs** to upload your job files.
+
+    {{<image filename="images/rc/rdi/rdi-transformation-jobs.png" alt="The transformation jobs section. Select Upload jobs to upload transformation jobs." >}}
+
+    When you upload job files, Redis Cloud will validate the job files to check for errors. 
 
-    {{<image filename="images/rc/rdi/rdi-configure-new-pipeline.png" alt="The Pipeline definition screen. Configure a new pipeline is selected." width=75% >}}
-    
     Select **Continue**.
-    
     {{<image filename="images/rc/rdi/rdi-continue-button.png" alt="The continue button." width=150px >}}
 
-1. Review the tables you selected in the **Summary** and select how you want to update the data pipeline:
+1. Review the tables you selected in and select how you want to update the data pipeline:
 
     {{<image filename="images/rc/rdi/rdi-update-preferences.png" alt="The Select update preferences section." width=25% >}}