RC RDI: Some typo fixes, scraping RDI metrics, and behavior fixes

content/embeds/rdi-aur-rds-pub-access.md

@@ -0,0 +1,5 @@
+## Allow public access
+
+For [RDI on Redis Cloud]({{< relref "/operate/rc/databases/rdi" >}}), you must allow public access to your source database to allow the proxy to connect to it. 
+
+To do this, [modify the database](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Overview.DBInstance.Modifying.html) and set **Publicly accessible** to **Yes**.

content/integrate/redis-data-integration/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-mysql.md

@@ -19,6 +19,8 @@ weight: 2
 Follow the steps in the sections below to prepare an [AWS Aurora MySQL](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/CHAP_GettingStartedAurora.CreatingConnecting.Aurora.html) or [AWS RDS MySQL](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_GettingStarted.CreatingConnecting.MySQL.html) database.
 database to work with RDI.
 
+{{< embed-md "rdi-aur-rds-pub-access.md" >}} 
+
 ## Create and apply parameter group
 
 RDI requires some changes to database parameters. On AWS RDS and AWS Aurora, you change these parameters via a parameter group.

content/integrate/redis-data-integration/data-pipelines/prepare-dbs/aws-aurora-rds/aws-aur-pgsql.md

@@ -23,6 +23,8 @@ Follow the steps in the sections below to prepare an
 [AWS Aurora PostgreSQL](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/CHAP_GettingStartedAurora.CreatingConnecting.AuroraPostgreSQL.html) or [AWS RDS PostgreSQL](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_GettingStarted.CreatingConnecting.PostgreSQL.html)
 database to work with RDI.
 
+{{< embed-md "rdi-aur-rds-pub-access.md" >}} 
+
 ## Create and apply parameter group
 
 RDI requires some changes to database parameters. On AWS RDS and AWS Aurora, you change these parameters via a parameter group.

content/integrate/redis-data-integration/data-pipelines/prepare-dbs/aws-aurora-rds/aws-rds-sqlserver.md

@@ -18,6 +18,12 @@ weight: 3
 
 Follow the steps in the sections below to prepare a [Microsoft SQL Server on AWS RDS](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_GettingStarted.CreatingConnecting.SQLServer.html) database to work with RDI.
 
+{{< note >}}
+Change Data Capture (CDC) is not supported on SQL Server Express Edition. Only the Standard, Enterprise, and Developer editions support CDC and are supported by RDI.
+{{< /note >}}
+
+{{< embed-md "rdi-aur-rds-pub-access.md" >}} 
+
 ## Create the Debezium user
 
 The Debezium connector needs a user account to connect to SQL Server. This

content/operate/rc/databases/rdi/define.md

@@ -82,6 +82,11 @@ After your pipeline is provisioned, you will be able to define your pipeline. Yo
 
 1. In the **Pipeline definition** section, select the Redis data type to write keys to the target. You can choose **Hash** or **JSON** if the target database supports JSON. 
     {{<image filename="images/rc/rdi/rdi-configure-new-pipeline.png" alt="The pipeline definition screen." width=75% >}}
+
+    You can also supply one or more [transformation job files]({{< relref "/integrate/redis-data-integration/data-pipelines/transform-examples" >}}) that specify how you want to transform the captured data before writing it to the target. Select **Upload jobs** to upload your job files.
+
+    {{<image filename="images/rc/rdi/rdi-transformation-jobs.png" alt="The transformation jobs section. Select Upload jobs to upload transformation jobs." >}}
+
     Select **Continue**.
     {{<image filename="images/rc/rdi/rdi-continue-button.png" alt="The continue button." width=150px >}}
 

content/operate/rc/databases/rdi/setup.md

@@ -81,7 +81,7 @@ In the [AWS Management Console](https://console.aws.amazon.com/), use the **Serv
             - **Target type**: Select **Instances**.
             - **Protocol : Port**: Select **TCP**, and then enter the port number where your database is exposed.
             - The **IP address type** and **VPC** should be selected already and match the VPC you selected earlier.
-        1. In **Register targets**, select the EC2 instance that runs your source database, enter the port, and select **Include as pending below**. Then, select **Create target group** to create your target group. Return **Listeners and routing** in the Network Load Balancer setup.
+        1. In **Register targets**, select the EC2 instance that runs your source database, enter the port, and select **Include as pending below**. Then, select **Create target group** to create your target group. Return to **Listeners and routing** in the Network Load Balancer setup.
     1. Set the following **Listener** properties:
         - **Protocol**: Select **TCP**.
         - **Port**: Enter your source database's port.
@@ -129,6 +129,13 @@ To do this:
 
 In the [AWS Management Console](https://console.aws.amazon.com/), use the **Services** menu to locate and select **Database** > **Aurora and RDS**. [Create an RDS proxy](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/rds-proxy-creating.html) that can access your database.
 
+The Proxy's IAM role must have the following permissions to access the database using the credentials secret and encryption key:
+- `secretsmanager:GetSecretValue`
+- `secretsmanager:DescribeSecret`
+- `kms:Decrypt`
+
+You can set the proxy's IAM role during creation in the **Authentication** section.
+
 #### Create network load balancer {#create-network-load-balancer-rds}
 
 In the [AWS Management Console](https://console.aws.amazon.com/), use the **Services** menu to locate and select **Compute** > **EC2**. [Create a network load balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/create-network-load-balancer.html#configure-load-balancer) with the following settings:
@@ -144,7 +151,7 @@ In the [AWS Management Console](https://console.aws.amazon.com/), use the **Serv
             - **Target type**: Select **IP Addresses**.
             - **Protocol : Port**: Select **TCP**, and then enter the port number where your database is exposed.
             - The **IP address type** and **VPC** should be selected already and match the VPC you selected earlier.
-        1. In **Register targets**, enter the static IP address of your RDS proxy, enter the port, and select **Include as pending below**. Then, select **Create target group** to create your target group. Return **Listeners and routing** in the Network Load Balancer setup.
+        1. In **Register targets**, enter the static IP address of your RDS proxy, enter the port, and select **Include as pending below**. Then, select **Create target group** to create your target group. Return to **Listeners and routing** in the Network Load Balancer setup.
             To get the static IP address of your RDS Proxy, run the following command on an EC2 instance in the same VPC as the Proxy:
             ```sh
             $ nslookup <proxy-endpoint>

content/operate/rc/databases/rdi/view-edit.md

@@ -99,6 +99,16 @@ At this point, the data pipeline will apply the changes. If you selected **Reset
 
 If you selected **Apply to new data changes only**, the data pipeline will enter the *change streaming* phase without ingesting data.
 
+## View metrics endpoints
+
+You can use [Prometheus and Grafana]({{< relref "/integrate/prometheus-with-redis-cloud/" >}}) to track and display metrics for the data pipeline. 
+
+To view the metrics endpoints for the source collector and pipeline processor, go to the **Data pipeline** tab and select **More actions**, and then **Show metrics**. You can add these endpoints as a Prometheus target to start tracking your RDI metrics.
+
+Prometheus endpoints are exposed on Redis Cloud's internal network. To access this network, enable [VPC peering]({{< relref "/operate/rc/security/vpc-peering" >}}) or [AWS Transit Gateway]({{< relref "/operate/rc/security/aws-transit-gateway" >}}). See [Prometheus and Grafana with Redis Cloud]({{< relref "/integrate/prometheus-with-redis-cloud/" >}}) for more information.
+
+For more information about available RDI metrics, see [Observability]({{< relref "/integrate/redis-data-integration/observability" >}}).
+
 ## Reset data pipeline
 
 Resetting the data pipeline creates a new baseline snapshot from the current state of your source database, and re-processes the data from the source database to the target Redis database. You may want to reset the pipeline if the source and target databases were disconnected or you made large changes to the data pipeline.