Skip to content
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion config.toml
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ rdi_redis_gears_version = "1.2.6"
rdi_debezium_server_version = "2.3.0.Final"
rdi_db_types = "cassandra|mysql|oracle|postgresql|sqlserver"
rdi_cli_latest = "latest"
rdi_current_version = "v1.6.7"
rdi_current_version = "1.8.0"

[params.clientsConfig]
"Python"={quickstartSlug="redis-py"}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -54,10 +54,10 @@ Complete the following steps before installing the RDI Helm chart:
more information).

- Download the RDI Helm chart tar file from the
[Redis download center](https://redis.io/downloads/) (in the *Modules, Tools & Integration* category) .
[Redis download center](https://redis-enterprise-software-downloads.s3.amazonaws.com/redis-di/rdi-{{< rdi-version >}}.tgz) (in the *Modules, Tools & Integration* category) .

```bash
export RDI_VERSION=<version>
export RDI_VERSION={{< rdi-version >}}
wget https://redis-enterprise-software-downloads.s3.amazonaws.com/redis-di/rdi-$RDI_VERSION.tgz
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ This guide explains how to install Redis Data Integration (RDI) on one or more V
your source database. You can also
[Install RDI on Kubernetes]({{< relref "/integrate/redis-data-integration/installation/install-k8s" >}}).

{{< note >}}We recommend you always use the latest version, which is RDI {{< field "rdi_current_version" >}}.
{{< note >}}We recommend you always use the latest version, which is RDI v{{< rdi-version >}}.
{{< /note >}}

## Create the RDI database
Expand Down Expand Up @@ -137,12 +137,12 @@ sudo ufw reload
Follow the steps below for each of your VMs:

1. Download the RDI installer from the
[Redis download center](https://redis.io/downloads/)
[Redis download center](https://redis-enterprise-software-downloads.s3.amazonaws.com/redis-di/rdi-installation-{{< rdi-version >}}.tar.gz)
(from the *Modules, Tools & Integration* category) and extract it to your preferred installation
folder.

```bash
export RDI_VERSION=<version>
export RDI_VERSION={{< rdi-version >}}
wget https://redis-enterprise-software-downloads.s3.amazonaws.com/redis-di/rdi-installation-$RDI_VERSION.tar.gz
tar -xvf rdi-installation-$RDI_VERSION.tar.gz
```
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,12 +22,12 @@ Follow the steps below to upgrade an existing
[VM installation]({{< relref "/integrate/redis-data-integration/installation/install-vm" >}})
of RDI:

1. Download the RDI installer from the [Redis download center](https://redis.io/downloads/)
1. Download the RDI installer from the [Redis download center](https://redis-enterprise-software-downloads.s3.amazonaws.com/redis-di/rdi-installation-{{< rdi-version >}}.tar.gz)
(in the *Modules, Tools & Integration* category) and extract it to your
preferred installation folder.

```bash
export RDI_VERSION=<version>
export RDI_VERSION={{< rdi-version >}}
wget https://redis-enterprise-software-downloads.s3.amazonaws.com/redis-di/rdi-installation-$RDI_VERSION.tar.gz
tar -xvf rdi-installation-$RDI_VERSION.tar.gz
```
Expand Down Expand Up @@ -92,11 +92,11 @@ installation of RDI:
[Using a private image registry]({{< relref "/integrate/redis-data-integration/installation/install-k8s#using-a-private-image-registry" >}})
and add them to your local registry.

1. Download the RDI Helm chart tar file from the [Redis download center](https://redis.io/downloads/)
1. Download the RDI Helm chart tar file from the [Redis download center](https://redis-enterprise-software-downloads.s3.amazonaws.com/redis-di/rdi-{{< rdi-version >}}.tgz)
(in the *Modules, Tools & Integration* category).

```bash
export RDI_VERSION=<version>
export RDI_VERSION={{< rdi-version >}}
wget https://redis-enterprise-software-downloads.s3.amazonaws.com/redis-di/rdi-$RDI_VERSION.tgz
```

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
---
Title: Redis Data Integration release notes 1.8.0 (May 2025)
alwaysopen: false
categories:
- docs
- operate
- rs
description: |
Enhanced RDI operator for better pipeline orchestration, resiliency, observability and flexibility;
External collector support;
Labels and annotations to RDI data plane pods;
Custom Debezium image;
Calculated TTL for target database.
linkTitle: 1.8.0 (May 2025)
toc: 'true'
weight: 984
---

{{< note >}}This minor release replaces the 1.6.7 release.{{< /note >}}

RDI’s mission is to help Redis customers sync Redis Enterprise with live data from their slow disk-based databases to:

- Meet the required speed and scale of read queries and provide an excellent and predictable user experience.
- Save resources and time when building pipelines and coding data transformations.
- Reduce the total cost of ownership by saving money on expensive database read replicas.

RDI keeps the Redis cache up to date with changes in the primary database, using a [_Change Data Capture (CDC)_](https://en.wikipedia.org/wiki/Change_data_capture) mechanism.
It also lets you _transform_ the data from relational tables into convenient and fast data structures that match your app's requirements. You specify the transformations using a configuration system, so no coding is required.

## Headlines

- Enhanced RDI operator for better pipeline orchestration, resiliency, observability and flexibility. It
will also enable many new features in the near future.
- You can now use an external collector that is not managed by RDI but writes into RDI streams
(Debezium compatible).
- You can now add labels and annotations to RDI data plane pods, for example to control service
mesh features.
- RDI now uses a custom image of Debezium (based on `3.0.8.Final`) to address known vulnerabilities.
- Added support for calculated TTL for target database keys via `expire` expressions.

## Detailed changes

### Helm chart changes

- All collector and processor values that were previously under `collector`, `collectorSourceMetricsExporter`, and `processor` have been moved to `operator.dataPlane.collector` and `operator.dataPlane.processor`.
- `global.collectorApiEnabled` has been moved to `operator.dataPlane.collectorApi.enabled`, and is now a boolean value, not `"0"` or `"1"`.
- `api.authEnabled` is also now a boolean value, not `"0"` or `"1"`.
- The following values have been deprecated: `rdiMetricsExporter.service.protocol`, `rdiMetricsExporter.service.port`, `rdiMetricsExporter.serviceMonitor.path`, `api.service.name`
- You can now add custom labels and annotations to all RDI components.
- You can now disable the creation of the RDI system secrets.

### Operator Improvements

The RDI operator has been significantly enhanced in the following areas:

- **Resilience**: The operator now always maintains the desired pipeline state. Manual changes or random disruptions are reverted automatically.
- **Automatic recovery**: When a configuration issue is resolved, the entire pipeline starts automatically, eliminating the need for manual redeployment.
- **Consistency**: A pipeline that has been stopped with `stop` will remain stopped after `deploy` or `reset`, until explicitly started again.
- **Enhanced configuration**: You can now configure data plane components in ways that were previously not supported, such as adding labels and annotations.
- **External collector support**: No collector resources are created for sources of type `external`.
- **Enhanced troubleshooting**: You can now gain extra insight into the pipeline state by examining the `Pipeline` and `PipelineRelease` custom K8s resources.

### Other Features, Improvements and Enhancements

- Added `expire` expression for target output in transformation jobs.
- Addressed security vulnerabilities: TLS certificate hostname verification is now ON by default.
- Improved Helm default values while preserving `values.yaml` formatting.
- Enhanced Helm values and templates for better configuration.
- Added a script to create or update secrets when using Helm (`rdi-secret.sh` in the Helm zip file).
- Improved validation schema and ensured backward compatibility.
- Fixed compatibility issues with newer versions of `requests` and `urllib3`.
- Improved error messages for JSON schema validation.
- Improved PostgreSQL documentation for mTLS.
- Added timestamps to the `status` command.
- Fixed issues with `primary_key` and `unique_constraint` attributes in Oracle metadata.
- Added `capture.mode` to MongoDB scaffolding.
- Improved Helm TLS setup for RDI database connections.
- Enhanced error handling and validation for transformation jobs.
- Improved documentation for supported platforms and configurations.

### Fixes

- Fixed HTTP 500 error when querying columns with tables parameter.
- Improved Helm TLS setup for RDI database connections.
- Fixed keystore overwrite when using mTLS on both source and RDI DBs in the collector.

## Limitations

RDI can write data to a Redis Active-Active database. However, it doesn't support writing data to two or more Active-Active replicas. Writing data from RDI to several Active-Active replicas could easily harm data integrity as RDI is not synchronous with the source database commits.
1 change: 1 addition & 0 deletions layouts/shortcodes/rdi-version.html
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{{- .Site.Params.rdi_current_version -}}