-
Notifications
You must be signed in to change notification settings - Fork 280
DOC-5223 RDI release notes for v1.8.0 #1549
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 2 commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
89 changes: 89 additions & 0 deletions
89
content/integrate/redis-data-integration/release-notes/rdi-1-8-0.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,89 @@ | ||
| --- | ||
| Title: Redis Data Integration release notes 1.8.0 (May 2025) | ||
| alwaysopen: false | ||
| categories: | ||
| - docs | ||
| - operate | ||
| - rs | ||
| description: | | ||
| Enhanced RDI operator for better pipeline orchestration, resiliency, observability and flexibility; | ||
| External collector support; | ||
| Labels and annotations to RDI data plane pods; | ||
| Custom Debezium image; | ||
| Calculated TTL for target database. | ||
| linkTitle: 1.8.0 (May 2025) | ||
| toc: 'true' | ||
| weight: 984 | ||
| --- | ||
|
|
||
| {{< note >}}This minor release replaces the 1.6.7 release.{{< /note >}} | ||
|
|
||
| RDI’s mission is to help Redis customers sync Redis Enterprise with live data from their slow disk-based databases to: | ||
|
|
||
| - Meet the required speed and scale of read queries and provide an excellent and predictable user experience. | ||
| - Save resources and time when building pipelines and coding data transformations. | ||
| - Reduce the total cost of ownership by saving money on expensive database read replicas. | ||
|
|
||
| RDI keeps the Redis cache up to date with changes in the primary database, using a [_Change Data Capture (CDC)_](https://en.wikipedia.org/wiki/Change_data_capture) mechanism. | ||
| It also lets you _transform_ the data from relational tables into convenient and fast data structures that match your app's requirements. You specify the transformations using a configuration system, so no coding is required. | ||
|
|
||
| ## Headlines | ||
|
|
||
| - Enhanced RDI operator for better pipeline orchestration, resiliency, observability and flexibility. It | ||
| will also enable many new features in the near future. | ||
| - You can now use an external collector that is not managed by RDI but writes into RDI streams | ||
| (Debezium compatible). | ||
| - You can now add labels and annotations to RDI data plane pods, for example to control service | ||
| mesh features. | ||
| - RDI now uses a custom image of Debezium (based on `3.0.8.Final`) to address known vulnerabilities. | ||
| - Added support for calculated TTL for target database keys via `expire` expressions. | ||
|
|
||
| ## Detailed changes | ||
|
|
||
| ### Helm chart changes | ||
|
|
||
| - All collector and processor values that were previously under `collector`, `collectorSourceMetricsExporter`, and `processor` have been moved to `operator.dataPlane.collector` and `operator.dataPlane.processor`. | ||
| - `global.collectorApiEnabled` has been moved to `operator.dataPlane.collectorApi.enabled`, and is now a boolean value, not `"0"` or `"1"`. | ||
ZdravkoDonev-redis marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
andy-stark-redis marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| - `api.authEnabled` is also now a boolean value, not `"0"` or `"1"`. | ||
| - The following values have been deprecated: `rdiMetricsExporter.service.protocol`, `rdiMetricsExporter.service.port`, `rdiMetricsExporter.serviceMonitor.path`, `api.service.name` | ||
| - You can now add custom labels and annotations to all RDI components. | ||
| - You can now disable the creation of the RDI system secrets. | ||
|
|
||
| ### Operator Improvements | ||
|
|
||
| The RDI operator has been significantly enhanced in the following areas: | ||
|
|
||
| - **Resilience**: The operator now always maintains the desired pipeline state. Manual changes or random disruptions are reverted automatically. | ||
| - **Automatic recovery**: When a configuration issue is resolved, the entire pipeline starts automatically, eliminating the need for manual redeployment. | ||
| - **Consistency**: A pipeline that has been stopped with `stop` will remain stopped after `deploy` or `reset`, until explicitly started again. | ||
| - **Enhanced configuration**: You can now configure data plane components in ways that were previously not supported, such as adding labels and annotations. | ||
| - **External collector support**: No collector resources are created for sources of type `external`. | ||
| - **Enhanced troubleshooting**: You can now gain extra insight into the pipeline state by examining the `Pipeline` and `PipelineRelease` custom K8s resources. | ||
|
|
||
| ### Other Features, Improvements and Enhancements | ||
|
|
||
| - Added `expire` expression for target output in transformation jobs. | ||
| - Addressed security vulnerabilities: TLS certificate hostname verification is now ON by default. | ||
| - Improved Helm default values while preserving `values.yaml` formatting. | ||
| - Enhanced Helm values and templates for better configuration. | ||
| - Added a script to create or update secrets when using Helm (`rdi-secret.sh` in the Helm zip file). | ||
| - Improved validation schema and ensured backward compatibility. | ||
| - Fixed compatibility issues with newer versions of `requests` and `urllib3`. | ||
| - Improved error messages for JSON schema validation. | ||
| - Improved PostgreSQL documentation for mTLS. | ||
| - Added timestamps to the `status` command. | ||
| - Fixed issues with `primary_key` and `unique_constraint` attributes in Oracle metadata. | ||
| - Added `capture.mode` to MongoDB scaffolding. | ||
| - Improved Helm TLS setup for RDI database connections. | ||
| - Enhanced error handling and validation for transformation jobs. | ||
| - Improved documentation for supported platforms and configurations. | ||
|
|
||
| ### Fixes | ||
|
|
||
| - Fixed HTTP 500 error when querying columns with tables parameter. | ||
| - Improved Helm TLS setup for RDI database connections. | ||
| - Fixed keystore overwrite when using mTLS on both source and RDI DBs in the collector. | ||
|
|
||
| ## Limitations | ||
|
|
||
| RDI can write data to a Redis Active-Active database. However, it doesn't support writing data to two or more Active-Active replicas. Writing data from RDI to several Active-Active replicas could easily harm data integrity as RDI is not synchronous with the source database commits. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| {{- .Site.Params.rdi_current_version -}} |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.