RDSC-4108: RDI 1.15.0 release notes

Author: ZdravkoDonev-redis

content/integrate/redis-data-integration/release-notes/rdi-1-15-0.md

@@ -0,0 +1,69 @@
+---
+Title: Redis Data Integration release notes 1.15.0 (October 2025)
+alwaysopen: false
+categories:
+- docs
+- operate
+- rs
+description: |
+  Flink collector enabled by default for improved performance.
+  Enhanced high availability with leader election and standby mode.
+  Support for sharded Redis databases.
+  Improved configuration validation and monitoring capabilities.
+  Better resource management and security enhancements.
+linkTitle: 1.15.0 (October 2025)
+toc: 'true'
+weight: 976
+---
+
+RDI's mission is to help Redis customers sync Redis Enterprise with live data from their slow disk-based databases to:
+
+- Meet the required speed and scale of read queries and provide an excellent and predictable user experience.
+- Save resources and time when building pipelines and coding data transformations.
+- Reduce the total cost of ownership by saving money on expensive database read replicas.
+
+RDI keeps the Redis cache up to date with changes in the primary database, using a [_Change Data Capture (CDC)_](https://en.wikipedia.org/wiki/Change_data_capture) mechanism.
+It also lets you _transform_ the data from relational tables into convenient and fast data structures that match your app's requirements. You specify the transformations using a configuration system, so no coding is required.
+
+## What's New in 1.15.0
+
+{{<warning>}}
+**Breaking change when using JSON with `json_update_strategy: merge`**
+
+RDI now uses the native `JSON.MERGE` command instead of Lua scripts for JSON merge operations. While this provides significant performance improvements (2x faster), there is a **functional difference** in how null values are handled:
+
+- **Previous behavior (Lua merge)**: When merging `{"field1": "value1", "field2": "value2"}` with `{"field2": null, "field3": "value3"}`, the result was `{"field1": "value1", "field2": null, "field3": "value3"}` (null value is preserved)
+- **New behavior (JSON.MERGE)**: The same merge produces `{"field1": "value1", "field3": "value3"}` (null value removes the field, following [RFC 7396](https://datatracker.ietf.org/doc/html/rfc7396))
+
+**Impact**: If your application logic distinguishes between a field with a `null` value and a missing field, you may need to adjust your data handling. This follows the JSON Merge Patch RFC standard but differs from the previous Lua implementation.
+
+**Configuration**: You can control this behavior using the `use_native_json_merge` property in the processors section of your configuration. Set it to `false` to revert to the previous Lua-based merge behavior if needed.
+{{</warning>}}
+
+- **Native JSON merge for improved performance**: RDI now automatically uses the native `JSON.MERGE` command from RedisJSON 2.6.0+ instead of Lua scripts for JSON merge operations, providing 2x performance improvement. This feature is enabled by default and can be controlled via the `use_native_json_merge` property in the processors section of the configuration.
+- **Support for sharded Redis databases**: RDI now supports writing to multi-sharded Redis Enterprise databases for the RDI database, resolving cross-slot violations when saving statistics.
+- **Enhanced processor performance metrics**: Detailed performance metrics are now exposed through the metrics exporter and statistics endpoint, with separate tracking for transformation time and write time.
+- **Resource management improvements**: Collector and processor pods now support configurable resource requests, limits, and node affinity/tolerations for better cluster resource utilization.
+
+- **Leadership status monitoring**: New metrics expose leadership status and pipeline phase information for better monitoring of HA deployments.
+- **Improved configuration validation**: More rigid validation for `config.yaml` and `jobs.yaml` files helps catch configuration errors earlier in the deployment process.
+- **Custom K3s installation options**: The installer now supports passing custom arguments to K3s installation for more flexible on-premises deployments.
+- **Workload Identity authentication**: Added support for Google Cloud Workload Identity authentication to Google Cloud Storage (GCS), eliminating the need for service account JSON files in many scenarios.
+- Processor performance metrics showing transformation and write times separately
+- Enhanced statistics endpoint with new metrics for transform and process time
+
+### Bug Fixes and Stability Improvements
+
+- **Fixed task reconciliation errors**: Resolved "The ID argument cannot be a complete ID because xadd-id-uniqueness-mode is strict" errors during task reconciliation.
+- **Fixed Debezium unavailable values**: Addressed issues where `__debezium_unavailable_value` was appearing in Redis data.
+- **Improved operator stability**: Disabled operator webhooks by default to simplify deployments and reduce potential issues.
+
+### Documentation Improvements
+
+- **Oracle XStream documentation**: Added documentation on how to prepare Oracle databases with XStream for RDI.
+- **Flink collector documentation**: Added the `rdi-flink-collector` to the public documentation.
+- **FAQ for common issues**: Created a comprehensive FAQ with common issues and error messages to help users troubleshoot problems more quickly.
+
+## Limitations
+
+RDI can write data to a Redis Active-Active database. However, it doesn't support writing data to two or more Active-Active replicas. Writing data from RDI to several Active-Active replicas could easily harm data integrity as RDI is not synchronous with the source database commits.