You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: _posts/2025-08-05-debezium-3-3-alpha1-released.adoc
+29-28Lines changed: 29 additions & 28 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -7,7 +7,7 @@ author: ccranfor
7
7
extraClasses: release-v2
8
8
---
9
9
10
-
Debezium **3.3.0.Alpha1** introduces an exiting wave of innovation, including exactly-once semantics for core connectors and a brand new CockroachDB connector led by the community.
10
+
Debezium **3.3.0.Alpha1** introduces an exciting wave of innovation, including exactly-once semantics for core connectors and a brand new CockroachDB connector led by the community.
11
11
Featuring support for emerging data types, deeper Quarkus integration, and enhanced tooling, this release raises the bar for modern change data capture solutions.
12
12
Ready to see what's new? Let's dive in.
13
13
@@ -41,7 +41,7 @@ The CockroachDB connector is a new connector, lead by the community in conjuncti
41
41
This connector is built on top of CockroachDB's change-feed subsystem, to capture row-level changes, and emit those as change events.
42
42
43
43
This new connector requires CockroachDB 25.2+ with the `rangefeed` option enabled and Java 21+.
44
-
This connector is in an incubating state and is under heavy development by the community.
44
+
This connector is currently incubating and is under heavy development by the community.
45
45
46
46
If you'd like to get started with the connector, you can get the connector with these coordinates:
47
47
@@ -50,7 +50,7 @@ If you'd like to get started with the connector, you can get the connector with
@@ -61,7 +61,7 @@ The documentation for this connector is still under development, but you can fin
61
61
There has been numerous requests from the community about Debezium with exactly-once semantics support.
62
62
Exactly-once semantics in layman's terms means that an event should only ever be delivered and written to a Kafka topic once, therefore avoiding the possibility of duplicates.
63
63
64
-
We are pleased to share that Debezium 3.3 introduces support for exactly-once semantics for all core connectors, including MariaDB, MySQL, Oracle, PostgreSQL, and SQL Server (https://issues.redhat.com/browse/DBZ-9177[DBZ-9177]).
64
+
We are pleased to share that Debezium 3.3 introduces support for exactly-once semantics for all core connectors, including MariaDB, MongoDB, MySQL, Oracle, PostgreSQL, and SQL Server (https://issues.redhat.com/browse/DBZ-9177[DBZ-9177]).
65
65
66
66
=== Disabling context headers
67
67
@@ -72,15 +72,14 @@ Based on community feedback, we have introduced a new configuration option, `ext
72
72
73
73
=== MariaDB 11.7+ vector data type support
74
74
75
-
MariaDB 11.7 introduced a new data type called `VECTOR(n)`, allowing the storage of values in the relational database as if it were a vector database.
75
+
MariaDB 11.7 introduced the `VECTOR(n)` data type, allowing the storage of values in the relational database as if it were a vector database.
76
76
Vector values that are generated by an AI model can be stored and searched in MariaDB using this new data type.
77
77
78
-
Debezium 3.3 introduces MariaDB Vector support for this new data type in both the MariaDB source connector and the JDBC sink connector when writing to a MariaDB target database (https://issues.redhat.com/browse/DBZ-8582[DBZ-8582]).
79
-
The only requirement to get started with the MariaDB vector data type support is to capture or write changes to a MariaDB 11.7 or later database.
78
+
Debezium 3.3 introduces MariaDB vector data type support in the MariaDB source connector and the JDBC sink connector when writing to a MariaDB target database (https://issues.redhat.com/browse/DBZ-8582[DBZ-8582]).
80
79
81
80
=== MariaDB and MySQL schema history improvements
82
81
83
-
The MariaDB and MySQL schema history topic is used by the connector to store a variety of DDL operations captured from the binary logs, allowing the connector to maintain a relational model of all captured tables.
82
+
The MariaDB and MySQL schema history topic is used by the connector to store a variety of DDL operations captured from the binary logs, allowing the connector to build a relational model from the chronological history of schema changes for tables.
84
83
85
84
In Debezium 3.3, the schema history topic's filters were updated to avoid storing specific statements in the topic that are not relevant to Debezium's function, such as procedure, function, view, trigger, and grant/revoke statements (https://issues.redhat.com/browse/DBZ-9186[DBZ-9186]).
86
85
@@ -95,7 +94,7 @@ For existing connectors, only newly captured DDL statements that match the impro
95
94
In Debezium 2.7, we introduced a bug fix that aligned the behavior of `decimal.handling.mode` when using the `double` or `string` modes in the Oracle connector to match other relational connectors.
96
95
This bug fix introduced a change in the event's schema and created numerous problems for those upgrading who were not expecting such changes.
97
96
98
-
We looked at a variety of options, whether to introduce legacy support via a `CustomConverter` or a `Transformation`, however, after looking at the changes required, we identified that the legacy support needed to be baked into the connector to provide a sufficient solution to the problem facing users who upgraded.
97
+
We looked at a variety of options, whether to introduce legacy support via a `CustomConverter` or a `Transformation`, however, we concluded that the legacy support needed to be baked into the connector to provide a sufficient solution.
99
98
100
99
In Debezium 3.3, a new configuration property `legacy.decimal.handling.strategy` was added allowing users to restore the use of the broken behavior, where numeric values with zero-scale would not be automatically converted to `double` or `string` and instead would be emitted as integer whole values as long as their length was 18 or less (https://issues.redhat.com/browse/DBZ-9166[DBZ-9166]).
101
100
@@ -123,7 +122,8 @@ For systems where there is a disproportionately higher volume of changes in non-
123
122
124
123
=== Postgres text-search vector support
125
124
126
-
In PostgreSQL 13+, full text search data types were added. The `tsvector` data type represents a document in the form optimized for text searches, providing a sorted list of distinct lexemes.
125
+
In PostgreSQL 13+, full text search data types were added.
126
+
The `tsvector` data type represents a document in a form optimized for text searches, providing a sorted list of distinct lexemes.
127
127
128
128
With Debezium 3.3, you can now capture changes made to columns using the `tsvector` data type (https://issues.redhat.com/browse/DBZ-8470[DBZ-8470]).
129
129
These column types are emitted using the logical type called `io.debezium.data.Tsvector` and is represented as a _string_ data type in the event.
@@ -137,7 +137,7 @@ With Debezium 3.3 (and backported to the upcoming Debezium 3.2.1), the Debezium
137
137
138
138
[NOTE]
139
139
====
140
-
Due to the compatibility changes, when using Debezium 3.2.0.Final or earlier, the Quarkus application must be based on 3.22.x or earlier.
140
+
Due to the compatibility changes, when using Debezium 3.2.0.Final or earlier, the Quarkus application must be based on Quarkus 3.22.x or earlier.
141
141
The use of Quarkus 3.24.x or later is only possible when using Debezium 3.2.1 / 3.3.x or later.
142
142
====
143
143
@@ -154,22 +154,23 @@ When enabled, the `debezium.sink.eventhubs.hashmessagekeyfunction` can be set to
154
154
155
155
=== Debezium Platform introduces a Smart Editor
156
156
157
-
Configuring a Debezium connector across runtimes can sometimes be difficult, particularly when different runtimes use different formats, i.e. Kafka Connect with JSON versus Debezium Server with key/value properties files.
158
-
The Debezium Platform also uses its own JSON-based format that differs slightly from Kafka Connect, which adds another layer of complexity when you may be working with connectors across two or more runtimes.
157
+
Managing similar Debezium connectors across different runtimes presents its own unique sets of maintenance challenges since each runtime uses different formats, i.e. Kafka Connect with JSON versus Debezium Server with key/value properties files.
158
+
The Debezium Platform also uses its own JSON-based format that differs slightly from Kafka Connect, which adds another layer of complexity.
159
159
160
-
To streamline the user experience with Debezium Platform, a new _Smart Editor_ feature is included, which allows you to write or paste configurations from Kafka Connect or Debezium Server, and convert those to the Platform's format, automatically (https://issues.redhat.com/browse/DBZ-8873[DBZ-8873] https://issues.redhat.com/browse/DBZ-8888[DBZ-8888]).
160
+
To streamline the user experience with Debezium Platform, a new _Smart Editor_ feature is available, which allows you to write or paste configurations from Kafka Connect or Debezium Server, and convert those to the Platform's format, automatically (https://issues.redhat.com/browse/DBZ-8873[DBZ-8873] https://issues.redhat.com/browse/DBZ-8888[DBZ-8888]).
161
161
162
162
=== Quarkus extension improvements
163
163
164
164
One of the most recent additions to Debezium's portfolio is the Debezium Quarkus extension, which enables developers to utilize Debezium's CDC capabilities directly within a Quarkus-based application in either JVM or Native builds.
165
+
We've added several new improvements, so let's cover each of those separately.
165
166
166
167
==== Heartbeat listeners
167
168
168
-
Debezium can be configured with `heartbeat.interval.ms` with a value greater than zero, forcing Debezium to emit a heartbeat event into the event stream.
169
-
Heartbeat events can be used for a variety of reasons, but their main use is to make sure that offsets remain aligned with the current read state on the source database, even during periods of no activity for captured tables.
169
+
Debezium can be configured with `heartbeat.interval.ms` to make Debezium emit a heartbeat event into the event stream periodically.
170
+
Heartbeat events can be used for a variety of reasons, but their main use is to make sure that offsets remain aligned with the current read state on the source database, even during periods of low/no activity for captured tables.
170
171
171
-
To allow the Quarkus application to perform addition actions when the `heartbeat.interval.ms` expires, the application can be written to observe the CDI event `DebeziumHeartbeat`.
172
-
This allows for application-specific code to be executed on each heartbeat (https://issues.redhat.com/browse/DBZ-8960[DBZ-8960]).
172
+
A Quarkus application using the Debezium extension can also perform additional actions when a heartbeat is emitted by observing the CDI event `DebeziumHeartbeat`.
173
+
This allows for application-specific code to be executed for each heartbeat (https://issues.redhat.com/browse/DBZ-8960[DBZ-8960]).
173
174
174
175
In the following example, the listener merely writes to the console when a heartbeat is observed.
175
176
@@ -186,11 +187,11 @@ public class HeartbeatListener {
186
187
==== Easily implement custom converters
187
188
188
189
A Debezium `CustomConverter` is a tool that can be used to change how Debezium emits the value for a given column type.
189
-
This can be an extremely powerful tool for consumers that require data in a specific format.
190
+
This can be an extremely powerful for consumers that require data in a specific format.
190
191
191
-
Within your Quarkus application, you can use the traditional way to define a `CustomConverter` by specifying it as part of the connector configuration in your `application.properties` file; however, Debezium 3.3 now provides an annotation-based approach that simplifies configuration and makes iterative development much easier and faster (https://issues.redhat.com/browse/DBZ-8966[DBZ-8966]).
192
+
Within your Quarkus application, you can use the traditional way to define a `CustomConverter` by specifying it as part of the connector configuration in your `application.properties` file; however, Debezium 3.3 adds a new annotation-based approach that simplifies configuration and makes iterative development much easier and faster (https://issues.redhat.com/browse/DBZ-8966[DBZ-8966]).
192
193
193
-
As an example, the following showcases defining a converter that takes a given raw field and return's its value always as a _String_-based field in the emitted event.
194
+
As an example, the following showcases defining a converter that takes a given raw field and return's its value converted as a _String_in the event using the Java `toString()` function.
194
195
195
196
[source,java]
196
197
----
@@ -203,10 +204,10 @@ class RawToStringConverter {
203
204
}
204
205
----
205
206
206
-
You may also need to implement custom converter behavior to be selectively applied to certain fields.
207
-
Developers may be inclined to implement this behavior in the custom converter annotated method, and while that's valid, there is another component-driven way you can do this by using a `FieldFilteringStrategy`.
207
+
In addition, custom converters can be selectively applied to fields.
208
+
Developers may be inclined to implement this behavior in the custom converter annotated method, and while that's valid, this can be done in a component-driven way using a `FieldFilteringStrategy`.
208
209
209
-
As an example, the following showcases the same custom converter, but it now selectively is applied based on the field filtering strategy:
210
+
As an example, the following showcases the same custom converter, but uses the field filtering strategy:
210
211
211
212
[source,java]
212
213
----
@@ -249,7 +250,7 @@ class ProductHandler {
249
250
----
250
251
251
252
For this to work, a `Deserializer<T>` needs to be implemented to convert the `ChangeEvent` into our target POJO type `Product`.
252
-
The extension provides an implementation to handle this using Jackson.
253
+
The extension provides an implementation to handle this using Jackson, `ObjectMapperDeserializer`.
253
254
254
255
[source,java]
255
256
----
@@ -260,17 +261,17 @@ public class ProductDeserializer extends ObjectMapperDeserializer<Product> {
260
261
}
261
262
----
262
263
263
-
The remaining step to glue this all together is to define the deserializer in the Quarkus configuration.
264
+
The last step involves defining the deserializing mapping in the Quarkus configuration to glue it all together.
This configuration tells the extension that when an event is destined for `prefix.inventory.products`, the `product` deserializer should be used.
272
+
The extension will use this configuration to map an event destined for the `prefix.inventory.products` topic to use the `product` deserializer.
272
273
This deserializer is mapped to the `<the-java-package>.ProductDeserializer` class, which uses Jackson to convert the `ChangeEvent` into a `Product` object.
273
-
When the conversion is applied, the `@Capturing` annotated method is called, providing the `Product` object instead of the emitted `ChangeEvent`.
274
+
After the conversion, the `@Capturing` annotated method is called with the `Product` object instead of the emitted `ChangeEvent`.
0 commit comments