Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions antora.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ asciidoc:
copyright: Neo4j Inc.
page-product: Neo4j Connector for Kafka
confluent-connect-version: '7.4'
connector-version: '5.1'
exact-connector-version: '5.1.19'
connector-version: '5.2'
exact-connector-version: '5.2.0'
kafka-connect-version: '3.4.1'
page-pagination: true
product-name: Neo4j Connector for Kafka
Expand Down
1 change: 1 addition & 0 deletions modules/ROOT/examples/docker-data/source.cdc.avro.json
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
"neo4j.start-from": "NOW",
"neo4j.cdc.poll-interval": "1s",
"neo4j.cdc.poll-duration": "5s",
"neo4j.cdc.topic.my-topic.key-strategy": "SKIP",
"neo4j.cdc.topic.my-topic.patterns.0.pattern": "(:Person)",
"neo4j.cdc.topic.my-topic.patterns.0.operation": "create",
"neo4j.cdc.topic.my-topic.patterns.0.changesTo": "name,surname",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
"neo4j.start-from": "NOW",
"neo4j.cdc.poll-interval": "1s",
"neo4j.cdc.poll-duration": "5s",
"neo4j.cdc.topic.my-topic.key-strategy": "SKIP",
"neo4j.cdc.topic.my-topic.patterns.0.pattern": "(:Person)",
"neo4j.cdc.topic.my-topic.patterns.0.operation": "create",
"neo4j.cdc.topic.my-topic.patterns.0.changesTo": "name,surname",
Expand Down
1 change: 1 addition & 0 deletions modules/ROOT/examples/docker-data/source.cdc.json.json
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
"neo4j.start-from": "NOW",
"neo4j.cdc.poll-interval": "1s",
"neo4j.cdc.poll-duration": "5s",
"neo4j.cdc.topic.my-topic.key-strategy": "SKIP",
"neo4j.cdc.topic.my-topic.patterns.0.pattern": "(:Person)",
"neo4j.cdc.topic.my-topic.patterns.0.operation": "create",
"neo4j.cdc.topic.my-topic.patterns.0.changesTo": "name,surname",
Expand Down
1 change: 1 addition & 0 deletions modules/ROOT/examples/docker-data/source.cdc.protobuf.json
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
"neo4j.start-from": "NOW",
"neo4j.cdc.poll-interval": "1s",
"neo4j.cdc.poll-duration": "5s",
"neo4j.cdc.topic.my-topic.key-strategy": "SKIP",
"neo4j.cdc.topic.my-topic.patterns.0.pattern": "(:Person)",
"neo4j.cdc.topic.my-topic.patterns.0.operation": "create",
"neo4j.cdc.topic.my-topic.patterns.0.changesTo": "name,surname",
Expand Down
51 changes: 40 additions & 11 deletions modules/ROOT/pages/changelog.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,40 @@

This page lists changes to the {product-name}.

== Version 5.2.0

[cols="1,2",options="header"]
|===
| Feature | Details

a|
label:feature[]
label:new[]

The CDC Source connector declares itself as Exactly Once capable.
| For environments that support link:{https://cwiki.apache.org/confluence/display/KAFKA/KIP-618%3A+Exactly-Once+Support+for+Source+Connectors}[KIP-618], the CDC source connector will take advantage of this feature without any additional configuration.

a|
label:feature[]
label:new[]

New batched CDC sink handlers.
| New CDC sink handlers are introduced to provide better performance when processing CDC events in the sink connector.
These handlers will process messages in batches and apply them into the database in a more efficient way.
New handlers use the APOC Core `apoc.cypher.doIt` procedure and Cypher `SET` and `REMOVE` clauses with dynamic labels, and will be preferred over the default handlers based on availability of these features.

a|
label:functionality[]
label:new[]

Improved logging for sink connector.
| Sink connectors are improved to provide useful logging to track the progress of the connector and identify potential issues.

|===

== Version 5.1.19
[cols="1,2", options="header"]

[cols="1,2",options="header"]
|===
| Feature | Details

Expand All @@ -17,7 +49,7 @@ Configure Neo4j Java Driver to use provider SLF4J configuration.

== Version 5.1.18

[cols="1,2", options="header"]
[cols="1,2",options="header"]
|===
| Feature | Details

Expand All @@ -32,7 +64,7 @@ This has been fixed so that the connector now correctly advances the in-memory o

== Version 5.1.16

[cols="1,2", options="header"]
[cols="1,2",options="header"]
|===
| Feature | Details

Expand All @@ -44,10 +76,9 @@ Fix wrong serialization of point types in `RAW_JSON_STRING` mode.
| For WGS-84 points, longitude and latitude values were mixed up in `RAW_JSON_STRING` mode.
|===


== Version 5.1.15

[cols="1,2", options="header"]
[cols="1,2",options="header"]
|===
| Feature | Details

Expand All @@ -65,7 +96,7 @@ This is a maintenance release which provides updated dependencies.

== Version 5.1.13

[cols="1,2", options="header"]
[cols="1,2",options="header"]
|===
| Feature | Details

Expand All @@ -91,7 +122,7 @@ This is a maintenance release which provides updated dependencies.

== Version 5.1.9

[cols="1,2", options="header"]
[cols="1,2",options="header"]
|===
| Feature | Details

Expand All @@ -103,10 +134,9 @@ Incorrect parsing of Neo4j version.
| Resolved an issue where the sink connector fails due to wrong handling of Neo4j version strings.
|===


== Version 5.1.8

[cols="1,2", options="header"]
[cols="1,2",options="header"]
|===
| Feature | Details

Expand Down Expand Up @@ -141,7 +171,6 @@ User provided Cypher queries that return results were failing in cypher sink str
The Cypher generation logic is now fixed and these type of queries are now supported.
|===


== Version 5.1.7

This is a maintenance release which provides updated dependencies.
Expand All @@ -154,7 +183,7 @@ This is a maintenance release which provides updated dependencies.

=== New and Updated Features

[cols="1,2", options="header"]
[cols="1,2",options="header"]
|===
| Feature | Details

Expand Down
18 changes: 18 additions & 0 deletions modules/ROOT/pages/sink/cdc.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,24 @@ This strategy allows to ingest CDC events coming from another Neo4j instance, ge
[IMPORTANT]
Change Data Capture events needs to be generated by the same corresponding version of the Source connector, which must be configured using a value converter that supports schemas.

== Batched Handlers

Starting from version 5.2.0, new batched CDC sink handlers are available to provide better performance when processing CDC events.
These handlers process messages in batches and apply them to the database more efficiently.

The batched handlers use the APOC Core `apoc.cypher.doIt` procedure and Cypher `SET` and `REMOVE` clauses with dynamic labels.
When these features are available in your target database, the connector will automatically prefer the batched handlers over the default handlers.

Batch size can be configured using the `neo4j.batch-size` setting, which defaults to `1000`.

[NOTE]
====
To take advantage of batched handlers, ensure that:

* APOC Core is installed and enabled in your database.
* Your database version supports dynamic labels in `SET` and `REMOVE` clauses.
====

Two sub-strategies are available:

* The <<schema, Schema>> strategy merges nodes and relationships by the constraints (node key, relationship key and/or property uniqueness + existence) defined in the source database.
Expand Down
11 changes: 11 additions & 0 deletions modules/ROOT/pages/source/cdc.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,17 @@ Usually will be the same as authenticated user, but could be different if impers
[NOTE]
Only `pattern` settings are mandatory in the above example, and others are optional and can be added based on your requirements.

== Exactly Once Semantics

Starting from version 5.2.0, the CDC source connector declares itself as Exactly Once capable.
For environments that support link:https://cwiki.apache.org/confluence/display/KAFKA/KIP-618%3A+Exactly-Once+Support+for+Source+Connectors[KIP-618], the CDC source connector will take advantage of this feature without any additional configuration.

[NOTE]
====
Exactly Once Semantics guarantees that each change event is delivered exactly once to the target Kafka topic, eliminating duplicate records that could occur during connector restarts or failures.
This feature requires a Kafka Connect cluster that supports KIP-618 (available in Apache Kafka 3.3.0 and later).
====

== Creating Source instance

Based on the above example, you can use one of the following configurations.
Expand Down