Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,63 @@ See the Debezium documentation for more information about the specific connector
| `lob.enabled` | `string` | Oracle | Enables capturing and serialization of large object (CLOB, NCLOB, and BLOB) column values in change events.<br/>Default: `false` |
| `unavailable.value.placeholder` | Special | Oracle | Specifies the constant that the connector provides to indicate that the original value is unchanged and not provided by the database (this has the type `__debezium_unavailable_value`). |

### Oracle heartbeat mechanism

{{< note >}} This section is only relevant for Oracle installations where many hours
or days may pass between changes to the source database.
{{< /note >}}

Oracle uses *system change numbers* (SCNs) to record the order in which events happen
in a database. Any given event always has a higher SCN than any event that happened
previously, so the SCNs form a logical "timeline". (See
[Transactions](https://docs.oracle.com/cd/E11882_01/server.112/e40540/transact.htm#CNCPT016)
in the Oracle docs for more information.)

For an Oracle source database, the RDI collector records the SCN of the most recent
transaction it has captured. When it checks the source for changes, it
uses this last recorded SCN to find all events that have happened in the meantime
and catch up with processing them. However, Oracle internally discards its SCN
information after a certain number of transactions have occurred. If these
transactions are against tables that RDI is *not* capturing, the last SCN
recorded by RDI might eventually be discarded by Oracle. When this happens,
the collector is unable to use its last SCN to detect new changes and data
may be lost as a result.

If you are using Oracle as a source database and you expect updates to the
data to be very infrequent, you should enable the *heartbeat mechanism* in
the RDI collector. This writes data to a "dummy" table in the source on a
regular basis purely to generate a new SCN. To enable the heartbeat,
first create the dummy table with the following SQL:

```sql
CREATE TABLE rdi_heartbeat (
id NUMBER PRIMARY KEY,
last_heartbeat TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);
```

Then, give WRITE/UPDATE permission to RDI on this table:

```sql
GRANT INSERT, UPDATE ON rdi_heartbeat TO rdi_user;
```

Finally, add the following lines to `config.yaml` under `sources.oracle.advanced`:

```yaml
sources:
oracle:
advanced:
source:
heartbeat.interval.ms: 60000
heartbeat.action.query: "MERGE INTO rdi_heartbeat t USING (SELECT 1 AS id FROM dual) s
ON (t.id = s.id)
WHEN MATCHED THEN UPDATE SET t.last_heartbeat = CURRENT_TIMESTAMP
WHEN NOT MATCHED THEN INSERT (id, last_heartbeat) VALUES (1, CURRENT_TIMESTAMP);"
```

This sets the appropriate heartbeat update to occur once every minute.

### Using custom queries in the initial snapshot {#custom-initial-query}

{{< note >}}This section is relevant only for MySQL, MariaDB, Oracle, PostgreSQL, and SQL Server.
Expand Down