Skip to content

Commit f88d47b

Browse files
DOC-5282 started job file restructure
1 parent 2816f49 commit f88d47b

File tree

1 file changed

+31
-0
lines changed
  • content/integrate/redis-data-integration/data-pipelines/transform-examples

1 file changed

+31
-0
lines changed

content/integrate/redis-data-integration/data-pipelines/transform-examples/_index.md

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,37 @@ configuration that controls the transformation for a particular table from the s
2424
database. You can also add a `default-job.yaml` file to provide
2525
a default transformation for tables that don't have a specific job file of their own.
2626

27+
## Change events
28+
29+
RDI captures changes to rows in the source database as *change events*. A change event
30+
is a JSON object with fields that describe which row was changed and how. A
31+
transformation job receives the change event as input and specifies how to transform
32+
it into the target data structure in Redis.
33+
34+
The job file can access the change event in either a `full` or a `partial` format. The
35+
fields of the `full` format are:
36+
37+
- `key`: An object containing the attributes of the primary key as fields. For example,
38+
you could access the value of a primary key column called `id` using `key.id`.
39+
- `before`: An object containing the value of the row before the change occurred.
40+
- `after`: An object containing the current value of the row, after the change.
41+
- `opcode`: A single character string that indicates the type of operation that generated the change,
42+
such as creation, update, or deletion. See
43+
[Using the operation code]({{< relref "/integrate/redis-data-integration/data-pipelines/transform-examples/redis-opcode-example" >}})
44+
for more information about the possible opcode values and how to use them.
45+
- `db`: The name of the source database.
46+
- `table`: The name of the table that contains the changed row.
47+
- `schema`: The name of the database schema.
48+
49+
{{< note >}} The `db` and `schema` fields are database-specific and may not be available in all databases. For example, MySQL doesn't use `schema` and uses `db` as the database name.
50+
{{< /note >}}
51+
52+
The `partial` format contains just the current value of the row, after the change. It
53+
is equivalent to the `after` field of the `full` format, but lets you omit the `after`
54+
prefix when you access a field.
55+
56+
## Job file format
57+
2758
The job files have a structure like the following example. This configures a default
2859
job that:
2960

0 commit comments

Comments
 (0)