RDSC RDSC-3626 - adding opcode examples

content/integrate/redis-data-integration/data-pipelines/transform-examples/redis-opcode-example.md

@@ -1,5 +1,5 @@
 ---
-Title: Add the opcode to the Redis output
+Title: Using the operation code
 alwaysopen: false
 categories:
 - docs
@@ -8,43 +8,91 @@ categories:
 - rdi
 description: null
 group: di
-linkTitle: Add the opcode to the Redis output
+linkTitle: Using the operation code
 summary: Redis Data Integration keeps Redis in sync with the primary database in near
   real time.
 type: integration
 weight: 100
 ---
 
-In the example below, the data is captured from the source table named `employee` and is written to the Redis database as a JSON document. When you specify the `data_type` parameter for the job, it overrides the system-wide setting `target_data_type` defined in `config.yaml`. 
+The operation code (`opcode`) is a metadata field that indicates the type of operation that generated the change in the source database. It can be useful for tracking changes and understanding the context of the data being processed.
+
+The opcode is only available in the [full row format]({{< relref "/integrate/redis-data-integration/data-pipelines/transform-examples/redis-row-format#full" >}}), and can be accessed in the `transform` and `output` sections of the job file.
+
+It has one of the following values:
+
+- r - Read (applies to only snapshots)
+- c - Create
+- u - Update
+- d - Delete
+- t = truncate (PostgreSQL specific)
+- m = message (PostgreSQL specific)
+
+
+You can both add the value of the operation code to the output, and use it as a conditional expression to modify the behaviour of the job. The following set of examples will demonstrate different use-cases.
+
+### Adding the operation code to the output
+
+
+Use the `add_field` transformation to add a new field called that contains the value of the `opcode` field from the source data. Note, that the fields must be prefixed with `after.` to be included in the output.
 
-Here, the result will be Redis JSON documents with fields captured from the source table
-(`employeeid`, `firstname`, `lastname`) and also with
-an extra field `my_opcode` added using the `merge` update strategy (see the
-[JSON job example]({{< relref "/integrate/redis-data-integration/data-pipelines/transform-examples/redis-json-example" >}})
-for more information). The `opcode` expression refers to the operation code captured from
-the source. This is a database-specific value that indicates which type of operation generated
-the change (insert, update, etc).
 
 ```yaml
 source:
   schema: public
   table: employee
   row_format: full
+
 transform:
+  # add the operation code to the data
   - uses: add_field
     with:
-      field: after.my_opcode
+      field: after.operation_code
       expression: opcode
       language: jmespath
-output:
-  - uses: redis.write
+```
+
+
+### Filtering operation by output code.
+
+In some cases you want to ignore certain operations, for example, you may want to ignore delete operations. You can use the `filter` transformation to filter out those operations.
+
+```yaml
+source:
+  schema: public
+  table: employee
+  row_format: full
+
+transform:
+  - uses: filter
+    with:
+      expression: opcode != 'd'
+      language: jmespath
+```
+
+### Modifying the output based on the operation code
+
+While the previous example filters out the operations, you can also modify the output based on the operation code. For example, you can add a new field showing the status of the record based on the operation code.
+
+Please note that in the case of deletion of the source record, you also need to modify the value of the `opcode` field, to prevent the automatic removal of the record from the target database.
+
+```yaml
+source:
+  schema: public
+  table: employee
+  row_format: full
+
+transform:
+  - uses: add_field
     with:
-      data_type: json
-      mapping:
-        - employeeid
-        - firstname
-        - lastname
-        - my_opcode
-      connection: target
-      on_update: merge
-```
+      fields:
+        # Here you set the value of the field based on the value of the opcode field
+        - field: after.status
+          expression: opcode == 'd' && 'inactive' || 'active'
+          language: jmespath
+
+        # You have to change the value of the opcode field to prevent deletion
+        - field: opcode
+          expression: opcode == 'd' && 'u' || opcode
+          language: jmespath
+```

content/integrate/redis-data-integration/data-pipelines/transform-examples/redis-row-format.md

@@ -63,7 +63,7 @@ With `row_format: full` the input value is a JSON object with the following stru
 - `key` - An object containing the attributes of the primary key. For example, `key.id` will give you the value of the `id` column as long as it is part of the primary key.
 - `before` - An object containing the previous value of the row.
 - `after` - An object containing the current value of the row.
-- `opcode` - The operation code. Different databases use different values for the operation code. See [operation code values]({{< relref "#operation-codes" >}}) below for more information.
+- `opcode` - The operation code. See the [using the operation code]({{< relref "/integrate/redis-data-integration/data-pipelines/transform-examples/redis-opcode-example" >}}) page for more information of the possible values and usage examples.
 - `db` - The database name.
 - `table` - The table name.
 - `schema` - The schema name. 
@@ -102,12 +102,3 @@ output:
         expression: concat(['addresses-full', '#', values(key)[0]])
         language: jmespath
 ```
-
-## Operation code values {#operation-codes}
-
-- r - Read (applies to only snapshots)
-- c - Create
-- u - Update
-- d - Delete
-- t = truncate (PostgreSQL specific)
-- m = message (PostgreSQL specific)