Skip to content

Commit f3404c7

Browse files
committed
Merge branch 'main' into latest
2 parents fe9420a + 52fdda0 commit f3404c7

File tree

5 files changed

+26
-24
lines changed

5 files changed

+26
-24
lines changed

content/integrate/redis-data-integration/data-pipelines/data-pipelines.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -28,8 +28,8 @@ However, you can also provide your own custom transformation [jobs](#job-files)
2828
for each source table, using your own data mapping and key pattern. You specify these
2929
jobs declaratively with YAML configuration files that require no coding.
3030

31-
The data tranformation involves two separate stages. First, the data ingested by
32-
[Debezium](https://debezium.io/) is automatically transformed to a JSON format. Then,
31+
The data tranformation involves two separate stages. First, the data ingested
32+
during CDC is automatically transformed to a JSON format. Then,
3333
this JSON data gets passed on to your custom transformation for further processing.
3434

3535
You can provide a job file for each source table you want to transform, but you
@@ -48,7 +48,7 @@ data in Redis as
4848

4949
The diagram below shows the flow of data through the pipeline:
5050

51-
{{< image filename="/images/rdi/data-transformation-pipeline.png" >}}
51+
{{< image filename="/images/rdi/RDIPipeDataflow.drawio.svg" >}}
5252

5353
## Pipeline configuration
5454

content/integrate/redis-data-integration/observability.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,7 @@ command from the CLI.{{< /note >}}
3939

4040
## Collector metrics
4141

42-
The endpoint for the collector metrics is `https://<RDI_HOST>:9121/metrics/collector-source`
42+
The endpoint for the collector metrics is `https://<RDI_HOST>/metrics/collector-source`
4343

4444
These metrics are divided into three groups:
4545

@@ -49,7 +49,7 @@ These metrics are divided into three groups:
4949

5050
## Stream processor metrics
5151

52-
The endpoint for the stream processor metrics is `https://<RDI_HOST>:9121/metrics/rdi`
52+
The endpoint for the stream processor metrics is `https://<RDI_HOST>/metrics/rdi`
5353

5454
RDI reports metrics during the two main phases of the ingest pipeline, the *snapshot*
5555
phase and the *change data capture (CDC)* phase. (See the

content/operate/oss_and_stack/stack-with-enterprise/gears-v1/jvm/classes/readers/commandoverrider.md

Lines changed: 15 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -51,26 +51,24 @@ CommandOverrider overrider = new CommandOverrider();
5151
overrider.setCommand("HSET");
5252
// Only override HSET for keys that start with "user:"
5353
overrider.setPrefix("user:");
54-
54+
5555
// Create the data pipe builder
5656
GearsBuilder.CreateGearsBuilder(overrider).map(r-> {
57-
// Extract the key from the command arguments
58-
String keyName = new String((byte[]) r[1], StandardCharsets.UTF_8);
59-
60-
// Add a last_modified timestamp to the user's profile
61-
GearsBuilder.execute("HSET", keyName, "last_modified", Long.toString(System.currentTimeMillis()));
62-
63-
// Get the original HSET arguments
64-
ArrayList<String> commandArray = new ArrayList<String>();
65-
for (int i=1; i < r.length; i++) {
66-
commandArray.add(new String((byte[]) r[i], StandardCharsets.UTF_8));
57+
// Get the operation arguments
58+
ArrayList<String> operationArguments = new ArrayList<String>();
59+
for (int i = 1; i < r.length; i++) {
60+
operationArguments.add(new String((byte[]) r[i], StandardCharsets.UTF_8));
6761
}
68-
69-
// Run the original HSET command
70-
GearsBuilder.callNext(commandArray.toArray(new String[0]));
71-
72-
return "OK";
73-
}).register();
62+
63+
// Add a last-modified field and a corresponding timestamp to the operation arguments
64+
operationArguments.add("last-modified");
65+
operationArguments.add(Long.toString(System.currentTimeMillis()));
66+
67+
// Run the enriched HSET operation
68+
Object reply = GearsBuilder.callNext(operationArguments.toArray(new String[0]));
69+
70+
return reply.toString();
71+
}).register(ExecutionMode.SYNC);
7472
```
7573

7674
After you register the previous code with the `RG.JEXECUTE` command, try to update the user's data with `HSET` to test it:

layouts/home.html

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -94,8 +94,8 @@ <h2 class="text-5xl font-medium" id="operate">Learn</h2>
9494
"LinksLeft" (slice
9595
(dict "Text" "RedisVL" "URL" "./integrate/redisvl/")
9696
(dict "Text" "RedisOM for Python" "URL" "./integrate/redisom-for-python")
97-
(dict "Text" "Ingest" "URL" "./integrate/redis-data-integration/quickstart/ingest-guide/")
98-
(dict "Text" "Write-behind" "URL" "./integrate/redis-data-integration/quickstart/write-behind-guide/")
97+
(dict "Text" "Ingest" "URL" "./integrate/redis-data-integration/")
98+
(dict "Text" "Write-behind" "URL" "./integrate/write-behind/quickstart/write-behind-guide/")
9999
)
100100
) }}
101101
{{ partial "docs-section.html" (dict

static/images/rdi/RDIPipeDataflow.drawio.svg

Lines changed: 4 additions & 0 deletions
Loading

0 commit comments

Comments
 (0)