Skip to content

Commit 99e56be

Browse files
rrelledgerrelledge
committed
Merge branch 'main' into release-rs-fuya-fuya-minor
2 parents 0566796 + 19baab7 commit 99e56be

File tree

45 files changed

+14275
-56
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

45 files changed

+14275
-56
lines changed

‎.github/workflows/rc_api_sync.yaml‎

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
name: rc_api_sync
2+
3+
on:
4+
schedule:
5+
- cron: '0 0 * * *' # run every day at midnight UTC time
6+
workflow_dispatch: # or run on manual trigger
7+
8+
jobs:
9+
rc_api_sync:
10+
runs-on: ubuntu-latest
11+
permissions:
12+
contents: write
13+
pull-requests: write
14+
actions: write
15+
steps:
16+
- name: 'Checkout'
17+
uses: 'actions/checkout@v3'
18+
19+
- name: 'Fetch openapi json file'
20+
env:
21+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
22+
run: |
23+
branch="rc_api_sync"
24+
spec_change=false
25+
26+
# check if remote branch already exists
27+
git fetch --all
28+
set +e
29+
git ls-remote --exit-code --heads origin "refs/heads/${branch}"
30+
if [ "$?" -eq 0 ]; then
31+
set -e
32+
# if it does, create local branch off existing remote branch
33+
git checkout -b "${branch}" "origin/${branch}"
34+
git branch --set-upstream-to="origin/${branch}" "${branch}"
35+
git pull
36+
else
37+
set -e
38+
# otherwise, create local branch from main
39+
git checkout -b "${branch}"
40+
fi
41+
42+
curl -Ls https://api.redislabs.com/v1/cloud-api-docs \
43+
| jq '(.. | .example? | try select(test("^{"))) |= fromjson' > content/operate/rc/api/api-reference/openapi.json
44+
45+
spec_is_different=$(git diff content/operate/rc/api/api-reference/openapi.json)
46+
47+
if [[ ! -z $spec_is_different ]]; then
48+
spec_change=true
49+
50+
git add "content/operate/rc/api/api-reference/openapi.json"
51+
git config user.email "177626021+redisdocsapp[bot]@users.noreply.github.com"
52+
git config user.name "redisdocsapp[bot]"
53+
git commit -m "Update content/operate/rc/api/api-reference/openapi.json"
54+
fi
55+
56+
if [ "$spec_change" = true ] ; then
57+
git push origin "${branch}"
58+
59+
# If a pr is not already open, create one
60+
set +e
61+
gh search prs -R redis/docs --state open --match title "update rc openapi spec" | grep -q "update rc openapi spec"
62+
if [ "$?" -eq 1 ]; then
63+
set -e
64+
gh pr create \
65+
--body "update rc openapi spec" \
66+
--title "update rc openapi spec" \
67+
--head "$branch" \
68+
--base "main"
69+
fi
70+
fi

‎content/apis/_index.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,7 @@ Redis Cloud is a fully managed Database as a Service offering and the fastest wa
4545

4646
- [Redis Cloud REST API introduction]({{< relref "/operate/rc/api/" >}})
4747
- [Redis Cloud REST API examples]({{< relref "/operate/rc/api/examples/" >}})
48-
- [Redis Cloud REST API reference](https://api.redislabs.com/v1/swagger-ui.html)
48+
- [Redis Cloud REST API reference]({{< relref "/operate/rc/api/api-reference" >}})
4949

5050

5151
### Redis Enterprise Software API

‎content/commands/client-list/index.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -114,6 +114,8 @@ Here is the meaning of the fields:
114114
* `user`: the authenticated username of the client
115115
* `redir`: client id of current client tracking redirection
116116
* `resp`: client RESP protocol version. Added in Redis 7.0
117+
* `rbp`: peak size of the client's read buffer since the client connected. Added in Redis 7.0
118+
* `rbs`: current size of the client's read buffer in bytes. Added in Redis 7.0
117119

118120
The client flags can be a combination of:
119121

‎content/commands/latency-histogram/index.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -54,9 +54,9 @@ Each histogram consists of the following fields:
5454
* Each bucket represents a latency range
5555
* Each bucket covers twice the previous bucket's range
5656
* Empty buckets are excluded from the reply
57-
* The tracked latencies are between 1 microsecond and roughly 1 second
57+
* The tracked latencies are between 1 nanosecond and roughly 1 second
5858
* Everything above 1 second is considered +Inf
59-
* At max, there will be log2(1,000,000,000)=30 buckets
59+
* At max, there will be log2(1,000,000,000) = 30 buckets
6060

6161
This command requires the extended latency monitoring feature to be enabled, which is the default.
6262
If you need to enable it, call `CONFIG SET latency-tracking yes`.

‎content/embeds/rdi-supported-source-versions.md‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
| Database | Versions | AWS RDS Versions | GCP SQL Versions |
2+
| :-- | :-- | :-- | :-- |
3+
| Oracle | 12c, 19c, 21c | 19c, 21c | - |
4+
| MariaDB | 10.5, 11.4.3 | 10.4 to 10.11, 11.4.3 | - |
5+
| MySQL | 5.7, 8.0.x, 8.2 | 8.0.x | 8.0 |
6+
| PostgreSQL | 10, 11, 12, 13, 14, 15, 16 | 11, 12, 13, 14, 15, 16 | 15 |
7+
| SQL Server | 2017, 2019, 2022 | 2016, 2017, 2019, 2022 | 2019 |
8+
| AlloyDB for PostgreSQL | 14.2, 15.7 | - | 14.2, 15.7 |
9+
| AWS Aurora/PostgreSQL | 15 | 15 | - |

‎content/integrate/prometheus-with-redis-enterprise/observability.md‎

Lines changed: 632 additions & 0 deletions
Large diffs are not rendered by default.

‎content/integrate/redis-data-integration/_index.md‎

Lines changed: 1 addition & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -106,18 +106,7 @@ You should *not* use RDI when:
106106

107107
RDI can capture data from any of the following sources:
108108

109-
| Database | Versions |
110-
| :-------------------------- | :--------------------- |
111-
| Oracle | 12c, 19c, 21c |
112-
| MariaDB | >= 10.5 |
113-
| MySQL | 5.7, 8.0.x |
114-
| Postgres | 10, 11, 12, 13, 14, 15 |
115-
| SQL Server | 2017, 2019 |
116-
| Google Cloud SQL MySQL | 8.0 |
117-
| Google Cloud SQL Postgres | 15 |
118-
| Google Cloud SQL SQL Server | 2019 |
119-
| Google Cloud AlloyDB for PostgreSQL | |
120-
109+
{{< embed-md "rdi-supported-source-versions.md" >}}
121110

122111
## Documentation
123112

‎content/integrate/redis-data-integration/architecture.md‎

Lines changed: 1 addition & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -89,17 +89,7 @@ the backpressure mechanism.
8989

9090
RDI supports the following database sources using [Debezium Server](https://debezium.io/documentation/reference/stable/operations/debezium-server.html) connectors:
9191

92-
| Database | Versions |
93-
| --------------------------- | ---------------------- |
94-
| Oracle | 12c, 19c, 21c |
95-
| MariaDB | >= 10.5 |
96-
| MySQL | 5.7, 8.0.x |
97-
| Postgres | 10, 11, 12, 13, 14, 15 |
98-
| SQL Server | 2017, 2019 |
99-
| Google Cloud SQL MySQL | 8.0 |
100-
| Google Cloud SQL Postgres | 15 |
101-
| Google Cloud SQL SQL Server | 2019 |
102-
| Google Cloud AlloyDB for PostgreSQL | |
92+
{{< embed-md "rdi-supported-source-versions.md" >}}
10393

10494
## How RDI is deployed
10595

‎content/integrate/redis-data-integration/data-pipelines/prepare-dbs/aws-aur-pgsql.md‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
---
2+
Title: Prepare AWS Aurora and PostgreSQL for RDI
3+
aliases: /integrate/redis-data-integration/ingest/data-pipelines/prepare-dbs/my-sql-mariadb/
4+
alwaysopen: false
5+
categories:
6+
- docs
7+
- integrate
8+
- rs
9+
- rdi
10+
description: Prepare AWS Aurora/PostgreSQL databases to work with RDI
11+
group: di
12+
linkTitle: Prepare AWS Aurora/PostgreSQL
13+
summary: Redis Data Integration keeps Redis in sync with the primary database in near
14+
real time.
15+
type: integration
16+
weight: 5
17+
---
18+
19+
Follow the steps in the sections below to prepare an
20+
[AWS Aurora PostgreSQL](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/CHAP_GettingStartedAurora.CreatingConnecting.AuroraPostgreSQL.html)
21+
database to work with RDI.
22+
23+
## 1. Create a parameter group
24+
25+
In the [Relational Database Service (RDS) console](https://console.aws.amazon.com/rds/),
26+
navigate to **Parameter groups > Create parameter group**. You will see the panel shown
27+
below:
28+
29+
{{<image filename="images/rdi/ingest/prepsrc/aurora-pgsql/CreateParamGroup.jpg" alt="Create parameter group panel" >}}
30+
31+
Enter the following information:
32+
33+
| Name | Value |
34+
| :-- | :-- |
35+
| **Parameter group name** | rdi-aurora-pg |
36+
| **Description** | Enable logical replication for RDI |
37+
| **Engine Type** | Aurora PostgreSQL |
38+
| **Parameter group family** | aurora-postgresql15 |
39+
| **Type** | DB Cluster Parameter Group |
40+
41+
Select **Create** to create the parameter group.
42+
43+
## 2. Edit the parameter group
44+
45+
Navigate to **Parameter groups** in the console. Select the `rdi-aurora-pg`
46+
group you have just created and then select **Edit** . You will see this panel:
47+
48+
{{<image filename="images/rdi/ingest/prepsrc/aurora-pgsql/EditParamGroup.jpg" alt="Edit parameter group panel" >}}
49+
50+
Search for the `rds.logical_replication` parameter and set its value to 1. Then,
51+
select **Save Changes**.
52+
53+
## 3. Select the new parameter group
54+
55+
Scroll down to **Additional Configuration**. Set the **DB Cluster Parameter Group**
56+
to the value `rdi-aurora-pg` that you have just added:
57+
58+
{{<image filename="images/rdi/ingest/prepsrc/aurora-pgsql/CreateDB6.jpg" alt="Additional Configuration panel" >}}

‎content/integrate/redis-data-integration/installation/install-k8s.md‎

Lines changed: 22 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,8 @@ information. *This requires Redis Enterprise v6.4 or greater*.
6565
[eviction policy]({{< relref "/operate/rs/databases/memory-performance/eviction-policy" >}}) to `noeviction` and set
6666
[data persistence]({{< relref "/operate/rs/databases/configure/database-persistence" >}})
6767
to AOF - fsync every 1 sec.
68+
- **Ensure that the RDI database is not clustered.** RDI will not work correctly if the
69+
RDI database is clustered, but it is OK for the target database to be clustered.
6870

6971
You should then provide the details of this database in the [`values.yaml`](#the-valuesyaml-file)
7072
file as described below.
@@ -95,20 +97,22 @@ To pull images from a local registry, you must provide the image pull secret and
9597
9698
## Install the RDI Helm chart
9799
98-
1. Decompress the tar file:
100+
1. Scaffold the default `values.yaml` file from the chart into a local
101+
`rdi-values.yaml` file:
99102

100103
```bash
101-
tar -xvf rdi-<rdi-tag>.tar.gz
104+
helm show values rdi-<rdi-tag>.tar.gz > rdi-values.yaml
102105
```
103106

104-
1. Open the `values.yaml` file and set the appropriate values for your installation
107+
1. Open the `rdi-values.yaml` file you just created and set the appropriate
108+
values for your installation
105109
(see [The `values.yaml` file](#the-valuesyaml-file) below for the full set of
106110
available values).
107111

108112
1. Start the installation:
109113

110114
```bash
111-
helm install <The logical chart name> ./rdi --create-namespace -n rdi
115+
helm install rdi rdi-<rdi-tag>.tar.gz -f rdi-values.yaml
112116
```
113117

114118
### The `values.yaml` file
@@ -117,8 +121,18 @@ The annotated [`values.yaml`](https://helm.sh/docs/topics/charts/#templates-and-
117121
file below describes the values you can set for the RDI Helm installation.
118122

119123
At a minimum, you must set the values of `RDI_REDIS_HOST` and `RDI_REDIS_PORT`
120-
in the `global.rdiSysConfig` section and also `RDI_REDIS_PASSWORD` in
121-
`global.rdiSysSecret` to enable the basic connection to the RDI database.
124+
in the `global.rdiSysConfig` section and also `RDI_REDIS_PASSWORD` and
125+
`JWT_SECRET_KEY` in `global.rdiSysSecret` to enable the basic connection to the
126+
RDI database. RDI uses the value in `JWT_SECRET_KEY` to encrypt the
127+
[JSON web token (JWT)](https://jwt.io/) token used by RDI API. Best practice is
128+
to generate a value containing 32 random bytes of data (equivalent to 256
129+
bits) and then encode this value as ASCII characters. Use the following
130+
command to generate the random key from the
131+
[`urandom` special file](https://en.wikipedia.org/wiki//dev/random):
132+
133+
```bash
134+
head -c 32 /dev/urandom | base64
135+
```
122136

123137
{{< note >}}If you want to use
124138
[Redis Insight]({{< relref "/develop/tools/insight/rdi-connector" >}})
@@ -220,7 +234,8 @@ global:
220234
# RDI_REDIS_KEY_PASSPHRASE: ""
221235
222236
# The key used to encrypt the JWT token used by RDI API. Best practice is for this
223-
# to contain 32 ASCII characters (equivalent to 256 bits of data).
237+
# to contain 32 random bytes encoded as ASCII characters (equivalent to 256 bits of
238+
# data). See `The values.yaml file` section above to learn how to generate the key.
224239
# JWT_SECRET_KEY: ""
225240

226241
rdiDbSSLSecret:

0 commit comments

Comments
 (0)