Skip to content

Commit 95207a2

Browse files
committed
update MOLT Replicator doc & related
1 parent ee821ed commit 95207a2

11 files changed

+422
-357
lines changed

src/current/_includes/molt/migration-prepare-database.md

Lines changed: 36 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
#### Create migration user on source database
22

3-
Create a dedicated migration user (e.g., `MIGRATION_USER`) on the source database. This user is responsible for reading data from source tables during the migration. You will pass this username in the [source connection string](#source-connection-string).
3+
Create a dedicated migration user (for example, `MIGRATION_USER`) on the source database. This user is responsible for reading data from source tables during the migration. You will pass this username in the [source connection string](#source-connection-string).
44

55
<section class="filter-content" markdown="1" data-scope="postgres">
66
{% include_cached copy-clipboard.html %}
@@ -12,11 +12,31 @@ Grant the user privileges to connect, view schema objects, and select the tables
1212

1313
{% include_cached copy-clipboard.html %}
1414
~~~ sql
15-
GRANT CONNECT ON DATABASE source_database TO MIGRATION_USER;
16-
GRANT USAGE ON SCHEMA migration_schema TO MIGRATION_USER;
17-
GRANT SELECT ON ALL TABLES IN SCHEMA migration_schema TO MIGRATION_USER;
18-
ALTER DEFAULT PRIVILEGES IN SCHEMA migration_schema GRANT SELECT ON TABLES TO MIGRATION_USER;
15+
GRANT CONNECT ON DATABASE source_database TO migration_user;
16+
GRANT USAGE ON SCHEMA migration_schema TO migration_user;
17+
GRANT SELECT ON ALL TABLES IN SCHEMA migration_schema TO migration_user;
18+
ALTER DEFAULT PRIVILEGES IN SCHEMA migration_schema GRANT SELECT ON TABLES TO migration_user;
1919
~~~
20+
21+
{% if page.name != "migrate-bulk-load.md" %}
22+
Grant the `SUPERUSER` role to the user (recommended for replication configuration):
23+
24+
{% include_cached copy-clipboard.html %}
25+
~~~ sql
26+
ALTER USER migration_user WITH SUPERUSER;
27+
~~~
28+
29+
Alternatively, grant the following permissions to create replication slots, access replication data, create publications, and add tables to publications:
30+
31+
{% include_cached copy-clipboard.html %}
32+
~~~ sql
33+
ALTER USER migration_user WITH LOGIN REPLICATION;
34+
GRANT CREATE ON DATABASE source_database TO migration_user;
35+
ALTER TABLE migration_schema.table_name OWNER TO migration_user;
36+
~~~
37+
38+
Run the `ALTER TABLE` command for each table to replicate.
39+
{% endif %}
2040
</section>
2141

2242
<section class="filter-content" markdown="1" data-scope="mysql">
@@ -29,9 +49,19 @@ Grant the user privileges to select only the tables you migrate:
2949

3050
{% include_cached copy-clipboard.html %}
3151
~~~ sql
32-
GRANT SELECT ON source_database.* TO MIGRATION_USER@'%';
52+
GRANT SELECT ON source_database.* TO 'migration_user'@'%';
53+
FLUSH PRIVILEGES;
54+
~~~
55+
56+
{% if page.name != "migrate-bulk-load.md" %}
57+
For replication, grant additional privileges for binlog access:
58+
59+
{% include_cached copy-clipboard.html %}
60+
~~~ sql
61+
GRANT REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'migration_user'@'%';
3362
FLUSH PRIVILEGES;
3463
~~~
64+
{% endif %}
3565
</section>
3666

3767
<section class="filter-content" markdown="1" data-scope="oracle">

src/current/_includes/molt/molt-docker.md

Lines changed: 5 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,9 @@
1-
For details on pulling Docker images, refer to [Docker images](#docker-images).
2-
3-
### Performance
1+
#### Performance
42

53
MOLT Fetch, Verify, and Replicator are likely to run more slowly in a Docker container than on a local machine. To improve performance, increase the memory or compute resources, or both, on your Docker container.
64

75
{% if page.name == "molt-fetch.md" %}
8-
### Authentication
6+
#### Authentication
97

108
When using MOLT Fetch with [cloud storage](#bucket-path), it is necessary to specify volumes and environment variables, as described in the following sections for [Google Cloud Storage](#google-cloud-storage) and [Amazon S3](#amazon-s3).
119

@@ -17,7 +15,7 @@ docker run -it cockroachdb/molt fetch ...
1715

1816
For more information on `docker run`, see the [Docker documentation](https://docs.docker.com/reference/cli/docker/container/run/).
1917

20-
#### Google Cloud Storage
18+
##### Google Cloud Storage
2119

2220
If you are using [Google Cloud Storage](https://cloud.google.com/storage/docs/access-control) for [cloud storage](#bucket-path):
2321

@@ -44,7 +42,7 @@ docker run \
4442

4543
For details on Google Cloud Storage authentication, see [Application Default Credentials](https://cloud.google.com/docs/authentication/application-default-credentials).
4644

47-
#### Amazon S3
45+
##### Amazon S3
4846

4947
If you are using [Amazon S3](https://docs.aws.amazon.com/AmazonS3/latest/userguide/security-iam.html) for [cloud storage](#bucket-path):
5048

@@ -61,7 +59,7 @@ docker run \
6159
~~~
6260
{% endif %}
6361

64-
### Local connection strings
62+
#### Local connection strings
6563

6664
When testing locally, specify the host as follows:
6765

src/current/_includes/molt/molt-install.md

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ The following binaries are included:
1515
- `replicator`
1616

1717
{{site.data.alerts.callout_success}}
18-
For ease of use, keep both `molt` and `replicator` in your current **working directory**.
18+
For ease of use, keep both `molt` and `replicator` in your current working directory.
1919
{{site.data.alerts.end}}
2020

2121
To display the current version of each binary, run `molt --version` and `replicator --version`.
@@ -71,6 +71,4 @@ To pull a specific version (for example, `v1.1.1`):
7171
~~~ shell
7272
docker pull cockroachdb/replicator:v1.1.1
7373
~~~
74-
{% endif %}
75-
76-
{% if page.name != "molt.md" %}For details on running in Docker, refer to [Docker usage](#docker-usage).{% endif %}
74+
{% endif %}

src/current/_includes/molt/oracle-migration-prerequisites.md

Lines changed: 2 additions & 49 deletions
Original file line numberDiff line numberDiff line change
@@ -23,52 +23,5 @@ Install Oracle Instant Client on the machine that will run `molt` and `replicato
2323
~~~
2424

2525
{{site.data.alerts.callout_success}}
26-
You can also download Oracle Instant Client directly from the Oracle site for [Linux ARM64](https://www.oracle.com/database/technologies/instant-client/linux-amd64-downloads.html) or [Linux x86-64](https://www.oracle.com/ca-en/database/technologies/instant-client/linux-x86-64-downloads.html).
27-
{{site.data.alerts.end}}
28-
29-
{% if page.name != "migrate-bulk-load.md" %}
30-
#### Enable `ARCHIVELOG`
31-
32-
Enable `ARCHIVELOG` mode on the Oracle database. This is required for Oracle LogMiner, Oracle's built-in changefeed tool that captures DML events for replication.
33-
34-
{% include_cached copy-clipboard.html %}
35-
~~~ sql
36-
SELECT log_mode FROM v$database;
37-
SHUTDOWN IMMEDIATE;
38-
STARTUP MOUNT;
39-
ALTER DATABASE ARCHIVELOG;
40-
ALTER DATABASE OPEN;
41-
SELECT log_mode FROM v$database;
42-
~~~
43-
44-
~~~
45-
LOG_MODE
46-
--------
47-
ARCHIVELOG
48-
49-
1 row selected.
50-
~~~
51-
52-
Enable supplemental primary key logging for logical replication:
53-
54-
{% include_cached copy-clipboard.html %}
55-
~~~ sql
56-
ALTER DATABASE ADD SUPPLEMENTAL LOG DATA (PRIMARY KEY) COLUMNS;
57-
SELECT supplemental_log_data_min, supplemental_log_data_pk FROM v$database;
58-
~~~
59-
60-
~~~
61-
SUPPLEMENTAL_LOG_DATA_MIN SUPPLEMENTAL_LOG_DATA_PK
62-
------------------------- ------------------------
63-
IMPLICIT YES
64-
65-
1 row selected.
66-
~~~
67-
68-
Enable `FORCE_LOGGING` to ensure that all data changes are captured for the tables to migrate:
69-
70-
{% include_cached copy-clipboard.html %}
71-
~~~ sql
72-
ALTER DATABASE FORCE LOGGING;
73-
~~~
74-
{% endif %}
26+
You can also download Oracle Instant Client directly from the Oracle site for [Linux ARM64](https://www.oracle.com/database/technologies/instant-client/linux-amd64-downloads.html) or [Linux x86-64](https://www.oracle.com/ca-en/database/technologies/instant-client/linux-x86-64-downloads.html).
27+
{{site.data.alerts.end}}

src/current/_includes/molt/replicator-flags-usage.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -63,7 +63,7 @@ When using `--table-filter`, you must also include `--userscript`. Refer to [Tab
6363
| `--tlsPrivateKey` | Path to the server TLS private key for the webhook sink. Refer to [TLS certificate and key](#tls-certificate-and-key). |
6464
| `--metricsAddr` | Enable Prometheus metrics at a specified `{host}:{port}`. Metrics are served at `http://{host}:{port}/_/varz`. |
6565

66-
- Failback requires `--stagingSchema`, which specifies the staging database name used for replication checkpoints and metadata. The staging schema is first created with [`--stagingCreateSchema`]({% link molt/migrate-load-replicate.md %}). For details on the staging schema, refer to [Staging schema]({% link molt/molt-replicator.md %}#staging-schema).
66+
- Failback requires `--stagingSchema`, which specifies the staging database name used for replication checkpoints and metadata. The staging schema is first created with [`--stagingCreateSchema`]({% link molt/migrate-load-replicate.md %}). For details on the staging schema, refer to [Staging database]({% link molt/molt-replicator.md %}#staging-database).
6767

6868
- When configuring a [secure changefeed](#tls-certificate-and-key) for failback, you **must** include `--tlsCertificate` and `--tlsPrivateKey`, which specify the paths to the server certificate and private key for the webhook sink connection.
6969

0 commit comments

Comments
 (0)