address review comments

Author: taroface

src/current/_includes/molt/fetch-data-load-output.md

@@ -1,5 +1,16 @@
 1. Check the output to observe `fetch` progress. 
 
+	{% if page.name == "migrate-load-replicate.md" %}
+	<section class="filter-content" markdown="1" data-scope="oracle">
+	The following message shows the appropriate values for the `--backfillFromSCN` and `--scn` replication flags to use when [starting Replicator](#start-replicator):
+
+	{% include_cached copy-clipboard.html %}
+	~~~ 
+	replication-only mode should include the following replicator flags: --backfillFromSCN 26685444 --scn 26685786
+	~~~
+	</section>
+	{% endif %}
+
 	A `starting fetch` message indicates that the task has started:
 
 	<section class="filter-content" markdown="1" data-scope="postgres">
@@ -16,7 +27,7 @@
 
 	<section class="filter-content" markdown="1" data-scope="oracle">
 	~~~ json
-	{"level":"info","type":"summary","num_tables":3,"cdc_cursor":"2358840","time":"2025-02-10T14:28:11-05:00","message":"starting fetch"}
+	{"level":"info","type":"summary","num_tables":3,"cdc_cursor":"26685786","time":"2025-02-10T14:28:11-05:00","message":"starting fetch"}
 	~~~
 	</section>
 
@@ -81,15 +92,4 @@
 	~~~ json
 	{"level":"info","type":"summary","fetch_id":"f5cb422f-4bb4-4bbd-b2ae-08c4d00d1e7c","num_tables":3,"tables":["migration_schema.employees","migration_schema.payments","migration_schema.payments"],"cdc_cursor":"2358840","net_duration_ms":6752.847625,"net_duration":"000h 00m 06s","time":"2024-03-18T12:30:37-04:00","message":"fetch complete"}
 	~~~
-	</section>
-
-	{% if page.name == "migrate-load-replicate.md" %}
-	<section class="filter-content" markdown="1" data-scope="oracle">
-	The following message shows the appropriate values for the `--backfillFromSCN` and `--scn` replication flags to use when [starting Replicator](#start-replicator):
-
-	{% include_cached copy-clipboard.html %}
-	~~~ 
-	replication-only mode should include the following replicator flags: --backfillFromSCN 26685444 --scn 26685786
-	~~~
-	</section>
-	{% endif %}
+	</section>

src/current/_includes/molt/migration-prepare-database.md

@@ -129,118 +129,42 @@ GRANT SELECT, FLASHBACK ON migration_schema.tbl TO MIGRATION_USER;
 {% if page.name != "migrate-bulk-load.md" %}
 #### Configure source database for replication
 
-<section class="filter-content" markdown="1" data-scope="postgres">
 {{site.data.alerts.callout_info}}
-Connect to the primary PostgreSQL instance, **not** a read replica. Read replicas cannot create or manage logical replication slots. Verify that you are connected to the primary server by running `SELECT pg_is_in_recovery();` and getting a `false` result.
+Connect to the primary instance (PostgreSQL primary, MySQL primary/master, or Oracle primary), **not** a replica. Replicas cannot provide the necessary replication checkpoints and transaction metadata required for ongoing replication.
 {{site.data.alerts.end}}
 
+<section class="filter-content" markdown="1" data-scope="postgres">
+Verify that you are connected to the primary server by running `SELECT pg_is_in_recovery();` and getting a `false` result.
+
 Enable logical replication by setting `wal_level` to `logical` in `postgresql.conf` or in the SQL shell. For example:
 
 {% include_cached copy-clipboard.html %}
 ~~~ sql
 ALTER SYSTEM SET wal_level = 'logical';
 ~~~
-
-Create a publication for the tables you want to replicate. Do this **before** creating the replication slot. 
-
-To create a publication for all tables:
-
-{% include_cached copy-clipboard.html %}
-~~~ sql
-CREATE PUBLICATION molt_publication FOR ALL TABLES;
-~~~
-
-To create a publication for specific tables:
-
-{% include_cached copy-clipboard.html %}
-~~~ sql
-CREATE PUBLICATION molt_publication FOR TABLE employees, payments, orders;
-~~~
-
-Create a logical replication slot:
-
-{% include_cached copy-clipboard.html %}
-~~~ sql
-SELECT pg_create_logical_replication_slot('molt_slot', 'pgoutput');
-~~~
-
-##### Verify logical replication setup
-
-Verify the publication was created successfully:
-
-{% include_cached copy-clipboard.html %}
-~~~ sql
-SELECT * FROM pg_publication;
-~~~
-
-~~~
-  oid  |     pubname      | pubowner | puballtables | pubinsert | pubupdate | pubdelete | pubtruncate | pubviaroot
--------+------------------+----------+--------------+-----------+-----------+-----------+-------------+------------
- 59084 | molt_publication |       10 | t            | t         | t         | t         | t           | f
-~~~
-
-Verify the replication slot was created:
-
-{% include_cached copy-clipboard.html %}
-~~~ sql
-SELECT * FROM pg_replication_slots;
-~~~
-
-~~~
- slot_name |  plugin  | slot_type | datoid | database | temporary | active | active_pid | xmin | catalog_xmin | restart_lsn | confirmed_flush_lsn | wal_status | safe_wal_size | two_phase
------------+----------+-----------+--------+----------+-----------+--------+------------+------+--------------+-------------+---------------------+------------+---------------+-----------
- molt_slot | pgoutput | logical   |  16385 | molt     | f         | f      |            |      |         2261 | 0/49913A20  | 0/49913A58          | reserved   |               | f
-~~~
 </section>
 
 <section class="filter-content" markdown="1" data-scope="mysql">
 Enable [global transaction identifiers (GTID)](https://dev.mysql.com/doc/refman/8.0/en/replication-options-gtids.html) and configure binary logging. Set `binlog-row-metadata` or `binlog-row-image` to `full` to provide complete metadata for replication.
 
+Configure binlog retention to ensure GTIDs remain available throughout the migration:
+
+- MySQL 8.0.1+: Set `binlog_expire_logs_seconds` (default: 2592000 = 30 days) based on your migration timeline.
+- MySQL < 8.0: Set `expire_logs_days`, or manually manage retention by setting `max_binlog_size` and using `PURGE BINARY LOGS BEFORE NOW() - INTERVAL 1 HOUR` (adjusting the interval as needed). Force binlog rotation with `FLUSH BINARY LOGS` if needed.
+- Managed services: Refer to provider-specific configuration for [Amazon RDS](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/mysql-stored-proc-configuring.html) or [Google Cloud SQL](https://cloud.google.com/sql/docs/mysql/flags#mysql-b).
+
+{% comment %}
 {{site.data.alerts.callout_info}}
-GTID replication sends all database changes to Replicator. To limit replication to specific tables or schemas, use the `--table-filter` and `--schema-filter` flags in the `replicator` command.
+GTID replication sends all database changes to Replicator. To limit replication to specific tables or schemas, use a userscript.
 {{site.data.alerts.end}}
+{% endcomment %}
 
 |  Version   |                                                                                         Configuration                                                                                          |
 |------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
 | MySQL 5.6  | `--gtid-mode=on`<br>`--enforce-gtid-consistency=on`<br>`--server-id={unique_id}`<br>`--log-bin=mysql-binlog`<br>`--binlog-format=row`<br>`--binlog-row-image=full`<br>`--log-slave-updates=ON` |
 | MySQL 5.7  | `--gtid-mode=on`<br>`--enforce-gtid-consistency=on`<br>`--binlog-row-image=full`<br>`--server-id={unique_id}`<br>`--log-bin=log-bin`                                                           |
 | MySQL 8.0+ | `--gtid-mode=on`<br>`--enforce-gtid-consistency=on`<br>`--binlog-row-metadata=full`                                                                                                            |
 | MariaDB    | `--log-bin`<br>`--server_id={unique_id}`<br>`--log-basename=master1`<br>`--binlog-format=row`<br>`--binlog-row-metadata=full`                                                                  |
-
-##### Verify MySQL GTID setup
-
-Get the current GTID set to use as the starting point for replication:
-
-{% include_cached copy-clipboard.html %}
-~~~ sql
--- For MySQL < 8.0:
-SHOW MASTER STATUS;
--- For MySQL 8.0+:
-SHOW BINARY LOG STATUS;
-~~~
-
-~~~
-+---------------+----------+--------------+------------------+-------------------------------------------+
-| File          | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set                         |
-+---------------+----------+--------------+------------------+-------------------------------------------+
-| binlog.000005 |      197 |              |                  | 77263736-7899-11f0-81a5-0242ac120002:1-38 |
-+---------------+----------+--------------+------------------+-------------------------------------------+
-~~~
-
-Use the `Executed_Gtid_Set` value for the `--defaultGTIDSet` flag in MOLT Replicator.
-
-To verify that a GTID set is valid and not purged, use the following queries:
-
-{% include_cached copy-clipboard.html %}
-~~~ sql
--- Verify the GTID set is in the executed set
-SELECT GTID_SUBSET('77263736-7899-11f0-81a5-0242ac120002:1-38', @@GLOBAL.gtid_executed) AS in_executed;
-
--- Verify the GTID set is not in the purged set
-SELECT GTID_SUBSET('77263736-7899-11f0-81a5-0242ac120002:1-38', @@GLOBAL.gtid_purged) AS in_purged;
-~~~
-
-If `in_executed` returns `1` and `in_purged` returns `0`, the GTID set is valid for replication.
 </section>
 
 <section class="filter-content" markdown="1" data-scope="oracle">
@@ -353,11 +277,11 @@ ON
 ~~~
 
 ~~~
-   GROUP# MEMBER                                       START_SCN                END_SCN 
-_________ _________________________________________ ____________ ______________________ 
-        3 /opt/oracle/oradata/ORCLCDB/redo03.log         1232896    9295429630892703743 
-        2 /opt/oracle/oradata/ORCLCDB/redo02.log         1155042                1232896 
-        1 /opt/oracle/oradata/ORCLCDB/redo01.log         1141934                1155042 
+   GROUP# MEMBER                                       START_SCN                END_SCN
+_________ _________________________________________ ____________ ______________________
+        3 /opt/oracle/oradata/ORCLCDB/redo03.log         1232896    9295429630892703743
+        2 /opt/oracle/oradata/ORCLCDB/redo02.log         1155042                1232896
+        1 /opt/oracle/oradata/ORCLCDB/redo01.log         1141934                1155042
 
 3 rows selected.
 ~~~
@@ -377,24 +301,6 @@ CURRENT_SCN
 1 row selected.
 ~~~
 
-##### Get SCNs for replication startup
-
-If you plan to use [initial data load](#start-fetch) followed by [replication](#start-replicator), obtain the correct SCNs **before** starting the initial data load to ensure no active transactions are missed. Run the following queries on the PDB in the order shown:
-
-{% include_cached copy-clipboard.html %}
-~~~ sql
--- Query the current SCN from Oracle
-SELECT CURRENT_SCN FROM V$DATABASE;
-
--- Query the starting SCN of the earliest active transaction
-SELECT MIN(t.START_SCNB) FROM V$TRANSACTION t;
-~~~
-
-Use the results as follows:
-
-- `--scn`: Use the result from the first query (current SCN)
-- `--backfillFromSCN`: Use the result from the second query (earliest active transaction SCN). If the second query returns no results, use the result from the first query instead.
-
 Add the redo log files to LogMiner, using the redo log file paths you queried:
 
 {% include_cached copy-clipboard.html %}

src/current/_includes/molt/molt-connection-strings.md

@@ -4,6 +4,12 @@ Define the connection strings for the [source](#source-connection-string) and [t
 
 The `--source` flag specifies the connection string for the source database:
 
+{% if page.name != "migrate-bulk-load.md" %}
+{{site.data.alerts.callout_info}}
+The source connection **must** point to the primary instance (PostgreSQL primary, MySQL primary/master, or Oracle primary). Replicas cannot provide the necessary replication checkpoints and transaction metadata required for ongoing replication.
+{{site.data.alerts.end}}
+{% endif %}
+
 <section class="filter-content" markdown="1" data-scope="postgres">
 ~~~
 --source 'postgres://{username}:{password}@{host}:{port}/{database}?sslmode=verify-full'
@@ -14,8 +20,6 @@ For example:
 ~~~
 --source 'postgres://migration_user:password@localhost:5432/molt?sslmode=verify-full'
 ~~~
-
-The source connection must point to the PostgreSQL primary instance, not a read replica.
 </section>
 
 <section class="filter-content" markdown="1" data-scope="mysql">

src/current/_includes/molt/molt-install.md

@@ -14,7 +14,9 @@ The following binaries are included:
 - `molt`
 - `replicator`
 
-Both `molt` and `replicator` must be in your current **working directory**.
+{{site.data.alerts.callout_success}}
+For ease of use, keep both `molt` and `replicator` in your current **working directory**.
+{{site.data.alerts.end}}
 
 To display the current version of each binary, run `molt --version` and `replicator --version`.
 

src/current/_includes/molt/molt-limitations.md

@@ -17,9 +17,7 @@
 {% if page.name != "migrate-bulk-load.md" %}
 #### Replicator limitations
 
-<section class="filter-content" markdown="1" data-scope="postgres">
-- Replication modes require write access to the PostgreSQL primary instance. MOLT cannot create replication slots or run replication against a read replica.
-</section>
+- Replication modes require connection to the primary instance (PostgreSQL primary, MySQL primary/master, or Oracle primary). MOLT cannot obtain replication checkpoints or transaction metadata from replicas.
 
 <section class="filter-content" markdown="1" data-scope="mysql">
 - MySQL replication is supported only with [GTID](https://dev.mysql.com/doc/refman/8.0/en/replication-gtids.html)-based configurations. Binlog-based features that do not use GTID are not supported.

src/current/_includes/molt/molt-setup.md

@@ -37,6 +37,32 @@
 
 {% include molt/migration-create-sql-user.md %}
 
+{% if page.name != "migrate-bulk-load.md" %}
+### Configure GC TTL
+
+Before starting the [initial data load](#start-fetch), configure the [garbage collection (GC) TTL]({% link {{ site.current_cloud_version }}/configure-replication-zones.md %}#gc-ttlseconds) on the source CockroachDB cluster to ensure that historical data remains available when replication begins. The GC TTL must be long enough to cover the full duration of the data load.
+
+Increase the GC TTL before starting the data load. For example, to set the GC TTL to 24 hours:
+
+{% include_cached copy-clipboard.html %}
+~~~ sql
+ALTER DATABASE defaultdb CONFIGURE ZONE USING gc.ttlseconds = 86400;
+~~~
+
+{{site.data.alerts.callout_info}}
+The GC TTL duration must be higher than your expected time for the initial data load.
+{{site.data.alerts.end}}
+
+Once replication has started successfully (which automatically protects its own data range), you can restore the GC TTL to its original value. For example, to restore to 5 minutes:
+
+{% include_cached copy-clipboard.html %}
+~~~ sql
+ALTER DATABASE defaultdb CONFIGURE ZONE USING gc.ttlseconds = 300;
+~~~
+
+For details, refer to [Protect Changefeed Data from Garbage Collection]({% link {{ site.current_cloud_version }}/protect-changefeed-data.md %}).
+{% endif %}
+
 ## Configure Fetch
 
 When you run `molt fetch`, you can configure the following options for data load:
@@ -46,7 +72,7 @@ When you run `molt fetch`, you can configure the following options for data load
 - [Table handling mode](#table-handling-mode): Determine how existing target tables are initialized before load.
 - [Schema and table filtering](#schema-and-table-filtering): Specify schema and table names to migrate.
 - [Data load mode](#data-load-mode): Choose between `IMPORT INTO` and `COPY FROM`.
-- [Fetch metrics](#fetch-metrics): Configure metrics collection during the load.
+- [Fetch metrics](#fetch-metrics): Configure metrics collection during initial data load.
 
 ### Connection strings
 

src/current/_includes/molt/molt-troubleshooting.md

@@ -87,9 +87,11 @@ If you shut down `molt` or `replicator` unexpectedly (e.g., with `kill -9` or a
 <section class="filter-content" markdown="1" data-scope="postgres">
 ##### Unable to create publication or slot
 
-This error occurs when the source database does not support logical replication.
+This error occurs when logical replication is not supported.
 
-**Resolution:** Verify that the source database supports logical replication by checking the `wal_level` parameter on PostgreSQL:
+**Resolution:** If you are connected to a replica, connect to the primary instance instead. Replicas cannot create or manage logical replication slots or publications.
+
+Verify that the source database supports logical replication by checking the `wal_level` parameter on PostgreSQL:
 
 {% include_cached copy-clipboard.html %}
 ~~~ sql
@@ -154,23 +156,35 @@ run SELECT pg_create_logical_replication_slot('molt_slot', 'pgoutput'); in sourc
 ~~~ sql
 SELECT pg_create_logical_replication_slot('molt_slot', 'pgoutput');
 ~~~
+</section>
 
 {% if page.name == "migrate-resume-replication.md" %}
 ##### Resuming from stale location
 
-**Resolution:** Clear the `_replicator.memo` table to remove stale LSN (log sequence number) checkpoints:
+<section class="filter-content" markdown="1" data-scope="postgres">
+For PostgreSQL, the replication slot on the source database tracks progress automatically. Clearing the memo table is only necessary if the replication slot was destroyed and you need to restart replication from a specific LSN.
+
+**Resolution:** Clear the `_replicator.memo` table:
+</section>
+
+<section class="filter-content" markdown="1" data-scope="mysql">
+**Resolution:** Clear the `_replicator.memo` table to remove stale GTID checkpoints:
+</section>
+
+<section class="filter-content" markdown="1" data-scope="oracle">
+**Resolution:** Clear the `_replicator.memo` table to remove stale SCN (System Change Number) checkpoints:
+</section>
 
 {% include_cached copy-clipboard.html %}
 ~~~ sql
 DELETE FROM _replicator.memo WHERE true;
 ~~~
 {% endif %}
-</section>
 
 <section class="filter-content" markdown="1" data-scope="mysql">
 ##### Repeated binlog syncing restarts
 
-If Replicator repeatedly restarts binlog syncing, this indicates an invalid or purged GTID.
+If Replicator repeatedly restarts binlog syncing or starts replication from an unexpectedly old location, this indicates an invalid or purged GTID. When an invalid GTID is provided, the binlog syncer will fall back to the first valid GTID.
 
 **Resolution:** Verify the GTID set is valid and **not** purged:
 
@@ -209,6 +223,16 @@ If the GTID is purged or invalid, follow these steps:
     SHOW BINARY LOG STATUS;
     ~~~
 
+    ~~~
+    +---------------+----------+--------------+------------------+-------------------------------------------+
+    | File          | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set                         |
+    +---------------+----------+--------------+------------------+-------------------------------------------+
+    | binlog.000005 |      197 |              |                  | 77263736-7899-11f0-81a5-0242ac120002:1-38 |
+    +---------------+----------+--------------+------------------+-------------------------------------------+
+    ~~~
+
+    Use the `Executed_Gtid_Set` value for the `--defaultGTIDSet` flag.
+
 ##### Invalid GTID format
 
 Invalid GTIDs can occur when GTIDs are purged due to insufficient binlog retention, when connecting to a replica instead of the primary host, or when passing a GTID that has valid format but doesn't exist in the binlog history.
@@ -276,7 +300,7 @@ WARNING: warning during tryCommit: ERROR: duplicate key value violates unique co
 ERROR: maximum number of retries (10) exceeded
 ~~~
 
-**Resolution:** Check target database constraints and connection stability. MOLT Replicator will log warnings for each retry attempt and surface a final error after exhausting all retry attempts, then restart the apply loop to continue processing.
+**Resolution:** Check target database constraints and connection stability. MOLT Replicator will log warnings for each retry attempt. If you see warnings but no final error, the apply succeeded after retrying. If all retry attempts are exhausted, Replicator will surface a final error and restart the apply loop to continue processing.
 
 ##### CockroachDB changefeed connection issues