Skip to content

DOC-5183, DOC-5170, DOC-5192 Astra Migration Toolkit, Migration index page, more Sideloader integration and clean up #195

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 23 commits into from
May 12, 2025
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions modules/ROOT/images/icons/astra-db-cloud-backup-restores.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
11 changes: 11 additions & 0 deletions modules/ROOT/images/icons/loading-data-into-astra-db.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
10 changes: 10 additions & 0 deletions modules/ROOT/images/icons/migrating-apps.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
6 changes: 6 additions & 0 deletions modules/ROOT/images/icons/what-is-astra-db.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
58 changes: 30 additions & 28 deletions modules/ROOT/nav.adoc
Original file line number Diff line number Diff line change
@@ -1,41 +1,41 @@
.{product}
* xref:introduction.adoc[]
* xref:components.adoc[]
* xref:ROOT:introduction.adoc[]
* xref:ROOT:components.adoc[]
* Planning
** xref:preliminary-steps.adoc[]
** xref:feasibility-checklists.adoc[]
** xref:deployment-infrastructure.adoc[]
** xref:create-target.adoc[]
** xref:rollback.adoc[]
** xref:ROOT:feasibility-checklists.adoc[]
** xref:ROOT:deployment-infrastructure.adoc[]
** xref:ROOT:create-target.adoc[]
** xref:ROOT:rollback.adoc[]
* Phase 1
** xref:phase1.adoc[]
** xref:setup-ansible-playbooks.adoc[]
** xref:deploy-proxy-monitoring.adoc[]
** xref:tls.adoc[]
** xref:connect-clients-to-proxy.adoc[]
** xref:metrics.adoc[]
** xref:manage-proxy-instances.adoc[]
** xref:ROOT:phase1.adoc[]
** xref:ROOT:setup-ansible-playbooks.adoc[]
** xref:ROOT:deploy-proxy-monitoring.adoc[]
** xref:ROOT:tls.adoc[]
** xref:ROOT:connect-clients-to-proxy.adoc[]
** xref:ROOT:metrics.adoc[]
** xref:ROOT:manage-proxy-instances.adoc[]
* Phase 2
** xref:migrate-and-validate-data.adoc[]
** xref:cassandra-data-migrator.adoc[{cass-migrator}]
** xref:dsbulk-migrator.adoc[{dsbulk-migrator}]
** xref:ROOT:migrate-and-validate-data.adoc[]
** xref:ROOT:cassandra-data-migrator.adoc[]
** xref:ROOT:dsbulk-migrator.adoc[]
** xref:sideloader:sideloader-zdm.adoc[]
* Phase 3
** xref:enable-async-dual-reads.adoc[]
** xref:ROOT:enable-async-dual-reads.adoc[]
* Phase 4
** xref:change-read-routing.adoc[]
** xref:ROOT:change-read-routing.adoc[]
* Phase 5
** xref:connect-clients-to-target.adoc[]
** xref:ROOT:connect-clients-to-target.adoc[]
* References
** xref:troubleshooting-tips.adoc[]
** xref:troubleshooting-scenarios.adoc[]
** xref:contributions.adoc[]
** xref:faqs.adoc[]
** xref:glossary.adoc[]
** xref:ROOT:troubleshooting-tips.adoc[]
** xref:ROOT:troubleshooting-scenarios.adoc[]
** xref:ROOT:contributions.adoc[]
** xref:ROOT:faqs.adoc[]
** xref:ROOT:glossary.adoc[]
** https://github.com/datastax/zdm-proxy/releases[{product-proxy} release notes]
** https://github.com/datastax/zdm-proxy-automation/releases[{product-automation} release notes]

.{cass-migrator}
* xref:cdm-overview.adoc[{cass-migrator}]
* xref:ROOT:cdm-overview.adoc[]
* https://github.com/datastax/cassandra-data-migrator/releases[{cass-migrator-short} release notes]

.{dsbulk-loader}
Expand All @@ -60,7 +60,9 @@

.Product-specific migration paths
* {astra-db}
** xref:astra-db-serverless:databases:migration-path-serverless.adoc[]
** xref:ROOT:astra-migration-paths.adoc[]
** https://www.datastax.com/products/datastax-astra/migration-toolkit[{astra} Migration Toolkit]
** xref:astra-db-serverless:databases:migration-path-serverless.adoc[Migrate to {astra-db}]
* {dse}
** {dse-short} 6.9
*** xref:6.9@dse:tooling:migration-path-dse.adoc[{dse-short} 6.9 migration tools]
Expand All @@ -72,4 +74,4 @@
*** xref:5.1@dse:managing:operations/migrate-data.adoc[Migrate data to {dse-short} 5.1]
* {mc}
** xref:mission-control:migrate:oss-cass-to-mission-control.adoc[Migrate {cass-short} clusters to {mc-short}]
** xref:mission-control:migrate:dse-to-mission-control.adoc[[Migrate {cass-short} clusters to {dse-short}]
** xref:mission-control:migrate:dse-to-mission-control.adoc[Migrate {cass-short} clusters to {dse-short}]
98 changes: 98 additions & 0 deletions modules/ROOT/pages/astra-migration-paths.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
= {astra-db} migration paths
:description: Learn which migration tools are compatible with your origin cluster.

The https://www.datastax.com/products/datastax-astra/migration-toolkit[{astra} Migration Toolkit] is designed to help you migrate your data to {astra-db}.

Use this matrix to choose a tool that is compatible with your current database provider or service.

[cols="2,1,1,1,1"]
|===
|Origin |{sstable-sideloader} |{cass-migrator} |{product-proxy} |{dsbulk-loader}

|Aiven for {cass-short}
|✅
|✅
|✅
|✅

|Amazon Keyspaces
|❌
|✅
|✅
|✅

|{cass-reg} OSS 3.11 or later
|✅
|✅
|✅
|✅

|{cass-reg} OSS 3.10 or earlier
|❌
|✅
|✅
|✅

|Azure Cosmos DB ({cass-short} API)
|❌
|✅
|✅
|✅

|Azure Managed Instance for {cass}
|✅
|✅
|✅
|✅

|{dse-short} 5.1 or later
|✅
|✅
|✅
|✅

|{dse-short} 5.0 or earlier
|❌
|✅
|✅
|✅

|Instaclustr Managed {cass-short}
|✅
|✅
|✅
|✅

|K8ssandra (self-managed)
|✅
|✅
|✅
|✅

|Scylla Cloud
|❌
|✅
|✅
|✅

|Scylla OSS or Enterprise
|❌
|✅
|✅
|✅

|Yugabyte Aeon (YCQL)
|❌
|✅
|✅
|✅

|Yugabyte OSS or Anywhere
|❌
|✅
|✅
|✅

|===

If you have questions about migrating from a specific source to {astra-db}, contact your {company} account representative, {support-url}[{company} Support], or an https://www.datastax.com/products/datastax-astra/migration-toolkit[{astra} Migration Toolkit expert].
4 changes: 3 additions & 1 deletion modules/ROOT/pages/cassandra-data-migrator.adoc
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
= {cass-migrator}
= Use {cass-migrator} with {product-short}
:navtitle: Use {cass-migrator}
:description: Use {cass-migrator} to migrate data with {product-short}
:page-aliases: cdm-parameters.adoc, ROOT:cdm-steps.adoc

//This page was an exact duplicate of cdm-overview.adoc and the (now deleted) cdm-steps.adoc, they are just in different parts of the nav.
Expand Down
1 change: 1 addition & 0 deletions modules/ROOT/pages/components.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,7 @@ For details, see:

* xref:setup-ansible-playbooks.adoc[]
* xref:deploy-proxy-monitoring.adoc[]
* https://github.com/datastax/zdm-proxy-automation[{product-automation} GitHub repository]

== Data migration tools

Expand Down
6 changes: 5 additions & 1 deletion modules/ROOT/pages/create-target.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -128,4 +128,8 @@ Make sure that all keyspaces and tables being migrated are identical to the corr

* To copy the schema, you can run CQL `describe` on the origin cluster to get the schema that is being migrated, and then run the output on your new cluster.
Bear in mind that, if you are migrating from an old version, you may need to adapt some CQL clauses that are no longer supported in newer versions (e.g. `COMPACT STORAGE`).
Please refer to the documentation of the relevant versions for more information.
Please refer to the documentation of the relevant versions for more information.

== Next steps

* xref:ROOT:rollback.adoc[]
6 changes: 6 additions & 0 deletions modules/ROOT/pages/deployment-infrastructure.adoc
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
= Deployment and infrastructure considerations
:page-tag: migration,zdm,zero-downtime,zdm-proxy,deploy,infrastructure

As part of planning your migration, you need to prepare your infrastructure.

== Choosing where to deploy the proxy

A typical {product-proxy} deployment is made up of multiple proxy instances.
Expand Down Expand Up @@ -153,3 +155,7 @@ Likewise, connecting to any {product-proxy} instance is as easy as this (replaci
----
ssh -F zdm_ssh_config zdm-proxy-0
----

== Next steps

* xref:ROOT:create-target.adoc[]
18 changes: 16 additions & 2 deletions modules/ROOT/pages/dsbulk-migrator.adoc
Original file line number Diff line number Diff line change
@@ -1,7 +1,21 @@
= {dsbulk-migrator}
= Use {dsbulk-migrator} with {product-short}
:navtitle: Use {dsbulk-migrator}
:description: Use {dsbulk-migrator} to migrate data with {product-short}.

Use {dsbulk-migrator} to perform simple migration of smaller data quantities, where data validation (other than post-migration row counts) is not necessary.

//when you can shard data from table rows into more manageable quantities.

{dsbulk-migrator} provides the following commands:

* `migrate-live`: Start a live data migration using the embedded version of {dsbulk-loader} or your own {dsbulk-loader} installation.
A live migration means that the data migration starts immediately and is performed by this migrator tool through the specified {dsbulk-loader} installation.

* `generate-script`: Generate a migration script that you can execute to perform a data migration with a your own {dsbulk-loader} installation.
This command _doesn't_ trigger the migration; it only generates the migration script that you must then execute.

* `generate-ddl`: Read the schema from origin, and then generate CQL files to recreate it in your target {astra-db} database.

[[prereqs-dsbulk-migrator]]
== {dsbulk-migrator} prerequisites

Expand All @@ -13,7 +27,7 @@ Use {dsbulk-migrator} to perform simple migration of smaller data quantities, wh
[[building-dsbulk-migrator]]
== Building {dsbulk-migrator}

Building {dsbulk-migrator} is accomplished with Maven. First, clone the git repo to your local machine.
Building {dsbulk-migrator} is accomplished with Maven. First, clone the https://github.com/datastax/dsbulk-migrator[{dsbulk-migrator} repository] to your local machine.
Example:

[source,bash]
Expand Down
5 changes: 5 additions & 0 deletions modules/ROOT/pages/feasibility-checklists.adoc
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
= Feasibility checks
:page-tag: migration,zdm,zero-downtime,zdm-proxy,feasibility
:page-aliases: ROOT:preliminary-steps.adoc

Before starting your migration, refer to the following considerations to ensure that your client application workload and xref:glossary.adoc#origin[**Origin**] are suitable for this {product} process.

Expand Down Expand Up @@ -245,3 +246,7 @@ The default retry policy retries all requests in case of a connection error *reg
Prior to version 2.5.0, this driver did *NOT* retry any requests after they have been written to the socket, it was up to the client application to handle these and retry them if they are suitable for a retry.

With the release of 2.5.0, the driver retries requests that are set as `idempotent`. See this C++ 2.x https://docs.datastax.com/en/developer/cpp-driver/2.16/topics/configuration/#query-idempotence[query idempotence documentation section].

== Next steps

* xref:ROOT:deployment-infrastructure.adoc[]
Loading