Skip to content

DOC-5183, DOC-5170, DOC-5192 Astra Migration Toolkit, Migration index page, more Sideloader integration and clean up #195

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 23 commits into from
May 12, 2025
Merged
Show file tree
Hide file tree
Changes from 16 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions antora.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ asciidoc:
cass-migrator-short: 'CDM'
dse: 'DataStax Enterprise (DSE)'
dse-short: 'DSE'
dse-brief: 'DataStax Enterprise'
hcd: 'Hyper-Converged Database (HCD)'
hcd-short: 'HCD'
astra-db: 'Astra DB'
Expand All @@ -40,5 +41,6 @@ asciidoc:
support-url: 'https://support.datastax.com'
mc: 'Mission Control (MC)'
mc-short: 'MC'
mc-brief: 'Mission Control'

#TODO: Bring SCB attributes
103 changes: 47 additions & 56 deletions modules/ROOT/nav.adoc
Original file line number Diff line number Diff line change
@@ -1,55 +1,55 @@
.Get started
* xref:ROOT:components.adoc[]
* xref:ROOT:zdm-proxy-migration-paths.adoc[]
* Product-specific migration paths
** xref:ROOT:astra-migration-paths.adoc[]
** {dse-short} 6.9
*** xref:6.9@dse:tooling:migration-path-dse.adoc[{dse-short} 6.9 migration tools]
*** xref:6.9@dse:managing:operations/migrate-data.adoc[Migrate data to {dse-short} 6.9]
** {dse-short} 6.8
*** xref:6.8@dse:tooling:migration-path-dse.adoc[{dse-short} 6.8 migration tools]
*** xref:6.8@dse:managing:operations/migrate-data.adoc[Migrate data to {dse-short} 6.8]
** {dse-short} 5.1
*** xref:5.1@dse:managing:operations/migrate-data.adoc[Migrate data to {dse-short} 5.1]
** {mc-brief} migrations
*** xref:mission-control:migrate:oss-cass-to-mission-control.adoc[Migrate {cass-short} clusters to {mc-short}]
*** xref:mission-control:migrate:dse-to-mission-control.adoc[Migrate {cass-short} clusters to {dse-short}]

.{product}
* xref:introduction.adoc[]
* xref:components.adoc[]
* xref:ROOT:introduction.adoc[]
* Planning
** xref:preliminary-steps.adoc[]
** xref:feasibility-checklists.adoc[]
** xref:deployment-infrastructure.adoc[]
** xref:create-target.adoc[]
** xref:rollback.adoc[]
** xref:ROOT:feasibility-checklists.adoc[]
** xref:ROOT:deployment-infrastructure.adoc[]
** xref:ROOT:create-target.adoc[]
** xref:ROOT:rollback.adoc[]
* Phase 1
** xref:phase1.adoc[]
** xref:setup-ansible-playbooks.adoc[]
** xref:deploy-proxy-monitoring.adoc[]
** xref:tls.adoc[]
** xref:connect-clients-to-proxy.adoc[]
** xref:metrics.adoc[]
** xref:manage-proxy-instances.adoc[]
** xref:ROOT:phase1.adoc[]
** xref:ROOT:setup-ansible-playbooks.adoc[]
** xref:ROOT:deploy-proxy-monitoring.adoc[]
** xref:ROOT:tls.adoc[]
** xref:ROOT:connect-clients-to-proxy.adoc[]
** xref:ROOT:metrics.adoc[]
** xref:ROOT:manage-proxy-instances.adoc[]
* Phase 2
** xref:migrate-and-validate-data.adoc[]
** xref:cassandra-data-migrator.adoc[{cass-migrator}]
** xref:dsbulk-migrator.adoc[{dsbulk-migrator}]
** xref:ROOT:migrate-and-validate-data.adoc[]
** xref:sideloader:sideloader-zdm.adoc[]
** xref:ROOT:cassandra-data-migrator.adoc[]
** xref:ROOT:dsbulk-migrator.adoc[]
* Phase 3
** xref:enable-async-dual-reads.adoc[]
** xref:ROOT:enable-async-dual-reads.adoc[]
* Phase 4
** xref:change-read-routing.adoc[]
** xref:ROOT:change-read-routing.adoc[]
* Phase 5
** xref:connect-clients-to-target.adoc[]
** xref:ROOT:connect-clients-to-target.adoc[]
* References
** xref:troubleshooting-tips.adoc[]
** xref:troubleshooting-scenarios.adoc[]
** xref:contributions.adoc[]
** xref:faqs.adoc[]
** xref:glossary.adoc[]
** xref:ROOT:troubleshooting-tips.adoc[]
** xref:ROOT:troubleshooting-scenarios.adoc[]
** xref:ROOT:contributions.adoc[]
** xref:ROOT:faqs.adoc[]
** xref:ROOT:glossary.adoc[]
** https://github.com/datastax/zdm-proxy/releases[{product-proxy} release notes]
** https://github.com/datastax/zdm-proxy-automation/releases[{product-automation} release notes]

.{cass-migrator}
* xref:cdm-overview.adoc[{cass-migrator}]
* https://github.com/datastax/cassandra-data-migrator/releases[{cass-migrator-short} release notes]

.{dsbulk-loader}
* xref:dsbulk:overview:dsbulk-about.adoc[{dsbulk-loader}]
* xref:dsbulk:installing:install.adoc[Installing {dsbulk-loader}]
* Loading and unloading data
** xref:dsbulk:getting-started:simple-load.adoc[Loading data without a configuration file]
** xref:dsbulk:getting-started:simple-unload.adoc[Unloading data without a configuration file]
** xref:dsbulk:developing:loading-unloading-vector-data.adoc[Loading and unloading vector data]
** xref:dsbulk:reference:load.adoc[Loading data examples]
** xref:dsbulk:reference:unload.adoc[Unloading data examples]
* xref:dsbulk:reference:dsbulk-cmd.adoc#escaping-and-quoting-command-line-arguments[Escaping and quoting command line arguments]
* https://github.com/datastax/dsbulk/releases[{dsbulk-loader} release notes]

.{sstable-sideloader}
* xref:sideloader:sideloader-overview.adoc[]
* xref:sideloader:prepare-sideloader.adoc[]
Expand All @@ -58,18 +58,9 @@
* xref:sideloader:cleanup-sideloader.adoc[]
* xref:sideloader:troubleshoot-sideloader.adoc[]

.Product-specific migration paths
* {astra-db}
** xref:astra-db-serverless:databases:migration-path-serverless.adoc[]
* {dse}
** {dse-short} 6.9
*** xref:6.9@dse:tooling:migration-path-dse.adoc[{dse-short} 6.9 migration tools]
*** xref:6.9@dse:managing:operations/migrate-data.adoc[Migrate data to {dse-short} 6.9]
** {dse-short} 6.8
*** xref:6.8@dse:tooling:migration-path-dse.adoc[{dse-short} 6.8 migration tools]
*** xref:6.8@dse:managing:operations/migrate-data.adoc[Migrate data to {dse-short} 6.8]
** {dse-short} 5.1
*** xref:5.1@dse:managing:operations/migrate-data.adoc[Migrate data to {dse-short} 5.1]
* {mc}
** xref:mission-control:migrate:oss-cass-to-mission-control.adoc[Migrate {cass-short} clusters to {mc-short}]
** xref:mission-control:migrate:dse-to-mission-control.adoc[[Migrate {cass-short} clusters to {dse-short}]
.{cass-migrator}
* xref:ROOT:cdm-overview.adoc[]
* https://github.com/datastax/cassandra-data-migrator/releases[{cass-migrator-short} release notes]

.{dsbulk-migrator}
* xref:ROOT:dsbulk-migrator-overview.adoc[]
106 changes: 106 additions & 0 deletions modules/ROOT/pages/astra-migration-paths.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
= {astra} Migration Toolkit
:description: Learn which migration tools are compatible with your origin cluster.

The {astra} Migration Toolkit includes all xref:ROOT:components.adoc[{company} migration tools] that are designed to help you migrate your data to {astra-db}.

== Migration tool compatibility

Use the following table to learn which tools are compatible with your current database provider or service:

[cols="2,1,1,1,1"]
|===
|Origin |{sstable-sideloader} |{cass-migrator} |{product-proxy} |{dsbulk-migrator}/{dsbulk-loader}

|Aiven for {cass-short}
|✅
|✅
|✅
|✅

|Amazon Keyspaces
|❌
|✅
|✅
|✅

|{cass-reg} OSS 3.11 or later
|✅
|✅
|✅
|✅

|{cass-reg} OSS 3.10 or earlier
|❌
|✅
|✅
|✅

|Azure Cosmos DB ({cass-short} API)
|❌
|✅
|✅
|✅

|Azure Managed Instance for {cass}
|✅
|✅
|✅
|✅

|{dse-short} 5.1 or later
|✅
|✅
|✅
|✅

|{dse-short} 5.0 or earlier
|❌
|✅
|✅
|✅

|Instaclustr Managed {cass-short}
|✅
|✅
|✅
|✅

|K8ssandra (self-managed)
|✅
|✅
|✅
|✅

|Scylla Cloud
|❌
|✅
|✅
|✅

|Scylla OSS or Enterprise
|❌
|✅
|✅
|✅

|Yugabyte Aeon (YCQL)
|❌
|✅
|✅
|✅

|Yugabyte OSS or Anywhere
|❌
|✅
|✅
|✅

|===

== Get support for your migration

If you have questions about migrating from a specific source to {astra-db}, contact your {company} account representative, {support-url}[{company} Support], or an https://www.datastax.com/products/datastax-astra/migration-toolkit[{astra} Migration Toolkit expert].

== See also

* xref:astra-db-serverless:databases:migration-path-serverless.adoc[Migrate to {astra-db}]
21 changes: 18 additions & 3 deletions modules/ROOT/pages/cassandra-data-migrator.adoc
Original file line number Diff line number Diff line change
@@ -1,15 +1,30 @@
= {cass-migrator}
= Use {cass-migrator} with {product-short}
:navtitle: Use {cass-migrator}
:description: Use {cass-migrator} to migrate data with {product-short}
:page-aliases: cdm-parameters.adoc, ROOT:cdm-steps.adoc

//This page was an exact duplicate of cdm-overview.adoc and the (now deleted) cdm-steps.adoc, they are just in different parts of the nav.

// tag::body[]
You can use {cass-migrator} ({cass-migrator-short}) to migrate and validate tables between the origin and target {cass-short} clusters, with optional logging and reconciliation support.
You can use {cass-migrator} ({cass-migrator-short}) to migrate and validate tables between {cass-short}-based clusters.
It is designed to connect to your target cluster, compare it with the origin cluster, log any differences, and, optionally, automatically reconcile inconsistencies and missing data.

{cass-migrator-short} facilitates data transfer by creating multiple jobs that access the {cass-short} cluster concurrently, making it an ideal choice for migrating large datasets.
It offers extensive configuration options, including logging, reconciliation, performance optimization, and more.

//TODO: Bring over content from the page that introduces the 3 options, and the features, limitations, and performance recommendations in the README https://github.com/datastax/cassandra-data-migrator?tab=readme-ov-file#features
{cass-migrator-short} features include the following:

* Validate migration accuracy and performance using examples that provide a smaller, randomized data set.
* Preserve internal `writetime` timestamps and Time To Live (TTL) values.
* Use advanced data types, including sets, lists, maps, and UDTs.
* Filter records from the origin cluster's data, using {cass-short}'s internal `writetime` timestamp.
* Use SSL Support, including custom cipher algorithms.

For more features and information, see the https://github.com/datastax/cassandra-data-migrator?tab=readme-ov-file#features[{cass-migrator-short} GitHub repository].

== {cass-migrator} requirements

To use {cass-migrator-short} successfully, your origin and target clusters must have matching schemas.

== Install {cass-migrator}

Expand Down
Loading