diff --git a/modules/ROOT/content-nav.adoc b/modules/ROOT/content-nav.adoc index cc9f5a892..dde23f590 100644 --- a/modules/ROOT/content-nav.adoc +++ b/modules/ROOT/content-nav.adoc @@ -111,6 +111,7 @@ *** xref:database-administration/standard-databases/seed-from-uri.adoc[] *** xref:database-administration/standard-databases/listing-databases.adoc[] *** xref:database-administration/standard-databases/alter-databases.adoc[] +*** xref:database-administration/standard-databases/recreate-database.adoc[] *** xref:database-administration/standard-databases/delete-databases.adoc[] *** xref:database-administration/standard-databases/wait-options.adoc[] *** xref:database-administration/standard-databases/configuration-parameters.adoc[] diff --git a/modules/ROOT/pages/clustering/databases.adoc b/modules/ROOT/pages/clustering/databases.adoc index 913b4a5e3..bbc64ad92 100644 --- a/modules/ROOT/pages/clustering/databases.adoc +++ b/modules/ROOT/pages/clustering/databases.adoc @@ -13,7 +13,7 @@ If a database is no longer needed, the command `DROP DATABASE` deletes the datab The command to create a database in a cluster is not significantly different from the command to create a database in a non-clustered environment (see xref:database-administration/standard-databases/create-databases.adoc[Create, start, and stop databases] for more information on database management on single servers). The difference in a clustered environment is that the topology can be specified, i.e. how many primaries and secondaries are desired for the database. -To create a database `foo` with 3 servers hosting the database in primary mode and 2 servers in secondary mode, the command looks like this: +To create a database `foo` with 3 servers hosting the database in primary mode and 2 servers in secondary mode, run the following command: [source, cypher] ---- @@ -44,11 +44,12 @@ For example, if the cluster's servers are set up with mode constraints to contai [NOTE] ==== If `TOPOLOGY` is not specified, the database is created according to `initial.dbms.default_primaries_count` and `initial.dbms.default_secondaries_count` specified in _neo4j.conf_. -After cluster startup, these values can be overwritten using the `dbms.setDefaultAllocationNumbers` procedure. +After cluster startup, you can overwrite these values using the `dbms.setDefaultAllocationNumbers` procedure. ==== + [NOTE] ==== -A `CREATE DATABASE` command allocates the database, therefore there is no requirement to execute `REALLOCATE DATABASES` (described in xref:clustering/servers.adoc#_hosting_databases_on_added_servers[Hosting databases on added servers]). +The `CREATE DATABASE` command allocates the database, therefore there is no requirement to execute `REALLOCATE DATABASES` (described in xref:clustering/servers.adoc#_hosting_databases_on_added_servers[Hosting databases on added servers]). However, over time, or after several `CREATE DATABASE` commands have been issued, the distribution of databases can become unbalanced. At this point you can run `REALLOCATE DATABASES` to make the cluster re-balance databases across all servers that are part of the cluster. ==== @@ -61,12 +62,13 @@ To alter the topology of or read/write access to a database after it has been cr [[alter-topology]] === Alter database topology -To change the topology of the database `foo` from the previous example, the command can look like this: +To change the topology of the database `foo` from the previous example, run the following command: [source, cypher] ---- ALTER DATABASE foo SET TOPOLOGY 2 PRIMARIES 1 SECONDARY ---- + Alternatively, starting from 5.26, you can use link:{neo4j-docs-base-uri}/cypher-manual/{page-version}/syntax/parameters[parameters] to provide the number of primaries and secondaries: .Parameters @@ -97,18 +99,18 @@ This normally happens when the cluster is configured with more servers than the It is not possible to automatically transition to a topology with a single primary host. Attempting to do so results in an error. -However, it is possible to _manually_ do this transition. -The first step is to back up the database, see xref:backup-restore/index.adoc[Backup and restore] for more information. -Once the database is backed up, the next step is to drop the database, see xref:database-administration/standard-databases/delete-databases.adoc[Delete databases] for more information. -The last step is to either seed a cluster from the backup with the new topology, or to restore the backup on a single server. +However, it is possible to _manually_ do this transition: + +. First, back up the database. +See xref:backup-restore/index.adoc[Backup and restore] for more information. +. Once the database is backed up, drop the database. +See xref:database-administration/standard-databases/delete-databases.adoc[Delete databases] for more information. +. The last step is to either seed a cluster from the backup with the new topology, or to restore the backup on a single server. See xref:clustering/databases.adoc#cluster-seed[Seed a cluster] further on for information on seeding. Also, it is possible to automatically transition _from_ a topology with a single primary host to multiple primary hosts. Keep in mind that during such a transition, the database will be unavailable for a short period of time. -`ALTER DATABASE` commands are optionally idempotent, with the default behavior to fail with an error if the database does not exist. -Appending `IF EXISTS` to the command ensures that no error is returned and nothing happens should the database not exist. - If the `ALTER DATABASE` command decreases the number of allocations of a database, allocations on xref:clustering/servers.adoc#cordoned-state[cordoned servers] are removed first. .Query @@ -259,128 +261,7 @@ neo4j@neo4j> DRYRUN REALLOCATE DATABASES; +----------------------------------------------------------------------------------------------------------------------------------------+ ---- -[role=label--new-5.24] -[[recreate-databases]] -== Recreate a database - -Neo4j 5.24 introduces the xref:procedures.adoc#procedure_dbms_cluster_recreateDatabase[`dbms.cluster.recreateDatabase()`] procedure, which allows you: - -* To change the database store to a specified backup, while keeping all the associated privileges for the database. - -* To make your database write-available again after it has been lost (for example, due to a disaster). -See xref:clustering/disaster-recovery.adoc[Disaster recovery] for more information. - -[CAUTION] -==== -The recreate procedure works only for real user databases and not for composite databases, or the `system` database. - -Remember that the recreate procedure results in downtime while the stores get updated. -The time is unbounded and may depend on different factors -- for example, the size of the store, network speed, etc. -==== - -The database in question can be in an `online` or `offline` state when it is recreated, but a successful operation starts the database regardless of its previous state. - -If your database has Change Data Capture (CDC) enabled, the CDC chain will stop when the database is recreated, even though CDC remains enabled in the recreated database. -To restore CDC functionality, follow the guide on how link:https://neo4j.com/docs/cdc/current/existing-databases/[to initialize CDC applications from an existing database]. - -Before recreating a database, any eventual quarantined states need to be addressed. -For more information, see xref:database-administration/standard-databases/errors.adoc#quarantine[Standard databases -> Error handling]. - -You need xref:authentication-authorization/dbms-administration.adoc#access-control-dbms-administration-database-management[the `CREATE DATABASE` and `DROP DATABASE` privileges] to run the recreate procedure. - -To check if the recreation is successful, use the `SHOW DATABASES` command and verify that all allocations have been started. - -Additionally, you have the option to modify <> during the recreation process. -However, note that the store format, access, and enrichment cannot be altered during recreation. - -[[recreate-seeding-options]] -=== Seeding options - -The store to be used during the recreation of a database can be defined in different ways. -One method uses a backup, while others use available allocations in the cluster. - -You can use either <> or <> to specify the source from which the database should be recreated. - -* If you define neither, an error is thrown. - -* If you define both of them, then `seedingServers` must be an empty list. -See <> for more details. - -* If `seedingServers` is not empty and `seedURI` is also defined, an error will occur. - -[[uri-seed]] -==== Use backup as a seed - -If you provide a URI to a backup or a dump, the stores on all allocations will be replaced by the backup or the dump at the given URI. -The new allocations can be put on any `ENABLED` server in the cluster. -See xref::database-administration/standard-databases/seed-from-uri.adoc[Seed from URI] for more details. - - -[source, shell] ----- -CALL dbms.cluster.recreateDatabase("neo4j", {seedURI: "s3://myBucket/myBackup.backup"}); ----- -[[seed-servers]] -==== Use available servers as a seed - -After the recreation is complete, the database will have the latest data store from the seeding servers. - -[CAUTION] -==== -Recreation is based on remaining stores or a store defined by the user. -This means that stores which were lost or not defined are not used for the recreation. -If not used stores were more up to date than the used ones, this results in data loss. -==== - -[[specified-servers]] -Specified servers:: - -You can specify a set of available servers. -The stores on all allocations will be synchronized to the most up-to-date store from the defined servers. -The number of defined servers cannot exceed the number of total allocations in the desired topology. -+ -[source, shell] ----- -CALL dbms.cluster.recreateDatabase("neo4j", {seedingServers: ["serverId1", "serverId2", "serverId3"]}); ----- - -[[undefined-servers]] -Undefined servers:: - -If you provide an empty list of seeding servers and do not specify a `seedURI`, Neo4j automatically selects all available allocations of the database as seeders. -The store will be replaced by the most up-to-date seeder available in the cluster. -+ -[source, shell] ----- -CALL dbms.cluster.recreateDatabase("neo4j", {seedingServers: []}); ----- - -[[undefined-servers-backup]] -Undefined servers with fallback backup:: - -If both an empty list of seeding servers and a `seedURI` are provided, Neo4j finds all available allocations of the database and use those as seeders. -However, if no available servers can be found, the database is recreated based on the backup or the dump defined by the URI. -This means the store is replaced by the most up-to-date seeder if available; otherwise, the backup is used. -+ -[source, shell] ----- -CALL dbms.cluster.recreateDatabase("neo4j", {seedingServers: [], seedURI: "s3://myBucket/myBackup.backup"}); ----- - -[[alter-topology-recreate]] -=== Change the topology - -There is an option to define a new topology when recreating a database. -This can be beneficial during a disaster, if enough servers are not available to recreate the database with the original topology. -When altering the total number of allocations down during a recreation, it is important to remember that the number of seeding servers cannot exceed the number of total allocations of the database. -This also holds true when using recreate with an empty list of seeders. -If there are more available servers in the cluster hosting the database than the number of new allocations, the recreation will fail. - -[source, shell] ----- -CALL dbms.cluster.recreateDatabase("neo4j", {seedingServers: [], primaries: 3, secondaries: 0}); ----- [[cluster-seed]] == Seed a cluster diff --git a/modules/ROOT/pages/clustering/disaster-recovery.adoc b/modules/ROOT/pages/clustering/disaster-recovery.adoc index 98f846c57..a404e1b3d 100644 --- a/modules/ROOT/pages/clustering/disaster-recovery.adoc +++ b/modules/ROOT/pages/clustering/disaster-recovery.adoc @@ -220,14 +220,14 @@ Instead, check that the primary is allocated on an available server and that it ===== . For each database that is not write-available, recreate it to move it from lost servers and regain write availability. -Go to xref:clustering/databases.adoc#recreate-databases[Recreate databases] for more information about recreate options. +Go to xref:database-administration/standard-databases/recreate-database.adoc[Recreate a database] for more information about recreate options. Remember to make sure there are recent backups for the databases before recreating them. See xref:backup-restore/online-backup.adoc[Online backup] for more information. -If any database has `currentStatus` = `quarantined` on an available server, recreate them from backup using xref:clustering/databases.adoc#uri-seed[Backup as seed]. +If any database has `currentStatus` = `quarantined` on an available server, recreate them from backup using xref:database-administration/standard-databases/recreate-database.adoc#uri-seed[Backup as seed]. + [CAUTION] ===== -If you recreate databases using xref:clustering/databases.adoc#undefined-servers[undefined servers] or xref:clustering/databases.adoc#undefined-servers-backup[undefined servers with fallback backup], the store might not be recreated as up-to-date as possible in certain edge cases where the `system` database has been restored. +If you recreate databases using xref:database-administration/standard-databases/recreate-database.adoc#undefined-servers[undefined servers] or xref:database-administration/standard-databases/recreate-database.adoc#undefined-servers-backup[undefined servers with fallback backup], the store might not be recreated as up-to-date as possible in certain edge cases where the `system` database has been restored. ===== . For each `Cordoned` server, run `DEALLOCATE DATABASES FROM SERVER cordoned-server-id` on one of the available servers. @@ -258,7 +258,7 @@ If they are still desired to be in stopped state, run `STOP DATABASE started-db [CAUTION] ==== -Remember, recreating a database takes an unbounded amount of time since it may involve copying the store to a new server, as described in xref:clustering/databases.adoc#recreate-databases[Recreate databases]. +Remember, recreating a database takes an unbounded amount of time since it may involve copying the store to a new server, as described in xref:database-administration/standard-databases/recreate-database.adoc[Recreate a database]. Therefore, an allocation with `currentStatus` = `starting` will probably reach the `requestedStatus` given some time. ==== @@ -291,14 +291,14 @@ Recreations might fail for different reasons, but one example is that the checks . Identify all write-unavailable databases by running `CALL dbms.cluster.statusCheck([])` as described in the xref:clustering/disaster-recovery.adoc#example-verification[Example verification] part of this disaster recovery step. Filter out all databases desired to be stopped, so that they are not recreated unnecessarily. . Recreate every database that is not write-available and has not been recreated previously. -See xref:clustering/databases.adoc#recreate-databases[Recreate databases] for more information. +See xref:database-administration/standard-databases/recreate-database.adoc[Recreate a database] for more information. Remember to make sure there are recent backups for the databases before recreating them. See xref:backup-restore/online-backup.adoc[Online backup] for more information. -If any database has `currentStatus` = `quarantined` on an available server, recreate them from backup using xref:clustering/databases.adoc#uri-seed[Backup as seed]. +If any database has `currentStatus` = `quarantined` on an available server, recreate them from backup using xref:database-administration/standard-databases/recreate-database.adoc#uri-seed[Backup as a seed]. + [CAUTION] ===== -If you recreate databases using xref:clustering/databases.adoc#undefined-servers[undefined servers] or xref:clustering/databases.adoc#undefined-servers-backup[undefined servers with fallback backup], the store might not be recreated as up-to-date as possible in certain edge cases where the `system` database has been restored. +If you recreate databases using xref:database-administration/standard-databases/recreate-database.adoc#undefined-servers[undefined servers] or xref:database-administration/standard-databases/recreate-database.adoc#undefined-servers-backup[undefined servers with fallback backup], the store might not be recreated as up-to-date as possible in certain edge cases where the `system` database has been restored. ===== . Run `SHOW DATABASES` and check any recreated databases that are not write-available. @@ -306,6 +306,6 @@ Recreating a database will not complete if one of the following messages is disp ** `Seeders ServerId1 and ServerId2 have different checksums for transaction TransactionId. All seeders must have the same checksum for the same append index.` ** `Seeders ServerId1 and ServerId2 have incompatible storeIds. All seeders must have compatible storeIds.` ** `No store found on any of the seeders ServerId1, ServerId2...` -. For each database which will not complete recreation, recreate them from backup using xref:clustering/databases.adoc#uri-seed[Backup as seed]. +. For each database which will not complete recreation, recreate them from backup using xref:database-administration/standard-databases/recreate-database.adoc#uri-seed[Backup as a seed]. ==== diff --git a/modules/ROOT/pages/database-administration/standard-databases/recreate-database.adoc b/modules/ROOT/pages/database-administration/standard-databases/recreate-database.adoc new file mode 100644 index 000000000..a10aefed3 --- /dev/null +++ b/modules/ROOT/pages/database-administration/standard-databases/recreate-database.adoc @@ -0,0 +1,164 @@ +:description: How to recreate your database in Neo4j. +:page-role: enterprise-edition new-5.24 + +[[recreate-databases]] += Recreate a database + +Neo4j 5.24 introduces the xref:procedures.adoc#procedure_dbms_cluster_recreateDatabase[`dbms.cluster.recreateDatabase()`] procedure, which allows you: + +* To change the database store to a specified backup, while keeping all the associated privileges for the database. + +* To make your database write-available again after it has been lost (for example, due to a disaster). +See xref:clustering/disaster-recovery.adoc[Disaster recovery] for more information. + +[CAUTION] +==== +The recreate procedure works only for real user databases and not for composite databases, or the `system` database. + +Remember that the recreate procedure results in downtime while the stores get updated. +The time is unbounded and may depend on different factors -- for example, the size of the store, network speed, etc. +==== + + +== Syntax + +[source, shell] +---- +dbms.cluster.recreateDatabase(database :: STRING, options = {} :: MAP) +---- + +Input arguments are the database name and list of the seeding and topology options used for recreating a database. + +.Seeding and topology options of the recreate procedure +[cols="2a,4",frame="topbot",options="header"] +|=== +|Option +|Description + +|seedingServers +|A list of possible seeding servers. +You can define available servers or provide an empty list. +For details, see <>. + +|seedURI +|External source specified by URI. + +|primaries +|Number of primary allocations for the recreated database. +If you set number of primaries without secondaries, then secondaries is set to `0`. +For more details, see <>. + +|secondaries +|Number of secondary allocations for the recreated database. +You cannot set secondaries without primaries. + +|=== + + +== Prerequisites and considerations + +The database in question can be in an `online` or `offline` state when it is recreated, but a successful operation starts the database regardless of its previous state. + +If your database has Change Data Capture (CDC) enabled, the CDC chain will stop when the database is recreated, even though CDC remains enabled in the recreated database. +To restore CDC functionality, follow the guide on how link:https://neo4j.com/docs/cdc/current/existing-databases/[to initialize CDC applications from an existing database]. + +Before recreating a database, any eventual quarantined states need to be addressed. +For more information, see xref:database-administration/standard-databases/errors.adoc#quarantine[Standard databases -> Error handling]. + +You need xref:authentication-authorization/dbms-administration.adoc#access-control-dbms-administration-database-management[the `CREATE DATABASE` and `DROP DATABASE` privileges] to run the recreate procedure. + +Additionally, in a cluster deployment, you have the option to modify <> during the recreation process. +However, note that the store format, access, and enrichment cannot be altered during recreation. + +To check if the recreation is successful, use the `SHOW DATABASES` command and verify that all allocations have been started. + + +[[recreate-seeding-options]] +== Seeding options + +The store to be used during the recreation of a database can be defined in different ways. +One method uses a backup, while others use available allocations in the cluster. + +You can use either <> or <> to specify the source from which the database should be recreated. + +* If you define neither, an error is thrown. + +* If you define both of them, then `seedingServers` must be an empty list. +See <> for more details. + +* If `seedingServers` is not empty and `seedURI` is also defined, an error will occur. + +[[uri-seed]] +=== Use backup as a seed + +If you provide a URI to a backup or a dump, the stores on all allocations will be replaced by the backup or the dump at the given URI. +The new allocations can be put on any `ENABLED` server in the cluster. +See xref::database-administration/standard-databases/seed-from-uri.adoc[Seed from URI] for more details. + + +[source, shell] +---- +CALL dbms.cluster.recreateDatabase("neo4j", {seedURI: "s3://myBucket/myBackup.backup"}); +---- + +[[seed-servers]] +=== Use available servers as a seed + +After the recreation is complete, the database will have the latest data store from the seeding servers. + +[CAUTION] +==== +Recreation is based on available remaining stores or specific stores explicitly defined by the user. +Stores that are lost or not explicitly specified are excluded from the recreation process. +Therefore, if the excluded stores contained more recent data than those used, data loss may occur. +==== + +[[specified-servers]] +Specified servers:: + +You can specify a set of available servers. +The stores on all allocations will be synchronized to the most up-to-date store from the defined servers. +The number of defined servers cannot exceed the number of total allocations in the desired topology. ++ +[source, shell] +---- +CALL dbms.cluster.recreateDatabase("neo4j", {seedingServers: ["serverId1", "serverId2", "serverId3"]}); +---- + +[[undefined-servers]] +Undefined servers:: + +If you provide an empty list of seeding servers and do not specify a `seedURI`, Neo4j automatically selects all available allocations of the database as seeders. +The store will be replaced by the most up-to-date seeder available in the cluster. ++ +[source, shell] +---- +CALL dbms.cluster.recreateDatabase("neo4j", {seedingServers: []}); +---- + +[[undefined-servers-backup]] +Undefined servers with fallback backup:: + +If both an empty list of seeding servers and a `seedURI` are provided, Neo4j finds all available allocations of the database and use those as seeders. +However, if no available servers can be found, the database is recreated based on the backup or the dump defined by the URI. +This means the store is replaced by the most up-to-date seeder if available; otherwise, the backup is used. ++ +[source, shell] +---- +CALL dbms.cluster.recreateDatabase("neo4j", {seedingServers: [], seedURI: "s3://myBucket/myBackup.backup"}); +---- + +[[alter-topology-recreate]] +== Change the topology + +In a cluster deployment, there is an option to define a new topology when recreating a database. +This can be beneficial during a disaster, if enough servers are not available to recreate the database with the original topology. +When altering the total number of allocations down during a recreation, it is important to remember that the number of seeding servers cannot exceed the number of total allocations of the database. +This also holds true when using recreate with an empty list of seeders. +If there are more available servers in the cluster hosting the database than the number of new allocations, the recreation will fail. + +[source, shell] +---- +CALL dbms.cluster.recreateDatabase("neo4j", {seedingServers: [], primaries: 3, secondaries: 0}); +---- + diff --git a/modules/ROOT/pages/procedures.adoc b/modules/ROOT/pages/procedures.adoc index b37d56f8c..21c240d8d 100644 --- a/modules/ROOT/pages/procedures.adoc +++ b/modules/ROOT/pages/procedures.adoc @@ -550,6 +550,15 @@ The procedure initiates a process, which when complete, will have synchronized a |=== +[NOTE] +==== +It is mandatory to specify either `seedURI` or `seedingServers` as seeding options in the `options` field. + +If no topology option is defined, the database will be recreated with the previous topology. + +Further details on how to use the `dbms.cluster.recreateDatabase()` procedure are provided in the section xref:database-administration/standard-databases/recreate-database.adoc[Database administration -> Recreate a database]. +==== + [role=label--enterprise-edition label--deprecated-5.21] [[procedure_dbms_cluster_routing_getroutingtable]] === dbms.cluster.routing.getRoutingTable()