From d44c5ce5a3a138494ad829ec728427695913af66 Mon Sep 17 00:00:00 2001 From: Rachel Elledge Date: Fri, 28 Jun 2024 15:53:28 -0500 Subject: [PATCH 1/9] DOC-3850 RS: Add GET shards and migrate shards REST API references --- .../rest-api/objects/statistics/_index.md | 2 +- .../rest-api/requests/shards/_index.md | 140 +++++++++++++++ .../requests/shards/actions/_index.md | 20 +++ .../requests/shards/actions/migrate.md | 165 ++++++++++++++++++ .../{shards-stats => shards/stats}/_index.md | 15 +- .../{shards-stats => shards/stats}/last.md | 13 +- 6 files changed, 340 insertions(+), 15 deletions(-) create mode 100644 content/operate/rs/references/rest-api/requests/shards/_index.md create mode 100644 content/operate/rs/references/rest-api/requests/shards/actions/_index.md create mode 100644 content/operate/rs/references/rest-api/requests/shards/actions/migrate.md rename content/operate/rs/references/rest-api/requests/{shards-stats => shards/stats}/_index.md (91%) rename content/operate/rs/references/rest-api/requests/{shards-stats => shards/stats}/last.md (89%) diff --git a/content/operate/rs/references/rest-api/objects/statistics/_index.md b/content/operate/rs/references/rest-api/objects/statistics/_index.md index 2b3ae4d65f..f8f04f026a 100644 --- a/content/operate/rs/references/rest-api/objects/statistics/_index.md +++ b/content/operate/rs/references/rest-api/objects/statistics/_index.md @@ -17,7 +17,7 @@ Clusters, databases, nodes, and shards collect various statistics at regular tim - [Cluster stats]({{< relref "/operate/rs/references/rest-api/requests/cluster/stats" >}}) - [Database stats]({{< relref "/operate/rs/references/rest-api/requests/bdbs/stats" >}}) - [Node stats]({{< relref "/operate/rs/references/rest-api/requests/nodes/stats" >}}) -- [Shard stats]({{< relref "/operate/rs/references/rest-api/requests/shards-stats" >}}) +- [Shard stats]({{< relref "/operate/rs/references/rest-api/requests/shards/stats" >}}) ### Response object diff --git a/content/operate/rs/references/rest-api/requests/shards/_index.md b/content/operate/rs/references/rest-api/requests/shards/_index.md new file mode 100644 index 0000000000..40a4e19328 --- /dev/null +++ b/content/operate/rs/references/rest-api/requests/shards/_index.md @@ -0,0 +1,140 @@ +--- +Title: Shard requests +alwaysopen: false +categories: +- docs +- operate +- rs +description: REST API requests for database shards +headerRange: '[1-2]' +hideListLinks: true +linkTitle: shards +weight: $weight +--- + +| Method | Path | Description | +|--------|------|-------------| +| [GET](#get-all-shards) | `/v1/shards` | Get all shards | +| [GET](#get-shard) | `/v1/shards/{uid}` | Get a specific shard | + +## Get all shards {#get-all-shards} + + GET /v1/shards + +Get information about all shards in the cluster. + +### Request {#get-all-request} + +#### Example HTTP request + + GET /shards?extra_info_keys=used_memory_rss&extra_info_keys=connected_clients + +#### Request headers + +| Key | Value | Description | +|-----|-------|-------------| +| Host | cnm.cluster.fqdn | Domain name | +| Accept | application/json | Accepted media type | + +#### Query parameters + +| Field | Type | Description | +|-------|------|-------------| +| extra_info_keys | list of strings | A list of extra keys to be fetched (optional) | + +### Response {#get-all-response} + +Returns a JSON array of [shard objects]({{}}). + +#### Example JSON body + +```json +[ + { + "uid": "1", + "role": "master", + "assigned_slots": "0-16383", + "bdb_uid": 1, + "detailed_status": "ok", + "loading": { + "status": "idle" + }, + "node_uid": "1", + "redis_info": {}, + "report_timestamp": "2024-06-28T18:44:01Z", + "status": "active" + }, + { + "uid": 2, + "role": "slave", + // additional fields... + } +] +``` + +### Status codes {#get-all-status-codes} + +| Code | Description | +|------|-------------| +| [200 OK](https://www.rfc-editor.org/rfc/rfc9110.html#name-200-ok) | No error. | + +## Get shard {#get-shard} + + GET /v1/shards/{int: uid} + +Gets information about a single shard. + +### Request {#get-request} + +#### Example HTTP request + + GET /shards/1?extra_info_keys=used_memory_rss&extra_info_keys=connected_clients + +#### Request headers + +| Key | Value | Description | +|-----|-------|-------------| +| Host | cnm.cluster.fqdn | Domain name | +| Accept | application/json | Accepted media type | + +#### URL parameters + +| Field | Type | Description | +|-------|------|-------------| +| uid | integer | The unique ID of the requested shard. | + +#### Query parameters + +| Field | Type | Description | +|-------|------|-------------| +| extra_info_keys | list of strings | A list of extra keys to be fetched (optional) | + +### Response {#get-response} + +Returns a [shard object]({{}}). + +#### Example JSON body + +```json +{ + "assigned_slots": "0-16383", + "bdb_uid": 1, + "detailed_status": "ok", + "loading": { + "status": "idle" + }, + "node_uid": "1", + "redis_info": {}, + "role": "master", + "report_timestamp": "2024-06-28T18:44:01Z", + "status": "active", + "uid": "1" +} +``` + +### Status codes {#get-status-codes} + +| Code | Description | +|------|-------------| +| [200 OK](https://www.rfc-editor.org/rfc/rfc9110.html#name-200-ok) | No error. | +| [404 Not Found](https://www.rfc-editor.org/rfc/rfc9110.html#name-404-not-found) | Shard UID does not exist. | diff --git a/content/operate/rs/references/rest-api/requests/shards/actions/_index.md b/content/operate/rs/references/rest-api/requests/shards/actions/_index.md new file mode 100644 index 0000000000..4550f9395f --- /dev/null +++ b/content/operate/rs/references/rest-api/requests/shards/actions/_index.md @@ -0,0 +1,20 @@ +--- +Title: Shard actions requests +alwaysopen: false +categories: +- docs +- operate +- rs +description: REST API requests to perform shard actions +headerRange: '[1-2]' +hideListLinks: true +linkTitle: actions +weight: $weight +--- + +## Migrate + +| Method | Path | Description | +|--------|------|-------------| +| [POST]({{}}) | `/v1/shards/actions/migrate` | Migrate multiple shards | +| [POST]({{}}) | `/v1/shards/{uid}/actions/migrate` | Migrate a specific shard | diff --git a/content/operate/rs/references/rest-api/requests/shards/actions/migrate.md b/content/operate/rs/references/rest-api/requests/shards/actions/migrate.md new file mode 100644 index 0000000000..9db429445f --- /dev/null +++ b/content/operate/rs/references/rest-api/requests/shards/actions/migrate.md @@ -0,0 +1,165 @@ +--- +Title: Migrate shards requests +alwaysopen: false +categories: +- docs +- operate +- rs +description: REST API requests to migrate database shards +headerRange: '[1-2]' +linkTitle: migrate +weight: $weight +--- + +| Method | Path | Description | +|--------|------|-------------| +| [POST](#post-multi-shards) | `/v1/shards/actions/migrate` | Migrate multiple shards | +| [POST](#post-shard) | `/v1/shards/{uid}/actions/migrate` | Migrate a specific shard | + +## Migrate multiple shards {#post-multi-shards} + + POST /v1/shards/actions/migrate + +Migrates the list of given shard UIDs to the node specified by `target_node_uid`. The shards can be from multiple databases. This request is asynchronous. + +#### Required permissions + +| Permission name | Roles | +|-----------------|-------| +| [migrate_shard]({{< relref "/operate/rs/references/rest-api/permissions#migrate_shard" >}}) | admin
cluster_member
db_member | + +### Request {#post-multi-request} + +#### Example HTTP request + + POST /shards/actions/migrate + +#### Example JSON body + +```json +{ + "shard_uids": ["2","4","6"], + "target_node_uid": 9, + "override_rack_policy": False, + "preserve_roles": False, + "max_concurrent_bdb_migrations": 3 +} +``` + +#### Request headers + +| Key | Value | Description | +|-----|-------|-------------| +| Host | cnm.cluster.fqdn | Domain name | +| Accept | application/json | Accepted media type | + +#### Request body + +The request body is a JSON object that can contain the following fields: + +| Field | Type | Description | +|-------|------|-------------| +| shard_uids | array of strings | List of shard UIDs to migrate. | +| target_node_uid | integer | UID of the node to where the shards should migrate. | +| override_rack_policy | boolean | If true, overrides and ignores rack-aware policy violations. | +| dry_run | boolean | Determines whether the migration is actually done. If true, will just do a dry-run. | +| preserve_roles | boolean | If true, preserves the migrated shards' roles after migration. | +| max_concurrent_bdb_migrations | integer | The number of concurrent databases that can migrate shards. | + +### Response {#post-multi-response} + +Returns a JSON object with an `action_uid`. You can track the action's progress with a [`GET /v1/actions/`]({{}}) request. + +#### Example JSON body + +```json +{ + "action_uid": "e5e24ddf-a456-4a7e-ad53-4463cd44880e", + "description": "Migrate was triggered" +} +``` + +### Status codes {#post-multi-status-codes} + +| Code | Description | +|------|-------------| +| [200 OK](https://www.rfc-editor.org/rfc/rfc9110.html#name-200-ok) | No error. | +| [400 Bad Request](https://www.rfc-editor.org/rfc/rfc9110.html#name-400-bad-request) | Conflicting parameters. | +| [404 Not Found](https://www.rfc-editor.org/rfc/rfc9110.html#name-404-not-found) | A list of shard uids is required and not given, a specified shard does not exist, or a node UID is required and not given. | +| [500 Internal Server Error](https://www.rfc-editor.org/rfc/rfc9110.html#name-500-internal-server-error) | Migration failed. | + + +## Migrate shard {#post-shard} + + POST /v1/shards/{int: uid}/actions/migrate + +Migrates the shard with the given `shard_uid` to the node specified by `target_node_uid`. If the shard is already on the target node, nothing happens. This request is asynchronous. + +#### Required permissions + +| Permission name | Roles | +|-----------------|-------| +| [migrate_shard]({{< relref "/operate/rs/references/rest-api/permissions#migrate_shard" >}}) | admin
cluster_member
db_member | + +### Request {#post-request} + +#### Example HTTP request + + POST /shards/1/actions/migrate + +#### Example JSON body + +```json +{ + "target_node_uid": 9, + "override_rack_policy": False, + "preserve_roles": False +} +``` + +#### Request headers + +| Key | Value | Description | +|-----|-------|-------------| +| Host | cnm.cluster.fqdn | Domain name | +| Accept | application/json | Accepted media type | + + +#### URL parameters + +| Field | Type | Description | +|-------|------|-------------| +| uid | integer | The unique ID of the shard to migrate. | + + +#### Request body + +The request body is a JSON object that can contain the following fields: + +| Field | Type | Description | +|-------|------|-------------| +| target_node_uid | integer | UID of the node to where the shard should migrate. | +| override_rack_policy | boolean | If true, overrides and ignores rack-aware policy violations. | +| dry_run | boolean | Determines whether the migration is actually done. If true, will just do a dry-run. | +| preserve_roles | boolean | If true, preserves the migrated shards' roles after migration. | + +### Response {#post-response} + +Returns a JSON object with an `action_uid`. You can track the action's progress with a [`GET /v1/actions/`]({{}}) request. + +#### Example JSON body + +```json +{ + "action_uid": "e5e24ddf-a456-4a7e-ad53-4463cd44880e", + "description": "Migrate was triggered" +} +``` + +### Status codes {#post-status-codes} + +| Code | Description | +|------|-------------| +| [200 OK](https://www.rfc-editor.org/rfc/rfc9110.html#name-200-ok) | No error. | +| [404 Not Found](https://www.rfc-editor.org/rfc/rfc9110.html#name-404-not-found) | Shard does not exist, or node UID is required and not given. | +| [409 Conflict](https://www.rfc-editor.org/rfc/rfc9110.html#name-409-conflict) | Database is currently busy. | diff --git a/content/operate/rs/references/rest-api/requests/shards-stats/_index.md b/content/operate/rs/references/rest-api/requests/shards/stats/_index.md similarity index 91% rename from content/operate/rs/references/rest-api/requests/shards-stats/_index.md rename to content/operate/rs/references/rest-api/requests/shards/stats/_index.md index e9f8ddfa79..f81b8e690e 100644 --- a/content/operate/rs/references/rest-api/requests/shards-stats/_index.md +++ b/content/operate/rs/references/rest-api/requests/shards/stats/_index.md @@ -8,8 +8,9 @@ categories: description: Shard statistics requests headerRange: '[1-2]' hideListLinks: true -linkTitle: shards/stats +linkTitle: stats weight: $weight +aliases: /operate/rs/references/rest-api/requests/shards-stats/ --- | Method | Path | Description | @@ -132,8 +133,8 @@ Returns a JSON array of [statistics]({{< relref "/operate/rs/references/rest-api | Code | Description | |------|-------------| -| [200 OK](http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.2.1) | No error | -| [404 Not Found](http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.5) | No shards exist | +| [200 OK](https://www.rfc-editor.org/rfc/rfc9110.html#name-200-ok) | No error | +| [404 Not Found](https://www.rfc-editor.org/rfc/rfc9110.html#name-404-not-found) | No shards exist | ## Get shard stats {#get-shard-stats} @@ -231,12 +232,10 @@ Returns [statistics]({{< relref "/operate/rs/references/rest-api/objects/statist } ``` - - ### Status codes {#get-status-codes} | Code | Description | |------|-------------| -| [200 OK](http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.2.1) | No error | -| [404 Not Found](http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.5) | Shard does not exist | -| [406 Not Acceptable](http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.7) | Shard isn't currently active | +| [200 OK](https://www.rfc-editor.org/rfc/rfc9110.html#name-200-ok) | No error | +| [404 Not Found](https://www.rfc-editor.org/rfc/rfc9110.html#name-404-not-found) | Shard does not exist | +| [406 Not Acceptable](https://www.rfc-editor.org/rfc/rfc9110.html#name-406-not-acceptable) | Shard isn't currently active | diff --git a/content/operate/rs/references/rest-api/requests/shards-stats/last.md b/content/operate/rs/references/rest-api/requests/shards/stats/last.md similarity index 89% rename from content/operate/rs/references/rest-api/requests/shards-stats/last.md rename to content/operate/rs/references/rest-api/requests/shards/stats/last.md index 96c890c815..133c53d3cb 100644 --- a/content/operate/rs/references/rest-api/requests/shards-stats/last.md +++ b/content/operate/rs/references/rest-api/requests/shards/stats/last.md @@ -9,6 +9,7 @@ description: Most recent shard statistics requests headerRange: '[1-2]' linkTitle: last weight: $weight +aliases: /operate/rs/references/rest-api/requests/shards-stats/last/ --- | Method | Path | Description | @@ -101,8 +102,8 @@ Returns most recent [statistics]({{< relref "/operate/rs/references/rest-api/obj | Code | Description | |------|-------------| -| [200 OK](http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.2.1) | No error | -| [404 Not Found](http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.5) | No shards exist | +| [200 OK](https://www.rfc-editor.org/rfc/rfc9110.html#name-200-ok) | No error | +| [404 Not Found](https://www.rfc-editor.org/rfc/rfc9110.html#name-404-not-found) | No shards exist | ## Get latest shard stats {#get-shard-stats-last} @@ -186,10 +187,10 @@ Returns the most recent [statistics]({{< relref "/operate/rs/references/rest-api } ``` -### Status codes {#get-all-status-codes} +### Status codes {#get-status-codes} | Code | Description | |------|-------------| -| [200 OK](http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.2.1) | No error | -| [404 Not Found](http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.5) | Shard does not exist | -| [406 Not Acceptable](http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.7) | Shard isn't currently active | +| [200 OK](https://www.rfc-editor.org/rfc/rfc9110.html#name-200-ok) | No error | +| [404 Not Found](https://www.rfc-editor.org/rfc/rfc9110.html#name-404-not-found) | Shard does not exist | +| [406 Not Acceptable](https://www.rfc-editor.org/rfc/rfc9110.html#name-406-not-acceptable) | Shard isn't currently active | From d9af3d802a4db7810ba810203647639db1613ffa Mon Sep 17 00:00:00 2001 From: Rachel Elledge Date: Fri, 28 Jun 2024 16:20:46 -0500 Subject: [PATCH 2/9] Fix boolean case for JSON examples --- .../rest-api/requests/shards/actions/migrate.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/content/operate/rs/references/rest-api/requests/shards/actions/migrate.md b/content/operate/rs/references/rest-api/requests/shards/actions/migrate.md index 9db429445f..db352354e4 100644 --- a/content/operate/rs/references/rest-api/requests/shards/actions/migrate.md +++ b/content/operate/rs/references/rest-api/requests/shards/actions/migrate.md @@ -40,8 +40,8 @@ Migrates the list of given shard UIDs to the node specified by `target_node_uid` { "shard_uids": ["2","4","6"], "target_node_uid": 9, - "override_rack_policy": False, - "preserve_roles": False, + "override_rack_policy": false, + "preserve_roles": false, "max_concurrent_bdb_migrations": 3 } ``` @@ -112,8 +112,8 @@ Migrates the shard with the given `shard_uid` to the node specified by `target_n ```json { "target_node_uid": 9, - "override_rack_policy": False, - "preserve_roles": False + "override_rack_policy": false, + "preserve_roles": false } ``` From 9be48c83324e60252962c9ea1c25f6dbb9d7b698 Mon Sep 17 00:00:00 2001 From: Rachel Elledge Date: Mon, 1 Jul 2024 08:12:26 -0500 Subject: [PATCH 3/9] Fix case of UIDs --- .../rs/references/rest-api/requests/shards/actions/migrate.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/operate/rs/references/rest-api/requests/shards/actions/migrate.md b/content/operate/rs/references/rest-api/requests/shards/actions/migrate.md index db352354e4..cb1f28a578 100644 --- a/content/operate/rs/references/rest-api/requests/shards/actions/migrate.md +++ b/content/operate/rs/references/rest-api/requests/shards/actions/migrate.md @@ -85,7 +85,7 @@ Returns a JSON object with an `action_uid`. You can track the action's progress |------|-------------| | [200 OK](https://www.rfc-editor.org/rfc/rfc9110.html#name-200-ok) | No error. | | [400 Bad Request](https://www.rfc-editor.org/rfc/rfc9110.html#name-400-bad-request) | Conflicting parameters. | -| [404 Not Found](https://www.rfc-editor.org/rfc/rfc9110.html#name-404-not-found) | A list of shard uids is required and not given, a specified shard does not exist, or a node UID is required and not given. | +| [404 Not Found](https://www.rfc-editor.org/rfc/rfc9110.html#name-404-not-found) | A list of shard UIDs is required and not given, a specified shard does not exist, or a node UID is required and not given. | | [500 Internal Server Error](https://www.rfc-editor.org/rfc/rfc9110.html#name-500-internal-server-error) | Migration failed. | From 70f40c343ad3c47b0322c443357e0294b504fa22 Mon Sep 17 00:00:00 2001 From: Rachel Elledge Date: Thu, 11 Jul 2024 13:58:13 -0500 Subject: [PATCH 4/9] DOC-3850 Add redis_info example for shard response --- .../rs/references/rest-api/requests/shards/_index.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/content/operate/rs/references/rest-api/requests/shards/_index.md b/content/operate/rs/references/rest-api/requests/shards/_index.md index 40a4e19328..4aba15697f 100644 --- a/content/operate/rs/references/rest-api/requests/shards/_index.md +++ b/content/operate/rs/references/rest-api/requests/shards/_index.md @@ -60,7 +60,9 @@ Returns a JSON array of [shard objects]({{ Date: Fri, 23 Aug 2024 17:27:09 -0500 Subject: [PATCH 5/9] DOC-3850 Feedback update to add GET shards connected_clients REST API example response --- .../rs/references/rest-api/requests/shards/_index.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/content/operate/rs/references/rest-api/requests/shards/_index.md b/content/operate/rs/references/rest-api/requests/shards/_index.md index 4aba15697f..7801b65838 100644 --- a/content/operate/rs/references/rest-api/requests/shards/_index.md +++ b/content/operate/rs/references/rest-api/requests/shards/_index.md @@ -61,8 +61,9 @@ Returns a JSON array of [shard objects]({{ Date: Fri, 23 Aug 2024 17:31:06 -0500 Subject: [PATCH 6/9] DOC-3850 Feedback update to add migrate shards REST API use cases and more info about dry_run --- .../requests/shards/actions/migrate.md | 20 +++++++++++++++++-- 1 file changed, 18 insertions(+), 2 deletions(-) diff --git a/content/operate/rs/references/rest-api/requests/shards/actions/migrate.md b/content/operate/rs/references/rest-api/requests/shards/actions/migrate.md index cb1f28a578..bba1c794f1 100644 --- a/content/operate/rs/references/rest-api/requests/shards/actions/migrate.md +++ b/content/operate/rs/references/rest-api/requests/shards/actions/migrate.md @@ -22,6 +22,14 @@ weight: $weight Migrates the list of given shard UIDs to the node specified by `target_node_uid`. The shards can be from multiple databases. This request is asynchronous. +Migrate shards in the following scenarios: + +- Before node removal. + +- To balance the database manually in case of latency issues or uneven load distribution across nodes. + +- To manage node resources, such as memory usage. + #### Required permissions | Permission name | Roles | @@ -62,7 +70,7 @@ The request body is a JSON object that can contain the following fields: | shard_uids | array of strings | List of shard UIDs to migrate. | | target_node_uid | integer | UID of the node to where the shards should migrate. | | override_rack_policy | boolean | If true, overrides and ignores rack-aware policy violations. | -| dry_run | boolean | Determines whether the migration is actually done. If true, will just do a dry-run. | +| dry_run | boolean | Determines whether the migration is actually done. If true, will just do a dry run. If the dry run succeeds, the request returns a `200 OK` status code. Otherwise, it returns a JSON object with an error code and description. | | preserve_roles | boolean | If true, preserves the migrated shards' roles after migration. | | max_concurrent_bdb_migrations | integer | The number of concurrent databases that can migrate shards. | @@ -95,6 +103,14 @@ Returns a JSON object with an `action_uid`. You can track the action's progress Migrates the shard with the given `shard_uid` to the node specified by `target_node_uid`. If the shard is already on the target node, nothing happens. This request is asynchronous. +Migrate shards in the following scenarios: + +- Before node removal. + +- To balance the database manually in case of latency issues or uneven load distribution across nodes. + +- To manage node resources, such as memory usage. + #### Required permissions | Permission name | Roles | @@ -140,7 +156,7 @@ The request body is a JSON object that can contain the following fields: |-------|------|-------------| | target_node_uid | integer | UID of the node to where the shard should migrate. | | override_rack_policy | boolean | If true, overrides and ignores rack-aware policy violations. | -| dry_run | boolean | Determines whether the migration is actually done. If true, will just do a dry-run. | +| dry_run | boolean | Determines whether the migration is actually done. If true, will just do a dry run. If the dry run succeeds, the request returns a `200 OK` status code. Otherwise, it returns a JSON object with an error code and description. | | preserve_roles | boolean | If true, preserves the migrated shards' roles after migration. | ### Response {#post-response} From f450bf5c8a8377a1253ae6d9d239a00f2027f046 Mon Sep 17 00:00:00 2001 From: Rachel Elledge Date: Fri, 23 Aug 2024 18:13:35 -0500 Subject: [PATCH 7/9] DOC-3850 Feedback update to add impact of shard migration --- .../requests/shards/actions/migrate.md | 34 +++++++++++-------- 1 file changed, 19 insertions(+), 15 deletions(-) diff --git a/content/operate/rs/references/rest-api/requests/shards/actions/migrate.md b/content/operate/rs/references/rest-api/requests/shards/actions/migrate.md index bba1c794f1..8ea42a71cd 100644 --- a/content/operate/rs/references/rest-api/requests/shards/actions/migrate.md +++ b/content/operate/rs/references/rest-api/requests/shards/actions/migrate.md @@ -16,13 +16,7 @@ weight: $weight | [POST](#post-multi-shards) | `/v1/shards/actions/migrate` | Migrate multiple shards | | [POST](#post-shard) | `/v1/shards/{uid}/actions/migrate` | Migrate a specific shard | -## Migrate multiple shards {#post-multi-shards} - - POST /v1/shards/actions/migrate - -Migrates the list of given shard UIDs to the node specified by `target_node_uid`. The shards can be from multiple databases. This request is asynchronous. - -Migrate shards in the following scenarios: +Migrate shards to a different node in the following scenarios: - Before node removal. @@ -30,6 +24,24 @@ Migrate shards in the following scenarios: - To manage node resources, such as memory usage. +For databases with replication: + +- Migrating a shard will not cause disruptions since a primary shard will still be available. + +- If you try to migrate a primary shard, it will be demoted to a replica shard and a replica shard will be promoted to primary before the migration. If you set `"preserve_roles": true` in the request, a second failover will occur after the migration finishes to change the migrated shard's role back to primary. + +For databases without replication, the migrated shard will not be available until the migration is done. + +Connected clients shouldn't be disconnected in either case. + +If too many primary shards are placed on the same node, it can impact database performance. + +## Migrate multiple shards {#post-multi-shards} + + POST /v1/shards/actions/migrate + +Migrates the list of given shard UIDs to the node specified by `target_node_uid`. The shards can be from multiple databases. This request is asynchronous. + #### Required permissions | Permission name | Roles | @@ -103,14 +115,6 @@ Returns a JSON object with an `action_uid`. You can track the action's progress Migrates the shard with the given `shard_uid` to the node specified by `target_node_uid`. If the shard is already on the target node, nothing happens. This request is asynchronous. -Migrate shards in the following scenarios: - -- Before node removal. - -- To balance the database manually in case of latency issues or uneven load distribution across nodes. - -- To manage node resources, such as memory usage. - #### Required permissions | Permission name | Roles | From 8ad365551dca1b81f7e3c129c078d36f76f80923 Mon Sep 17 00:00:00 2001 From: Rachel Elledge Date: Fri, 13 Sep 2024 11:36:19 -0500 Subject: [PATCH 8/9] DOC-4062 Add /bdbs/uid/shards reference --- .../rest-api/requests/bdbs/shards.md | 81 +++++++++++++++++++ 1 file changed, 81 insertions(+) create mode 100644 content/operate/rs/references/rest-api/requests/bdbs/shards.md diff --git a/content/operate/rs/references/rest-api/requests/bdbs/shards.md b/content/operate/rs/references/rest-api/requests/bdbs/shards.md new file mode 100644 index 0000000000..4ea6207cb6 --- /dev/null +++ b/content/operate/rs/references/rest-api/requests/bdbs/shards.md @@ -0,0 +1,81 @@ +--- +Title: Database shards requests +alwaysopen: false +categories: +- docs +- operate +- rs +description: REST API requests for database shards +headerRange: '[1-2]' +linkTitle: shards +weight: $weight +--- + +| Method | Path | Description | +|--------|------|-------------| +| [GET](#get-bdb-shards) | `/v1/bdbs/{bdb_uid}/shards` | Get the status of a database's shards | + +## Get database shards {#get-bdb-shards} + + GET /v1/bdbs/{int: bdb_uid}/shards + +Gets the status for all shards that belong to the specified database. + +### Request {#get-request} + +#### Example HTTP request + + GET /bdbs/1/shards?extra_info_keys=used_memory_rss&extra_info_keys=connected_clients + +#### Request headers + +| Key | Value | Description | +|-----|-------|-------------| +| Host | cnm.cluster.fqdn | Domain name | +| Accept | application/json | Accepted media type | + +#### URL parameters + +| Field | Type | Description | +|-------|------|-------------| +| bdb_uid | integer | The unique ID of the database. | + +### Response {#get-response} + +The response body contains a JSON array with all shards, represented as [shard objects]({{}}). + +#### Example JSON body + +```json +[ + { + "uid": "1", + "role": "master", + "assigned_slots": "0-8191", + "bdb_uid": 1, + "detailed_status": "ok", + "loading": { + "status": "idle" + }, + "node_uid": "1", + "redis_info": { + "connected_clients": 14, + "used_memory_rss": 12460032 + }, + "report_timestamp": "2024-09-13T15:28:10Z", + "status": "active" + }, + { + "uid": 2, + "role": "slave", + // additional fields... + } +] +``` + +### Status codes {#get-status-codes} + +| Code | Description | +|------|-------------| +| [200 OK](https://www.rfc-editor.org/rfc/rfc9110.html#name-200-ok) | No error. | +| [404 Not Found](https://www.rfc-editor.org/rfc/rfc9110.html#name-404-not-found) | bdb uid does not exist. | From 7662b4109bf66e00903a443ab1c9e6474e8e605f Mon Sep 17 00:00:00 2001 From: Rachel Elledge Date: Fri, 13 Sep 2024 16:00:03 -0500 Subject: [PATCH 9/9] DOC-3850 Create separate shard migration article and crosslink with rladmin & REST API references --- .../operate/rs/databases/migrate-shards.md | 144 ++++++++++++++++++ .../cli-utilities/rladmin/migrate.md | 2 + .../requests/shards/actions/migrate.md | 28 +--- 3 files changed, 152 insertions(+), 22 deletions(-) create mode 100644 content/operate/rs/databases/migrate-shards.md diff --git a/content/operate/rs/databases/migrate-shards.md b/content/operate/rs/databases/migrate-shards.md new file mode 100644 index 0000000000..12469ffb99 --- /dev/null +++ b/content/operate/rs/databases/migrate-shards.md @@ -0,0 +1,144 @@ +--- +alwaysopen: false +categories: +- docs +- operate +- rs +db_type: database +description: How to migrate database shards to other nodes in a Redis Software cluster. +linkTitle: Migrate shards +title: Migrate database shards +toc: 'true' +weight: 32 +--- + +To migrate database shards to other nodes in the cluster, you can use the [`rladmin migrate`]({{}}) command or [REST API requests]({{}}). + +## Use cases for shard migration + +Migrate database shards to a different node in the following scenarios: + +- Before node removal. + +- To balance the database manually in case of latency issues or uneven load distribution across nodes. + +- To manage node resources, such as memory usage. + +## Considerations for shard migration + +For databases with replication: + +- Migrating a shard will not cause disruptions since a primary shard will still be available. + +- If you try to migrate a primary shard, it will be demoted to a replica shard and a replica shard will be promoted to primary before the migration. If you set `"preserve_roles": true` in the request, a second failover will occur after the migration finishes to change the migrated shard's role back to primary. + +For databases without replication, the migrated shard will not be available until the migration is done. + +Connected clients shouldn't be disconnected in either case. + +If too many primary shards are placed on the same node, it can impact database performance. + +## Migrate specific shard + +To migrate a specific database shard, use one of the following methods: + +- [`rladmin migrate shard`]({{}}): + + ```sh + rladmin migrate shard target_node + ``` + +- [Migrate shard]({{}}) REST API request: + + Specify the ID of the shard to migrate in the request path and the destination node's ID as the `target_node_uid` in the request body. See the [request reference]({{}}) for more options. + + ```sh + POST /v1/shards//actions/migrate + { + "target_node_uid": + } + ``` + + Example JSON response body: + + ```json + { + "action_uid": "", + "description": "Migrate was triggered" + } + ``` + + You can track the action's progress with a [`GET /v1/actions/`]({{}}) request. + +## Migrate multiple shards + +To migrate multiple database shards, use one of the following methods: + +- [`rladmin migrate shard`]({{}}): + + ```sh + rladmin migrate shard target_node + ``` + +- [Migrate multiple shards]({{}}) REST API request: + + Specify the IDs of the shards to migrate in the `shard_uids` list and the destination node's ID as the `target_node_uid` in the request body. See the [request reference]({{}}) for more options. + + ```sh + POST /v1/shards/actions/migrate + { + "shard_uids": ["","",""], + "target_node_uid": + } + ``` + + Example JSON response body: + + ```json + { + "action_uid": "", + "description": "Migrate was triggered" + } + ``` + + You can track the action's progress with a [`GET /v1/actions/`]({{}}) request. + +## Migrate all shards from a node + +To migrate all shards from a specific node to another node, run [`rladmin migrate all_shards`]({{}}): + +```sh +rladmin migrate node all_shards target_node +``` + +## Migrate primary shards + +You can use the [`rladmin migrate all_master_shards`]({{}}) command to migrate all primary shards for a specific database or node to another node in the cluster. + +To migrate a specific database's primary shards: + +```sh +rladmin migrate db db: all_master_shards target_node +``` + +To migrate all primary shards from a specific node: + +```sh +rladmin migrate node all_master_shards target_node +``` + +## Migrate replica shards + +You can use the [`rladmin migrate all_slave_shards`]({{}}) command to migrate all replica shards for a specific database or node to another node in the cluster. + +To migrate a specific database's replica shards: + +```sh +rladmin migrate db db: all_slave_shards target_node +``` + +To migrate all replica shards from a specific node: + +```sh +rladmin migrate node all_slave_shards target_node +``` diff --git a/content/operate/rs/references/cli-utilities/rladmin/migrate.md b/content/operate/rs/references/cli-utilities/rladmin/migrate.md index 913b5334e4..dc33ea3532 100644 --- a/content/operate/rs/references/cli-utilities/rladmin/migrate.md +++ b/content/operate/rs/references/cli-utilities/rladmin/migrate.md @@ -15,6 +15,8 @@ weight: $weight Moves Redis Enterprise shards or endpoints to a new node in the same cluster. +For more information about shard migration use cases and considerations, see [Migrate database shards]({{}}). + ## `migrate all_master_shards` Moves all primary shards of a specified database or node to a new node in the same cluster. diff --git a/content/operate/rs/references/rest-api/requests/shards/actions/migrate.md b/content/operate/rs/references/rest-api/requests/shards/actions/migrate.md index 8ea42a71cd..7c535291da 100644 --- a/content/operate/rs/references/rest-api/requests/shards/actions/migrate.md +++ b/content/operate/rs/references/rest-api/requests/shards/actions/migrate.md @@ -16,32 +16,14 @@ weight: $weight | [POST](#post-multi-shards) | `/v1/shards/actions/migrate` | Migrate multiple shards | | [POST](#post-shard) | `/v1/shards/{uid}/actions/migrate` | Migrate a specific shard | -Migrate shards to a different node in the following scenarios: - -- Before node removal. - -- To balance the database manually in case of latency issues or uneven load distribution across nodes. - -- To manage node resources, such as memory usage. - -For databases with replication: - -- Migrating a shard will not cause disruptions since a primary shard will still be available. - -- If you try to migrate a primary shard, it will be demoted to a replica shard and a replica shard will be promoted to primary before the migration. If you set `"preserve_roles": true` in the request, a second failover will occur after the migration finishes to change the migrated shard's role back to primary. - -For databases without replication, the migrated shard will not be available until the migration is done. - -Connected clients shouldn't be disconnected in either case. - -If too many primary shards are placed on the same node, it can impact database performance. - ## Migrate multiple shards {#post-multi-shards} POST /v1/shards/actions/migrate Migrates the list of given shard UIDs to the node specified by `target_node_uid`. The shards can be from multiple databases. This request is asynchronous. +For more information about shard migration use cases and considerations, see [Migrate database shards]({{}}). + #### Required permissions | Permission name | Roles | @@ -73,7 +55,7 @@ Migrates the list of given shard UIDs to the node specified by `target_node_uid` | Host | cnm.cluster.fqdn | Domain name | | Accept | application/json | Accepted media type | -#### Request body +#### Request body {#post-multi-request-body} The request body is a JSON object that can contain the following fields: @@ -115,6 +97,8 @@ Returns a JSON object with an `action_uid`. You can track the action's progress Migrates the shard with the given `shard_uid` to the node specified by `target_node_uid`. If the shard is already on the target node, nothing happens. This request is asynchronous. +For more information about shard migration use cases and considerations, see [Migrate database shards]({{}}). + #### Required permissions | Permission name | Roles | @@ -152,7 +136,7 @@ Migrates the shard with the given `shard_uid` to the node specified by `target_n | uid | integer | The unique ID of the shard to migrate. | -#### Request body +#### Request body {#post-request-body} The request body is a JSON object that can contain the following fields: