From ced63b89d62b7d77935466aad905c1e8513deaa8 Mon Sep 17 00:00:00 2001 From: mich-elle-luna Date: Thu, 18 Sep 2025 09:55:04 -0700 Subject: [PATCH 1/4] Add resharding performance factors section to memory-limit documentation - Add comprehensive resharding process overview explaining 3-stage process - Document performance characteristics and factors affecting resharding duration - Include flex sharding performance improvements and key length considerations - Add traffic impact analysis during resharding operations - Use proper Hugo note shortcode for atomic operation warning - Align content with Google technical writing style for clarity and consistency This addition provides Redis Enterprise administrators with essential information for planning resharding operations and understanding performance implications. --- .../memory-performance/memory-limit.md | 42 +++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/content/operate/rs/databases/memory-performance/memory-limit.md b/content/operate/rs/databases/memory-performance/memory-limit.md index 59248269db..2932be3ad5 100644 --- a/content/operate/rs/databases/memory-performance/memory-limit.md +++ b/content/operate/rs/databases/memory-performance/memory-limit.md @@ -79,6 +79,48 @@ Databases should return to their configured memory limits after data is removed If you observe this behavior, consider [monitoring]({{< relref "/operate/rs/monitoring/" >}}) memory usage and controlling the traffic load on the application side. +## Resharding duration factors + +Resharding enables you to increase database capacity by adding shards and utilizing more cluster memory. Understanding resharding duration helps you plan maintenance operations and minimize database risk. + +### Resharding process overview + +Resharding follows a three-stage process for each new shard: + +1. **Shard creation**: Creates a new shard and performs partial synchronization based on assigned hash slots +2. **Key trimming**: Removes keys from the original shard according to new hash slot assignments +3. **Defragmentation**: Executes failovers to optimize new shard performance + +When resharding multiple shards (for example, expanding from 4 to 8 shards), this process runs serially for each new shard. The operation completes only after all stages finish for every shard. + +{{< note >}} +Resharding is an atomic operation that cannot be interrupted. Database corruption can occur if the process fails or stops unexpectedly. Minimizing resharding time reduces this risk. +{{< /note >}} + +Flexible sharding optimizes the key trimming stage, significantly reducing the time required to remove unnecessary keys from original shards. + +### Duration characteristics + +Resharding duration increases linearly with the number of keys in your database. Several factors affect resharding duration: + +### Flex sharding impact + +Flex sharding improves resharding duration compared to standard sharding. + +### Key length considerations + +Key length directly affects resharding duration. Longer keys require more time to process due to increased hash calculation overhead per key. + +The decrease in duration provided by flex sharding varies based on key length: + +- **Short keys (10 bytes)**: Flex sharding provides up to 50% improvement +- **Long keys (2000 KB)**: Flex sharding provides minimal improvement (approximately 11%) +- **Critical threshold**: Between 50-100 bytes, flex sharding advantages begin to diminish + +### Traffic impact during resharding + +Network traffic has a measurable but limited effect on resharding duration. Since resharding operations typically don't reach CPU limits, the impact on both resharding time and ongoing traffic remains minimal. + ## Memory metrics The Cluster Manager UI provides metrics that can help you evaluate your memory use. From c7e078e1fda2109dfdbd155c65485e7efa674ab6 Mon Sep 17 00:00:00 2001 From: mich-elle-luna Date: Fri, 19 Sep 2025 07:27:57 -0700 Subject: [PATCH 2/4] Incorporate feedback on resharding performance section - Refine content based on review feedback - Improve clarity and technical accuracy of resharding documentation - Update performance characteristics and factor descriptions - Enhance readability and flow of the resharding process explanation --- .../memory-performance/memory-limit.md | 18 +++++------------- 1 file changed, 5 insertions(+), 13 deletions(-) diff --git a/content/operate/rs/databases/memory-performance/memory-limit.md b/content/operate/rs/databases/memory-performance/memory-limit.md index 2932be3ad5..7753f758ea 100644 --- a/content/operate/rs/databases/memory-performance/memory-limit.md +++ b/content/operate/rs/databases/memory-performance/memory-limit.md @@ -83,7 +83,7 @@ If you observe this behavior, consider [monitoring]({{< relref "/operate/rs/moni Resharding enables you to increase database capacity by adding shards and utilizing more cluster memory. Understanding resharding duration helps you plan maintenance operations and minimize database risk. -### Resharding process overview +### Resharding overview Resharding follows a three-stage process for each new shard: @@ -99,26 +99,18 @@ Resharding is an atomic operation that cannot be interrupted. Database corruptio Flexible sharding optimizes the key trimming stage, significantly reducing the time required to remove unnecessary keys from original shards. -### Duration characteristics +### Resharding duration factors -Resharding duration increases linearly with the number of keys in your database. Several factors affect resharding duration: +Resharding duration increases linearly with the number of keys in your database. Flexible sharding, key length, and network traffic affect resharding duration: -### Flex sharding impact +Flexible sharding improves resharding duration compared to standard sharding. -Flex sharding improves resharding duration compared to standard sharding. - -### Key length considerations - -Key length directly affects resharding duration. Longer keys require more time to process due to increased hash calculation overhead per key. - -The decrease in duration provided by flex sharding varies based on key length: +Key length directly affects resharding duration. Longer keys require more time to process due to increased hash calculation overhead per key. The decrease in duration provided by flex sharding varies based on key length: - **Short keys (10 bytes)**: Flex sharding provides up to 50% improvement - **Long keys (2000 KB)**: Flex sharding provides minimal improvement (approximately 11%) - **Critical threshold**: Between 50-100 bytes, flex sharding advantages begin to diminish -### Traffic impact during resharding - Network traffic has a measurable but limited effect on resharding duration. Since resharding operations typically don't reach CPU limits, the impact on both resharding time and ongoing traffic remains minimal. ## Memory metrics From fbb396c790fd5a10d38f4fbe2bee98412248e197 Mon Sep 17 00:00:00 2001 From: mich-elle-luna Date: Fri, 19 Sep 2025 07:29:35 -0700 Subject: [PATCH 3/4] Update terminology consistency: use 'Flexible sharding' instead of 'Flex sharding' - Standardize terminology throughout the resharding performance section - Ensure consistent naming convention for flexible sharding feature --- .../operate/rs/databases/memory-performance/memory-limit.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/operate/rs/databases/memory-performance/memory-limit.md b/content/operate/rs/databases/memory-performance/memory-limit.md index 7753f758ea..76dd0bc426 100644 --- a/content/operate/rs/databases/memory-performance/memory-limit.md +++ b/content/operate/rs/databases/memory-performance/memory-limit.md @@ -107,8 +107,8 @@ Flexible sharding improves resharding duration compared to standard sharding. Key length directly affects resharding duration. Longer keys require more time to process due to increased hash calculation overhead per key. The decrease in duration provided by flex sharding varies based on key length: -- **Short keys (10 bytes)**: Flex sharding provides up to 50% improvement -- **Long keys (2000 KB)**: Flex sharding provides minimal improvement (approximately 11%) +- **Short keys (10 bytes)**: Flexible sharding provides up to 50% improvement +- **Long keys (2000 KB)**: Flexible sharding provides minimal improvement (approximately 11%) - **Critical threshold**: Between 50-100 bytes, flex sharding advantages begin to diminish Network traffic has a measurable but limited effect on resharding duration. Since resharding operations typically don't reach CPU limits, the impact on both resharding time and ongoing traffic remains minimal. From 785dbbb21dbd52c7da19f66594a396655c27e1c8 Mon Sep 17 00:00:00 2001 From: mich-elle-luna <153109578+mich-elle-luna@users.noreply.github.com> Date: Thu, 25 Sep 2025 11:09:33 -0700 Subject: [PATCH 4/4] Update memory-limit.md --- .../rs/databases/memory-performance/memory-limit.md | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-) diff --git a/content/operate/rs/databases/memory-performance/memory-limit.md b/content/operate/rs/databases/memory-performance/memory-limit.md index 76dd0bc426..430d487b2b 100644 --- a/content/operate/rs/databases/memory-performance/memory-limit.md +++ b/content/operate/rs/databases/memory-performance/memory-limit.md @@ -97,19 +97,11 @@ When resharding multiple shards (for example, expanding from 4 to 8 shards), thi Resharding is an atomic operation that cannot be interrupted. Database corruption can occur if the process fails or stops unexpectedly. Minimizing resharding time reduces this risk. {{< /note >}} -Flexible sharding optimizes the key trimming stage, significantly reducing the time required to remove unnecessary keys from original shards. - ### Resharding duration factors -Resharding duration increases linearly with the number of keys in your database. Flexible sharding, key length, and network traffic affect resharding duration: - -Flexible sharding improves resharding duration compared to standard sharding. - -Key length directly affects resharding duration. Longer keys require more time to process due to increased hash calculation overhead per key. The decrease in duration provided by flex sharding varies based on key length: +Resharding duration increases linearly with the number of keys in your database. Key length and network traffic affect resharding duration: -- **Short keys (10 bytes)**: Flexible sharding provides up to 50% improvement -- **Long keys (2000 KB)**: Flexible sharding provides minimal improvement (approximately 11%) -- **Critical threshold**: Between 50-100 bytes, flex sharding advantages begin to diminish +Key length directly affects resharding duration. Longer keys (2000 KB) require more time to process due to increased hash calculation overhead per key compared to short keys (10 bytes). Network traffic has a measurable but limited effect on resharding duration. Since resharding operations typically don't reach CPU limits, the impact on both resharding time and ongoing traffic remains minimal.