Skip to content

Commit 67199c6

Browse files
Copilotavifenesh
andcommitted
Replace MSET batching with individual SET operations to avoid cross-slot issues
Co-authored-by: avifenesh <[email protected]>
1 parent b527e88 commit 67199c6

File tree

1 file changed

+10
-32
lines changed

1 file changed

+10
-32
lines changed

topics/cluster-tutorial.md

Lines changed: 10 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -414,32 +414,19 @@ async function runExample() {
414414

415415
console.log(`Starting from counter: ${last}`);
416416

417-
// Write keys in batches using mset for better performance
418-
const batchSize = 100;
419-
for (let start = last + 1; start <= 1000000000; start += batchSize) {
417+
// Write keys sequentially with individual SET operations
418+
for (let x = last + 1; x <= 1000000000; x++) {
420419
try {
421-
const keyValuePairs = [];
422-
const end = Math.min(start + batchSize - 1, 1000000000);
420+
await client.set(`foo${x}`, x.toString());
423421

424-
// Prepare batch of key-value pairs as array
425-
for (let x = start; x <= end; x++) {
426-
keyValuePairs.push(`foo${x}`, x.toString());
422+
// Update counter every 1000 operations and display progress
423+
if (x % 1000 === 0) {
424+
await client.set("__last__", x.toString());
425+
console.log(`Progress: ${x} keys written`);
427426
}
428427

429-
// Execute batch mset with array format
430-
await client.mset(keyValuePairs);
431-
432-
// Update counter and display progress
433-
await client.set("__last__", end.toString());
434-
console.log(`Batch completed: ${start} to ${end}`);
435-
436-
// Verify a sample key from the batch
437-
const sampleKey = `foo${start}`;
438-
const value = await client.get(sampleKey);
439-
console.log(`Sample verification - ${sampleKey}: ${value}`);
440-
441428
} catch (error) {
442-
console.log(`Error in batch starting at ${start}: ${error.message}`);
429+
console.log(`Error writing key foo${x}: ${error.message}`);
443430
}
444431
}
445432
} catch (error) {
@@ -452,11 +439,7 @@ async function runExample() {
452439
runExample().catch(console.error);
453440
```
454441

455-
The application writes keys in the format `foo<number>` with their corresponding numeric values, using batched `MSET` operations for better performance. The `MSET` command accepts an array of alternating keys and values. So if you run the program the result is batches of `MSET` commands:
456-
457-
* `MSET foo1 1 foo2 2 foo3 3 ... foo100 100` (batch of 100 keys)
458-
* `MSET foo101 101 foo102 102 ... foo200 200` (next batch)
459-
* And so forth...
442+
The application writes keys in the format `foo<number>` with their corresponding numeric values using individual `SET` operations. This approach ensures compatibility with cluster deployments where keys may be distributed across different nodes based on their hash slots.
460443

461444
The program includes comprehensive error handling to display errors instead of
462445
crashing, so all cluster operations are wrapped in try-catch blocks.
@@ -476,12 +459,7 @@ The **counter initialization section** reads a counter so that when we restart t
476459
we don't start again with `foo0`, but continue from where we left off.
477460
The counter is stored in Valkey itself using the key `__last__`.
478461

479-
The **main processing loop** sets keys in batches using `MSET` operations
480-
for better performance, processing 100 keys at a time and displaying progress or
481-
any errors that occur.
482-
you'll get the usually 10k ops/second in the best of the conditions).
483-
484-
you'll get optimal performance).
462+
The **main processing loop** sets keys sequentially using individual `SET` operations, updating progress every 1000 keys and displaying any errors that occur.
485463

486464
Starting the application produces the following output:
487465

0 commit comments

Comments
 (0)