Speed up merge

[JakeRoggenbuck]

fixes #150

[JakeRoggenbuck]

(venv) benchmarks (speed-up-merge) λ RUST_LOG=debug p scaling_tester.py
Time taken to update 10 records: 0.0001 seconds
Time taken to update 100 records: 0.0002 seconds
Time taken to update 1000 records: 0.0021 seconds
Time taken to update 10000 records: 0.0234 seconds
Time taken to update 100000 records: 0.2720 seconds
[2025-03-01T00:43:18Z INFO  redoxql::pagerange] Starting merge!
[2025-03-01T00:43:18Z INFO  redoxql::pagerange] Finished first clone of base_container and tail_container.
[2025-03-01T00:43:18Z INFO  redoxql::pagerange] Starting loop over tail_rids_to_process with len = 100001.
[2025-03-01T00:43:52Z INFO  redoxql::pagerange] Finished first loop section.
[2025-03-01T00:43:52Z INFO  redoxql::pagerange] About to get from and insert into page_directory 100001 times. Which are both O(log n). As well as reading and writing O(n) pages.
[2025-03-01T00:43:52Z INFO  redoxql::pagerange] Merge finished!
[2025-03-01T00:43:52Z INFO  redoxql::pagerange] Starting merge!
[2025-03-01T00:43:52Z INFO  redoxql::pagerange] Finished first clone of base_container and tail_container.
[2025-03-01T00:43:52Z INFO  redoxql::pagerange] Starting loop over tail_rids_to_process with len = 200002.
[2025-03-01T00:45:08Z INFO  redoxql::pagerange] Finished first loop section.
[2025-03-01T00:45:08Z INFO  redoxql::pagerange] About to get from and insert into page_directory 200002 times. Which are both O(log n). As well as reading and writing O(n) pages.
[2025-03-01T00:45:08Z INFO  redoxql::pagerange] Merge finished!
[2025-03-01T00:45:09Z INFO  redoxql::pagerange] Starting merge!
[2025-03-01T00:45:09Z INFO  redoxql::pagerange] Finished first clone of base_container and tail_container.
[2025-03-01T00:45:09Z INFO  redoxql::pagerange] Starting loop over tail_rids_to_process with len = 300003.
[2025-03-01T00:47:05Z INFO  redoxql::pagerange] Finished first loop section.
[2025-03-01T00:47:05Z INFO  redoxql::pagerange] About to get from and insert into page_directory 300003 times. Which are both O(log n). As well as reading and writing O(n) pages.
[2025-03-01T00:47:05Z INFO  redoxql::pagerange] Merge finished!
[2025-03-01T00:47:05Z INFO  redoxql::pagerange] Starting merge!
[2025-03-01T00:47:05Z INFO  redoxql::pagerange] Finished first clone of base_container and tail_container.
[2025-03-01T00:47:05Z INFO  redoxql::pagerange] Starting loop over tail_rids_to_process with len = 400004.
[2025-03-01T00:49:50Z INFO  redoxql::pagerange] Finished first loop section.
[2025-03-01T00:49:50Z INFO  redoxql::pagerange] About to get from and insert into page_directory 400004 times. Which are both O(log n). As well as reading and writing O(n) pages.
[2025-03-01T00:49:50Z INFO  redoxql::pagerange] Merge finished!
[2025-03-01T00:49:51Z INFO  redoxql::pagerange] Starting merge!
[2025-03-01T00:49:51Z INFO  redoxql::pagerange] Finished first clone of base_container and tail_container.
[2025-03-01T00:49:51Z INFO  redoxql::pagerange] Starting loop over tail_rids_to_process with len = 500005.
[2025-03-01T00:53:25Z INFO  redoxql::pagerange] Finished first loop section.
[2025-03-01T00:53:25Z INFO  redoxql::pagerange] About to get from and insert into page_directory 500005 times. Which are both O(log n). As well as reading and writing O(n) pages.
[2025-03-01T00:53:25Z INFO  redoxql::pagerange] Merge finished!
[2025-03-01T00:53:25Z INFO  redoxql::pagerange] Starting merge!
[2025-03-01T00:53:25Z INFO  redoxql::pagerange] Finished first clone of base_container and tail_container.
[2025-03-01T00:53:25Z INFO  redoxql::pagerange] Starting loop over tail_rids_to_process with len = 600006.

[JakeRoggenbuck]

PageRange_Merge/Original/100
time:   [35.338 µs 35.554 µs 35.820 µs]
change: [+1.2285% +1.9299% +2.6338%] (p = 0.00 < 0.05)
Performance has regressed.

PageRange_Merge/Optimized/100
time:   [36.692 µs 36.798 µs 36.862 µs]
change: [+0.3306% +0.7394% +1.1385%] (p = 0.00 < 0.05)
Change within noise threshold.
Found 2 outliers among 10 measurements (20.00%)
2 (20.00%) low mild


PageRange_Merge/Original/1000
time:   [82.048 µs 83.336 µs 84.033 µs]
change: [-1.9076% -0.7334% +0.3362%] (p = 0.26 > 0.05)
No change in performance detected.

PageRange_Merge/Optimized/1000
time:   [83.050 µs 83.272 µs 83.543 µs]
change: [-2.0228% -0.7568% +0.4783%] (p = 0.30 > 0.05)
No change in performance detected.
Found 2 outliers among 10 measurements (20.00%)
2 (20.00%) high mild


PageRange_Merge/Original/10000
time:   [681.81 µs 685.57 µs 691.53 µs]
Found 1 outliers among 10 measurements (10.00%)
1 (10.00%) high mild

PageRange_Merge/Optimized/10000
time:   [672.23 µs 676.68 µs 681.36 µs]


PageRange_Merge/Original/100000
time:   [16.162 ms 16.269 ms 16.431 ms]

PageRange_Merge/Optimized/100000
time:   [15.874 ms 15.927 ms 16.013 ms]

[JakeRoggenbuck]

PageRange_Merge/Original/100
time:   [34.885 µs 34.981 µs 35.044 µs]
change: [-2.2890% -1.6167% -0.9163%] (p = 0.00 < 0.05)
Change within noise threshold.

PageRange_Merge/New/100 time:   [50.268 µs 50.503 µs 50.662 µs]

PageRange_Merge/Optimized/100
time:   [41.244 µs 41.559 µs 41.907 µs]
change: [+11.975% +13.011% +14.170%] (p = 0.00 < 0.05)
Performance has regressed.
Found 1 outliers among 10 measurements (10.00%)
1 (10.00%) high mild

PageRange_Merge/Original/1000
time:   [86.359 µs 86.577 µs 86.889 µs]
change: [+3.6061% +4.6689% +5.7049%] (p = 0.00 < 0.05)
Performance has regressed.
Found 2 outliers among 10 measurements (20.00%)
2 (20.00%) high mild

PageRange_Merge/New/1000
time:   [149.77 µs 150.22 µs 150.54 µs]

PageRange_Merge/Optimized/1000
time:   [88.372 µs 88.853 µs 89.246 µs]
change: [+5.0816% +5.9741% +6.7722%] (p = 0.00 < 0.05)
Performance has regressed.

PageRange_Merge/Original/10000
time:   [674.49 µs 676.98 µs 679.90 µs]
change: [-1.6878% -0.7619% +0.2993%] (p = 0.18 > 0.05)
No change in performance detected.
Found 1 outliers among 10 measurements (10.00%)
1 (10.00%) high severe

PageRange_Merge/New/10000
time:   [1.1918 ms 1.2988 ms 1.4509 ms]
Found 2 outliers among 10 measurements (20.00%)
2 (20.00%) high severe

PageRange_Merge/Optimized/10000
time:   [674.87 µs 679.71 µs 684.31 µs]
change: [-0.7660% +0.0908% +1.0356%] (p = 0.85 > 0.05)
No change in performance detected.

PageRange_Merge/Original/100000
time:   [16.836 ms 16.977 ms 17.183 ms]
change: [+1.4276% +3.2404% +5.0597%] (p = 0.00 < 0.05)
Performance has regressed.
Found 1 outliers among 10 measurements (10.00%)
1 (10.00%) high severe

PageRange_Merge/New/100000
time:   [25.086 ms 25.180 ms 25.344 ms]

PageRange_Merge/Optimized/100000
time:   [16.540 ms 16.690 ms 16.893 ms]
change: [+2.2168% +4.3335% +6.3968%] (p = 0.00 < 0.05)
Performance has regressed.