Skip to content

Conversation

@benluddy
Copy link
Contributor

@benluddy benluddy commented Dec 18, 2025

Set deserialization with children in reverse order performs an insertion sort. Sets serialized by structured-merge-diff are sorted, but in practice the serialization of sets is controlled by clients and they can (intentionally or not) exercise this worst case. Switching to a bulk insert followed by a sort appears to perform no worse in the best case (sorted) and is significantly better in the worst case:

                                              │     master     │                 bulk                 │
                                              │     sec/op     │    sec/op     vs base                │
SetFromJSON/children_in_ascending_order            1.286m ± 3%   1.341m ±  1%   +4.27% (p=0.000 n=10)
SetFromJSON/children_in_descending_order          20.189m ± 0%   1.385m ±  1%  -93.14% (p=0.000 n=10)
SetFromJSON/grandchildren_in_ascending_order       7.821m ± 1%   8.211m ±  1%   +4.98% (p=0.000 n=10)
SetFromJSON/grandchildren_in_descending_order      75.16m ± 4%   13.31m ± 13%  -82.28% (p=0.000 n=10)
geomean                                            11.11m        3.774m        -66.04%

                                              │     master     │                 bulk                 │
                                              │      B/op      │     B/op      vs base                │
SetFromJSON/children_in_ascending_order           1.320Mi ± 0%   1.320Mi ± 0%        ~ (p=1.000 n=10)
SetFromJSON/children_in_descending_order          1.320Mi ± 0%   1.320Mi ± 0%   +0.00% (p=0.000 n=10)
SetFromJSON/grandchildren_in_ascending_order      5.395Mi ± 0%   5.395Mi ± 0%   +0.00% (p=0.000 n=10)
SetFromJSON/grandchildren_in_descending_order     6.187Mi ± 0%   5.395Mi ± 0%  -12.80% (p=0.000 n=10)
geomean                                           2.761Mi        2.668Mi        -3.37%

                                              │     master     │                 bulk                  │
                                              │   allocs/op    │  allocs/op   vs base                  │
SetFromJSON/children_in_ascending_order            25.99k ± 0%   25.99k ± 0%        ~ (p=1.000 n=10) ¹
SetFromJSON/children_in_descending_order           25.99k ± 0%   25.99k ± 0%        ~ (p=1.000 n=10) ¹
SetFromJSON/grandchildren_in_ascending_order       138.4k ± 0%   138.4k ± 0%        ~ (p=1.000 n=10) ¹
SetFromJSON/grandchildren_in_descending_order      155.7k ± 0%   138.4k ± 0%  -11.11% (p=0.000 n=10)
geomean                                            61.77k        59.98k        -2.90%
¹ all samples are equal

This was originally proposed in #292. All I've done here is added benchmarks to demonstrate the worst case performance and separated it from the json-iterator migration.

@k8s-ci-robot k8s-ci-robot requested a review from yliaog December 18, 2025 20:33
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: benluddy
Once this PR has been reviewed and has the lgtm label, please assign jpbetz for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Dec 18, 2025
@benluddy
Copy link
Contributor Author

/cc @jpbetz @liggitt @inteon

@benluddy benluddy force-pushed the fromjson-post-accumulation-sort branch from 6c869cc to 367149b Compare December 19, 2025 15:58
@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Dec 19, 2025
Set deserialization with children in reverse order performs an insertion sort. Sets serialized by
structured-merge-diff are sorted, but in practice the serialization of sets is controlled by clients
and they can (intentionally or not) exercise this worst case. Switching to a bulk insert followed by
a sort appears to perform no worse in the best case (sorted) and is significantly better in the
worst case:

                                              │     master     │                 bulk                 │
                                              │     sec/op     │    sec/op     vs base                │
SetFromJSON/children_in_ascending_order            1.286m ± 3%   1.341m ±  1%   +4.27% (p=0.000 n=10)
SetFromJSON/children_in_descending_order          20.189m ± 0%   1.385m ±  1%  -93.14% (p=0.000 n=10)
SetFromJSON/grandchildren_in_ascending_order       7.821m ± 1%   8.211m ±  1%   +4.98% (p=0.000 n=10)
SetFromJSON/grandchildren_in_descending_order      75.16m ± 4%   13.31m ± 13%  -82.28% (p=0.000 n=10)
geomean                                            11.11m        3.774m        -66.04%

                                              │     master     │                 bulk                 │
                                              │      B/op      │     B/op      vs base                │
SetFromJSON/children_in_ascending_order           1.320Mi ± 0%   1.320Mi ± 0%        ~ (p=1.000 n=10)
SetFromJSON/children_in_descending_order          1.320Mi ± 0%   1.320Mi ± 0%   +0.00% (p=0.000 n=10)
SetFromJSON/grandchildren_in_ascending_order      5.395Mi ± 0%   5.395Mi ± 0%   +0.00% (p=0.000 n=10)
SetFromJSON/grandchildren_in_descending_order     6.187Mi ± 0%   5.395Mi ± 0%  -12.80% (p=0.000 n=10)
geomean                                           2.761Mi        2.668Mi        -3.37%

                                              │     master     │                 bulk                  │
                                              │   allocs/op    │  allocs/op   vs base                  │
SetFromJSON/children_in_ascending_order            25.99k ± 0%   25.99k ± 0%        ~ (p=1.000 n=10) ¹
SetFromJSON/children_in_descending_order           25.99k ± 0%   25.99k ± 0%        ~ (p=1.000 n=10) ¹
SetFromJSON/grandchildren_in_ascending_order       138.4k ± 0%   138.4k ± 0%        ~ (p=1.000 n=10) ¹
SetFromJSON/grandchildren_in_descending_order      155.7k ± 0%   138.4k ± 0%  -11.11% (p=0.000 n=10)
geomean                                            61.77k        59.98k        -2.90%
¹ all samples are equal
@benluddy benluddy force-pushed the fromjson-post-accumulation-sort branch from 367149b to 26509bc Compare December 19, 2025 16:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants