Performance difference between builder & derive

[crowlKats]

Over at Deno we have been wondering if there is a sizeable performance gain for using the derive macro instead of the builder, as we have a large set of arguments and subcommands. Has performance between the two different solutions been benchmarked somewhere?

[epage]

We have not done direct performance comparisons but they should be similar. The derive API just generates the builder objects, calls the regular parser, and then extracts values from ArgMatches. We haven't done any work to bypass this. It is a consideration to do so but that is lower down in our priority list.

[epage]

Has the performance of clap been noticeable enough to be a concern?

[crowlKats]

I have tried #4792, and it seems to have a considerable performance benefit

[epage]

Any quantification to that? How's it do in absolute terms and relative to pre-allocating? Are you all wanting pre-allocating in addition to it, and if so, how much of a difference does it still make for me to evaluate whether I want to take on pre-allocating in the API?

[crowlKats]

Sorry for the late reply! Here is a flamegraph:

This is with all subcommands deferred except the run subcommand (which is the most commonly used one, and the one we benchmark on)

[epage]

After (too much) delay, #4792 is merged and available in v4.3.3. I'm sorry for howl long this took.

[frol]

Just for the reference, I just want to mention that there is a performance difference between builder & derive if defer API is used as that one is only available through manual builder pattern and the fix for the derive was not accepted to the upstream: #6225. In my case, 330 deeply nested structs that use derives were contributing to 5 seconds start up time, which I got down to 10 milliseconds with the mentioned fix.

[epage]

First, sorry I haven't finished up #4792.

I did have another question though. If I slowed down start up time for debug builds (see #4838), is that something you'd notice or care about?

[bartlomieju]

Hey @epage, no worries, we got side-tracked with other work too.

I did have another question though. If I slowed down start up time for debug builds (see #4838), is that something you'd notice or care about?

If it doesn't cause multi-second latency then it's completely fine for us :)