Is there a macro like `Base.@timed` or `BenchmarkTools.@btimed`?

[singularitti]

According to Base.@timed docs, @timed is

A macro to execute an expression, and return the value of the expression, elapsed time in seconds, total bytes allocated, garbage collection time, an object with various memory allocation counters, compilation time in seconds, and recompilation time in seconds.

Since version v1.6.0, BenchmarkTools.jl has added macro @btimed.
I wonder if Chairmarks.jl has the same tool or has it planned. I am migrating from BenchmarkTools.jl to Chairmarks.jl, it would be nice to have the same tool without changing much of my code.

[LilithHafner]

What is the specific use-case that motivates this? I generally recommend @time and @timed (here's a longer explanation) and am happy to hear about important use cases that @time and @timed don't cover.

[singularitti]

Hi @LilithHafner, thank you for your timely response! The use-case I am having is that I am calculating the same function executed given different input sizes, and I want to save the result and the timing information. You can think of an example like below:

timed_results = map(100:100:1000) do iter
    value, time = @btimed f($iter) samples=3
end

Then, I want to plot the computed values or their time cost v.s. number of iterations. But the time could range from milliseconds to minutes, and the value accuracy could range from 1e-4 to 1e-9. I want to do all these things with a simple macro like @btimed in 1 for-loop. I am not sure @timed will give me the accurate time as @btimed.

The example you mentioned in the other issue:

julia> x = Ref{Float64}()
Base.RefValue{Float64}(0.0)

julia> @b rand(100) sum x[] = _
12.166 ns

julia> x[]
52.6901305539253

requires to specify the type of x explicitly. This may work for simple types, but my f may return complicated types.

[LilithHafner]

You could use a Ref{Any}, but that is certainly not as clean as @btimed.

There's a real, but not common usecase for this; I'll have to think about how I want to balance feature-rich-ness vs scope creep, maintainability, and concision of src and docs.

[singularitti]

Will the Ref{Any} assignment be counted in the benchmark? I guess not, because that is in the teardown process?

[LilithHafner]

It will not be counted. This can also be easily verified:

julia> x = Ref{Any}()
Base.RefValue{Any}(#undef)

julia> @b rand hash
2.554 ns

julia> @b rand hash x[] = _
2.563 ns

julia> x[]
0x376f126f3b0be0e8

[singularitti]

Thanks for your help! Although I still think this syntax is a bit overdoing a simple thing. I would appreciate it if you could consider my request.

[singularitti]

I have another reason for returning the result:

result = Ref{Any}()
@b a_very_long_function(
    args...
) result[] = _ samples = 1 evals = 1

When we give arguments like samples & evals to @b or @be, they mingle with the statement result[] = _, which makes people confused about where _ applies to.

[LilithHafner]

This is now available in CharimarksExtras.jl

[singularitti]

Great, thank you!