Skip to content
This repository was archived by the owner on Jul 4, 2023. It is now read-only.

Commit 8ed825d

Browse files
ericphansonericphanson
authored
Generate README automatically (#2)
* wip * fix
1 parent 9e8465e commit 8ed825d

File tree

5 files changed

+306
-20
lines changed

5 files changed

+306
-20
lines changed

‎README.md

Lines changed: 36 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -14,49 +14,65 @@ Providing this functionality in BenchmarkTools itself was discussed in <https://
1414
## Example
1515

1616
One just uses `BenchmarkPlots` instead of `BenchmarkTools`, e.g.
17+
1718
```julia
18-
julia> using BenchmarkPlots
19+
using BenchmarkPlots
20+
21+
@benchmark sin(x) setup=(x=rand())
22+
```
1923

20-
julia> @benchmark sin(x) setup=(x=rand())
21-
samples: 10000; evals/sample: 1000; memory estimate: 0 bytes; allocs estimate: 0
24+
```
25+
samples: 10000; evals/sample: 999; memory estimate: 0 bytes; allocs estimate: 0
2226
┌ ┐
23-
[ 0.0, 5.0) ┤ 131
24-
[ 5.0, 10.0) ┤▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 9848
25-
ns [10.0, 15.0) ┤ 18
26-
[15.0, 20.0) ┤ 2
27+
[ 5.0, 10.0) ┤▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 7857
28+
ns [10.0, 15.0) ┤▇▇▇▇▇▇▇▇▇ 2134
29+
[15.0, 20.0) ┤ 8
2730
[20.0, 25.0) ┤ 1
2831
└ ┘
2932
Counts
30-
min: 4.917 ns (0.00% GC); mean: 5.578 ns (0.00% GC); median: 5.042 ns (0.00% GC); max: 22.375 ns (0.00% GC).
33+
min: 8.967 ns (0.00% GC); mean: 9.564 ns (0.00% GC); median: 9.092 ns (0.00% GC); max: 20.145 ns (0.00% GC).
3134
```
35+
3236
That benchmark does not have a very interesting distribution, but it's not hard to find more interesting cases.
3337

3438
```julia
35-
julia> @benchmark 5 ∈ v setup=(v = sort(rand(1:10000, 10000)))
36-
samples: 3169; evals/sample: 1000; memory estimate: 0 bytes; allocs estimate: 0
39+
@benchmark 5 ∈ v setup=(v = sort(rand(1:10000, 10000)))
40+
```
41+
42+
```
43+
samples: 3094; evals/sample: 1000; memory estimate: 0 bytes; allocs estimate: 0
3744
┌ ┐
38-
[ 0.0, 1000.0) ┤▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 2020
45+
[ 0.0, 1000.0) ┤▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 1936
3946
ns [1000.0, 2000.0) ┤ 0
4047
[2000.0, 3000.0) ┤ 0
41-
[3000.0, 4000.0) ┤▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 1149
48+
[3000.0, 4000.0) ┤▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 1158
4249
└ ┘
4350
Counts
44-
min: 1.875 ns (0.00% GC); mean: 1.152 μs (0.00% GC); median: 4.708 ns (0.00% GC); max: 3.588 μs (0.00% GC).
51+
min: 4.333 ns (0.00% GC); mean: 1.188 μs (0.00% GC); median: 7.208 ns (0.00% GC); max: 3.711 μs (0.00% GC).
4552
```
53+
4654
Here, we see a bimodal distribution; in the case `5` is indeed in the vector, we find it very quickly, in the 0-1000 ns range (thanks to `sort` which places it at the front). In the case 5 is not present, we need to check every entry to be sure, and we end up in the 3000-4000 ns range.
4755

4856
Without the `sort`, we end up with more of a uniform distribution:
57+
4958
```julia
50-
julia> @benchmark 5 ∈ v setup=(v = rand(1:10000, 10000))
51-
samples: 2379; evals/sample: 1000; memory estimate: 0 bytes; allocs estimate: 0
59+
@benchmark 5 ∈ v setup=(v = rand(1:10000, 10000))
60+
```
61+
62+
```
63+
samples: 2394; evals/sample: 1000; memory estimate: 0 bytes; allocs estimate: 0
5264
┌ ┐
53-
[ 0.0, 1000.0) ┤▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 619
54-
ns [1000.0, 2000.0) ┤▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 458
55-
[2000.0, 3000.0) ┤▇▇▇▇▇▇▇▇▇▇▇▇▇ 356
56-
[3000.0, 4000.0) ┤▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 946
65+
[ 0.0, 2000.0) ┤▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 1113
66+
ns [2000.0, 4000.0) ┤▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 1273
67+
[4000.0, 6000.0) ┤ 8
5768
└ ┘
5869
Counts
59-
min: 1.917 ns (0.00% GC); mean: 2.040 μs (0.00% GC); median: 2.257 μs (0.00% GC); max: 3.552 μs (0.00% GC).
70+
min: 2.000 ns (0.00% GC); mean: 2.035 μs (0.00% GC); median: 2.215 μs (0.00% GC); max: 5.932 μs (0.00% GC).
6071
```
6172

6273
See also <https://tratt.net/laurie/blog/entries/minimum_times_tend_to_mislead_when_benchmarking.html> for another example of where looking at the whole histogram can be useful in benchmarking.
74+
75+
---
76+
77+
*This page was generated using [Literate.jl](https://github.com/fredrikekre/Literate.jl).*
78+

‎generate_readme/Manifest.toml

Lines changed: 233 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,233 @@
1+
# This file is machine-generated - editing it directly is not advised
2+
3+
[[ArgTools]]
4+
uuid = "0dad84c5-d112-42e6-8d28-ef12dabb789f"
5+
6+
[[Artifacts]]
7+
uuid = "56f22d72-fd6d-98f1-02f0-08ddc0907c33"
8+
9+
[[Base64]]
10+
uuid = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f"
11+
12+
[[BenchmarkPlots]]
13+
deps = ["BenchmarkTools", "Printf", "Statistics", "UnicodePlots"]
14+
path = ".."
15+
uuid = "a80a1652-aad8-438d-b80b-ecb1a674e33b"
16+
version = "0.1.0"
17+
18+
[[BenchmarkTools]]
19+
deps = ["JSON", "Logging", "Printf", "Statistics", "UUIDs"]
20+
git-tree-sha1 = "068fda9b756e41e6c75da7b771e6f89fa8a43d15"
21+
uuid = "6e4b80f9-dd63-53aa-95a3-0cdb28fa8baf"
22+
version = "0.7.0"
23+
24+
[[Compat]]
25+
deps = ["Base64", "Dates", "DelimitedFiles", "Distributed", "InteractiveUtils", "LibGit2", "Libdl", "LinearAlgebra", "Markdown", "Mmap", "Pkg", "Printf", "REPL", "Random", "SHA", "Serialization", "SharedArrays", "Sockets", "SparseArrays", "Statistics", "Test", "UUIDs", "Unicode"]
26+
git-tree-sha1 = "0a817fbe51c976de090aa8c997b7b719b786118d"
27+
uuid = "34da2185-b29b-5c13-b0c7-acf172513d20"
28+
version = "3.28.0"
29+
30+
[[Crayons]]
31+
git-tree-sha1 = "3f71217b538d7aaee0b69ab47d9b7724ca8afa0d"
32+
uuid = "a8cc5b0e-0ffa-5ad4-8c14-923d3ee1735f"
33+
version = "4.0.4"
34+
35+
[[DataAPI]]
36+
git-tree-sha1 = "dfb3b7e89e395be1e25c2ad6d7690dc29cc53b1d"
37+
uuid = "9a962f9c-6df0-11e9-0e5d-c546b8b5ee8a"
38+
version = "1.6.0"
39+
40+
[[DataStructures]]
41+
deps = ["Compat", "InteractiveUtils", "OrderedCollections"]
42+
git-tree-sha1 = "4437b64df1e0adccc3e5d1adbc3ac741095e4677"
43+
uuid = "864edb3b-99cc-5e75-8d2d-829cb0a9cfe8"
44+
version = "0.18.9"
45+
46+
[[Dates]]
47+
deps = ["Printf"]
48+
uuid = "ade2ca70-3891-5945-98fb-dc099432e06a"
49+
50+
[[DelimitedFiles]]
51+
deps = ["Mmap"]
52+
uuid = "8bb1440f-4735-579b-a4ab-409b98df4dab"
53+
54+
[[Distributed]]
55+
deps = ["Random", "Serialization", "Sockets"]
56+
uuid = "8ba89e20-285c-5b6f-9357-94700520ee1b"
57+
58+
[[Downloads]]
59+
deps = ["ArgTools", "LibCURL", "NetworkOptions"]
60+
uuid = "f43a241f-c20a-4ad4-852c-f6b1247861c6"
61+
62+
[[IOCapture]]
63+
deps = ["Logging"]
64+
git-tree-sha1 = "1868e4e7ad2f93d8de0904d89368c527b46aa6a1"
65+
uuid = "b5f81e59-6552-4d32-b1f0-c071b021bf89"
66+
version = "0.2.1"
67+
68+
[[InteractiveUtils]]
69+
deps = ["Markdown"]
70+
uuid = "b77e0a4c-d291-57a0-90e8-8db25a27a240"
71+
72+
[[JSON]]
73+
deps = ["Dates", "Mmap", "Parsers", "Unicode"]
74+
git-tree-sha1 = "81690084b6198a2e1da36fcfda16eeca9f9f24e4"
75+
uuid = "682c06a0-de6a-54ab-a142-c8b1cf79cde6"
76+
version = "0.21.1"
77+
78+
[[LibCURL]]
79+
deps = ["LibCURL_jll", "MozillaCACerts_jll"]
80+
uuid = "b27032c2-a3e7-50c8-80cd-2d36dbcbfd21"
81+
82+
[[LibCURL_jll]]
83+
deps = ["Artifacts", "LibSSH2_jll", "Libdl", "MbedTLS_jll", "Zlib_jll", "nghttp2_jll"]
84+
uuid = "deac9b47-8bc7-5906-a0fe-35ac56dc84c0"
85+
86+
[[LibGit2]]
87+
deps = ["Base64", "NetworkOptions", "Printf", "SHA"]
88+
uuid = "76f85450-5226-5b5a-8eaa-529ad045b433"
89+
90+
[[LibSSH2_jll]]
91+
deps = ["Artifacts", "Libdl", "MbedTLS_jll"]
92+
uuid = "29816b5a-b9ab-546f-933c-edad1886dfa8"
93+
94+
[[Libdl]]
95+
uuid = "8f399da3-3557-5675-b5ff-fb832c97cbdb"
96+
97+
[[LinearAlgebra]]
98+
deps = ["Libdl"]
99+
uuid = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"
100+
101+
[[Literate]]
102+
deps = ["Base64", "IOCapture", "JSON", "REPL"]
103+
git-tree-sha1 = "501a1a74a0c825037860d36d87d703e987d39dbc"
104+
uuid = "98b081ad-f1c9-55d3-8b20-4c87d4299306"
105+
version = "2.8.1"
106+
107+
[[Logging]]
108+
uuid = "56ddb016-857b-54e1-b83d-db4d58db5568"
109+
110+
[[Markdown]]
111+
deps = ["Base64"]
112+
uuid = "d6f4376e-aef5-505a-96c1-9c027394607a"
113+
114+
[[MbedTLS_jll]]
115+
deps = ["Artifacts", "Libdl"]
116+
uuid = "c8ffd9c3-330d-5841-b78e-0817d7145fa1"
117+
118+
[[Missings]]
119+
deps = ["DataAPI"]
120+
git-tree-sha1 = "4ea90bd5d3985ae1f9a908bd4500ae88921c5ce7"
121+
uuid = "e1d29d7a-bbdc-5cf2-9ac0-f12de2c33e28"
122+
version = "1.0.0"
123+
124+
[[Mmap]]
125+
uuid = "a63ad114-7e13-5084-954f-fe012c677804"
126+
127+
[[MozillaCACerts_jll]]
128+
uuid = "14a3606d-f60d-562e-9121-12d972cd8159"
129+
130+
[[NetworkOptions]]
131+
uuid = "ca575930-c2e3-43a9-ace4-1e988b2c1908"
132+
133+
[[OrderedCollections]]
134+
git-tree-sha1 = "4fa2ba51070ec13fcc7517db714445b4ab986bdf"
135+
uuid = "bac558e1-5e72-5ebc-8fee-abe8a469f55d"
136+
version = "1.4.0"
137+
138+
[[Parsers]]
139+
deps = ["Dates"]
140+
git-tree-sha1 = "c8abc88faa3f7a3950832ac5d6e690881590d6dc"
141+
uuid = "69de0a69-1ddd-5017-9359-2bf0b02dc9f0"
142+
version = "1.1.0"
143+
144+
[[Pkg]]
145+
deps = ["Artifacts", "Dates", "Downloads", "LibGit2", "Libdl", "Logging", "Markdown", "Printf", "REPL", "Random", "SHA", "Serialization", "TOML", "Tar", "UUIDs", "p7zip_jll"]
146+
uuid = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f"
147+
148+
[[Printf]]
149+
deps = ["Unicode"]
150+
uuid = "de0858da-6303-5e67-8744-51eddeeeb8d7"
151+
152+
[[REPL]]
153+
deps = ["InteractiveUtils", "Markdown", "Sockets", "Unicode"]
154+
uuid = "3fa0cd96-eef1-5676-8a61-b3b8758bbffb"
155+
156+
[[Random]]
157+
deps = ["Serialization"]
158+
uuid = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"
159+
160+
[[SHA]]
161+
uuid = "ea8e919c-243c-51af-8825-aaa63cd721ce"
162+
163+
[[Serialization]]
164+
uuid = "9e88b42a-f829-5b0c-bbe9-9e923198166b"
165+
166+
[[SharedArrays]]
167+
deps = ["Distributed", "Mmap", "Random", "Serialization"]
168+
uuid = "1a1011a3-84de-559e-8e89-a11a2f7dc383"
169+
170+
[[Sockets]]
171+
uuid = "6462fe0b-24de-5631-8697-dd941f90decc"
172+
173+
[[SortingAlgorithms]]
174+
deps = ["DataStructures"]
175+
git-tree-sha1 = "2ec1962eba973f383239da22e75218565c390a96"
176+
uuid = "a2af1166-a08f-5f64-846c-94a0d3cef48c"
177+
version = "1.0.0"
178+
179+
[[SparseArrays]]
180+
deps = ["LinearAlgebra", "Random"]
181+
uuid = "2f01184e-e22b-5df5-ae63-d93ebab69eaf"
182+
183+
[[Statistics]]
184+
deps = ["LinearAlgebra", "SparseArrays"]
185+
uuid = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"
186+
187+
[[StatsAPI]]
188+
git-tree-sha1 = "1958272568dc176a1d881acb797beb909c785510"
189+
uuid = "82ae8749-77ed-4fe6-ae5f-f523153014b0"
190+
version = "1.0.0"
191+
192+
[[StatsBase]]
193+
deps = ["DataAPI", "DataStructures", "LinearAlgebra", "Missings", "Printf", "Random", "SortingAlgorithms", "SparseArrays", "Statistics", "StatsAPI"]
194+
git-tree-sha1 = "2f6792d523d7448bbe2fec99eca9218f06cc746d"
195+
uuid = "2913bbd2-ae8a-5f71-8c99-4fb6c76f3a91"
196+
version = "0.33.8"
197+
198+
[[TOML]]
199+
deps = ["Dates"]
200+
uuid = "fa267f1f-6049-4f14-aa54-33bafae1ed76"
201+
202+
[[Tar]]
203+
deps = ["ArgTools", "SHA"]
204+
uuid = "a4e569a6-e804-4fa4-b0f3-eef7a1d5b13e"
205+
206+
[[Test]]
207+
deps = ["InteractiveUtils", "Logging", "Random", "Serialization"]
208+
uuid = "8dfed614-e22c-5e08-85e1-65c5234f0b40"
209+
210+
[[UUIDs]]
211+
deps = ["Random", "SHA"]
212+
uuid = "cf7118a7-6976-5b1a-9a39-7adc72f591a4"
213+
214+
[[Unicode]]
215+
uuid = "4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5"
216+
217+
[[UnicodePlots]]
218+
deps = ["Crayons", "Dates", "SparseArrays", "StatsBase"]
219+
git-tree-sha1 = "1a63e6eea76b291378ff9f95801f8b6d96213208"
220+
uuid = "b8865327-cd53-5732-bb35-84acbb429228"
221+
version = "1.3.0"
222+
223+
[[Zlib_jll]]
224+
deps = ["Libdl"]
225+
uuid = "83775a58-1f1d-513f-b197-d71354ab007a"
226+
227+
[[nghttp2_jll]]
228+
deps = ["Artifacts", "Libdl"]
229+
uuid = "8e850ede-7688-5339-a07c-302acd2aaf8d"
230+
231+
[[p7zip_jll]]
232+
deps = ["Artifacts", "Libdl"]
233+
uuid = "3f19e933-33d8-53b3-aaab-bd5110c3b7a0"

‎generate_readme/Project.toml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
[deps]
2+
BenchmarkPlots = "a80a1652-aad8-438d-b80b-ecb1a674e33b"
3+
Literate = "98b081ad-f1c9-55d3-8b20-4c87d4299306"

‎generate_readme/README.jl

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
# [![CI](https://github.com/ericphanson/BenchmarkPlots.jl/actions/workflows/CI.yml/badge.svg)](https://github.com/ericphanson/BenchmarkPlots.jl/actions/workflows/CI.yml)
2+
# [![codecov](https://codecov.io/gh/ericphanson/BenchmarkPlots.jl/branch/main/graph/badge.svg?token=v0aca89xRi)](https://codecov.io/gh/ericphanson/BenchmarkPlots.jl)
3+
4+
# # BenchmarkPlots
5+
6+
# Wraps [BenchmarkTools.jl](https://github.com/JuliaCI/BenchmarkTools.jl/) to provide a UnicodePlots.jl-powered `show` method for `@benchmark`. This is accomplished by a custom `@benchmark` method which wraps the output in a `BenchmarkPlot` struct with a custom show method.
7+
8+
# This means one should not call `using` on both BenchmarkPlots and BenchmarkTools in the same namespace, or else these `@benchmark` macros will conflict ("WARNING: using `BenchmarkTools.@benchmark` in module Main conflicts with an existing identifier.")
9+
10+
# However, BenchmarkPlots re-exports all the export of BenchmarkTools, so you can simply call `using BenchmarkPlots`.
11+
12+
# Providing this functionality in BenchmarkTools itself was discussed in <https://github.com/JuliaCI/BenchmarkTools.jl/pull/180>.
13+
14+
# ## Example
15+
16+
# One just uses `BenchmarkPlots` instead of `BenchmarkTools`, e.g.
17+
18+
using BenchmarkPlots
19+
20+
@benchmark sin(x) setup=(x=rand())
21+
22+
# That benchmark does not have a very interesting distribution, but it's not hard to find more interesting cases.
23+
24+
@benchmark 5 ∈ v setup=(v = sort(rand(1:10000, 10000)))
25+
26+
# Here, we see a bimodal distribution; in the case `5` is indeed in the vector, we find it very quickly, in the 0-1000 ns range (thanks to `sort` which places it at the front). In the case 5 is not present, we need to check every entry to be sure, and we end up in the 3000-4000 ns range.
27+
28+
# Without the `sort`, we end up with more of a uniform distribution:
29+
30+
@benchmark 5 ∈ v setup=(v = rand(1:10000, 10000))
31+
32+
# See also <https://tratt.net/laurie/blog/entries/minimum_times_tend_to_mislead_when_benchmarking.html> for another example of where looking at the whole histogram can be useful in benchmarking.

‎generate_readme/make.jl

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
using Literate
2+
Literate.markdown(joinpath(@__DIR__, "README.jl"), outputdir="..", execute=true, documenter=false)

0 commit comments

Comments
 (0)