Skip to content

Commit 3b8ca17

Browse files
erikcserikcs
authored
Add release log to GitHub (#170)
1 parent 4eef4df commit 3b8ca17

File tree

1 file changed

+377
-1
lines changed

1 file changed

+377
-1
lines changed

‎releases/README.md‎

Lines changed: 377 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,6 @@
1-
### Preparing a new CRAN release
1+
# policytree Release Process
2+
3+
This is just a minor summary document, practically all details are the same as the [GRF release process](https://github.com/grf-labs/grf/tree/master/releases). Memory/perf timings for releases are recorded at the end.
24

35
0. Open a "Prepare the v.x release PR" and update CHANGELOG
46

@@ -39,3 +41,377 @@
3941
git tag -a v1.1.0 "release-commit-hash"
4042
git push --tags
4143
```
44+
45+
Make sure to submit a test tarball containing everything under tests/ to different R-Hub images.
46+
For a big release, can also run `R CMD check --as-cran --run-donttest --use-valgrind <development tarball>` on a Linux machine with Valgrind for a thorough stress test (will take very long).
47+
48+
In case of a major release with breaking functionality, or a subsequent minor release, update the removed functions list in `deprecated.R`.
49+
50+
## Make sure there are no performance regressions
51+
52+
**check performance**
53+
54+
`perf.R`:
55+
56+
```R
57+
options(width=120)
58+
library(microbenchmark)
59+
library(policytree)
60+
set.seed(42)
61+
62+
# depth <- 3
63+
n <- 100
64+
p <- 5
65+
d <- 4
66+
X.c <- matrix(rnorm(n * p), n, p)
67+
X.d <- matrix(sample(10:15, n * p, replace = TRUE), n, p)
68+
X <- cbind(X.c, X.d, rbinom(n, 1, 0.5))
69+
Y <- matrix(rnorm(n * d), n, d)
70+
b1 <- microbenchmark(
71+
tree <- policy_tree(X, Y, depth = 3),
72+
unit = "ms",
73+
times = 15)
74+
b2 <- microbenchmark(
75+
pp <- predict(tree, X),
76+
unit = "ms",
77+
times = 50)
78+
79+
# depth <- 2
80+
n <- 10000
81+
p <- 5
82+
d <- 4
83+
X.c <- matrix(rnorm(n * p), n, p)
84+
X.d <- matrix(sample(10:15, n * p, replace = TRUE), n, p)
85+
X <- cbind(X.c, X.d, rbinom(n, 1, 0.5))
86+
Y <- matrix(rnorm(n * d), n, d)
87+
b3 <- microbenchmark(
88+
tree <- policy_tree(X, Y, depth = 2),
89+
unit = "ms",
90+
times = 15)
91+
b4 <- microbenchmark(
92+
pp <- predict(tree, X),
93+
unit = "ms",
94+
times = 50)
95+
b5 <- microbenchmark(
96+
tree <- policy_tree(X, Y, depth = 2, split.step = 10),
97+
unit = "ms",
98+
times = 15)
99+
100+
print(b1)
101+
print(b2)
102+
print(b3)
103+
print(b4)
104+
print(b5)
105+
```
106+
107+
**check memory usage**
108+
109+
```
110+
> R -d "valgrind --tool=massif" -f memory.R
111+
> ms_print massif.out.34681 &> ms_print.out
112+
```
113+
114+
The plot should look smooth, and should not contain large spikes in memory consumption and should not change drastically between releases.
115+
116+
`memory1.R`:
117+
118+
```R
119+
library(policytree)
120+
set.seed(42)
121+
depth <- 2
122+
n <- 10000
123+
p <- 2
124+
d <- 4
125+
X.c <- matrix(rnorm(n * p), n, p)
126+
X.d <- matrix(sample(10:15, n * p, replace = TRUE), n, p)
127+
X <- cbind(X.c, X.d, rbinom(n, 1, 0.5))
128+
Y <- matrix(rnorm(n * d), n, d)
129+
tree <- policy_tree(X, Y, depth = depth)
130+
pp <- predict(tree, X)
131+
```
132+
133+
`memory2.R`:
134+
135+
```R
136+
library(policytree)
137+
set.seed(42)
138+
depth <- 3
139+
n <- 100
140+
p <- 5
141+
d <- 4
142+
X.c <- matrix(rnorm(n * p), n, p)
143+
X.d <- matrix(sample(10:15, n * p, replace = TRUE), n, p)
144+
X <- cbind(X.c, X.d, rbinom(n, 1, 0.5))
145+
Y <- matrix(rnorm(n * d), n, d)
146+
tree <- policy_tree(X, Y, depth = depth)
147+
pp <- predict(tree, X)
148+
```
149+
150+
## Previous performance test results
151+
152+
**1.2.2**
153+
154+
This is only a patch release dropping CXX in Makevars per latest CRAN guidelines.
155+
156+
**1.2.1**
157+
158+
This is only a patch release updating the printed leaf labels for hybrid policy tree. No C++ changes.
159+
160+
**1.2.0**
161+
162+
(Machine: Sherlock 12 cores/50G/R version 3.5.1)
163+
164+
No changes to C++, only added hybrid_policy_tree.
165+
166+
perf
167+
168+
```
169+
> print(b1)
170+
Unit: milliseconds
171+
expr min lq mean median uq max neval
172+
tree <- policy_tree(X, Y, depth = 3) 4564.033 4573.721 4588.593 4578.084 4597.359 4638.117 15
173+
> print(b2)
174+
Unit: milliseconds
175+
expr min lq mean median uq max neval
176+
pp <- predict(tree, X) 0.0253 0.025779 0.03207804 0.026279 0.02687 0.291661 50
177+
> print(b3)
178+
Unit: milliseconds
179+
expr min lq mean median uq max neval
180+
tree <- policy_tree(X, Y, depth = 2) 67672.76 68403.61 68538.73 68570.28 68733.21 68991.17 15
181+
> print(b4)
182+
Unit: milliseconds
183+
expr min lq mean median uq max neval
184+
pp <- predict(tree, X) 0.680048 0.732875 0.7332164 0.7373515 0.742225 0.788218 50
185+
> print(b5)
186+
Unit: milliseconds
187+
expr min lq mean median uq max neval
188+
tree <- policy_tree(X, Y, depth = 2, split.step = 10) 8937.686 8967.728 8998.685 9004.088 9023.854 9042.722 15
189+
>
190+
```
191+
192+
memory1
193+
194+
```
195+
MB
196+
54.87^ #
197+
|@:::::::::::::::@:::::::::::::::::#:::::::::::::::@:::::@::::@:::::@::::
198+
|@:::::::: ::::: @: ::: :::::::::::#:::::::::: ::::@:::::@::::@:::::@::::
199+
|@:::::::: ::::: @: ::: :::::::::::#:::::::::: ::::@:::::@::::@:::::@::::
200+
|@:::::::: ::::: @: ::: :::::::::::#:::::::::: ::::@:::::@::::@:::::@::::
201+
|@:::::::: ::::: @: ::: :::::::::::#:::::::::: ::::@:::::@::::@:::::@::::
202+
|@:::::::: ::::: @: ::: :::::::::::#:::::::::: ::::@:::::@::::@:::::@::::
203+
|@:::::::: ::::: @: ::: :::::::::::#:::::::::: ::::@:::::@::::@:::::@::::
204+
|@:::::::: ::::: @: ::: :::::::::::#:::::::::: ::::@:::::@::::@:::::@::::
205+
|@:::::::: ::::: @: ::: :::::::::::#:::::::::: ::::@:::::@::::@:::::@::::
206+
|@:::::::: ::::: @: ::: :::::::::::#:::::::::: ::::@:::::@::::@:::::@::::
207+
|@:::::::: ::::: @: ::: :::::::::::#:::::::::: ::::@:::::@::::@:::::@::::
208+
|@:::::::: ::::: @: ::: :::::::::::#:::::::::: ::::@:::::@::::@:::::@::::
209+
|@:::::::: ::::: @: ::: :::::::::::#:::::::::: ::::@:::::@::::@:::::@::::
210+
|@:::::::: ::::: @: ::: :::::::::::#:::::::::: ::::@:::::@::::@:::::@::::
211+
|@:::::::: ::::: @: ::: :::::::::::#:::::::::: ::::@:::::@::::@:::::@::::
212+
|@:::::::: ::::: @: ::: :::::::::::#:::::::::: ::::@:::::@::::@:::::@::::
213+
|@:::::::: ::::: @: ::: :::::::::::#:::::::::: ::::@:::::@::::@:::::@::::
214+
|@:::::::: ::::: @: ::: :::::::::::#:::::::::: ::::@:::::@::::@:::::@::::
215+
|@:::::::: ::::: @: ::: :::::::::::#:::::::::: ::::@:::::@::::@:::::@::::
216+
0 +----------------------------------------------------------------------->Gi
217+
0 93.02
218+
```
219+
220+
memory2
221+
222+
```
223+
MB
224+
54.31^ #
225+
| #
226+
| #
227+
| #::@@:::::::@::@:::::@:::::::::::::::::::::::::::::@:::::@:::::@:::::@
228+
| #::@ :: ::: @: @:::::@::: ::::: ::::::::::::: :::::@:::::@:::::@:::::@
229+
| #::@ :: ::: @: @:::::@::: ::::: ::::::::::::: :::::@:::::@:::::@:::::@
230+
| #::@ :: ::: @: @:::::@::: ::::: ::::::::::::: :::::@:::::@:::::@:::::@
231+
| #::@ :: ::: @: @:::::@::: ::::: ::::::::::::: :::::@:::::@:::::@:::::@
232+
| #::@ :: ::: @: @:::::@::: ::::: ::::::::::::: :::::@:::::@:::::@:::::@
233+
| :#::@ :: ::: @: @:::::@::: ::::: ::::::::::::: :::::@:::::@:::::@:::::@
234+
| :#::@ :: ::: @: @:::::@::: ::::: ::::::::::::: :::::@:::::@:::::@:::::@
235+
| :#::@ :: ::: @: @:::::@::: ::::: ::::::::::::: :::::@:::::@:::::@:::::@
236+
| :#::@ :: ::: @: @:::::@::: ::::: ::::::::::::: :::::@:::::@:::::@:::::@
237+
| :#::@ :: ::: @: @:::::@::: ::::: ::::::::::::: :::::@:::::@:::::@:::::@
238+
| :#::@ :: ::: @: @:::::@::: ::::: ::::::::::::: :::::@:::::@:::::@:::::@
239+
| :#::@ :: ::: @: @:::::@::: ::::: ::::::::::::: :::::@:::::@:::::@:::::@
240+
| :#::@ :: ::: @: @:::::@::: ::::: ::::::::::::: :::::@:::::@:::::@:::::@
241+
| :#::@ :: ::: @: @:::::@::: ::::: ::::::::::::: :::::@:::::@:::::@:::::@
242+
| :#::@ :: ::: @: @:::::@::: ::::: ::::::::::::: :::::@:::::@:::::@:::::@
243+
| :#::@ :: ::: @: @:::::@::: ::::: ::::::::::::: :::::@:::::@:::::@:::::@
244+
0 +----------------------------------------------------------------------->Gi
245+
0 24.37
246+
```
247+
248+
**1.1.0**
249+
250+
(Machine: Sherlock 12 cores/50G/R version 3.5.1)
251+
252+
Notable changes since previous version: faster predict(), split.step skips observations instead of unique values, add min.node.size.
253+
254+
perf
255+
256+
```
257+
> print(b1)
258+
Unit: milliseconds
259+
expr min lq mean median uq max neval
260+
tree <- policy_tree(X, Y, depth = 3) 4608.389 4611.736 4653.743 4613.628 4648.493 4907.877 15
261+
> print(b2)
262+
Unit: milliseconds
263+
expr min lq mean median uq max neval
264+
pp <- predict(tree, X) 0.024629 0.025264 0.03059398 0.0257975 0.026444 0.240188 50
265+
> print(b3)
266+
Unit: milliseconds
267+
expr min lq mean median uq max neval
268+
tree <- policy_tree(X, Y, depth = 2) 66994.34 67054.26 67187.36 67148.37 67267.58 67598.78 15
269+
> print(b4)
270+
Unit: milliseconds
271+
expr min lq mean median uq max neval
272+
pp <- predict(tree, X) 0.617329 0.6959 0.6944944 0.7075835 0.717939 0.726327 50
273+
> print(b5)
274+
Unit: milliseconds
275+
expr min lq mean median uq max neval
276+
tree <- policy_tree(X, Y, depth = 2, split.step = 10) 8226.363 8229.001 8260.271 8231.613 8263.748 8403.574 15
277+
```
278+
279+
memory1
280+
281+
```
282+
MB
283+
54.24^##
284+
|# :::::::::::::@@::::@@:@::::::::@:::@::::::::::::::@:::@:::::@::::@::::
285+
|# : :::::::::::@ :: :@ :@:::: :::@:::@: ::::::::: ::@:::@:::::@::::@::::
286+
|# : :::::::::::@ :: :@ :@:::: :::@:::@: ::::::::: ::@:::@:::::@::::@::::
287+
|# : :::::::::::@ :: :@ :@:::: :::@:::@: ::::::::: ::@:::@:::::@::::@::::
288+
|# : :::::::::::@ :: :@ :@:::: :::@:::@: ::::::::: ::@:::@:::::@::::@::::
289+
|# : :::::::::::@ :: :@ :@:::: :::@:::@: ::::::::: ::@:::@:::::@::::@::::
290+
|# : :::::::::::@ :: :@ :@:::: :::@:::@: ::::::::: ::@:::@:::::@::::@::::
291+
|# : :::::::::::@ :: :@ :@:::: :::@:::@: ::::::::: ::@:::@:::::@::::@::::
292+
|# : :::::::::::@ :: :@ :@:::: :::@:::@: ::::::::: ::@:::@:::::@::::@::::
293+
|# : :::::::::::@ :: :@ :@:::: :::@:::@: ::::::::: ::@:::@:::::@::::@::::
294+
|# : :::::::::::@ :: :@ :@:::: :::@:::@: ::::::::: ::@:::@:::::@::::@::::
295+
|# : :::::::::::@ :: :@ :@:::: :::@:::@: ::::::::: ::@:::@:::::@::::@::::
296+
|# : :::::::::::@ :: :@ :@:::: :::@:::@: ::::::::: ::@:::@:::::@::::@::::
297+
|# : :::::::::::@ :: :@ :@:::: :::@:::@: ::::::::: ::@:::@:::::@::::@::::
298+
|# : :::::::::::@ :: :@ :@:::: :::@:::@: ::::::::: ::@:::@:::::@::::@::::
299+
|# : :::::::::::@ :: :@ :@:::: :::@:::@: ::::::::: ::@:::@:::::@::::@::::
300+
|# : :::::::::::@ :: :@ :@:::: :::@:::@: ::::::::: ::@:::@:::::@::::@::::
301+
|# : :::::::::::@ :: :@ :@:::: :::@:::@: ::::::::: ::@:::@:::::@::::@::::
302+
|# : :::::::::::@ :: :@ :@:::: :::@:::@: ::::::::: ::@:::@:::::@::::@::::
303+
0 +----------------------------------------------------------------------->Gi
304+
0 89.42
305+
```
306+
307+
memory2
308+
309+
```
310+
MB
311+
54.24^ #
312+
| #
313+
| #
314+
| #::::::::::::::::::::::@@:::::::::@@::::::@::::::::@:::::@:::::@:::::@
315+
| #::::::: ::::::: ::::::@ ::: :::::@ ::::::@:: :::::@:::::@:::::@:::::@
316+
| #::::::: ::::::: ::::::@ ::: :::::@ ::::::@:: :::::@:::::@:::::@:::::@
317+
| #::::::: ::::::: ::::::@ ::: :::::@ ::::::@:: :::::@:::::@:::::@:::::@
318+
| #::::::: ::::::: ::::::@ ::: :::::@ ::::::@:: :::::@:::::@:::::@:::::@
319+
| #::::::: ::::::: ::::::@ ::: :::::@ ::::::@:: :::::@:::::@:::::@:::::@
320+
| #::::::: ::::::: ::::::@ ::: :::::@ ::::::@:: :::::@:::::@:::::@:::::@
321+
| #::::::: ::::::: ::::::@ ::: :::::@ ::::::@:: :::::@:::::@:::::@:::::@
322+
| #::::::: ::::::: ::::::@ ::: :::::@ ::::::@:: :::::@:::::@:::::@:::::@
323+
|::#::::::: ::::::: ::::::@ ::: :::::@ ::::::@:: :::::@:::::@:::::@:::::@
324+
|: #::::::: ::::::: ::::::@ ::: :::::@ ::::::@:: :::::@:::::@:::::@:::::@
325+
|: #::::::: ::::::: ::::::@ ::: :::::@ ::::::@:: :::::@:::::@:::::@:::::@
326+
|: #::::::: ::::::: ::::::@ ::: :::::@ ::::::@:: :::::@:::::@:::::@:::::@
327+
|: #::::::: ::::::: ::::::@ ::: :::::@ ::::::@:: :::::@:::::@:::::@:::::@
328+
|: #::::::: ::::::: ::::::@ ::: :::::@ ::::::@:: :::::@:::::@:::::@:::::@
329+
|: #::::::: ::::::: ::::::@ ::: :::::@ ::::::@:: :::::@:::::@:::::@:::::@
330+
|: #::::::: ::::::: ::::::@ ::: :::::@ ::::::@:: :::::@:::::@:::::@:::::@
331+
0 +----------------------------------------------------------------------->Gi
332+
0 23.95
333+
```
334+
335+
**1.0**
336+
337+
(Machine: Sherlock 12 cores/50G/R version 3.5.1)
338+
339+
perf
340+
```
341+
> print(b1)
342+
Unit: milliseconds
343+
expr min lq mean median uq max neval
344+
tree <- policy_tree(X, Y, depth = 3) 4097.366 4099.058 4137.154 4109.452 4167.606 4224.911 15
345+
> print(b2)
346+
Unit: milliseconds
347+
expr min lq mean median uq max neval
348+
pp <- predict(tree, X) 1.935283 1.964984 2.066606 1.984469 2.046768 3.191294 50
349+
> print(b3)
350+
Unit: milliseconds
351+
expr min lq mean median uq max neval
352+
tree <- policy_tree(X, Y, depth = 2) 59961.76 60042.89 60110.17 60128.08 60154.42 60290.77 15
353+
> print(b4)
354+
Unit: milliseconds
355+
expr min lq mean median uq max neval
356+
pp <- predict(tree, X) 133.6293 135.0255 135.9349 135.4114 135.8708 157.6832 50
357+
> print(b5)
358+
Unit: milliseconds
359+
expr min lq mean median uq max neval
360+
tree <- policy_tree(X, Y, depth = 2, split.step = 10) 7905.692 7929.35 7974.073 7933.476 7984.098 8223.15 15
361+
```
362+
363+
memory1
364+
365+
```
366+
MB
367+
54.24^#
368+
|#:::@::::::@::::::::@:::@::@::::::::::::::::::@:::::@:::::@:::::@::::::@
369+
|#:::@::::::@::: ::: @:::@::@:: :: ::::::::::::@:::::@:::::@:::::@::::::@
370+
|#:::@::::::@::: ::: @:::@::@:: :: ::::::::::::@:::::@:::::@:::::@::::::@
371+
|#:::@::::::@::: ::: @:::@::@:: :: ::::::::::::@:::::@:::::@:::::@::::::@
372+
|#:::@::::::@::: ::: @:::@::@:: :: ::::::::::::@:::::@:::::@:::::@::::::@
373+
|#:::@::::::@::: ::: @:::@::@:: :: ::::::::::::@:::::@:::::@:::::@::::::@
374+
|#:::@::::::@::: ::: @:::@::@:: :: ::::::::::::@:::::@:::::@:::::@::::::@
375+
|#:::@::::::@::: ::: @:::@::@:: :: ::::::::::::@:::::@:::::@:::::@::::::@
376+
|#:::@::::::@::: ::: @:::@::@:: :: ::::::::::::@:::::@:::::@:::::@::::::@
377+
|#:::@::::::@::: ::: @:::@::@:: :: ::::::::::::@:::::@:::::@:::::@::::::@
378+
|#:::@::::::@::: ::: @:::@::@:: :: ::::::::::::@:::::@:::::@:::::@::::::@
379+
|#:::@::::::@::: ::: @:::@::@:: :: ::::::::::::@:::::@:::::@:::::@::::::@
380+
|#:::@::::::@::: ::: @:::@::@:: :: ::::::::::::@:::::@:::::@:::::@::::::@
381+
|#:::@::::::@::: ::: @:::@::@:: :: ::::::::::::@:::::@:::::@:::::@::::::@
382+
|#:::@::::::@::: ::: @:::@::@:: :: ::::::::::::@:::::@:::::@:::::@::::::@
383+
|#:::@::::::@::: ::: @:::@::@:: :: ::::::::::::@:::::@:::::@:::::@::::::@
384+
|#:::@::::::@::: ::: @:::@::@:: :: ::::::::::::@:::::@:::::@:::::@::::::@
385+
|#:::@::::::@::: ::: @:::@::@:: :: ::::::::::::@:::::@:::::@:::::@::::::@
386+
|#:::@::::::@::: ::: @:::@::@:: :: ::::::::::::@:::::@:::::@:::::@::::::@
387+
0 +----------------------------------------------------------------------->Gi
388+
0 87.41
389+
```
390+
391+
memory2
392+
393+
```
394+
MB
395+
54.24^ #
396+
| #
397+
| #
398+
| #:::::::@@:::::@@:::::::@@::::@::::::::::::::::::::::::@:::::::::@::::
399+
| #: ::: :@ : :::@ : :::: @ : : @:::::::: :: :: ::: : :::@: :: : ::@::::
400+
| #: ::: :@ : :::@ : :::: @ : : @:::::::: :: :: ::: : :::@: :: : ::@::::
401+
| #: ::: :@ : :::@ : :::: @ : : @:::::::: :: :: ::: : :::@: :: : ::@::::
402+
| #: ::: :@ : :::@ : :::: @ : : @:::::::: :: :: ::: : :::@: :: : ::@::::
403+
| #: ::: :@ : :::@ : :::: @ : : @:::::::: :: :: ::: : :::@: :: : ::@::::
404+
| #: ::: :@ : :::@ : :::: @ : : @:::::::: :: :: ::: : :::@: :: : ::@::::
405+
| :#: ::: :@ : :::@ : :::: @ : : @:::::::: :: :: ::: : :::@: :: : ::@::::
406+
| :#: ::: :@ : :::@ : :::: @ : : @:::::::: :: :: ::: : :::@: :: : ::@::::
407+
| :#: ::: :@ : :::@ : :::: @ : : @:::::::: :: :: ::: : :::@: :: : ::@::::
408+
| :#: ::: :@ : :::@ : :::: @ : : @:::::::: :: :: ::: : :::@: :: : ::@::::
409+
| :#: ::: :@ : :::@ : :::: @ : : @:::::::: :: :: ::: : :::@: :: : ::@::::
410+
| :#: ::: :@ : :::@ : :::: @ : : @:::::::: :: :: ::: : :::@: :: : ::@::::
411+
| :#: ::: :@ : :::@ : :::: @ : : @:::::::: :: :: ::: : :::@: :: : ::@::::
412+
| :#: ::: :@ : :::@ : :::: @ : : @:::::::: :: :: ::: : :::@: :: : ::@::::
413+
| :#: ::: :@ : :::@ : :::: @ : : @:::::::: :: :: ::: : :::@: :: : ::@::::
414+
| :#: ::: :@ : :::@ : :::: @ : : @:::::::: :: :: ::: : :::@: :: : ::@::::
415+
0 +----------------------------------------------------------------------->Gi
416+
0 23.11
417+
```

0 commit comments

Comments
 (0)