Reduce package(s) size

[pawelru]

Have a look at the formatters:

❯ R CMD BUILD formatters
* checking for file ‘formatters/DESCRIPTION’ ... OK
* preparing ‘formatters’:
* checking DESCRIPTION meta-information ... OK
* installing the package to build vignettes
* creating vignettes ... OK
* checking for LF line-endings in source and make files and shell scripts
* checking for empty or unneeded directories
* building ‘formatters_0.5.9.9003.tar.gz’

❯ tar -tzvf formatters_0.5.9.9003.tar.gz | sort -k3 -hr
-rw-r--r--  0 ruckip staff 1847812 Dec  6  2023 formatters/man/figures/logo.png
-rw-r--r--  0 ruckip staff  925622 Nov  6  2023 formatters/data/ex_advs.rda
-rw-r--r--  0 ruckip staff  629211 Nov  6  2023 formatters/data/ex_adqs.rda
-rw-r--r--  0 ruckip staff  493724 Nov  6  2023 formatters/data/ex_adlb.rda
-rw-r--r--  0 ruckip staff   58553 Jun 10 11:05 formatters/R/pagination.R
-rw-r--r--  0 ruckip staff   58457 Nov  6  2023 formatters/data/ex_adae.rda
-rw-r--r--  0 ruckip staff   56998 Jun 10 11:05 formatters/R/tostring.R
-rw-r--r--  0 ruckip staff   50367 Nov  6  2023 formatters/data/ex_admh.rda
-rw-r--r--  0 ruckip staff   48949 Nov  6  2023 formatters/data/ex_adcm.rda
-rw-r--r--  0 ruckip staff   45794 Oct 11 10:16 formatters/R/matrix_form.R
-rw-r--r--  0 ruckip staff   43801 Nov  6  2023 formatters/data/ex_adaette.rda
-rw-r--r--  0 ruckip staff   42991 Nov  6  2023 formatters/data/ex_adrs.rda
-rw-r--r--  0 ruckip staff   42770 Nov  6  2023 formatters/data/ex_adtte.rda
-rw-r--r--  0 ruckip staff   28158 Aug 13 14:33 formatters/tests/testthat/test-formatters.R
-rw-r--r--  0 ruckip staff   25210 Nov 14 18:02 formatters/inst/doc/formatters.html
-rw-r--r--  0 ruckip staff   23892 Jun 10 11:05 formatters/R/mpf_exporters.R
(...)

when untared:

❯ total_size=$(du -sk . | awk '{print $1}')
find . -type f -exec du -sk {} + | awk -v total_size=$total_size '{printf "%s %d KB (%.2f%%)\n", $2, $1, ($1/total_size)*100}' | sort -k2,2nr
./man/figures/logo.png 1808 KB (37.05%)
./data/ex_advs.rda 904 KB (18.52%)
./data/ex_adqs.rda 616 KB (12.62%)
./data/ex_adlb.rda 484 KB (9.92%)
./R/pagination.R 60 KB (1.23%)

• 37% of package size is consumed by the logo alone!
• looking into the codebase of that repo - we are not using ex_ datasets at all. But it might be used elsewhere. There are few packages that are using ex_adlb but I cannot find any usage of ex_advs and ex_adqs.

Adding this in general task repo to repeat the analysis for multiple packages and find a common solution.

It's important because of:

• R CMD CHECK note about size
• WebR with package downloads on the client (website user) side so we aim to have small and fast installable packages. I'm seeing two ways of doing this: limiting deps (not discussed here) and reducing the size (this issue). Currently it's neither good or bad. Have a look at the below extract from a random shinylive teal app in the tlg-catalog (devtools in chrome - network tab - open shinylive tab). This is filtered to downloaded .tgz files and order by size.
  

[llrs-roche]

Reducing the logo file size is relatively easy (using an online tool reduced the size by 70% to ~500KB). I also think that the size of the logo should be checked, as I found that for formatters is quite big 1299x1501 (compared it with teal logo at teal/man/figures/teal.png of 278x321 it is >20 times bigger).

Dimensions of the logos found:

teal/man/figures/teal.png
[1] 278 321
rtables/man/figures/logo.png:
[1] 1501 1501
formatters/man/figures/logo.png:
[1] 1299 1501

> file.size("../teal/man/figures/teal.png")
[1] 79859
> file.size("../rtables/man/figures/logo.png")
[1] 1929788
> file.size("../formatters/man/figures/logo.png")
[1] 498770

I think that only compressing can be enough, but we should reduce the file size of the logos for rtables and formatters.
Note also that teal and formatters' logos have the same proportion but rtables one is a square.

Checking for objects/datasets not used can be more complicated because it is harder to automate as they might be used internally or by other repositories, and I don't know a good method to automatically extract which objects are datasets from an R package (checking https://pharmaverse.r-universe.dev/datasets, reports 33 datasets from teal.* packages but it is harder to explore) .

[llrs-roche]

Logos were compressed to reduce size. After the two compressors only a third of the size was left ~500Kb. As described above, logos have different dimension size, reducing it to the minimum would help too to avoid bloating packages.

Manual check of datasets & usage

Go to r-universe.dev/datasets and filter by "teal." => There are 33 datasets
11 from formatters.
Those with an x are those already reviewed. If there is no usage at all edit the line to add "No usage" or something similar.
Note, Github search is not reliable when filtering out some code ("ex_advs -tmc_ex_advs"): some results that only have ex_advs (as per a search with just "ex_advs") will disappear when negating tmc_ex_advs.

• rADTTE @m7pr
• default_cdisc_join_keys @m7pr
• rADAE @m7pr
• rADCM @m7pr
• rADEX @m7pr
• rADLB @m7pr
• rADRS @m7pr
• rADSL @m7pr
• rADTR @m7pr
• rADTTE @m7pr
• rADVS @m7pr
• rADAE @m7pr
• rADLB @m7pr
• rADRS @m7pr
• rADSL @m7pr
• rADTTE @m7pr
• tmc_ex_adae @llrs-roche
• tmc_ex_adaette @llrs-roche
• tmc_ex_adcm @llrs-roche
• tmc_ex_adeg @llrs-roche
• tmc_ex_adex @llrs-roche
• tmc_ex_adlb @llrs-roche
• tmc_ex_admh @llrs-roche
• tmc_ex_adqs @llrs-roche No usage beyond the package
• tmc_ex_adrs @llrs-roche
• tmc_ex_adsl @llrs-roche
• tmc_ex_adtte @llrs-roche
• tmc_ex_advs: @llrs-roche no usage beyond
• rADAE @llrs-roche Duplicated from csdisc.random.data, only used for tests
• rADLB @llrs-roche Duplicated from [csdisc.random.data], only used for tests (https://github.com/insightsengineering/teal.modules.general/blob/46e5267951f341d7245042a1826d31406c0f2417/data-raw/data.R#L1)
• rADRS @llrs-roche Duplicated from csdisc.random.data, only used for tests
• rADSL @llrs-roche Duplicated from csdisc.random.data, labeled as internal but used on examples, README and vignette.
• rADTTE @llrs-roche Duplicated from csdisc.random.data, labeled as internal but used on examples, tests and vignettes
• DM @llrs-roche
• ex_adae @llrs-roche
• ex_adaette @llrs-roche
• ex_adcm @llrs-roche
• ex_adlb @llrs-roche
• ex_admh @llrs-roche
• ex_adqs @llrs-roche
• ex_adrs @llrs-roche
• ex_adsl @llrs-roche
• ex_adtte @llrs-roche No usage
• ex_advs @llrs-roche No usage (different dataset than from tmc_ex_adae?)

From the datasets name (and size) there seems to be some data duplication or reexport (for example rADTTE is in 4 different packages). One way to reduce package size would be to move all these datasets to a single data package.

[m7pr]

From the datasets name (and size) there seems to be some data duplication or reexport (for example rADTTE is in 4 different packages). One way to reduce package size would be to move all these datasets to a single data package.

@llrs-roche in the past we had all datasets in one package, but we decided to delete this package and just move data directly to packages where the data is used. That's why you see some duplications

[m7pr]

However for cases like this, where multiple packages already depend on each other, and they all are using the same dataset we can delete rADAE from teal.modules.general and reuse this data from teal.data

[m7pr]

@llrs-roche 3 PRs where we can remove datasets becuase they all exist in teal.data already

• 87 remove datasets - decrease package size teal.modules.general#818
• 87 remove datasets - decrease package size teal.modules.hermes#396
• 87 remove datasets - decrease package size teal.transform#228