[Bug]: Saving/loading model objects #42
Description
Describe the bug
Is there a preferred save and load procedure and class type for SDMmodel files? I am running numerous SDMtune models and saving them to file before loading them later on to then extract various metrics like auc() and predict() to output raster surfaces. But, I am having issues with being able to use the reloaded model objects and intermittently getting errors from them. It seems to be a class issue with the error commonly being:
"Error in UseMethod("predict") : no applicable method for 'predict' applied to an object of class "randomForest""
But, this goes for maxent, maxnet and BRT methods also. I am using saveRDS() and readRDS() to save and load SDMmodelCV class models. When I read in these objects in they look identical to when they were originally created and are of class SDMmodel or SDMmodelCV, but they are not working (sometimes) with code they have done previously when not saved and reloaded. I have written some example code below, which frustratingly worked when I ran it, but is indicative of my workflow.
For reference, these model files are based on training data and have 4 k-folds as below. I am also applying an optimisation process such that:
methods <- c('Maxnet', 'BRT', 'RF')
m1 <- SDMtune::train(method = methods[j],data = train, sampsize = RF_wts, replace =T, weights = BRT_wts, folds = folds)
m2 <- varSel(m1, metric = "auc", test = TRUE, bg4cor = d_env, method = "spearman", cor_th = 0.9, permut = 1, interactive = F)m3 m3 <- reduceVar(m2, th = 5, metric = "auc", test = TRUE, permut = 1, use_jk = T, interactive = F)
m4 <- optimizeModel(m3, hypers = paramsL[[j]], metric = "auc", interactive = F)
m5 <- combineCV(m4@models[[1]])
Any feedback on this process (which is taking alot of computing time) more broadly would also be appreciated.
Steps to reproduce the bug
library(SDMtune)
files <- list.files(path = file.path(system.file(package = "dismo"), "ex"),
pattern = "grd",
full.names = TRUE)
predictors <- terra::rast(files)
# Prepare presence and background locations
p_coords <- virtualSp$presence
bg_coords <- virtualSp$background
# Create SWD object
data <- prepareSWD(species = "Virtual species",
p = p_coords,
a = bg_coords,
env = predictors,
categorical = "biome")
c(train, test) %<-% trainValTest(data,
test = 0.2,
only_presence = TRUE,
seed = 25)
flds <- randomFolds(train,
k = 4,
seed = 25,
only_presence = T)
## Train a Maxent model
model <- train(method = "Maxent",
data = train,
fc = "l",
reg = 1.5,
iter = 700,
folds = flds)
saveRDS(model, 'eg_model.rda')
class(model)
auc(model)
pred <- raster::as.data.frame(SDMtune::predict(model,data = predictors, type = "cloglog"),xy=T)
model2 <- readRDS('eg_model.rda')
class(model2)
auc(model2)
pred <- raster::as.data.frame(SDMtune::predict(model2,data = predictors, type = "cloglog"),xy=T)Session information
R version 4.3.2 (2023-10-31 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19045)
Matrix products: default
locale:
[1] LC_COLLATE=English_Australia.utf8 LC_CTYPE=English_Australia.utf8 LC_MONETARY=English_Australia.utf8
[4] LC_NUMERIC=C LC_TIME=English_Australia.utf8
time zone: Pacific/Guadalcanal
tzcode source: internal
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] zeallot_0.1.0 SDMtune_1.3.1 patchwork_1.1.3 flextable_0.9.4 RColorBrewer_1.1-3 sdm_1.1-8
[7] viridis_0.6.4 viridisLite_0.4.2 lubridate_1.9.3 forcats_1.0.0 stringr_1.5.1 dplyr_1.1.3
[13] purrr_1.0.2 readr_2.1.4 tidyr_1.3.0 tibble_3.2.1 tidyverse_2.0.0 ggplot2_3.4.4
[19] yardstick_1.3.1 raster_3.6-26 sp_2.1-1 modEvA_3.13.3
loaded via a namespace (and not attached):
[1] DBI_1.1.3 gridExtra_2.3 rlang_1.1.2 magrittr_2.0.3 e1071_1.7-13
[6] compiler_4.3.2 maps_3.4.1.1 sfheaders_0.4.4 systemfonts_1.0.5 vctrs_0.6.4
[11] httpcode_0.3.0 pkgconfig_2.0.3 crayon_1.5.2 fastmap_1.1.1 ellipsis_0.3.2
[16] utf8_1.2.4 promises_1.2.1 rmarkdown_2.25 tzdb_0.4.0 ragg_1.2.6
[21] xfun_0.41 randomForest_4.7-1.1 jsonlite_1.8.7 later_1.3.1 uuid_1.1-1
[26] terra_1.7-55 parallel_4.3.2 R6_2.5.1 stringi_1.7.12 hoardr_0.5.3
[31] Rcpp_1.0.11 knitr_1.45 usethis_2.2.2 httpuv_1.6.12 timechange_0.2.0
[36] tidyselect_1.2.0 rnaturalearth_0.3.4 rstudioapi_0.15.0 yaml_2.3.7 codetools_0.2-19
[41] curl_5.1.0 lattice_0.22-5 shiny_1.8.0 withr_2.5.2 askpass_1.2.0
[46] evaluate_0.23 sf_1.0-14 units_0.8-4 proxy_0.4-27 zip_2.3.0
[51] xml2_1.3.5 pillar_1.9.0 KernSmooth_2.23-22 ncdf4_1.21 generics_0.1.3
[56] hms_1.1.3 munsell_0.5.0 scales_1.2.1 xtable_1.8-4 class_7.3-22
[61] glue_1.6.2 gdtools_0.3.4 tools_4.3.2 gfonts_0.2.0 data.table_1.14.8
[66] fs_1.6.3 grid_4.3.2 colorspace_2.1-0 rerddap_1.0.4 cli_3.6.1
[71] rappdirs_0.3.3 textshaping_0.3.7 plotROC_2.3.1 officer_0.6.3 fontBitstreamVera_0.1.1
[76] fansi_1.0.5 gtable_0.3.4 digest_0.6.33 fontquiver_0.2.1 classInt_0.4-10
[81] crul_1.4.0 htmltools_0.5.7 lifecycle_1.0.4 dismo_1.3-14 httr_1.4.7
[86] mime_0.12 fontLiberation_0.1.0 openssl_2.1.1Additional information
This error is happening intermittently but I cannot determine the root cause. I have tried running SDMtune::auc(), SDMtune::predict, raster::predict(), terra::predict() to attempt to overcome any package issues, but with no success. The error also persists no matter if I use SDMmodelCV class object, or if I subset it such as model@models[[1]] etc
Reproducible example
- I have done my best to provide the steps to reproduce the bug