Error in FUN(X[[i]], ...) : subscript out of bounds in boot_specieslevel

[valentinitnelav]

Error appeared in the Poland dataset when computing closeness and betweenness

dt <- read.csv2("../data/Interactions_PL18_ver2.csv")
setDT(dt) # convert to data.table; to convert back just run setDF(dt_raw)

# Prepare a table with all observed interactions, replicated by the number of
# visits observed.
set.seed(123)
dt_interactions <- 
  dt[, .(abund = sum(Individuals, na.rm = TRUE)),
     by = c("Meadow.type", "Plant", "Insect.species")] %>%
  .[rep(1:.N, times = abund)] %>% # expand/explode
  .[sample(1:nrow(.))] # shuffle

dt_meadow <- dt_interactions[Meadow.type == "Meadow"]
dt_pasture <- dt_interactions[Meadow.type == "Pasture"]

# Get an idea about the total number of interactions
dt_meadow$abund %>% unique %>% sum
dt_pasture$abund %>% unique %>% sum
start_spl <- 30
step_spl <- 20

system.time({
  lst_closeness <- 
    boot_specieslevel(lst = list(meadow = dt_meadow,
                                 pasture = dt_pasture),
                      col_lower = "Plant",           # column name for plants
                      col_higher = "Insect.species", # column name for insects
                      index = "closeness",
                      level = "both", 
                      start = start_spl,
                      step = step_spl,
                      n_boot = 100,
                      n_cpu = 3)
})

## Error in FUN(X[[i]], ...) : subscript out of bounds

[valentinitnelav]

Problem was solved by increasing the start sample size start_spl with 10 in this case. The error message could be more informative, though even if is very probable I think is not always this cause for triggering a computation to fail.

So, what happened: it can happen by chance that when sampling from the interactions to build a first sample network, closeness might fail to compute, breaking the parallel chain. Possibly try to pass the error messages coming from bipartite::specieslevel to the console. One should be able to see this in the steps of the track error.

system.time({
  lst_closeness <- 
    boot_specieslevel(lst = list(meadow = dt_meadow,
                                 pasture = dt_pasture),
                      col_lower = "Plant",           # column name for plants
                      col_higher = "Insect.species", # column name for insects
                      index = "closeness",
                      level = "both", 
                      start = start_spl + 10,
                      step = step_spl,
                      n_boot = 100,
                      n_cpu = 3)
})
# user  system elapsed 
# 7.20    0.17  687.08 

[valentinitnelav]

Though the solution presented above fixes the problem, I will keep this open to try to give more informative error messages. This is not urgent though.

[valentinitnelav]

This error appeared again and the fast fix was again to increase the start sample to an even bigger size, almost to more than 50% of the smallest network from the list of networks to compare.

The error message is uninformative and the trace-back of the error messages is also not informative for the user (nor for me anymore...)

 Error in FUN(X[[i]], ...) : subscript out of bounds 
11.
lapply(., "[[", metric_names[i]) 
10.
lengths(x) 
9.
unique(lengths(x)) 
8.
simplify2array(.) 
7.
boot_lst %>% lapply("[[", metric_names[i]) %>% simplify2array 
6.
get_list_of_arrays(boot_lst, metric_names, n, iter_spl_size) 
5.
boot_specieslevel_n(., col_lower = col_lower, col_higher = col_higher, 
    index = index, level = level, start = start, step = step, 
    n_boot = n_boot, n_cpu = n_cpu, ...) 
4.
lapply(., FUN = get_stats_single, probs = probs) 
3.
lst[[i]] %>% boot_specieslevel_n(col_lower = col_lower, col_higher = col_higher, 
    index = index, level = level, start = start, step = step, 
    n_boot = n_boot, n_cpu = n_cpu, ...) %>% lapply(FUN = get_stats_single, 
    probs = probs) 
2.
boot_specieslevel(lst = list(M1 = dt_M1_PL18, M2 = dt_M2_PL18), 
    col_lower = "Plant", col_higher = "Insect.species", index = "closeness", 
    level = "both", start = 10, step = 10, n_boot = 10, n_cpu = 5) 
1.
system.time({
    lst_closeness <- boot_specieslevel(lst = list(M1 = dt_M1_PL18, 
        M2 = dt_M2_PL18), col_lower = "Plant", col_higher = "Insect.species", 
        index = "closeness", level = "both", start = 10, step = 10,  ...